Status of Windows 2016 LTSC support

[slonopotamus]

1. Docker Desktop dropped support in 2.1.x:

Windows 14393 is marked as deprecated ; it will not be supported anymore in the next major stable release (2.1.0.0 and further)

Latest Docker Desktop version that can be installed on 2016 LTSC was 2.0.0.3. That old version neither has fixes for 8GB nor for 20GB issues.

2. Microsoft mainstream support ends on 2021-10-12

3. @adamrehn says he stopped running tests on Windows 2016 LTSC some time ago.

I see two options here:
a. Drop support for Windows 2016 LTSC
b. Document how to install Docker on it and then do option A somewhere around October :)

[slonopotamus]

UPD: I suddenly noticed that I am mixing Windows Server 2016 LTSC and non-server version.

The non-server has additional issue - it lacks process isolation.

[TBBle]

To be clear, we are meant to be talking about Windows Server LTSC 2016, right? The link in

2. Microsoft mainstream support ends on 2021-10-12

points at the Windows 10 lifecycles, which are different from the Windows Server lifecycles.

Windows Server LTSC 2016's "Mainstream End Date" is 11th January 2022 with "Extended End Date" (security updates) 5 years after that.

I don't think Docker Desktop is supported at all on Windows Server, but I'm not certain of that claim. It probably works, but the support level will probably be "If it breaks, you get to keep both pieces". And the current version certainly won't run on Windows Server LTSC 2016, but can probably be beaten into working on Windows Server LTSC 2019 (Desktop Experience) if someone is so-motivated.

It looks like the current leaning for Docker Engine is to keep HCS v1 support around after all, in parallel to HCS v2 (i.e. Windows Server 2019 or later) support via containerd.

So we may well get Docker Engine 21.xx available for Windows Server LTSC 2016, although that also depends on Mirantis supporting it in the DockerMSFTProvider, which presumably depends on whether MS wants to pay them to support Docker Engine on Windows Server LTSC 2016 once it leaves mainstream support.

If Docker Engine 21.xx hasn't shipped by January 2022, they may well just keep shipping 20.10 patch releases for Windows Server LTSC 2016 during Extended Support, and only deploy 21.xx to Windows Server LTSC 2019 and Windows Server LTSC 2022 (which should be out before Windows Server LTSC 2016 leaves Mainstream support).

It's also possible they'll just ship the same thing to all Windows Server versions, and leave it to the in-engine support for older Windows. I think that's what they do now, I don't think DockerMSFTProvider actually checks the version of Windows Server it's running on before choosing a package.

---

Since I was looking at the Windows 10 support dates anyway, we have some options here too:

1. only care about in-support SAC Professional versions, i.e. 1909 until May 2021, 2004 until December 2021, and 20H2 until May 2022.
2. only care about in-support SAC Professional and Enterprise versions, i.e. 1803-1809 until May 2021, 1909 until May 2022, 2004 until December 2021, and 20H2 until May 2023.
3. just support whatever Docker Desktop for Windows supports, which is, as of Docker Desktop 3.2.0, is 1803 or newer.

• I'm not sure if Docker Desktop are doing option 2, or it's just a coincidence.

4. keep the currently documented "64-bit Windows 10 Pro, Enterprise, or Education (Version 1607 or newer)", aka the "no one's tried this and reported it broken" approach. ^_^

[slonopotamus]

we are meant to be talking about Windows Server LTSC 2016, right?

Right. As I said, I confused myself over server/client flavors.

So, forget what I said about Docker Desktop, we only care about DockerMSFTProvider in context of 2016 LTSC.

[slonopotamus]

I'm failing to install DockerMSFTProvider on 2016 Server LTSC because it hits OneGet/MicrosoftDockerProvider#78. So, currently Microsoft instructions on installing Docker on Windows Server are not working for 2016 LTSC.

UPD: Okay, I managed to use workaround.

For the reference:

#This addresses issue defined in https://github.com/OneGet/MicrosoftDockerProvider/issues/78
[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12
Register-PSRepository -Default -Verbose
Set-PSRepository -Name "PSGallery" -InstallationPolicy Trusted

#Install the DockerMicrosoft PackageManagement provider
Install-Module -Name DockerMsftProvider -Force

#Install the Docker package
Install-Package -Name docker -ProviderName DockerMsftProvider -Force

[slonopotamus]

More problems... Chocolatey fails to install in ue4-build-prerequisites, most likely this is the same TLS issue, but now inside Docker container:

PS C:\Users\Administrator> ue4-docker build 4.21.2 --no-engine --exclude debug --exclude ddc --exclude templates
[ue4-docker build] COMMAND-LINE INVOCATION:
[ue4-docker build] ['ue4-docker', '4.21.2', '--no-engine', '--exclude', 'debug', '--exclude', 'ddc', '--exclude', 'templates']
​
[ue4-docker build] UNREAL ENGINE VERSION SETTINGS:
[ue4-docker build] Custom build:  No
[ue4-docker build] Release:       4.21.2
[ue4-docker build] Repository:    https://github.com/EpicGames/UnrealEngine.git
[ue4-docker build] Branch/tag:    4.21.2-release
​
[ue4-docker build] WINDOWS CONTAINER SETTINGS
[ue4-docker build] Isolation mode:               process
[ue4-docker build] Base OS image tag:            ltsc2016 (host OS is Windows Server version 1607)
[ue4-docker build] Memory limit:                 No limit
[ue4-docker build] Detected max image size:      500GB
[ue4-docker build] Directory to copy DLLs from:  C:\Windows\System32
​
[ue4-docker build] GENERAL SETTINGS
[ue4-docker build] Excluding the following Engine components:
[ue4-docker build] - Debug symbols
[ue4-docker build] - Derived Data Cache (DDC)
[ue4-docker build] - Template projects and samples
​
Retrieving the Git credentials that will be used to clone the UE4 repo
Username: slonopotamus
Password:
​
​
[ue4-docker build] Building image "adamrehn/ue4-build-prerequisites:ltsc2016"...
Sending build context to Docker daemon  2.429MB
Step 1/27 : ARG BASEIMAGE
Step 2/27 : FROM ${BASEIMAGE} AS dlls
 ---> 43241022ff34
Step 3/27 : SHELL ["cmd", "/S", "/C"]
 ---> Running in e8649bcd323c
Removing intermediate container e8649bcd323c
 ---> 19f835cfeb75
Step 4/27 : LABEL com.adamrehn.ue4-docker.sentinel="1"
 ---> Running in e0712d54091e
Removing intermediate container e0712d54091e
 ---> 24a8f46ca337
Step 5/27 : RUN mkdir C:\GatheredDlls && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Not
e that for large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.
 ---> Running in 56e65b6b49a5
​
RUN directive complete. Docker will now commit the filesystem layer to disk.
Note that for large filesystem layers this can take quite some time.
Performing filesystem layer commit...
​
Removing intermediate container 56e65b6b49a5
 ---> 01fffd7cbba7
Step 6/27 : RUN powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolat
ey.org/install.ps1'))" && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that for larg
e filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.
 ---> Running in 01bbb13f13ff
Exception calling "DownloadString" with "1" argument(s): "The request was
aborted: Could not create SSL/TLS secure channel."
At line:1 char:1
+ iex ((New-Object System.Net.WebClient).DownloadString('https://chocol ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : WebException
​
The command 'cmd /S /C powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://c
hocolatey.org/install.ps1'))" && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that f
or large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.' returned a non-zero code: 1
[ue4-docker build] Error: failed to build image "adamrehn/ue4-build-prerequisites:ltsc2016".

https://blog.chocolatey.org/2020/01/remove-support-for-old-tls-versions/ says that chocolatey.org switched off pre-TLS1.2 on 2020-02-03, so this thing is broken for a year already.

Also, see MicrosoftDocs/windowsserverdocs#2783. There's some mess with TLS1.2 on 2019 LTSC. It is claimed to be enabled by default, but in fact it isn't.

And here's how to enable TLS1.2 persistently (via a registry key): https://docs.microsoft.com/en-us/windows-server/identity/ad-fs/operations/manage-ssl-protocols-in-ad-fs#enable-and-disable-tls-12

[TBBle]

I think those last couple of links are talking about ADFS specifically. The fetch-chocolatey RUN probably needs [Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12 added to it too, as documented by Chocolatey themselves:

Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

so that's a flaw in our Dockerfile, it's using old Chocolately install instructions from before the TLS1.2 minimum was enforced on their end.

We could also bundle a copy of install.ps1 locally, rather than fetching it during the build. I suspect it doesn't change much, and when install.ps1 is run, it sets the right values to use TLS1.2 internally, so it's only getting install.ps1 that's tricky.

[slonopotamus]

Created #152 with a fix for TLS-1.2 issue.

[slonopotamus]

Oh nice, next error:

Step 14/27 : RUN curl --progress -L "https://download.microsoft.com/download/8/4/A/84A35BF1-DAFE-4AE8-82AF-AD2AE20B6B14/directx_Jun2010_redist.exe" --output %TEMP%\directx_redist.exe && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that for large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.
 ---> Running in f8a7abfc90e0
curl: option --progress: is ambiguous
curl: try 'curl --help' or 'curl --manual' for more information
The command 'cmd /S /C curl --progress -L "https://download.microsoft.com/download/8/4/A/84A35BF1-DAFE-4AE8-82AF-AD2AE20B6B14/directx_Jun2010_redist.exe" --output %TEMP%\directx_redist.exe && echo. && echo.RUN directive complete. Docker will now commit the filesystem layer to disk. && echo.Note that for large filesystem layers this can take quite some time. && echo.Performing filesystem layer commit... && echo.' returned a non-zero code: 2
[ue4-docker build] Error: failed to build image "adamrehn/ue4-build-prerequisites:ltsc2016".

I'm not sure why --progress is gone.

Inside Docker container:

C:\>where curl
C:\ProgramData\chocolatey\bin\curl.exe

C:\>curl --version
curl 7.76.0 (x86_64-pc-win32) libcurl/7.76.0 OpenSSL/1.1.1k (Schannel) zlib/1.2.11 brotli/1.0.9 zstd/1.4.9 WinIDN libssh2/1.9.0 nghttp2/1.43.0 libgsasl/1.10.0
Release-Date: 2021-03-31
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli gsasl HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL SSPI TLS-SRP Unicode UnixSockets zstd

UPD: According to moby/moby#40914, curl never had --progress flag and it only worked due some weird heuristics. The flag is actually called --progress-bar.

[slonopotamus]

Created #153 with a fix for curl --progress.

[slonopotamus]

Okay, I managed to build 4.21 with changes from #144 (and it passes ue4-docker test). I also had to apply #152 and #153 on top of that, but both of them are unrelated to #144.

[adamrehn]

@slonopotamus thanks for your fixes, those are now merged in ue4-docker version 0.0.84. I'd still like to drop support for Windows Server 2016 at some point in the future so we can eliminate the various awkward fixes that accommodate it (copying DLL files from the host system, using copy.py instead of xcopy, etc.), but that can be left until after Docker drops support for it or mainstream support from Microsoft for Server 2016 ends.

[slonopotamus]

Given that #152 and #153 were merged, 2016 LTSC Server works again.