Windows TeamCity images don't seem to stop correctly

[wtigotw]

Using: TeamCity Server 2018.1.1 running on Windows 10 Enterprise 1803
Using: Docker Cloud plugin to launch agents
Docker image: jetbrains/teamcity-agent:2018.1.1-nanoserver-sac2016

When a build agent is commanded to shut down by TC, either because of manual request ("Stop" clicked from Agents->Cloud screen) or automatic shutdown via plugin configuration ("Terminate after first build" setting), the container is immediately shut down successfully, but TeamCity reports "Cloud image is not configured to stop agents gracefully. The agent that was not stopped gracefully might be assigned with the build."

I have set a shutdown timeout of two minutes on the container but that does not help. From everything I can tell, the container is gracefully shutting down but I believe the agent process within is not.

[dtretyakov]

@wtigotw, did you tried to stop the same container on this machine via regular docker run ... / docker stop ... / docker rm ... commands? Does it stop build agents gracefully?

Note: since Docker Cloud plugin is 3rd party plugin we don't know how it terminates the docker container.

[wtigotw]

Ah, good point. Yes, I failed to recognize the shutdown mechanism of the plugin could should be considered.

Using docker run ... / docker stop ... / docker rm ..., I see the following in the TC logs:

[2018-08-28 15:41:03,285]   WARN -    jetbrains.buildServer.AGENT - Agent "ip_xx.xx.xx.xx" {id=158} is unauthorized on registration
[2018-08-28 15:45:23,322]   INFO -    jetbrains.buildServer.AGENT - Agent can't be pinged, last communication time: 2018-08-28 15:44:22.457, agent: "ip_xx.xx.xx.xx" {id=158, protocol=unidirectional, host=xx.xx.xx.xx:9090, agentTypeId=11, pool=Default, registered since 2018-08-28 15:41:03.175}
[2018-08-28 15:45:23,322]   INFO - ldServer.AGENT.PollingProtocol - Polling Agent session 'LRkyHfm...' was closed, agent: "ip_xx.xx.xx.xx" {id=158}
[2018-08-28 15:45:23,328]   INFO -    jetbrains.buildServer.AGENT - Agent "ip_xx.xx.xx.xx" {id=158} has been unregistered. Reason: Cannot access agent