Impact: All nodes are inactive from grid

Experienced: After upgrading our QB test server from 8.0.5 to 8.0.11, all qb agent nodes are shown as inactive on grid page (62 windows agents). Inspection of agent log shows that after the agent updated, it failed to start with the error: java.net.BindException: Address already in use: bind. We looked for a hung java process on the box, but couldnt find one. Restarting the box resolved the issue.

Agent Log:

2018-07-30 10:09:00,938 [Wrapper-Restarter] INFO  com.pmease.quickbuild.Quickbuild - Waiting for finish of existing jobs...
2018-07-30 10:09:00,938 [Wrapper-Restarter] INFO  com.pmease.quickbuild.Quickbuild - Stopping plugins...
2018-07-30 10:09:00,954 [Wrapper-Restarter] INFO  com.pmease.quickbuild.Quickbuild - Stopping task scheduler...
2018-07-30 10:14:02,402 [Wrapper-Restarter] INFO  com.pmease.quickbuild.Quickbuild - QuickBuild agent stopped.
2018-07-30 10:14:19,068 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.Bootstrap - Cleaning cached plugin libraries...
2018-07-30 10:14:21,104 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.Bootstrap - Cleaning temp directory...
2018-07-30 10:14:21,119 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.Bootstrap - Benchmarking...
2018-07-30 10:14:23,925 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.Bootstrap - Connecting to server 'https:/***server address removed***:1443'...
2018-07-30 10:14:24,896 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Updating framework...
2018-07-30 10:14:24,943 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Updating plugins...
2018-07-30 10:14:24,943 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.github_8.0.7.jar'...
2018-07-30 10:14:24,974 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.report.jmeter_8.0.5.jar'...
2018-07-30 10:14:24,974 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.gerrit_8.0.1.jar'...
2018-07-30 10:14:24,974 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.perforce_8.0.2.jar'...
2018-07-30 10:14:24,974 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild_8.0.14.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.notifier.slack_8.0.6.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.bitbucketserver_8.0.1.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.bitbucketcloud_8.0.1.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.git_8.0.9.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.builder.gradle_8.0.7.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.tracker.jira_8.0.10.jar'...
2018-07-30 10:14:25,677 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.report.jacoco_8.0.11.jar'...
2018-07-30 10:14:25,677 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.gitlab_8.0.10.jar'...
2018-07-30 10:14:25,677 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.measurement.core_8.0.10.jar'...
2018-07-30 10:14:25,724 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.alert.engine_8.0.10.jar'...
2018-07-30 10:14:25,740 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.report.engine_8.0.5.jar'...
2018-07-30 10:14:25,756 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.cloud.ec2_8.0.2.jar'...
2018-07-30 10:14:25,912 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.gerrit_8.0.0.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.gitlab_8.0.4.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.git_8.0.3.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.bitbucketserver_8.0.0.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.report.engine_8.0.0.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.report.jmeter_8.0.0.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.github_8.0.1.jar'...
2018-07-30 10:14:25,959 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.tracker.jira_8.0.4.jar'...
2018-07-30 10:14:25,990 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.bitbucketcloud_8.0.0.jar'...
2018-07-30 10:14:25,990 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.builder.gradle_8.0.1.jar'...
2018-07-30 10:14:25,990 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.report.jacoco_8.0.5.jar'...
2018-07-30 10:14:25,990 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.cloud.ec2_8.0.0.jar'...
2018-07-30 10:14:26,084 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild_8.0.8.jar'...
2018-07-30 10:14:26,099 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.measurement.core_8.0.4.jar'...
2018-07-30 10:14:26,099 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.notifier.slack_8.0.0.jar'...
2018-07-30 10:14:26,099 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.alert.engine_8.0.4.jar'...
2018-07-30 10:14:26,099 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.perforce_8.0.0.jar'...
2018-07-30 10:14:26,115 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Updating plugins/com.pmease.quickbuild.libs...
2018-07-30 10:14:26,115 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Updating plugins/site...
2018-07-30 10:14:26,131 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Updating asset...
2018-07-30 10:14:26,368 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'release'...
2018-07-30 10:14:26,368 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.Bootstrap - Booting QuickBuild from 'C:\QBAgent'...
2018-07-30 10:14:26,383 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.bootstrap.Bootstrap - Starting framework...
2018-07-30 10:14:34,761 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.equinoxadapter.EquinoxAdapter - Caching plugin libraries...
2018-07-30 10:14:43,393 [WrapperStartStopAppMain] WARN  org.eclipse.jetty.util.component.AbstractLifeCycle - FAILED ServerConnector@70cc8492{SSL-HTTP/1.1}{0.0.0.0:443}: java.net.BindException: Address already in use: bind
    java.net.BindException: Address already in use: bind
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Unknown Source)
        at sun.nio.ch.Net.bind(Unknown Source)
        at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
        at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
        at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
        at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
        at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.server.Server.doStart(Server.java:366)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at com.pmease.quickbuild.Quickbuild.startJettyServer(Quickbuild.java:388)
        at com.pmease.quickbuild.Quickbuild.start(Quickbuild.java:249)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at com.pmease.quickbuild.equinoxadapter.EquinoxAdapter.startup(EquinoxAdapter.java:59)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at com.pmease.quickbuild.bootstrap.Bootstrap.boot(Bootstrap.java:576)
        at com.pmease.quickbuild.bootstrap.Bootstrap.main(Bootstrap.java:126)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.tanukisoftware.wrapper.WrapperStartStopApp.run(WrapperStartStopApp.java:400)
        at java.lang.Thread.run(Unknown Source)
2018-07-30 10:14:43,425 [WrapperStartStopAppMain] WARN  org.eclipse.jetty.util.component.AbstractLifeCycle - FAILED org.eclipse.jetty.server.Server@b137f7d: java.net.BindException: Address already in use: bind
    java.net.BindException: Address already in use: bind
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Unknown Source)
        at sun.nio.ch.Net.bind(Unknown Source)
        at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
        at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
        at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
        at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
        at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.server.Server.doStart(Server.java:366)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at com.pmease.quickbuild.Quickbuild.startJettyServer(Quickbuild.java:388)
        at com.pmease.quickbuild.Quickbuild.start(Quickbuild.java:249)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at com.pmease.quickbuild.equinoxadapter.EquinoxAdapter.startup(EquinoxAdapter.java:59)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at com.pmease.quickbuild.bootstrap.Bootstrap.boot(Bootstrap.java:576)
        at com.pmease.quickbuild.bootstrap.Bootstrap.main(Bootstrap.java:126)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.tanukisoftware.wrapper.WrapperStartStopApp.run(WrapperStartStopApp.java:400)
        at java.lang.Thread.run(Unknown Source)
2018-07-30 10:14:44,459 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.Quickbuild - Stopping QuickBuild agent...
2018-07-30 10:14:44,709 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.Quickbuild - Disconnecting from server...
2018-07-30 10:14:44,725 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.Quickbuild - Waiting for finish of existing jobs...
2018-07-30 10:14:44,725 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.Quickbuild - Stopping plugins...
2018-07-30 10:14:44,741 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.Quickbuild - Stopping task scheduler...
2018-07-30 10:14:44,788 [WrapperStartStopAppMain] INFO  com.pmease.quickbuild.Quickbuild - QuickBuild agent stopped.java.net.BindException: Address already in use: bind