Impact: All nodes are inactive from grid
Experienced: After upgrading our QB test server from 8.0.5 to 8.0.11, all qb agent nodes are shown as inactive on grid page (62 windows agents). Inspection of agent log shows that after the agent updated, it failed to start with the error: java.net.BindException: Address already in use: bind
. We looked for a hung java process on the box, but couldnt find one. Restarting the box resolved the issue.
Agent Log:
2018-07-30 10:09:00,938 [Wrapper-Restarter] INFO com.pmease.quickbuild.Quickbuild - Waiting for finish of existing jobs...
2018-07-30 10:09:00,938 [Wrapper-Restarter] INFO com.pmease.quickbuild.Quickbuild - Stopping plugins...
2018-07-30 10:09:00,954 [Wrapper-Restarter] INFO com.pmease.quickbuild.Quickbuild - Stopping task scheduler...
2018-07-30 10:14:02,402 [Wrapper-Restarter] INFO com.pmease.quickbuild.Quickbuild - QuickBuild agent stopped.
2018-07-30 10:14:19,068 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.Bootstrap - Cleaning cached plugin libraries...
2018-07-30 10:14:21,104 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.Bootstrap - Cleaning temp directory...
2018-07-30 10:14:21,119 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.Bootstrap - Benchmarking...
2018-07-30 10:14:23,925 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.Bootstrap - Connecting to server 'https:/***server address removed***:1443'...
2018-07-30 10:14:24,896 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Updating framework...
2018-07-30 10:14:24,943 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Updating plugins...
2018-07-30 10:14:24,943 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.github_8.0.7.jar'...
2018-07-30 10:14:24,974 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.report.jmeter_8.0.5.jar'...
2018-07-30 10:14:24,974 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.gerrit_8.0.1.jar'...
2018-07-30 10:14:24,974 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.perforce_8.0.2.jar'...
2018-07-30 10:14:24,974 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild_8.0.14.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.notifier.slack_8.0.6.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.bitbucketserver_8.0.1.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.bitbucketcloud_8.0.1.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.git_8.0.9.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.builder.gradle_8.0.7.jar'...
2018-07-30 10:14:25,646 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.tracker.jira_8.0.10.jar'...
2018-07-30 10:14:25,677 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.report.jacoco_8.0.11.jar'...
2018-07-30 10:14:25,677 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.scm.gitlab_8.0.10.jar'...
2018-07-30 10:14:25,677 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.measurement.core_8.0.10.jar'...
2018-07-30 10:14:25,724 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.alert.engine_8.0.10.jar'...
2018-07-30 10:14:25,740 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.report.engine_8.0.5.jar'...
2018-07-30 10:14:25,756 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'com.pmease.quickbuild.plugin.cloud.ec2_8.0.2.jar'...
2018-07-30 10:14:25,912 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.gerrit_8.0.0.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.gitlab_8.0.4.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.git_8.0.3.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.bitbucketserver_8.0.0.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.report.engine_8.0.0.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.report.jmeter_8.0.0.jar'...
2018-07-30 10:14:25,927 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.github_8.0.1.jar'...
2018-07-30 10:14:25,959 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.tracker.jira_8.0.4.jar'...
2018-07-30 10:14:25,990 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.bitbucketcloud_8.0.0.jar'...
2018-07-30 10:14:25,990 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.builder.gradle_8.0.1.jar'...
2018-07-30 10:14:25,990 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.report.jacoco_8.0.5.jar'...
2018-07-30 10:14:25,990 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.cloud.ec2_8.0.0.jar'...
2018-07-30 10:14:26,084 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild_8.0.8.jar'...
2018-07-30 10:14:26,099 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.measurement.core_8.0.4.jar'...
2018-07-30 10:14:26,099 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.notifier.slack_8.0.0.jar'...
2018-07-30 10:14:26,099 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.alert.engine_8.0.4.jar'...
2018-07-30 10:14:26,099 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Removing file 'com.pmease.quickbuild.plugin.scm.perforce_8.0.0.jar'...
2018-07-30 10:14:26,115 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Updating plugins/com.pmease.quickbuild.libs...
2018-07-30 10:14:26,115 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Updating plugins/site...
2018-07-30 10:14:26,131 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Updating asset...
2018-07-30 10:14:26,368 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.AgentUpdater - Downloading file 'release'...
2018-07-30 10:14:26,368 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.Bootstrap - Booting QuickBuild from 'C:\QBAgent'...
2018-07-30 10:14:26,383 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.bootstrap.Bootstrap - Starting framework...
2018-07-30 10:14:34,761 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.equinoxadapter.EquinoxAdapter - Caching plugin libraries...
2018-07-30 10:14:43,393 [WrapperStartStopAppMain] WARN org.eclipse.jetty.util.component.AbstractLifeCycle - FAILED ServerConnector@70cc8492{SSL-HTTP/1.1}{0.0.0.0:443}: java.net.BindException: Address already in use: bind
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at com.pmease.quickbuild.Quickbuild.startJettyServer(Quickbuild.java:388)
at com.pmease.quickbuild.Quickbuild.start(Quickbuild.java:249)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.pmease.quickbuild.equinoxadapter.EquinoxAdapter.startup(EquinoxAdapter.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.pmease.quickbuild.bootstrap.Bootstrap.boot(Bootstrap.java:576)
at com.pmease.quickbuild.bootstrap.Bootstrap.main(Bootstrap.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.tanukisoftware.wrapper.WrapperStartStopApp.run(WrapperStartStopApp.java:400)
at java.lang.Thread.run(Unknown Source)
2018-07-30 10:14:43,425 [WrapperStartStopAppMain] WARN org.eclipse.jetty.util.component.AbstractLifeCycle - FAILED org.eclipse.jetty.server.Server@b137f7d: java.net.BindException: Address already in use: bind
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at com.pmease.quickbuild.Quickbuild.startJettyServer(Quickbuild.java:388)
at com.pmease.quickbuild.Quickbuild.start(Quickbuild.java:249)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.pmease.quickbuild.equinoxadapter.EquinoxAdapter.startup(EquinoxAdapter.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.pmease.quickbuild.bootstrap.Bootstrap.boot(Bootstrap.java:576)
at com.pmease.quickbuild.bootstrap.Bootstrap.main(Bootstrap.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.tanukisoftware.wrapper.WrapperStartStopApp.run(WrapperStartStopApp.java:400)
at java.lang.Thread.run(Unknown Source)
2018-07-30 10:14:44,459 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.Quickbuild - Stopping QuickBuild agent...
2018-07-30 10:14:44,709 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.Quickbuild - Disconnecting from server...
2018-07-30 10:14:44,725 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.Quickbuild - Waiting for finish of existing jobs...
2018-07-30 10:14:44,725 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.Quickbuild - Stopping plugins...
2018-07-30 10:14:44,741 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.Quickbuild - Stopping task scheduler...
2018-07-30 10:14:44,788 [WrapperStartStopAppMain] INFO com.pmease.quickbuild.Quickbuild - QuickBuild agent stopped.java.net.BindException: Address already in use: bind
- solved #5
- replies 6
- views 4085
- stars 1