[jboss-jira] [JBoss JIRA] (SWSQE-374) When starting 40> more jenkins slaves some of them fail to start
Filip Brychta (JIRA)
issues at jboss.org
Wed Aug 8 09:09:00 EDT 2018
Filip Brychta created SWSQE-374:
-----------------------------------
Summary: When starting 40> more jenkins slaves some of them fail to start
Key: SWSQE-374
URL: https://issues.jboss.org/browse/SWSQE-374
Project: Kiali QE
Issue Type: QE Task
Reporter: Filip Brychta
Assignee: Filip Brychta
Priority: Minor
Some slaves fail to start with following errors:
from jenkins log:
ARNING: Error in provisioning; agent=KubernetesSlave name: jenkins-slave-kiali-ui-tests-fcdsz, template=PodTemplate{inheritFrom='', name='jenkins-slave-kiali-ui-tests', namespace='jenkins-slaves', label='python kiali-ui-tests', nodeSelector='', nodeUsageMode=NORMAL, workspaceVolume=EmptyDirWorkspaceVolume [memory=false], containers=[ContainerTemplate{name='jnlp', image='docker-registry.default.svc:5000/jenkins-slaves/jenkins-slave-kiali-ui-tests', alwaysPullImage=true, workingDir='/home/jenkins', command='', args='${computer.jnlpmac} ${computer.name} ', resourceRequestCpu='', resourceRequestMemory='', resourceLimitCpu='', resourceLimitMemory='', livenessProbe=org.csanchez.jenkins.plugins.kubernetes.ContainerLivenessProbe at 3106f760}], yaml=}
java.lang.IllegalStateException: Node was deleted, computer is null
at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:177)
at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:292)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Aug 08, 2018 8:38:46 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminating Kubernetes instance for agent jenkins-slave-kiali-ui-tests-fcdsz
Aug 08, 2018 8:38:46 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
SEVERE: Computer for agent is null: jenkins-slave-kiali-ui-tests-fcdsz
WARNING: Unable to move atomically, falling back to non-atomic move.
java.nio.file.NoSuchFileException: /var/lib/jenkins/nodes/jenkins-slave-kiali-ui-tests-fcdsz/atomic9143280938281774341tmp -> /var/lib/jenkins/nodes/jenkins-slave-kiali-ui-tests-fcdsz/config.xml
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:396)
at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
at java.nio.file.Files.move(Files.java:1395)
at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:191)
at hudson.XmlFile.write(XmlFile.java:198)
at jenkins.model.Nodes.persistNode(Nodes.java:175)
at jenkins.model.Nodes.addNode(Nodes.java:144)
at jenkins.model.Jenkins.addNode(Jenkins.java:2058)
at hudson.slaves.NodeProvisioner$2.run(NodeProvisioner.java:241)
at hudson.model.Queue._withLock(Queue.java:1380)
at hudson.model.Queue.withLock(Queue.java:1257)
at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:207)
at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:61)
at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(No
WARNING: Unable to move /var/lib/jenkins/nodes/jenkins-slave-kiali-ui-tests-fcdsz/atomic9143280938281774341tmp to /var/lib/jenkins/nodes/jenkins-slave-kiali-ui-tests-fcdsz/config.xml. Attempting to delete /var/lib/jenkins/nodes/jenkins-slave-kiali-ui-tests-fcdsz/atomic9143280938281774341tmp and abandoning.
Aug 08, 2018 8:38:34 AM hudson.slaves.NodeProvisioner$2 run
WARNING: Provisioned agent Kubernetes Pod Template failed to launch
java.nio.file.NoSuchFileException: /var/lib/jenkins/nodes/jenkins-slave-kiali-ui-tests-fcdsz/atomic9143280938281774341tmp
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:409)
at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
at java.nio.file.Files.move(Files.java:1395)
at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:206)
at hudson.XmlFile.write(XmlFile.java:198)
at jenkins.model.Nodes.persistNode(Nodes.java:175)
at jenkins.model.Nodes.addNode(Nodes.java:144)
at jenkins.model.Jenkins.addNode(Jenkins.java:2058)
at hudson.slaves.NodeProvisioner$2.run(NodeProvisioner.java:241)
>From docker log:
INFO: Handshaking
Aug 08, 2018 12:38:43 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins2.bc.jonqe.lab.eng.bos.redhat.com:39765
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: b0:58:42:9c:19:76:a1:78:81:79:d9:fc:9a:e9:19:fd
Aug 08, 2018 12:38:44 PM org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer onRecv
INFO: [JNLP4-connect connection to jenkins2.bc.jonqe.lab.eng.bos.redhat.com/10.16.23.71:39765] Local headers refused by remote: Unknown client name: jenkins-slave-kiali-ui-tests-fcdsz
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: jenkins-slave-kiali-ui-tests-fcdsz
at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
at hudson.remoting.Engine.innerRun(Engine.java:609)
at hudson.remoting.Engine.run(Engine.java:469)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: jenkins-slave-kiali-ui-tests-fcdsz
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.newAbortCause(ConnectionHeadersFilterLayer.java:378)
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecvClosed(ConnectionHeadersFilterLayer.java:433)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:172)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$1500(BIONetworkLayer.java:48)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:247)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
at java.lang.Thread.run(Thread.java:748)
Suppressed: java.nio.channels.ClosedChannelException
... 7 more
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins2.bc.jonqe.lab.eng.bos.redhat.com:39765
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP4-plaintext not supported, skipping
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP3-connect not supported, skipping
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP2-connect not supported, skipping
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP-connect not supported, skipping
Aug 08, 2018 12:38:44 PM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: The server rejected the connection: None of the protocols were accepted
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
at hudson.remoting.Engine.onConnectionRejected(Engine.java:670)
at hudson.remoting.Engine.innerRun(Engine.java:634)
at hudson.remoting.Engine.run(Engine.java:469)
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
More information about the jboss-jira
mailing list