[JBoss JIRA] (JBTM-1383) LogStoreRecoveryTest hang
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1383?page=com.atlassian.jira.plugin.... ]
Paul Robinson edited comment on JBTM-1383 at 12/13/12 11:33 AM:
----------------------------------------------------------------
The problem is that narayana.sh doesn't kill existing JBoss instances when it starts. The bug is in the command:
{code}
#Make sure no JBoss processes running
for i in `ps -eaf | grep java | grep "standalone*.xml" | grep -v grep | cut -c10-15`; do kill $i; done
{code}
You need to grep on {code}"standalone.*.xml"{code} as ".*" means zero or more of anything.
was (Author: paul.robinson):
The problem is that narayana.sh doesn't kill existing JBoss instances when it starts. The bug is in the command:
{code}
#Make sure no JBoss processes running
for i in `ps -eaf | grep java | grep "standalone*.xml" | grep -v grep | cut -c10-15`; do kill $i; done
{code}
You need to grep on "standalone.*.xml" as ".*" means zero or more of anything.
> LogStoreRecoveryTest hang
> -------------------------
>
> Key: JBTM-1383
> URL: https://issues.jboss.org/browse/JBTM-1383
> Project: JBoss Transaction Manager
> Issue Type: Enhancement
> Security Level: Public(Everyone can see)
> Components: Testing
> Affects Versions: 5.0.0.M2
> Reporter: Michael Musgrove
> Assignee: Paul Robinson
> Attachments: debug.25494
>
>
> http://172.17.131.2/job/jbossts-narayana-java6/123 job is stuck running LogStoreRecoveryTest
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years
[JBoss JIRA] (JBTM-1383) LogStoreRecoveryTest hang
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1383?page=com.atlassian.jira.plugin.... ]
Paul Robinson commented on JBTM-1383:
-------------------------------------
See: http://172.17.131.2/computer/tenafly/builds
Here you can see the last test to fail on tenafly was 'jbossts-narayana-java6' and it was due to JBTM-1386 (ATBridgeTest hangs during Deployment). Here the job was timed out by Hudson and presumably the JBoss process was not killed.
I also remember that when I tried to run narayana.sh on my laptop (paul's Macbook Pro) it first failed with JBTM-1386 and then this issue.
This started happening on Hoboken after ATBridgeTest was timed out by Hudson due to JBTM-1380.
So it's pretty clear that We are not tidying up processes properly when Hudson timesout a job.
> LogStoreRecoveryTest hang
> -------------------------
>
> Key: JBTM-1383
> URL: https://issues.jboss.org/browse/JBTM-1383
> Project: JBoss Transaction Manager
> Issue Type: Enhancement
> Security Level: Public(Everyone can see)
> Components: Testing
> Affects Versions: 5.0.0.M2
> Reporter: Michael Musgrove
> Assignee: Paul Robinson
> Attachments: debug.25494
>
>
> http://172.17.131.2/job/jbossts-narayana-java6/123 job is stuck running LogStoreRecoveryTest
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years
[JBoss JIRA] (JBTM-1383) LogStoreRecoveryTest hang
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1383?page=com.atlassian.jira.plugin.... ]
Paul Robinson commented on JBTM-1383:
-------------------------------------
Looks like this error happened because 'jbossts-narayana-java6' didn't tidy up properly.
{code}
ps aux | grep jboss-modules
hudson 10147 0.1 4.8 455064 247304 ? Sl Dec10 7:53 /usr/local/jdk1.6.0_37/bin/java -Dorg.jboss.byteman.verbose -Djboss.modules.system.pkgs=org.jboss.byteman -Dorg.jboss.byteman.transform.all -javaagent:../lib/byteman.jar=listener:true -ea -Djboss.home.dir=/home/hudson/workspace/jbossts-narayana-java6/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT -Dorg.jboss.boot.log.file=/home/hudson/workspace/jbossts-narayana-java6/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/standalone/log/boot.log -Dlogging.configuration=file:/home/hudson/workspace/jbossts-narayana-java6/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/standalone/configuration/logging.properties -Djboss.bundles.dir=/home/hudson/workspace/jbossts-narayana-java6/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/bundles -jar /home/hudson/workspace/jbossts-narayana-java6/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/jboss-modules.jar -mp /home/hudson/workspace/jbossts-narayana-java6/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/modules -jaxpmodule javax.xml.jaxp-provider org.jboss.as.standalone -server-config standalone-xts.xml
{code}
> LogStoreRecoveryTest hang
> -------------------------
>
> Key: JBTM-1383
> URL: https://issues.jboss.org/browse/JBTM-1383
> Project: JBoss Transaction Manager
> Issue Type: Enhancement
> Security Level: Public(Everyone can see)
> Components: Testing
> Affects Versions: 5.0.0.M2
> Reporter: Michael Musgrove
> Assignee: Paul Robinson
> Attachments: debug.25494
>
>
> http://172.17.131.2/job/jbossts-narayana-java6/123 job is stuck running LogStoreRecoveryTest
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years
[JBoss JIRA] (JBTM-1383) LogStoreRecoveryTest hang
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1383?page=com.atlassian.jira.plugin.... ]
Paul Robinson edited comment on JBTM-1383 at 12/13/12 10:11 AM:
----------------------------------------------------------------
See this output on tenafly http://172.17.131.2/job/JBTM-1383-LogStoreRecoveryTest/5/console
{code}
Running com.hp.mwtests.ts.arjuna.objectstore.LogStoreRecoveryTest
byteman jar is /home/hudson/.m2/repository/org/jboss/byteman/byteman/2.1.2/byteman-2.1.2.jar
TransformListener() : unexpected exception opening server socket java.net.BindException: Address already in use
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383)
at java.net.ServerSocket.bind(ServerSocket.java:328)
at java.net.ServerSocket.bind(ServerSocket.java:286)
at org.jboss.byteman.agent.TransformListener.initialize(TransformListener.java:69)
at org.jboss.byteman.agent.Retransformer.addTransformListener(Retransformer.java:204)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.jboss.byteman.agent.Main.premain(Main.java:216)
at org.jboss.byteman.agent.Main.agentmain(Main.java:230)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:323)
at sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:348)
{code}
It's possible that it was caused by a previous job not tidying up properly or it could be the cause of this issue.
was (Author: paul.robinson):
See this output on tenafly http://172.17.131.2/job/JBTM-1383-LogStoreRecoveryTest/5/console
{code}
Running com.hp.mwtests.ts.arjuna.objectstore.LogStoreRecoveryTest
byteman jar is /home/hudson/.m2/repository/org/jboss/byteman/byteman/2.1.2/byteman-2.1.2.jar
TransformListener() : unexpected exception opening server socket java.net.BindException: Address already in use
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383)
at java.net.ServerSocket.bind(ServerSocket.java:328)
at java.net.ServerSocket.bind(ServerSocket.java:286)
at org.jboss.byteman.agent.TransformListener.initialize(TransformListener.java:69)
at org.jboss.byteman.agent.Retransformer.addTransformListener(Retransformer.java:204)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.jboss.byteman.agent.Main.premain(Main.java:216)
at org.jboss.byteman.agent.Main.agentmain(Main.java:230)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:323)
at sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:348)
{code}
> LogStoreRecoveryTest hang
> -------------------------
>
> Key: JBTM-1383
> URL: https://issues.jboss.org/browse/JBTM-1383
> Project: JBoss Transaction Manager
> Issue Type: Enhancement
> Security Level: Public(Everyone can see)
> Components: Testing
> Affects Versions: 5.0.0.M2
> Reporter: Michael Musgrove
> Assignee: Paul Robinson
> Attachments: debug.25494
>
>
> http://172.17.131.2/job/jbossts-narayana-java6/123 job is stuck running LogStoreRecoveryTest
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years
[JBoss JIRA] (JBTM-1383) LogStoreRecoveryTest hang
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1383?page=com.atlassian.jira.plugin.... ]
Paul Robinson commented on JBTM-1383:
-------------------------------------
See this output on tenafly http://172.17.131.2/job/JBTM-1383-LogStoreRecoveryTest/5/console
{code}
Running com.hp.mwtests.ts.arjuna.objectstore.LogStoreRecoveryTest
byteman jar is /home/hudson/.m2/repository/org/jboss/byteman/byteman/2.1.2/byteman-2.1.2.jar
TransformListener() : unexpected exception opening server socket java.net.BindException: Address already in use
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383)
at java.net.ServerSocket.bind(ServerSocket.java:328)
at java.net.ServerSocket.bind(ServerSocket.java:286)
at org.jboss.byteman.agent.TransformListener.initialize(TransformListener.java:69)
at org.jboss.byteman.agent.Retransformer.addTransformListener(Retransformer.java:204)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.jboss.byteman.agent.Main.premain(Main.java:216)
at org.jboss.byteman.agent.Main.agentmain(Main.java:230)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:323)
at sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:348)
{code}
> LogStoreRecoveryTest hang
> -------------------------
>
> Key: JBTM-1383
> URL: https://issues.jboss.org/browse/JBTM-1383
> Project: JBoss Transaction Manager
> Issue Type: Enhancement
> Security Level: Public(Everyone can see)
> Components: Testing
> Affects Versions: 5.0.0.M2
> Reporter: Michael Musgrove
> Assignee: Paul Robinson
> Attachments: debug.25494
>
>
> http://172.17.131.2/job/jbossts-narayana-java6/123 job is stuck running LogStoreRecoveryTest
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years