8:30 a.m.
In case you usually don't look at the result of the AS testsuite, here
is the list of the 39 remaining failing tests:
* org.jboss.test.naming.test.SimpleUnitTestCase.unknown
*
org.jboss.test.classloader.leak.test.AopPreparedClassesClassloaderLeakUnitTestCase.testNoCacheReplicableWar
*
org.jboss.test.cluster.classloader.leak.test.ClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testSimpleWar
*
org.jboss.test.jbossmessaging.clustertest.ClusteredTestCase(tests-jbossmessaging-cluster).testSimpleCluster
*
org.jboss.test.profileservice.test.DeployUnitTestCase(profileservice).testEjb3xDeployment
* org.jboss.test.util.test.SchedulerUnitTestCase.testDefaultScheduler
* org.jboss.test.jacc.test.EJBSpecUnitTestCase(JACC).testRunAsWithRoles
* org.jboss.test.jacc.test.EJBSpecUnitTestCase(JACC).testMDBDeepRunAs
*
org.jboss.test.cluster.classloader.leak.test.ClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testNoPassivationWar
*
org.jboss.test.cluster.classloader.leak.test.FieldGranularityClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testSimpleReplicableWar
*
org.jboss.test.cluster.classloader.leak.test.FieldGranularityClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testFieldWar
*
org.jboss.test.cluster.classloader.leak.test.FieldGranularityClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testFieldNoReplicableWar
* org.jboss.test.timer.test.BasicTimerUnitTestCase.testMDBTimer
*
org.jboss.test.cluster.defaultcfg.test.PartitionRestartUnitTestCase(Default-udp).testStatefulBeanFailover
*
org.jboss.test.compatibility.test.SerialVersionUIDUnitTestCase(tests-compatibility).test401Compatibility
*
org.jboss.test.compatibility.test.SerialVersionUIDUnitTestCase(tests-compatibility).testJ2EERI141Compatibility
*
org.jboss.test.jacc.test.FormAuthUnitTestCase(JACC).testPostDataFormAuth
*
org.jboss.test.web.security.AuthenticatorsExternalizationTestCase(tomcat-federation).testHeaderBasedAuthentication
*
org.jboss.test.web.test.FormAuthUnitTestCase(tomcat-federation).testPostDataFormAuth
*
org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testRoleSecuredSLSB
*
org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testBasicSecuredSLSB
*
org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testBasicSecuredServiceAccess
*
org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testBasicSecuredDIIAccess
*
org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testRoleSecuredServiceAccess
*
org.jboss.test.security.test.EJBSpecUnitTestCase(tests-security-basic-unit).testMDBRunAs
*
org.jboss.test.security.test.EJBSpecUnitTestCase(tests-security-basic-unit).testMDBDeepRunAs
* org.jboss.test.web.test.FormAuthUnitTestCase.testPostDataFormAuth
*
org.jboss.test.refs.test.CircularRefResolutionUnitTestCase.testClientSessionRefByInterface
* org.jboss.test.asynch.AsynchTestCase.
*
org.jboss.test.refs.test.RefResolutionUnitTestCase.testClientSessionRefByInterface
* org.jboss.test.jmx.test.DeployXMBeanUnitTestCase.testSecuredJndiXMBean
*
org.jboss.test.jmx.test.DeployXMBeanUnitTestCase.testPersistentJndiXMBean
* org.jboss.test.naming.test.NamingRestartUnitTestCase.testLookupLink
* org.jboss.test.web.test.JASPIFormAuthUnitTestCase.unknown
* org.jboss.test.naming.test.EjbLinkUnitTestCase.testEjbNoLink
*
org.jboss.test.naming.test.InternalNamingClassReplacementUnitTestCase.testEjbNoLink
*
org.jboss.test.jcaprops.test.JBossRaXmlOverrideUnitTestCase.testJBossRaXmlOverride
* org.jboss.test.jmx.test.JMXConnectorUnitTestCase.testNotification
*
org.jboss.test.jmx.test.JMXConnectorUnitTestCase.testNotificationWithBadListener
8:38 a.m.
On Thu, 2008-06-26 at 14:30 +0200, Sacha Labourey wrote:
In case you usually don't look at the result of the AS testsuite,
here
is the list of the 39 remaining failing tests:
Some of these already have fixes they've just not been committed.
*
org.jboss.test.jmx.test.DeployXMBeanUnitTestCase.testSecuredJndiXMBean
*
org.jboss.test.jmx.test.DeployXMBeanUnitTestCase.testPersistentJndiXMBean
Anil has a fix for this (and probably some other tests that use
the legacy SecurityAssociation in the client?).
http://www.jboss.com/index.html?module=bb&op=viewtopic&t=138047
*
org.jboss.test.naming.test.NamingRestartUnitTestCase.testLookupLink
Brian has a fix for this here:
http://jira.jboss.com/jira/browse/JBNAME-3
*
org.jboss.test.jcaprops.test.JBossRaXmlOverrideUnitTestCase.testJBossRaXmlOverride
Vicky has an uncommitted fix for this as well:
http://jira.jboss.com/jira/browse/JBAS-3415
*
org.jboss.test.jmx.test.JMXConnectorUnitTestCase.testNotification
*
org.jboss.test.jmx.test.JMXConnectorUnitTestCase.testNotificationWithBadListener
There's a long discussion about this here:
http://www.jboss.com/index.html?module=bb&op=viewtopic&t=137781
_______________________________________________
jboss-development mailing list
jboss-development(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-development
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
8:50 a.m.
Maybe we should put the qualified failure in the exclude list with a comment point to the
related JIRA issue. I know some projects do that :-)
Adrian Brock wrote:
On Thu, 2008-06-26 at 14:30 +0200, Sacha Labourey wrote:
> In case you usually don't look at the result of the AS testsuite, here
> is the list of the 39 remaining failing tests:
Some of these already have fixes they've just not been committed.
> * org.jboss.test.jmx.test.DeployXMBeanUnitTestCase.testSecuredJndiXMBean
> *
> org.jboss.test.jmx.test.DeployXMBeanUnitTestCase.testPersistentJndiXMBean
Anil has a fix for this (and probably some other tests that use
the legacy SecurityAssociation in the client?).
http://www.jboss.com/index.html?module=bb&op=viewtopic&t=138047
> *
> org.jboss.test.naming.test.NamingRestartUnitTestCase.testLookupLink
Brian has a fix for this here:
http://jira.jboss.com/jira/browse/JBNAME-3
> *
> org.jboss.test.jcaprops.test.JBossRaXmlOverrideUnitTestCase.testJBossRaXmlOverride
Vicky has an uncommitted fix for this as well:
http://jira.jboss.com/jira/browse/JBAS-3415
> * org.jboss.test.jmx.test.JMXConnectorUnitTestCase.testNotification
> *
> org.jboss.test.jmx.test.JMXConnectorUnitTestCase.testNotificationWithBadListener
>
There's a long discussion about this here:
http://www.jboss.com/index.html?module=bb&op=viewtopic&t=137781
> _______________________________________________
> jboss-development mailing list
> jboss-development(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-development
9:03 a.m.
On Thu, 2008-06-26 at 15:50 +0300, Dimitris Andreadis wrote:
Maybe we should put the qualified failure in the exclude list with a
comment point to the
related JIRA issue. I know some projects do that :-)
I'd rather know the problem still exists then we won't forget
to actually commit the fixes. ;-)
Adrian Brock wrote:
> On Thu, 2008-06-26 at 14:30 +0200, Sacha Labourey wrote:
>> In case you usually don't look at the result of the AS testsuite, here
>> is the list of the 39 remaining failing tests:
>
> Some of these already have fixes they've just not been committed.
>
>> * org.jboss.test.jmx.test.DeployXMBeanUnitTestCase.testSecuredJndiXMBean
>> *
>> org.jboss.test.jmx.test.DeployXMBeanUnitTestCase.testPersistentJndiXMBean
>
> Anil has a fix for this (and probably some other tests that use
> the legacy SecurityAssociation in the client?).
>
> http://www.jboss.com/index.html?module=bb&op=viewtopic&t=138047
>
>> *
>> org.jboss.test.naming.test.NamingRestartUnitTestCase.testLookupLink
>
> Brian has a fix for this here:
>
> http://jira.jboss.com/jira/browse/JBNAME-3
>
>> *
>>
org.jboss.test.jcaprops.test.JBossRaXmlOverrideUnitTestCase.testJBossRaXmlOverride
>
> Vicky has an uncommitted fix for this as well:
>
> http://jira.jboss.com/jira/browse/JBAS-3415
>
>> * org.jboss.test.jmx.test.JMXConnectorUnitTestCase.testNotification
>> *
>> org.jboss.test.jmx.test.JMXConnectorUnitTestCase.testNotificationWithBadListener
>>
>
> There's a long discussion about this here:
>
> http://www.jboss.com/index.html?module=bb&op=viewtopic&t=137781
>
>> _______________________________________________
>> jboss-development mailing list
>> jboss-development(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jboss-development
_______________________________________________
jboss-development mailing list
jboss-development(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-development
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
10:29 a.m.
Adrian Brock wrote:
On Thu, 2008-06-26 at 14:30 +0200, Sacha Labourey wrote:
> In case you usually don't look at the result of the AS testsuite, here
> is the list of the 39 remaining failing tests:
Some of these already have fixes they've just not been committed.
...
Besides the ones Adrian mentioned:
org.jboss.test.cluster.classloader.leak.test.ClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testSimpleWar
org.jboss.test.cluster.classloader.leak.test.ClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testNoPassivationWar
org.jboss.test.cluster.classloader.leak.test.FieldGranularityClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testSimpleReplicableWar
org.jboss.test.cluster.classloader.leak.test.FieldGranularityClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testFieldWar
org.jboss.test.cluster.classloader.leak.test.FieldGranularityClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testFieldNoReplicableWar
Fixed in JGroups; will put in new JGroups once AS 5 CR1 is done.
org.jboss.test.cluster.defaultcfg.test.PartitionRestartUnitTestCase(Default-udp).testStatefulBeanFailover
JBAS-5590. Fixed but there was a .0001% chance the fix caused the
testsuite run to fail, so I reverted the fix until after CR1 since the
failed test is of a very much edge case.
http://lists.jboss.org/pipermail/jboss-development/2008-June/012159.html
--
Brian Stansberry
Lead, AS Clustering
JBoss, a division of Red Hat
brian.stansberry(a)redhat.com
9:25 a.m.
New subject: Naming test timeout was Re: PUSH
On Thu, 2008-06-26 at 14:30 +0200, Sacha Labourey wrote:
In case you usually don't look at the result of the AS testsuite,
here
is the list of the 39 remaining failing tests:
* org.jboss.test.naming.test.SimpleUnitTestCase.unknown
This test passes for me locally, so there's obviously something
happening different on the test machine.
When I look at the log around the time of this test
http://hudson.jboss.org/hudson/job/JBoss-AS-5.0.x-TestSuite-sun15/lastBui...
I see some warnings from the TM and some errors from jgroups:
1) end of previous test
2008-06-26 04:05:00,541 DEBUG
[org.jboss.deployers.plugins.deployers.DeployersImpl] (RMI TCP
Connection(487)-10.18.94.175) Fully Undeployed
vfszip:/qa/services/hudson/
hudson_workspace/workspace/JBoss-AS-5.0.x-TestSuite-sun15/trunk/testsuite/output/lib/naming-pooled.sar
2) TM Warnings (these are repeated)
2008-06-26 04:05:18,631 WARN [com.arjuna.ats.jta.logging.loggerI18N]
(Thread-19)
[com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa]
[com.arjuna.ats.inter
nal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource
to use for recovering non-serializable XAResource < 131075, 28, 26,
49459749504848565458989998
48585256545152521029758511015748974950484856545898999848585256545152521029758511015753
>
2008-06-26 04:05:18,631 WARN
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.internal.arjuna.gandiva.inventory.StaticInventory_1] -
cannot
find null implementation.
2008-06-26 04:05:18,631 WARN
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.arjuna.recovery.RecoverAtomicAction_4] -
RecoverAtomicAction: t
ransaction a120086:bcb0:486344fa:3e90 not activated, unable to replay
phase 2 commit
2008-06-26 04:05:18,632 DEBUG
[com.arjuna.ats.arjuna.logging.arjLoggerI18N]
(Server.Connection:10.18.0.134:34373)
[com.arjuna.ats.arjuna.recovery.ActionStatusService_1
] -
transactionType: /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid:
a120086:bcb0:486344fa:3e99
Status is 7
2008-06-26 04:05:18,637 WARN [com.arjuna.ats.jta.logging.loggerI18N]
(Thread-19)
[com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa]
[com.arjuna.ats.inter
nal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource
to use for recovering non-serializable XAResource < 131075, 28, 26,
49459749504848565458989998
485852565451525210297585110157579749504848565458989998485852565451525210297585110157101
>
2008-06-26 04:05:18,637 WARN
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.internal.arjuna.gandiva.inventory.StaticInventory_1] -
cannot
find null implementation.
2008-06-26 04:05:18,637 WARN
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.arjuna.recovery.RecoverAtomicAction_4] -
RecoverAtomicAction: t
ransaction a120086:bcb0:486344fa:3e99 not activated, unable to replay
phase 2 commit
2008-06-26 04:05:18,637 DEBUG
[com.arjuna.ats.arjuna.logging.arjLoggerI18N]
(Server.Connection:10.18.0.134:34373)
[com.arjuna.ats.arjuna.recovery.ActionStatusService_1
] -
transactionType: /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid:
a120086:bcb0:486344fa:3ea2
Status is 7
2008-06-26 04:05:18,639 WARN [com.arjuna.ats.jta.logging.loggerI18N]
(Thread-19)
[com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa]
[com.arjuna.ats.inter
nal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource
to use for recovering non-serializable XAResource < 131075, 28, 26,
49459749504848565458989998
48585256545152521029758511019750974950484856545898999848585256545152521029758511019755
>
2008-06-26 04:05:18,639 WARN
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.internal.arjuna.gandiva.inventory.StaticInventory_1] -
cannot
find null implementation.
2008-06-26 04:05:18,639 WARN
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.arjuna.recovery.RecoverAtomicAction_4] -
RecoverAtomicAction: t
ransaction a120086:bcb0:486344fa:3ea2 not activated, unable to replay
phase 2 commit
3) JGroups error (there are repeated)
2008-06-26 04:06:02,959 ERROR [org.jgroups.protocols.MPING]
(ReceiverThread) failed receiving packet (from /10.18.0.232:45567)
java.io.IOException: length has to be 4 or 16 bytes (was 10 bytes)
at org.jgroups.stack.IpAddress.readFrom(IpAddress.java:306)
at org.jgroups.Message.readFrom(Message.java:737)
at org.jgroups.protocols.MPING.run(MPING.java:355)
at java.lang.Thread.run(Thread.java:595)
4) Start of next test (5 minutes after step 1)
2008-06-26 04:10:06,294 WARN [org.jboss.deployment.MainDeployer] (RMI
TCP Connection(491)-10.18.94.175) undeploy
'file:/qa/services/hudson/hudson_workspace/workspace/
JBoss-AS-5.0.x-TestSuite-sun15/trunk/testsuite/output/lib/probe.jar' :
package not deployed
2008-06-26 04:10:06,294 INFO [org.jboss.deployment.MainDeployer] (RMI
TCP Connection(491)-10.18.94.175) deploy,
url=file:/qa/services/hudson/hudson_workspace/workspac
e/JBoss-AS-5.0.x-TestSuite-sun15/trunk/testsuite/output/lib/probe.jar
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
10:53 a.m.
New subject: Naming test timeout was Re: PUSH
Adrian Brock wrote:
On Thu, 2008-06-26 at 14:30 +0200, Sacha Labourey wrote:
> In case you usually don't look at the result of the AS testsuite, here
> is the list of the 39 remaining failing tests:
>
>
> * org.jboss.test.naming.test.SimpleUnitTestCase.unknown
This test passes for me locally, so there's obviously something
happening different on the test machine.
When I look at the log around the time of this test
http://hudson.jboss.org/hudson/job/JBoss-AS-5.0.x-TestSuite-sun15/lastBui...
I see some warnings from the TM and some errors from jgroups:
1) end of previous test
2008-06-26 04:05:00,541 DEBUG
[org.jboss.deployers.plugins.deployers.DeployersImpl] (RMI TCP
Connection(487)-10.18.94.175) Fully Undeployed
vfszip:/qa/services/hudson/
hudson_workspace/workspace/JBoss-AS-5.0.x-TestSuite-sun15/trunk/testsuite/output/lib/naming-pooled.sar
2) TM Warnings (these are repeated)
<snip/>
3) JGroups error (there are repeated)
2008-06-26 04:06:02,959 ERROR [org.jgroups.protocols.MPING]
(ReceiverThread) failed receiving packet (from /10.18.0.232:45567)
java.io.IOException: length has to be 4 or 16 bytes (was 10 bytes)
at org.jgroups.stack.IpAddress.readFrom(IpAddress.java:306)
at org.jgroups.Message.readFrom(Message.java:737)
at org.jgroups.protocols.MPING.run(MPING.java:355)
at java.lang.Thread.run(Thread.java:595)
This is logged from the MPING multicast discovery protocol used by JBM's
data channel. Means something else in the cluster is multicasting on
port 45567 (probably other non-MPING JGroups traffic from a UDP
protocol.) MPING just recognizes it doesn't understand the incoming
packet, logs an error and drops it. Should have no meaningful impact on
this server.
These were occuring occasionally throughout the course of the test run.
Seeing more of them during the 5 minute lag here is just a continuation
of what had been happening all along.
4) Start of next test (5 minutes after step 1)
2008-06-26 04:10:06,294 WARN [org.jboss.deployment.MainDeployer] (RMI
TCP Connection(491)-10.18.94.175) undeploy
'file:/qa/services/hudson/hudson_workspace/workspace/
JBoss-AS-5.0.x-TestSuite-sun15/trunk/testsuite/output/lib/probe.jar' :
package not deployed
2008-06-26 04:10:06,294 INFO [org.jboss.deployment.MainDeployer] (RMI
TCP Connection(491)-10.18.94.175) deploy,
url=file:/qa/services/hudson/hudson_workspace/workspac
e/JBoss-AS-5.0.x-TestSuite-sun15/trunk/testsuite/output/lib/probe.jar
--
Brian Stansberry
Lead, AS Clustering
JBoss, a division of Red Hat
brian.stansberry(a)redhat.com
11:10 a.m.
New subject: Naming test timeout was Re: PUSH
Adrian Brock wrote:
This test passes for me locally, so there's obviously something
happening different on the test machine.
. . .
2) TM Warnings (these are repeated)
2008-06-26 04:05:18,631 WARN [com.arjuna.ats.jta.logging.loggerI18N]
(Thread-19)
[com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa]
[com.arjuna.ats.inter
nal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource
to use for recovering non-serializable XAResource < 131075, 28, 26,
49459749504848565458989998
48585256545152521029758511015748974950484856545898999848585256545152521029758511015753
>
Hmm, this normally only happens after a crash and restart. Could a
previous test have crashed and left entries in the tx object store?
2008-06-26 04:05:18,631 WARN
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.internal.arjuna.gandiva.inventory.StaticInventory_1] -
cannot
find null implementation.
Oh boy, null is meant to be a classname -- the log was obviously hosed!
This accounts for the rest of the messages -- garbage out garbage in. If
the TX store is unreadable like this then there must have been a crash
in the middle of writing it on a previous run or else something worse
like running out of disk space or a disk error during the previous or
current run. My money is on it not being a bug in the TX code :-)
regards,
Andrew Dinn
-----------
11:26 a.m.
New subject: XAResource problem was Re: Naming test timeout was Re: PUSH
The first mention of this "broken" XAResource looks to be after the
ha invoker tests, can you have a look at the log to see
if there are any other clues?
This is about 13 minutes before the naming test hangs.
http://hudson.jboss.org/hudson/job/JBoss-AS-5.0.x-TestSuite-sun15/lastBui...
2008-06-26 03:52:16,940 DEBUG
[org.jboss.deployers.plugins.deployers.DeployersImpl] (RMI TCP
Connection(349)-10.18.94.175) Fully Undeployed
vfszip:/qa/services/hudson/
hudson_workspace/workspace/JBoss-AS-5.0.x-TestSuite-sun15/trunk/testsuite/output/lib/ha-invoker.sar
2008-06-26 03:52:18,312 DEBUG [com.arjuna.ats.arjuna.logging.arjLogger]
(Thread-19) Periodic recovery - second pass <Thu, 26 Jun 2008 03:52:18>
2008-06-26 03:52:18,312 DEBUG [com.arjuna.ats.arjuna.logging.arjLogger]
(Thread-19) AtomicActionRecoveryModule: Second pass
2008-06-26 03:52:18,319 DEBUG
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.internal.arjuna.recovery.TransactionStatusConnector_4] -
Transa
ctionStatusManager process for uid bcb0 is ALIVE. connected to host:
10.18.0.134, port: 34714 on socket:
Socket[addr=/10.18.0.134,port=34714,localport=34373]
2008-06-26 03:52:18,321 DEBUG
[com.arjuna.ats.arjuna.logging.arjLoggerI18N]
(Server.Connection:10.18.0.134:34373)
[com.arjuna.ats.arjuna.recovery.ActionStatusService_1
] -
transactionType: /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid:
a120086:bcb0:486344fa:3e90
Status is 7
2008-06-26 03:52:18,335 WARN [com.arjuna.ats.jta.logging.loggerI18N]
(Thread-19)
[com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa]
[com.arjuna.ats.inter
nal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource
to use for recovering non-serializable XAResource < 131075, 28, 26,
49459749504848565458989998
48585256545152521029758511015748974950484856545898999848585256545152521029758511015753
>
2008-06-26 03:52:18,336 WARN
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.internal.arjuna.gandiva.inventory.StaticInventory_1] -
cannot
find null implementation.
On Thu, 2008-06-26 at 16:10 +0100, Andrew Dinn wrote:
Adrian Brock wrote:
> This test passes for me locally, so there's obviously something
> happening different on the test machine.
>
> . . .
> 2) TM Warnings (these are repeated)
>
> 2008-06-26 04:05:18,631 WARN [com.arjuna.ats.jta.logging.loggerI18N]
> (Thread-19)
> [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa]
> [com.arjuna.ats.inter
> nal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource
> to use for recovering non-serializable XAResource < 131075, 28, 26,
> 49459749504848565458989998
>
48585256545152521029758511015748974950484856545898999848585256545152521029758511015753
>
Hmm, this normally only happens after a crash and restart. Could a
previous test have crashed and left entries in the tx object store?
> 2008-06-26 04:05:18,631 WARN
> [com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
> [com.arjuna.ats.internal.arjuna.gandiva.inventory.StaticInventory_1] -
> cannot
> find null implementation.
Oh boy, null is meant to be a classname -- the log was obviously hosed!
This accounts for the rest of the messages -- garbage out garbage in. If
the TX store is unreadable like this then there must have been a crash
in the middle of writing it on a previous run or else something worse
like running out of disk space or a disk error during the previous or
current run. My money is on it not being a bug in the TX code :-)
regards,
Andrew Dinn
-----------
_______________________________________________
jboss-development mailing list
jboss-development(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-development
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
12:55 p.m.
New subject: XAResource problem was Re: Naming test timeout was Re: PUSH
Adrian Brock wrote:
2008-06-26 03:52:18,344 WARN
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.internal.arjuna.gandiva.inventory.StaticInventory_1] -
cannot find null implementation.
2008-06-26 03:52:18,344 WARN
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.arjuna.recovery.RecoverAtomicAction_4] -
RecoverAtomicAction: transaction a120086:bcb0:486344fa:3ea2 not
activated, unable to replay phase 2 commit
Now, when the heuristic outcome happens the failed TX is written to the
object store in order that it can be retrieved for later analysis. The
error happens when the recovery thread tries to scan it and finds a
corrupt entry. So, somewhere between these two messages the object store
appears to have been trashed. The same appears to be the case for the
other TXs which also have commitwhenaborted failures.
In between the two messages there are a whole series of errors with
messages from
org.jboss.test.jca.adapter.TestManagedConnectionFactory.createManagedConnection
Throwable while attempting to get a new connection: null
followed by yet more ResourceExceptions
javax.resource.ResourceException:
at
org.jboss.test.jca.adapter.TestManagedConnectionFactory.matchManagedConnections(TestManagedConnectionFactory.java:150)
and even RuntimeExceptions
java.lang.RuntimeException:
at
org.jboss.test.jca.adapter.TestManagedConnectionFactory.matchManagedConnections(TestManagedConnectionFactory.java:152)
These are also followed by some TX is not active and rollback errors. I
have no idea if these are related. I doubt it though, because none of
them should have any effect on the object store contents for the now
rolled back heuristic TX. It's dead, Jim -- at least until recovery
kicks in.
Is this running clustered? Are the clusters using independent object
stores? Could there be a caching issue here which allows another server
to think the TX is active? This is probably a more fruitful line of
enquiry than anything I can see in the log.
regards,
Andrew Dinn
-----------
The first mention of this "broken" XAResource looks to be
after the
ha invoker tests, can you have a look at the log to see
if there are any other clues?
This is about 13 minutes before the naming test hangs.
http://hudson.jboss.org/hudson/job/JBoss-AS-5.0.x-TestSuite-sun15/lastBui...
Well, working backwards from the first occurence of the recovery
failures I found this report of a heuristic outcome:
2008-06-26 03:50:58,957 INFO
[org.jboss.test.jca.adapter.TestManagedConnection]
(WorkerThread#39[10.18.94.175:52882]) rollback with
xid=1-a120086:bcb0:486344fa:3ea2 for TestManagedConnection#1{xid=null
destroyed=false}
2008-06-26 03:50:58,969 ERROR [org.jboss.ejb.plugins.LogInterceptor]
(WorkerThread#39[10.18.94.175:52882])
TransactionRolledbackLocalException in method: public abstract void
org.jboss.test.jca.interfaces.XAExceptionSessionLocal.testXAExceptionToTransactionRolledbackException(),
causedBy:
javax.transaction.HeuristicMixedException
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commit(TransactionImple.java:252)
at
org.jboss.ejb.plugins.TxInterceptorCMT.endTransaction(TxInterceptorCMT.java:501)
at
org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:361)
. . .
2008-06-26 03:50:58,970 INFO
[org.jboss.test.jca.ejb.XAExceptionTestSessionBean]
(WorkerThread#39[10.18.94.175:52882]) Test worked
The next mention of the associated transaction
(a120086:bcb0:486344fa:3ea2) is in the recovery failure message
2008-06-26 03:52:18,342 DEBUG
[com.arjuna.ats.arjuna.logging.arjLoggerI18N]
(Server.Connection:10.18.0.134:34373)
[com.arjuna.ats.arjuna.recovery.ActionStatusService_1] -
transactionType:
/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid:
a120086:bcb0:486344fa:3ea2
Status is 7
2008-06-26 03:52:18,344 WARN [com.arjuna.ats.jta.logging.loggerI18N]
(Thread-19)
[com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa]
[com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] Could
not find new XAResource to use for recovering non-serializable
XAResource < 131075, 28, 26,
4945974950484856545898999848585256545152521029758511019750974950484856545898999848585256545152521029758511019755
1:28 p.m.
New subject: XAResource problem was Re: Naming test timeout was Re: PUSH
Andrew Dinn wrote:
<snip/>
Is this running clustered? Are the clusters using independent object
stores? Could there be a caching issue here which allows another server
to think the TX is active? This is probably a more fruitful line of
enquiry than anything I can see in the log.
The tests are running the 'all' config, so the clustering services are
enabled, but no, these aren't clustered tests -- the server is running
as a standalone server.
The logs also don't show any other mistakenly forming a cluster with
this one.
regards,
Andrew Dinn
-----------
_______________________________________________
jboss-development mailing list
jboss-development(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-development
--
Brian Stansberry
Lead, AS Clustering
JBoss, a division of Red Hat
brian.stansberry(a)redhat.com
8:02 a.m.
New subject: XAResource problem was Re: Naming test timeout was Re: PUSH
To make sure this doesn't "fall the cracks"
you can reproduce this problem by running the following
in the testsuite in jboss-head.
./build.sh one-test
-Dtest=org.jboss.test.jca.test.XAExceptionUnitTestCase
(and wait for periodic recovery to kick in).
The idea of this test is to make sure that
the server is stable to "random" XAExceptions from
an XAResource, so their's clearly something wrong
somewhere.
NOTE: I don't see a problem running the same
test in jboss-4.2.x
On Thu, 2008-06-26 at 17:55 +0100, Andrew Dinn wrote:
Adrian Brock wrote:
> The first mention of this "broken" XAResource looks to be after the
> ha invoker tests, can you have a look at the log to see
> if there are any other clues?
>
> This is about 13 minutes before the naming test hangs.
>
>
http://hudson.jboss.org/hudson/job/JBoss-AS-5.0.x-TestSuite-sun15/lastBui...
Well, working backwards from the first occurence of the recovery
failures I found this report of a heuristic outcome:
2008-06-26 03:50:58,957 INFO
[org.jboss.test.jca.adapter.TestManagedConnection]
(WorkerThread#39[10.18.94.175:52882]) rollback with
xid=1-a120086:bcb0:486344fa:3ea2 for TestManagedConnection#1{xid=null
destroyed=false}
2008-06-26 03:50:58,969 ERROR [org.jboss.ejb.plugins.LogInterceptor]
(WorkerThread#39[10.18.94.175:52882])
TransactionRolledbackLocalException in method: public abstract void
org.jboss.test.jca.interfaces.XAExceptionSessionLocal.testXAExceptionToTransactionRolledbackException(),
causedBy:
javax.transaction.HeuristicMixedException
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commit(TransactionImple.java:252)
at
org.jboss.ejb.plugins.TxInterceptorCMT.endTransaction(TxInterceptorCMT.java:501)
at
org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:361)
. . .
2008-06-26 03:50:58,970 INFO
[org.jboss.test.jca.ejb.XAExceptionTestSessionBean]
(WorkerThread#39[10.18.94.175:52882]) Test worked
The next mention of the associated transaction
(a120086:bcb0:486344fa:3ea2) is in the recovery failure message
2008-06-26 03:52:18,342 DEBUG
[com.arjuna.ats.arjuna.logging.arjLoggerI18N]
(Server.Connection:10.18.0.134:34373)
[com.arjuna.ats.arjuna.recovery.ActionStatusService_1] -
transactionType:
/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid:
a120086:bcb0:486344fa:3ea2
Status is 7
2008-06-26 03:52:18,344 WARN [com.arjuna.ats.jta.logging.loggerI18N]
(Thread-19)
[com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa]
[com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] Could
not find new XAResource to use for recovering non-serializable
XAResource < 131075, 28, 26,
4945974950484856545898999848585256545152521029758511019750974950484856545898999848585256545152521029758511019755
>
2008-06-26 03:52:18,344 WARN
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.internal.arjuna.gandiva.inventory.StaticInventory_1] -
cannot find null implementation.
2008-06-26 03:52:18,344 WARN
[com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Thread-19)
[com.arjuna.ats.arjuna.recovery.RecoverAtomicAction_4] -
RecoverAtomicAction: transaction a120086:bcb0:486344fa:3ea2 not
activated, unable to replay phase 2 commit
Now, when the heuristic outcome happens the failed TX is written to the
object store in order that it can be retrieved for later analysis. The
error happens when the recovery thread tries to scan it and finds a
corrupt entry. So, somewhere between these two messages the object store
appears to have been trashed. The same appears to be the case for the
other TXs which also have commitwhenaborted failures.
In between the two messages there are a whole series of errors with
messages from
org.jboss.test.jca.adapter.TestManagedConnectionFactory.createManagedConnection
Throwable while attempting to get a new connection: null
followed by yet more ResourceExceptions
javax.resource.ResourceException:
at
org.jboss.test.jca.adapter.TestManagedConnectionFactory.matchManagedConnections(TestManagedConnectionFactory.java:150)
and even RuntimeExceptions
java.lang.RuntimeException:
at
org.jboss.test.jca.adapter.TestManagedConnectionFactory.matchManagedConnections(TestManagedConnectionFactory.java:152)
These are also followed by some TX is not active and rollback errors. I
have no idea if these are related. I doubt it though, because none of
them should have any effect on the object store contents for the now
rolled back heuristic TX. It's dead, Jim -- at least until recovery
kicks in.
Is this running clustered? Are the clusters using independent object
stores? Could there be a caching issue here which allows another server
to think the TX is active? This is probably a more fruitful line of
enquiry than anything I can see in the log.
regards,
Andrew Dinn
-----------
_______________________________________________
jboss-development mailing list
jboss-development(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-development
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
12:38 p.m.
From the Security project perspective:
Test Batch 1:
====================================
*org.jboss.test.security.test.EJBSpecUnitTestCase(tests-security-basic-unit).testMDBRunAs
*org.jboss.test.security.test.EJBSpecUnitTestCase(tests-security-basic-unit).testMDBDeepRunAs
*org.jboss.test.jacc.test.EJBSpecUnitTestCase(JACC).testRunAsWithRoles
*org.jboss.test.jacc.test.EJBSpecUnitTestCase(JACC).testMDBDeepRunAs
====================================
Reason: There is some issue with the lookup of ejb local refs here. I
need to dig further in metadata project for these??
org.jboss.test.security.ejb.CallerBean.java#invokeEcho(...)
Context enc = (Context) ic.lookup("java:comp/env");
Object ref = enc.lookup("ejb/local/CalleeHome"); <==== NPE
enc is null so there is a NPE. Why would the lookup of env become null?
Test Batch 2:
==================================
*org.jboss.test.web.test.FormAuthUnitTestCase(tomcat-federation).testPostDataFormAuth
*org.jboss.test.web.test.JASPIFormAuthUnitTestCase.unknown
*org.jboss.test.web.test.FormAuthUnitTestCase(tomcat-federation).testPostDataFormAuth
*org.jboss.test.jacc.test.FormAuthUnitTestCase(JACC).testPostDataFormAuth
===================================
Reason: http://jira.jboss.com/jira/browse/JBAS-5645 (Fix available when
Remy releases next CR of JBossWeb)
Test Batch 3:
====================================
*org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testRoleSecuredSLSB
*org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testBasicSecuredSLSB
*org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testBasicSecuredServiceAccess
*org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testBasicSecuredDIIAccess
*org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testRoleSecuredServiceAccess
=======================================
Reason: http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4157811
Something to do with WS deployments not getting deployed properly or such.
Other Security related Tests: I have fixes for a few.
The above will be dealt immediately after 5.0.0.CR1
Sacha Labourey wrote:
In case you usually don't look at the result of the AS testsuite,
here
is the list of the 39 remaining failing tests:
* org.jboss.test.naming.test.SimpleUnitTestCase.unknown
*
org.jboss.test.classloader.leak.test.AopPreparedClassesClassloaderLeakUnitTestCase.testNoCacheReplicableWar
*
org.jboss.test.cluster.classloader.leak.test.ClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testSimpleWar
*
org.jboss.test.jbossmessaging.clustertest.ClusteredTestCase(tests-jbossmessaging-cluster).testSimpleCluster
*
org.jboss.test.profileservice.test.DeployUnitTestCase(profileservice).testEjb3xDeployment
* org.jboss.test.util.test.SchedulerUnitTestCase.testDefaultScheduler
* org.jboss.test.jacc.test.EJBSpecUnitTestCase(JACC).testRunAsWithRoles
* org.jboss.test.jacc.test.EJBSpecUnitTestCase(JACC).testMDBDeepRunAs
*
org.jboss.test.cluster.classloader.leak.test.ClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testNoPassivationWar
*
org.jboss.test.cluster.classloader.leak.test.FieldGranularityClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testSimpleReplicableWar
*
org.jboss.test.cluster.classloader.leak.test.FieldGranularityClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testFieldWar
*
org.jboss.test.cluster.classloader.leak.test.FieldGranularityClassloaderLeakUnitTestCase(ClusteredClassLoaderLeak).testFieldNoReplicableWar
* org.jboss.test.timer.test.BasicTimerUnitTestCase.testMDBTimer
*
org.jboss.test.cluster.defaultcfg.test.PartitionRestartUnitTestCase(Default-udp).testStatefulBeanFailover
*
org.jboss.test.compatibility.test.SerialVersionUIDUnitTestCase(tests-compatibility).test401Compatibility
*
org.jboss.test.compatibility.test.SerialVersionUIDUnitTestCase(tests-compatibility).testJ2EERI141Compatibility
*
org.jboss.test.jacc.test.FormAuthUnitTestCase(JACC).testPostDataFormAuth
*
org.jboss.test.web.security.AuthenticatorsExternalizationTestCase(tomcat-federation).testHeaderBasedAuthentication
*
org.jboss.test.web.test.FormAuthUnitTestCase(tomcat-federation).testPostDataFormAuth
*
org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testRoleSecuredSLSB
*
org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testBasicSecuredSLSB
*
org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testBasicSecuredServiceAccess
*
org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testBasicSecuredDIIAccess
*
org.jboss.test.webservice.jbws309.JBWS309TestCase(JACC).testRoleSecuredServiceAccess
*
org.jboss.test.security.test.EJBSpecUnitTestCase(tests-security-basic-unit).testMDBRunAs
*
org.jboss.test.security.test.EJBSpecUnitTestCase(tests-security-basic-unit).testMDBDeepRunAs
* org.jboss.test.web.test.FormAuthUnitTestCase.testPostDataFormAuth
*
org.jboss.test.refs.test.CircularRefResolutionUnitTestCase.testClientSessionRefByInterface
* org.jboss.test.asynch.AsynchTestCase.
*
org.jboss.test.refs.test.RefResolutionUnitTestCase.testClientSessionRefByInterface
* org.jboss.test.jmx.test.DeployXMBeanUnitTestCase.testSecuredJndiXMBean
*
org.jboss.test.jmx.test.DeployXMBeanUnitTestCase.testPersistentJndiXMBean
* org.jboss.test.naming.test.NamingRestartUnitTestCase.testLookupLink
* org.jboss.test.web.test.JASPIFormAuthUnitTestCase.unknown
* org.jboss.test.naming.test.EjbLinkUnitTestCase.testEjbNoLink
*
org.jboss.test.naming.test.InternalNamingClassReplacementUnitTestCase.testEjbNoLink
*
org.jboss.test.jcaprops.test.JBossRaXmlOverrideUnitTestCase.testJBossRaXmlOverride
* org.jboss.test.jmx.test.JMXConnectorUnitTestCase.testNotification
*
org.jboss.test.jmx.test.JMXConnectorUnitTestCase.testNotificationWithBadListener
5776
days inactive
5781
days old
jboss-development@lists.jboss.org
12 comments
6 participants
participants (6)
-
Adrian Brock -
Andrew Dinn -
Anil Saldhana -
Brian Stansberry -
Dimitris Andreadis -
Sacha Labourey