[JBoss JIRA] (WFLY-5451) LdapCacheInSecurityRealmsTestCase rarely fails with No additional LDAP search was expected
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/WFLY-5451?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on WFLY-5451:
-----------------------------------------------
Ondrej Lukas <olukas(a)redhat.com> changed the Status of [bug 1268220|https://bugzilla.redhat.com/show_bug.cgi?id=1268220] from ASSIGNED to POST
> LdapCacheInSecurityRealmsTestCase rarely fails with No additional LDAP search was expected
> ------------------------------------------------------------------------------------------
>
> Key: WFLY-5451
> URL: https://issues.jboss.org/browse/WFLY-5451
> Project: WildFly
> Issue Type: Bug
> Components: Test Suite
> Affects Versions: 10.0.0.CR2
> Reporter: Ondrej Lukas
> Assignee: Ondrej Lukas
>
> LdapCacheInSecurityRealmsTestCase rarely fails with "No additional LDAP search was expected". Following tests are affected:
> testGroupToPrincipal
> testPrincipalToGroup
> testUsernameToDnFilter
> testUsernameToDnAdvanced
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 2 months
- 1
- 0
[JBoss JIRA] (WFLY-5009) TimeoutException: Could not acquire lock during server startup causes operation ("deploy") to fail
by Michal Vinkler (JIRA)
[ https://issues.jboss.org/browse/WFLY-5009?page=com.atlassian.jira.plugin.... ]
Michal Vinkler commented on WFLY-5009:
--------------------------------------
Still present in EAP 7.0.0.DR11 with all above mentioned causes + one new cause:
{code}
[JBossINF] [0m[31m23:37:53,201 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 67) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./clusterbench: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./clusterbench: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote
[JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:85)
[JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
[JBossINF] at org.jboss.threads.JBossThread.run(JBossThread.java:320)
[JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote
[JBossINF] at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:236)
[JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAndRecord(AbstractLockingInterceptor.java:189)
[JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockKey(AbstractTxLockingInterceptor.java:192)
[JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockOrRegisterBackupLock(AbstractTxLockingInterceptor.java:115)
[JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataReadCommand(PessimisticLockingInterceptor.java:69)
[JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitGetKeyValueCommand(AbstractLockingInterceptor.java:76)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:345)
[JBossINF] at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:330)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:403)
[JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleDefault(StateTransferInterceptor.java:388)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitDataReadCommand(CacheMgmtInterceptor.java:103)
[JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitGetKeyValueCommand(CacheMgmtInterceptor.java:91)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
[JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
[JBossINF] at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:412)
[JBossINF] at org.infinispan.cache.impl.DecoratedCache.get(DecoratedCache.java:437)
[JBossINF] at org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:286)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:164)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:66)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.lambda$schedule$13(InfinispanSessionManager.java:376)
[JBossINF] at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
[JBossINF] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
[JBossINF] at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
[JBossINF] at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
[JBossINF] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
[JBossINF] at java.util.Iterator.forEachRemaining(Iterator.java:116)
[JBossINF] at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
[JBossINF] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
[JBossINF] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
[JBossINF] at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
[JBossINF] at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
[JBossINF] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
[JBossINF] at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
[JBossINF] at org.infinispan.stream.impl.local.AbstractLocalCacheStream.forEach(AbstractLocalCacheStream.java:216)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.schedule(InfinispanSessionManager.java:372)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.start(InfinispanSessionManager.java:156)
[JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.start(DistributableSessionManager.java:71)
[JBossINF] at io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:490)
[JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:101)
[JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:82)
[JBossINF] ... 6 more
[JBossINF]
[JBossINF] [0m[31m23:37:53,233 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "clusterbench-ee7.ear")]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.undertow.deployment.default-server.default-host./clusterbench" => "org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./clusterbench: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote
[JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote"}}
{code}
Server log:
http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-http-...
Probably caused by
https://issues.jboss.org/browse/WFLY-5186 TimeoutException: Unable to acquire lock after 15 seconds
https://issues.jboss.org/browse/WFLY-5128 TimeoutException: Could not acquire lock: Waiting to complete tx
> TimeoutException: Could not acquire lock during server startup causes operation ("deploy") to fail
> --------------------------------------------------------------------------------------------------
>
> Key: WFLY-5009
> URL: https://issues.jboss.org/browse/WFLY-5009
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 10.0.0.Alpha5, 10.0.0.Alpha6, 10.0.0.Beta1
> Reporter: Michal Vinkler
> Assignee: Paul Ferraro
> Priority: Critical
>
> Seen regularly in these scenarios of failover tests:
> ejb-ejbservlet-jvmkill-dist-sync
> ejb-ejbservlet-shutdown-dist-async
> The error is present in EAP 7.0.0.DR5, EAP 7.0.0.DR6. We did not see it in EAP 7.0.0.DR4.
> This error happens every time for the specified scenarios and is seen on one of the nodes in a 4-nodes cluster - usually it is the second node to fail, perf19.
> When the affected server is restarting after failover, it logs these error messages, ending up with *Operation ("deploy") failed*:
> {code}
> [JBossINF] [0m[0m03:49:26,594 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started clusterbench-ee7.ear.clusterbench-ee7-web-passivating.war cache from web container
> [JBossINF] [0m[0m03:49:26,666 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 62) WFLYCLINF0002: Started clusterbench-ee7.ear.clusterbench-ee7-web-default.war cache from web container
> [JBossINF] [0m[0m03:49:26,667 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0002: Started dist cache from web container
> [JBossINF] [0m[0m03:49:27,364 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 65) WFLYCLINF0002: Started dist cache from ejb container
> [JBossINF] [0m[0m03:49:27,409 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 65) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [JBossINF] [0m[0m03:49:27,410 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 65) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [JBossINF] [0m[0m03:49:27,487 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 65) WFLYCLINF0002: Started clusterbench-ee7.ear/clusterbench-ee7-ejb.jar cache from ejb container
> [JBossINF] [0m[0m03:49:28,506 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 67) Initializing Mojarra 2.2.11-jbossorg-1 20150505-1501 for context '/clusterbench'
> [JBossINF] [0m[0m03:49:28,506 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 69) Initializing Mojarra 2.2.11-jbossorg-1 20150505-1501 for context '/clusterbench-passivating'
> [JBossINF] [0m[0m03:49:29,435 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 69) WFLYUT0021: Registered web context: /clusterbench-passivating
> [JBossINF] [0m[31m03:49:44,489 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (ServerService Thread Pool -- 67) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Could not acquire lock on 1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF on behalf of transaction GlobalTransaction:<perf19>:203:local. Waiting to complete tx: RemoteTransaction{modifications=[ReplaceCommand{key=1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF, oldValue=null, newValue=[B@2b5a5fb2, metadata=EmbeddedMetadata{version=null}, flags=[IGNORE_RETURN_VALUES], successful=true, valueMatcher=MATCH_ALWAYS}], lookedUpEntries={}, lockedKeys=null, backupKeyLocks=[1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF], lookedUpEntriesTopology=15, isMarkedForRollback=false, tx=GlobalTransaction:<perf21>:3605:remote, state=null}.
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.newTimeoutException(AbstractTxLockingInterceptor.java:219)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.waitForTransactionsToComplete(AbstractTxLockingInterceptor.java:207)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:171)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataReadCommand(PessimisticLockingInterceptor.java:68)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitGetKeyValueCommand(AbstractLockingInterceptor.java:70)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:86)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:346)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:318)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:86)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:364)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleDefault(StateTransferInterceptor.java:349)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:86)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitDataReadCommand(CacheMgmtInterceptor.java:103)
> [JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitGetKeyValueCommand(CacheMgmtInterceptor.java:91)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:86)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:430)
> [JBossINF] at org.infinispan.cache.impl.DecoratedCache.get(DecoratedCache.java:427)
> [JBossINF] at org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:287)
> [JBossINF] at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:120)
> [JBossINF] at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:56)
> [JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.schedule(InfinispanSessionManager.java:363)
> [JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.start(InfinispanSessionManager.java:156)
> [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.start(DistributableSessionManager.java:69)
> [JBossINF] at io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:485)
> [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:101)
> [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:82)
> [JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> [JBossINF] at org.jboss.threads.JBossThread.run(JBossThread.java:320)
> [JBossINF]
> [JBossINF] [0m[31m03:49:44,492 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 67) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./clusterbench: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./clusterbench: org.infinispan.util.concurrent.TimeoutException: Could not acquire lock on 1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF on behalf of transaction GlobalTransaction:<perf19>:203:local. Waiting to complete tx: RemoteTransaction{modifications=[ReplaceCommand{key=1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF, oldValue=null, newValue=[B@2b5a5fb2, metadata=EmbeddedMetadata{version=null}, flags=[IGNORE_RETURN_VALUES], successful=true, valueMatcher=MATCH_ALWAYS}], lookedUpEntries={}, lockedKeys=null, backupKeyLocks=[1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF], lookedUpEntriesTopology=15, isMarkedForRollback=false, tx=GlobalTransaction:<perf21>:3605:remote, state=null}.
> [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:85)
> [JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> [JBossINF] at org.jboss.threads.JBossThread.run(JBossThread.java:320)
> [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: Could not acquire lock on 1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF on behalf of transaction GlobalTransaction:<perf19>:203:local. Waiting to complete tx: RemoteTransaction{modifications=[ReplaceCommand{key=1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF, oldValue=null, newValue=[B@2b5a5fb2, metadata=EmbeddedMetadata{version=null}, flags=[IGNORE_RETURN_VALUES], successful=true, valueMatcher=MATCH_ALWAYS}], lookedUpEntries={}, lockedKeys=null, backupKeyLocks=[1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF], lookedUpEntriesTopology=15, isMarkedForRollback=false, tx=GlobalTransaction:<perf21>:3605:remote, state=null}.
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.newTimeoutException(AbstractTxLockingInterceptor.java:219)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.waitForTransactionsToComplete(AbstractTxLockingInterceptor.java:207)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:171)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataReadCommand(PessimisticLockingInterceptor.java:68)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitGetKeyValueCommand(AbstractLockingInterceptor.java:70)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:86)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:346)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:318)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:86)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:364)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleDefault(StateTransferInterceptor.java:349)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:86)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitDataReadCommand(CacheMgmtInterceptor.java:103)
> [JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitGetKeyValueCommand(CacheMgmtInterceptor.java:91)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:86)
> [JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:430)
> [JBossINF] at org.infinispan.cache.impl.DecoratedCache.get(DecoratedCache.java:427)
> [JBossINF] at org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:287)
> [JBossINF] at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:120)
> [JBossINF] at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:56)
> [JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.schedule(InfinispanSessionManager.java:363)
> [JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.start(InfinispanSessionManager.java:156)
> [JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.start(DistributableSessionManager.java:69)
> [JBossINF] at io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:485)
> [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:101)
> [JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:82)
> [JBossINF] ... 6 more
> [JBossINF]
> [JBossINF] [0m[31m03:49:44,503 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "clusterbench-ee7.ear")]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.undertow.deployment.default-server.default-host./clusterbench" => "org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./clusterbench: org.infinispan.util.concurrent.TimeoutException: Could not acquire lock on 1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF on behalf of transaction GlobalTransaction:<perf19>:203:local. Waiting to complete tx: RemoteTransaction{modifications=[ReplaceCommand{key=1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF, oldValue=null, newValue=[B@2b5a5fb2, metadata=EmbeddedMetadata{version=null}, flags=[IGNORE_RETURN_VALUES], successful=true, valueMatcher=MATCH_ALWAYS}], lookedUpEntries={}, lockedKeys=null, backupKeyLocks=[1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF], lookedUpEntriesTopology=15, isMarkedForRollback=false, tx=GlobalTransaction:<perf21>:3605:remote, state=null}.
> [JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: Could not acquire lock on 1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF on behalf of transaction GlobalTransaction:<perf19>:203:local. Waiting to complete tx: RemoteTransaction{modifications=[ReplaceCommand{key=1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF, oldValue=null, newValue=[B@2b5a5fb2, metadata=EmbeddedMetadata{version=null}, flags=[IGNORE_RETURN_VALUES], successful=true, valueMatcher=MATCH_ALWAYS}], lookedUpEntries={}, lockedKeys=null, backupKeyLocks=[1tlWOZAPqztbiLjjrJHTL_6OK7fRTPBhVvyB4BLF], lookedUpEntriesTopology=15, isMarkedForRollback=false, tx=GlobalTransaction:<perf21>:3605:remote, state=null}."}}
> {code}
> Server log (failover-ejb-ejbservlet-jvmkill-dist-sync):
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-failo...
> Server log (failover-ejb-ejbservlet-shutdown-dist-async):
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-failo...
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 2 months
- 1
- 0
[JBoss JIRA] (WFLY-5186) TimeoutException: Unable to acquire lock after 15 seconds
by Michal Vinkler (JIRA)
[ https://issues.jboss.org/browse/WFLY-5186?page=com.atlassian.jira.plugin.... ]
Michal Vinkler edited comment on WFLY-5186 at 10/2/15 5:50 AM:
---------------------------------------------------------------
Also, this error resulted in Operation ("deploy") failed during server restart after a failover:
{code}
[JBossINF] [0m[31m23:37:53,201 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 67) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./clusterbench: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./clusterbench: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote
[JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:85)
[JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
[JBossINF] at org.jboss.threads.JBossThread.run(JBossThread.java:320)
[JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote
[JBossINF] at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:236)
[JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAndRecord(AbstractLockingInterceptor.java:189)
[JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockKey(AbstractTxLockingInterceptor.java:192)
[JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockOrRegisterBackupLock(AbstractTxLockingInterceptor.java:115)
[JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataReadCommand(PessimisticLockingInterceptor.java:69)
[JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitGetKeyValueCommand(AbstractLockingInterceptor.java:76)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:345)
[JBossINF] at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:330)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:403)
[JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleDefault(StateTransferInterceptor.java:388)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitDataReadCommand(CacheMgmtInterceptor.java:103)
[JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitGetKeyValueCommand(CacheMgmtInterceptor.java:91)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
[JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
[JBossINF] at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:412)
[JBossINF] at org.infinispan.cache.impl.DecoratedCache.get(DecoratedCache.java:437)
[JBossINF] at org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:286)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:164)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:66)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.lambda$schedule$13(InfinispanSessionManager.java:376)
[JBossINF] at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
[JBossINF] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
[JBossINF] at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
[JBossINF] at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
[JBossINF] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
[JBossINF] at java.util.Iterator.forEachRemaining(Iterator.java:116)
[JBossINF] at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
[JBossINF] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
[JBossINF] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
[JBossINF] at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
[JBossINF] at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
[JBossINF] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
[JBossINF] at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
[JBossINF] at org.infinispan.stream.impl.local.AbstractLocalCacheStream.forEach(AbstractLocalCacheStream.java:216)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.schedule(InfinispanSessionManager.java:372)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.start(InfinispanSessionManager.java:156)
[JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.start(DistributableSessionManager.java:71)
[JBossINF] at io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:490)
[JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:101)
[JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:82)
[JBossINF] ... 6 more
[JBossINF]
[JBossINF] [0m[31m23:37:53,233 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "clusterbench-ee7.ear")]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.undertow.deployment.default-server.default-host./clusterbench" => "org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./clusterbench: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote
[JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote"}}
{code}
Server log:
http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-http-...
May be related to https://issues.jboss.org/browse/JBEAP-505
was (Author: mvinkler):
Also, this error resulted in Operation ("deploy") failed during server restart after a failover:
{code}
[JBossINF] [0m[31m23:37:53,201 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 67) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./clusterbench: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./clusterbench: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote
[JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:85)
[JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
[JBossINF] at org.jboss.threads.JBossThread.run(JBossThread.java:320)
[JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote
[JBossINF] at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:236)
[JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAndRecord(AbstractLockingInterceptor.java:189)
[JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockKey(AbstractTxLockingInterceptor.java:192)
[JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockOrRegisterBackupLock(AbstractTxLockingInterceptor.java:115)
[JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataReadCommand(PessimisticLockingInterceptor.java:69)
[JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitGetKeyValueCommand(AbstractLockingInterceptor.java:76)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:345)
[JBossINF] at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:330)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:403)
[JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleDefault(StateTransferInterceptor.java:388)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitDataReadCommand(CacheMgmtInterceptor.java:103)
[JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitGetKeyValueCommand(CacheMgmtInterceptor.java:91)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
[JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
[JBossINF] at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:412)
[JBossINF] at org.infinispan.cache.impl.DecoratedCache.get(DecoratedCache.java:437)
[JBossINF] at org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:286)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:164)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:66)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.lambda$schedule$13(InfinispanSessionManager.java:376)
[JBossINF] at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
[JBossINF] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
[JBossINF] at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
[JBossINF] at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
[JBossINF] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
[JBossINF] at java.util.Iterator.forEachRemaining(Iterator.java:116)
[JBossINF] at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
[JBossINF] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
[JBossINF] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
[JBossINF] at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
[JBossINF] at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
[JBossINF] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
[JBossINF] at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
[JBossINF] at org.infinispan.stream.impl.local.AbstractLocalCacheStream.forEach(AbstractLocalCacheStream.java:216)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.schedule(InfinispanSessionManager.java:372)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.start(InfinispanSessionManager.java:156)
[JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.start(DistributableSessionManager.java:71)
[JBossINF] at io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:490)
[JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:101)
[JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:82)
[JBossINF] ... 6 more
[JBossINF]
[JBossINF] [0m[31m23:37:53,233 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "clusterbench-ee7.ear")]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.undertow.deployment.default-server.default-host./clusterbench" => "org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./clusterbench: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote
[JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote"}}
{code}
May be related to https://issues.jboss.org/browse/JBEAP-505
> TimeoutException: Unable to acquire lock after 15 seconds
> ---------------------------------------------------------
>
> Key: WFLY-5186
> URL: https://issues.jboss.org/browse/WFLY-5186
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 10.0.0.Beta1
> Reporter: Michal Vinkler
> Assignee: Paul Ferraro
>
> Seen in these scenarios:
> ejb-ejbservlet-shutdown-repl-async
> ejb-ejbservlet-jvmkill-repl-async
> http-session-shutdown-repl-async-tcpStack
> http-session-undeploy-dist-sync
> With no obvious connection to failover of the nodes, usually all the 4 servers log this error few times during the testrun (each server different time):
> {code}
> [JBossINF] [0m[31m17:21:36,562 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (remote-thread--p4-t1) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key D89rThGoUJDT6kuJ-MV7w_4TeNsSpQvBoL2wKi0l and requestor GlobalTransaction:<perf20>:56262:remote. Lock is held by GlobalTransaction:<perf20>:51723:remote, while request came from perf20
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:180)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:118)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:200)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:141)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:199)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitLockControlCommand(TransactionSynchronizerInterceptor.java:75)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:196)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.visitLockControlCommand(StateTransferInterceptor.java:99)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.visitLockControlCommand(InvocationContextInterceptor.java:76)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.perform(LockControlCommand.java:129)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:85)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> The error is usually followed by a warning:
> {code}
> [JBossINF] [0m[33m17:21:36,563 WARN [org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler] (remote-thread--p4-t1) ISPN000071: Caught exception when handling command LockControlCommand{cache=clusterbench-ee7.ear.clusterbench-ee7-web-default.war, keys=[D89rThGoUJDT6kuJ-MV7w_4TeNsSpQvBoL2wKi0l], flags=[IGNORE_RETURN_VALUES], unlock=false, gtx=GlobalTransaction:<perf20>:56262:remote}: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key D89rThGoUJDT6kuJ-MV7w_4TeNsSpQvBoL2wKi0l and requestor GlobalTransaction:<perf20>:56262:remote. Lock is held by GlobalTransaction:<perf20>:51723:remote, while request came from perf20
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:180)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:118)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:200)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:141)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:199)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitLockControlCommand(TransactionSynchronizerInterceptor.java:75)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:196)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.visitLockControlCommand(StateTransferInterceptor.java:99)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.visitLockControlCommand(InvocationContextInterceptor.java:76)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.perform(LockControlCommand.java:129)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:85)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> While the error was logged 15 times by all servers in this particular run, the warning message was logged 505 times.
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
> The same error with different stacktrace:
> {code}
> [JBossINF] [0m[31m17:21:16,585 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-45) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key UnknownSessionID [5365576765655250685155485253536866686865666767514850676951495057] and requestor GlobalTransaction:<perf20>:52228:local. Lock is held by GlobalTransaction:<perf20>:49990:local, while request came from local
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:180)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.lockAndRegisterBackupLock(PessimisticLockingInterceptor.java:261)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataWriteCommand(PessimisticLockingInterceptor.java:131)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitReplaceCommand(AbstractLockingInterceptor.java:60)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:54)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:308)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.visitReplaceCommand(TxInterceptor.java:219)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:54)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTxWriteCommand(StateTransferInterceptor.java:261)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:243)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.visitReplaceCommand(StateTransferInterceptor.java:124)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> [JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitReplaceCommand(CacheMgmtInterceptor.java:182)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:54)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1613)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.replaceInternal(CacheImpl.java:1158)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.replace(CacheImpl.java:1151)
> [JBossINF] at org.infinispan.cache.impl.DecoratedCache.replace(DecoratedCache.java:502)
> [JBossINF] at org.infinispan.cache.impl.AbstractDelegatingCache.replace(AbstractDelegatingCache.java:262)
> [JBossINF] at org.wildfly.clustering.ee.infinispan.CacheEntryMutator.mutate(CacheEntryMutator.java:52)
> [JBossINF] at org.wildfly.clustering.ejb.infinispan.bean.InfinispanBean.close(InfinispanBean.java:113)
> [JBossINF] at org.wildfly.clustering.ejb.infinispan.InfinispanBeanManager$SchedulableBean.close(InfinispanBeanManager.java:406)
> [JBossINF] at org.jboss.as.ejb3.cache.distributable.DistributableCache.release(DistributableCache.java:136)
> [JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.releaseInstance(StatefulSessionSynchronizationInterceptor.java:168)
> [JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor$StatefulSessionSynchronization.afterCompletion(StatefulSessionSynchronizationInterceptor.java:250)
> [JBossINF] at org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList.afterCompletion(JCAOrderedLastSynchronizationList.java:147)
> [JBossINF] at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96)
> [JBossINF] at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:545)
> [JBossINF] at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:101)
> [JBossINF] at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> [JBossINF] at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
> [JBossINF] at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> [JBossINF] at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91)
> [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279)
> [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327)
> [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:66)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
> [JBossINF] at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)
> [JBossINF] at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
> [JBossINF] at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
> [JBossINF] at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:195)
> [JBossINF] at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
> [JBossINF] at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$$$view3.getSerialAndIncrement(Unknown Source)
> [JBossINF] at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
> [JBossINF] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [JBossINF] at java.lang.reflect.Method.invoke(Method.java:497)
> [JBossINF] at org.jboss.weld.util.reflection.Reflections.invokeAndUnwrap(Reflections.java:434)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:127)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)
> [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
> [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
> The timeout stated in the exception is sometimes not exactly 15 seconds, so far I have seen these variants: 14.99 seconds, 14.97 seconds, and also only few milliseconds (!), see this server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-http-...
>
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 2 months
- 1
- 0
[JBoss JIRA] (WFLY-5186) TimeoutException: Unable to acquire lock after 15 seconds
by Michal Vinkler (JIRA)
[ https://issues.jboss.org/browse/WFLY-5186?page=com.atlassian.jira.plugin.... ]
Michal Vinkler commented on WFLY-5186:
--------------------------------------
Also, this error resulted in Operation ("deploy") failed during server restart after a failover:
{code}
[JBossINF] [0m[31m23:37:53,201 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 67) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./clusterbench: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./clusterbench: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote
[JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:85)
[JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
[JBossINF] at org.jboss.threads.JBossThread.run(JBossThread.java:320)
[JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote
[JBossINF] at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:236)
[JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAndRecord(AbstractLockingInterceptor.java:189)
[JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockKey(AbstractTxLockingInterceptor.java:192)
[JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockOrRegisterBackupLock(AbstractTxLockingInterceptor.java:115)
[JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataReadCommand(PessimisticLockingInterceptor.java:69)
[JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitGetKeyValueCommand(AbstractLockingInterceptor.java:76)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:345)
[JBossINF] at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:330)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:403)
[JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleDefault(StateTransferInterceptor.java:388)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitDataReadCommand(CacheMgmtInterceptor.java:103)
[JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitGetKeyValueCommand(CacheMgmtInterceptor.java:91)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
[JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
[JBossINF] at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
[JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
[JBossINF] at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:412)
[JBossINF] at org.infinispan.cache.impl.DecoratedCache.get(DecoratedCache.java:437)
[JBossINF] at org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:286)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:164)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:66)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.lambda$schedule$13(InfinispanSessionManager.java:376)
[JBossINF] at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
[JBossINF] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
[JBossINF] at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
[JBossINF] at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
[JBossINF] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
[JBossINF] at java.util.Iterator.forEachRemaining(Iterator.java:116)
[JBossINF] at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
[JBossINF] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
[JBossINF] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
[JBossINF] at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
[JBossINF] at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
[JBossINF] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
[JBossINF] at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
[JBossINF] at org.infinispan.stream.impl.local.AbstractLocalCacheStream.forEach(AbstractLocalCacheStream.java:216)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.schedule(InfinispanSessionManager.java:372)
[JBossINF] at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.start(InfinispanSessionManager.java:156)
[JBossINF] at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.start(DistributableSessionManager.java:71)
[JBossINF] at io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:490)
[JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:101)
[JBossINF] at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:82)
[JBossINF] ... 6 more
[JBossINF]
[JBossINF] [0m[31m23:37:53,233 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "clusterbench-ee7.ear")]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.undertow.deployment.default-server.default-host./clusterbench" => "org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./clusterbench: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote
[JBossINF] Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 14.52 seconds for key t01lwcG5RwFtd5RENoF8R1uDGGO0o_642OOMvbEj and requestor GlobalTransaction:<perf18>:3220:local. Lock is held by GlobalTransaction:<perf20>:28507:remote"}}
{code}
May be related to https://issues.jboss.org/browse/JBEAP-505
> TimeoutException: Unable to acquire lock after 15 seconds
> ---------------------------------------------------------
>
> Key: WFLY-5186
> URL: https://issues.jboss.org/browse/WFLY-5186
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 10.0.0.Beta1
> Reporter: Michal Vinkler
> Assignee: Paul Ferraro
>
> Seen in these scenarios:
> ejb-ejbservlet-shutdown-repl-async
> ejb-ejbservlet-jvmkill-repl-async
> http-session-shutdown-repl-async-tcpStack
> http-session-undeploy-dist-sync
> With no obvious connection to failover of the nodes, usually all the 4 servers log this error few times during the testrun (each server different time):
> {code}
> [JBossINF] [0m[31m17:21:36,562 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (remote-thread--p4-t1) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key D89rThGoUJDT6kuJ-MV7w_4TeNsSpQvBoL2wKi0l and requestor GlobalTransaction:<perf20>:56262:remote. Lock is held by GlobalTransaction:<perf20>:51723:remote, while request came from perf20
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:180)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:118)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:200)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:141)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:199)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitLockControlCommand(TransactionSynchronizerInterceptor.java:75)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:196)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.visitLockControlCommand(StateTransferInterceptor.java:99)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.visitLockControlCommand(InvocationContextInterceptor.java:76)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.perform(LockControlCommand.java:129)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:85)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> The error is usually followed by a warning:
> {code}
> [JBossINF] [0m[33m17:21:36,563 WARN [org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler] (remote-thread--p4-t1) ISPN000071: Caught exception when handling command LockControlCommand{cache=clusterbench-ee7.ear.clusterbench-ee7-web-default.war, keys=[D89rThGoUJDT6kuJ-MV7w_4TeNsSpQvBoL2wKi0l], flags=[IGNORE_RETURN_VALUES], unlock=false, gtx=GlobalTransaction:<perf20>:56262:remote}: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key D89rThGoUJDT6kuJ-MV7w_4TeNsSpQvBoL2wKi0l and requestor GlobalTransaction:<perf20>:56262:remote. Lock is held by GlobalTransaction:<perf20>:51723:remote, while request came from perf20
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:180)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:118)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:200)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:141)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:199)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitLockControlCommand(TransactionSynchronizerInterceptor.java:75)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:196)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.visitLockControlCommand(StateTransferInterceptor.java:99)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.visitLockControlCommand(InvocationContextInterceptor.java:76)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.perform(LockControlCommand.java:129)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:85)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> While the error was logged 15 times by all servers in this particular run, the warning message was logged 505 times.
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
> The same error with different stacktrace:
> {code}
> [JBossINF] [0m[31m17:21:16,585 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-45) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key UnknownSessionID [5365576765655250685155485253536866686865666767514850676951495057] and requestor GlobalTransaction:<perf20>:52228:local. Lock is held by GlobalTransaction:<perf20>:49990:local, while request came from local
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:180)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.lockAndRegisterBackupLock(PessimisticLockingInterceptor.java:261)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataWriteCommand(PessimisticLockingInterceptor.java:131)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitReplaceCommand(AbstractLockingInterceptor.java:60)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:54)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:308)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.visitReplaceCommand(TxInterceptor.java:219)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:54)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTxWriteCommand(StateTransferInterceptor.java:261)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:243)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.visitReplaceCommand(StateTransferInterceptor.java:124)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> [JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitReplaceCommand(CacheMgmtInterceptor.java:182)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:54)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1613)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.replaceInternal(CacheImpl.java:1158)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.replace(CacheImpl.java:1151)
> [JBossINF] at org.infinispan.cache.impl.DecoratedCache.replace(DecoratedCache.java:502)
> [JBossINF] at org.infinispan.cache.impl.AbstractDelegatingCache.replace(AbstractDelegatingCache.java:262)
> [JBossINF] at org.wildfly.clustering.ee.infinispan.CacheEntryMutator.mutate(CacheEntryMutator.java:52)
> [JBossINF] at org.wildfly.clustering.ejb.infinispan.bean.InfinispanBean.close(InfinispanBean.java:113)
> [JBossINF] at org.wildfly.clustering.ejb.infinispan.InfinispanBeanManager$SchedulableBean.close(InfinispanBeanManager.java:406)
> [JBossINF] at org.jboss.as.ejb3.cache.distributable.DistributableCache.release(DistributableCache.java:136)
> [JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.releaseInstance(StatefulSessionSynchronizationInterceptor.java:168)
> [JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor$StatefulSessionSynchronization.afterCompletion(StatefulSessionSynchronizationInterceptor.java:250)
> [JBossINF] at org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList.afterCompletion(JCAOrderedLastSynchronizationList.java:147)
> [JBossINF] at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96)
> [JBossINF] at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:545)
> [JBossINF] at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:101)
> [JBossINF] at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> [JBossINF] at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
> [JBossINF] at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> [JBossINF] at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91)
> [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279)
> [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327)
> [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:66)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
> [JBossINF] at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)
> [JBossINF] at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
> [JBossINF] at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
> [JBossINF] at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:195)
> [JBossINF] at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
> [JBossINF] at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$$$view3.getSerialAndIncrement(Unknown Source)
> [JBossINF] at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
> [JBossINF] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [JBossINF] at java.lang.reflect.Method.invoke(Method.java:497)
> [JBossINF] at org.jboss.weld.util.reflection.Reflections.invokeAndUnwrap(Reflections.java:434)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:127)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)
> [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
> [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
> The timeout stated in the exception is sometimes not exactly 15 seconds, so far I have seen these variants: 14.99 seconds, 14.97 seconds, and also only few milliseconds (!), see this server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-http-...
>
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 2 months
- 1
- 0
[JBoss JIRA] (DROOLS-933) Allow to optionally pass a dependency filter when creating KieModuleMetaData
by Michael Anstis (JIRA)
[ https://issues.jboss.org/browse/DROOLS-933?page=com.atlassian.jira.plugin... ]
Michael Anstis commented on DROOLS-933:
---------------------------------------
See https://github.com/droolsjbpm/kie-wb-common/pull/97 for changes to *kie-wb-common* following the change in behaviour introduced by this enhancement.
> Allow to optionally pass a dependency filter when creating KieModuleMetaData
> ----------------------------------------------------------------------------
>
> Key: DROOLS-933
> URL: https://issues.jboss.org/browse/DROOLS-933
> Project: Drools
> Issue Type: Feature Request
> Reporter: Mario Fusco
> Assignee: Mario Fusco
> Fix For: 6.4.x
>
>
> When creating the meta data for a given KieModule it is required to optionally filter its dependencies with the following filtering interface:
> {code}
> public interface DependencyFilter {
> boolean accept(ReleaseId releaseId, String scope);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 2 months
- 1
- 0
[JBoss JIRA] (WFLY-5208) UT005023: Exception handling request: NullPointerException
by Michal Vinkler (JIRA)
[ https://issues.jboss.org/browse/WFLY-5208?page=com.atlassian.jira.plugin.... ]
Michal Vinkler edited comment on WFLY-5208 at 10/2/15 5:41 AM:
---------------------------------------------------------------
In EAP 7.0.0.DR11 run, I saw this exception which appeared near the Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133)
{code}
[JBossINF] [0m[31m16:48:34,010 ERROR [io.undertow.request] (default task-8) UT005023: Exception handling request to /clusterbench/ejbservlet: org.jboss.weld.exceptions.CreationException: WELD-000079: Could not instantiate a proxy for a session bean: Session bean [class org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB with qualifiers [@Any @Default]; local interfaces are [LocalStatefulSB]
[JBossINF] Proxy: class org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.newInstance(SessionBeanProxyInstantiator.java:72)
[JBossINF] at org.jboss.weld.bean.SessionBean.create(SessionBean.java:149)
[JBossINF] at org.jboss.weld.util.bean.IsolatedForwardingBean.create(IsolatedForwardingBean.java:45)
[JBossINF] at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:96)
[JBossINF] at org.jboss.weld.context.PassivatingContextWrapper$AbstractPassivatingContextWrapper.get(PassivatingContextWrapper.java:76)
[JBossINF] at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:101)
[JBossINF] at org.jboss.weld.bean.ContextualInstanceStrategy$CachingContextualInstanceStrategy.get(ContextualInstanceStrategy.java:178)
[JBossINF] at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50)
[JBossINF] at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:99)
[JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.getInstance(ProxyMethodHandler.java:125)
[JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
[JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
[JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
[JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
[JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
[JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
[JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
[JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
[JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
[JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
[JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
[JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
[JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
[JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
[JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:198)
[JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:784)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
[JBossINF] Caused by: java.lang.NullPointerException
[JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionComponent.createSession(StatefulSessionComponent.java:238)
[JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.<init>(StatefulSessionObjectReferenceImpl.java:70)
[JBossINF] at org.jboss.as.weld.services.bootstrap.WeldEjbServices.resolveEjb(WeldEjbServices.java:53)
[JBossINF] at org.jboss.weld.bean.SessionBean.createReference(SessionBean.java:230)
[JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.<init>(EnterpriseBeanProxyMethodHandler.java:83)
[JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.<init>(EnterpriseBeanProxyMethodHandler.java:63)
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.createEnterpriseTargetBeanInstance(SessionBeanProxyInstantiator.java:80)
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.newInstance(SessionBeanProxyInstantiator.java:61)
[JBossINF] ... 40 more
{code}
Server log:
http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
Can you confirm it is the same case as the other NPE (meaning it is related and the recommended solution is to use suspend/resume or graceful shutdown)?
Is there really no way how to avoid NPEs? Throwing some other reasonable exception perhaps?
was (Author: mvinkler):
In EAP 7.0.0.DR11 run, I saw this exception which appeared near the Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133)
{code}
[JBossINF] [0m[31m16:48:34,010 ERROR [io.undertow.request] (default task-8) UT005023: Exception handling request to /clusterbench/ejbservlet: org.jboss.weld.exceptions.CreationException: WELD-000079: Could not instantiate a proxy for a session bean: Session bean [class org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB with qualifiers [@Any @Default]; local interfaces are [LocalStatefulSB]
[JBossINF] Proxy: class org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.newInstance(SessionBeanProxyInstantiator.java:72)
[JBossINF] at org.jboss.weld.bean.SessionBean.create(SessionBean.java:149)
[JBossINF] at org.jboss.weld.util.bean.IsolatedForwardingBean.create(IsolatedForwardingBean.java:45)
[JBossINF] at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:96)
[JBossINF] at org.jboss.weld.context.PassivatingContextWrapper$AbstractPassivatingContextWrapper.get(PassivatingContextWrapper.java:76)
[JBossINF] at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:101)
[JBossINF] at org.jboss.weld.bean.ContextualInstanceStrategy$CachingContextualInstanceStrategy.get(ContextualInstanceStrategy.java:178)
[JBossINF] at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50)
[JBossINF] at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:99)
[JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.getInstance(ProxyMethodHandler.java:125)
[JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
[JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
[JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
[JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
[JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
[JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
[JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
[JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
[JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
[JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
[JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
[JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
[JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
[JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
[JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:198)
[JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:784)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
[JBossINF] Caused by: java.lang.NullPointerException
[JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionComponent.createSession(StatefulSessionComponent.java:238)
[JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.<init>(StatefulSessionObjectReferenceImpl.java:70)
[JBossINF] at org.jboss.as.weld.services.bootstrap.WeldEjbServices.resolveEjb(WeldEjbServices.java:53)
[JBossINF] at org.jboss.weld.bean.SessionBean.createReference(SessionBean.java:230)
[JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.<init>(EnterpriseBeanProxyMethodHandler.java:83)
[JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.<init>(EnterpriseBeanProxyMethodHandler.java:63)
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.createEnterpriseTargetBeanInstance(SessionBeanProxyInstantiator.java:80)
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.newInstance(SessionBeanProxyInstantiator.java:61)
[JBossINF] ... 40 more
{code}
Server log:
http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
Can you confirm it is the same case as the other NPE (meaning it is related and the recommended solution is to use suspend/resume or graceful shutdown)?
> UT005023: Exception handling request: NullPointerException
> ----------------------------------------------------------
>
> Key: WFLY-5208
> URL: https://issues.jboss.org/browse/WFLY-5208
> Project: WildFly
> Issue Type: Bug
> Components: CDI / Weld
> Reporter: Michal Vinkler
> Assignee: Stuart Douglas
>
> Setup: 4-node cluster, one node at time either shuts down or undeploys application (clusterbench-ee7.ear), and then recovers after period of time, while standalone clients keep calling the application.
> This error was seen during clustering failover testing in the following scenarios:
> ejb-ejbservlet-shutdown-repl-async (failover of HTTP traffic accessing a servlet that locally invokes a stateful clustered EJB, failover type: server shutdown, with asynchronously replicated cache)
> ejb-ejbservlet-shutdown-repl-sync (the same as above, with synchronously replicated cache)
> ejb-ejbservlet-undeploy-dist-sync (failover type: application undeploy, with distributed cache which is replicated synchronously)
> It occurs in these two situations (intermittently):
> 1. server startup after previous shutdown
> {code}
> [JBossINF] [0m[0m17:23:14,240 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started repl cache from web container
> [JBossINF] [0m[0m17:23:14,247 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 67) WFLYCLINF0002: Started clusterbench-ee7.ear.clusterbench-ee7-web-default.war cache from web container
> [JBossINF] [0m[0m17:23:14,374 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 66) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [JBossINF] [0m[0m17:23:14,375 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 66) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [JBossINF] [0m[0m17:23:15,708 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 65) Initializing Mojarra 2.2.12-jbossorg-2 20150729-1131 for context '/clusterbench'
> [JBossINF] [0m[0m17:23:15,708 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 71) Initializing Mojarra 2.2.12-jbossorg-2 20150729-1131 for context '/clusterbench-passivating'
> [JBossINF] [0m[0m17:23:16,784 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 71) WFLYUT0021: Registered web context: /clusterbench-passivating
> [JBossINF] [0m[0m17:23:17,198 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 65) WFLYUT0021: Registered web context: /clusterbench
> [JBossINF] [0m[31m17:23:18,931 ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException
> [JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:122)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)
> [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
> [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
> 2. Server shutdown/application undeploy
> {code}
> [JBossINF] [0m[0m18:04:35,977 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0211: Suspending server
> [JBossINF] [0m[0m18:04:35,981 INFO [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested.
> 2015/08/18 18:04:36:037 EDT [DEBUG][RMI TCP Connection(14)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - [SHUTDOWN] JBossShutdown command executed successfully.
> 2015/08/18 18:04:36:037 EDT [DEBUG][RMI TCP Connection(14)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Waiting for server to shutdown.
> [JBossINF] [0m[0m18:04:36,048 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 77) WFLYUT0022: Unregistered web context: /clusterbench-passivating
> [JBossINF] [0m[0m18:04:36,063 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 81) WFLYUT0022: Unregistered web context: /clusterbench
> [JBossINF] [0m[0m18:04:36,072 INFO [org.infinispan.eviction.impl.PassivationManagerImpl] (ServerService Thread Pool -- 82) ISPN000029: Passivating all entries to disk
> [JBossINF] [0m[31m18:04:36,106 ERROR [io.undertow.request] (default task-63) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException
> [JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:122)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)
> [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
> [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
> During server shutdown, it occurred also with different stacktrace:
> {code}
> [JBossINF] [0m[31m17:19:11,728 ERROR [io.undertow.request] (default task-128) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException
> [JBossINF] at org.jboss.weld.servlet.WeldInitialListener.requestInitialized(WeldInitialListener.java:143)
> [JBossINF] at io.undertow.servlet.core.ApplicationListeners.requestInitialized(ApplicationListeners.java:216)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:281)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> [JBossINF]
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 2 months
- 1
- 0
[JBoss JIRA] (WFLY-5208) UT005023: Exception handling request: NullPointerException
by Michal Vinkler (JIRA)
[ https://issues.jboss.org/browse/WFLY-5208?page=com.atlassian.jira.plugin.... ]
Michal Vinkler edited comment on WFLY-5208 at 10/2/15 5:36 AM:
---------------------------------------------------------------
In EAP 7.0.0.DR11 run, I saw this exception which appeared near the Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133)
{code}
[JBossINF] [0m[31m16:48:34,010 ERROR [io.undertow.request] (default task-8) UT005023: Exception handling request to /clusterbench/ejbservlet: org.jboss.weld.exceptions.CreationException: WELD-000079: Could not instantiate a proxy for a session bean: Session bean [class org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB with qualifiers [@Any @Default]; local interfaces are [LocalStatefulSB]
[JBossINF] Proxy: class org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.newInstance(SessionBeanProxyInstantiator.java:72)
[JBossINF] at org.jboss.weld.bean.SessionBean.create(SessionBean.java:149)
[JBossINF] at org.jboss.weld.util.bean.IsolatedForwardingBean.create(IsolatedForwardingBean.java:45)
[JBossINF] at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:96)
[JBossINF] at org.jboss.weld.context.PassivatingContextWrapper$AbstractPassivatingContextWrapper.get(PassivatingContextWrapper.java:76)
[JBossINF] at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:101)
[JBossINF] at org.jboss.weld.bean.ContextualInstanceStrategy$CachingContextualInstanceStrategy.get(ContextualInstanceStrategy.java:178)
[JBossINF] at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50)
[JBossINF] at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:99)
[JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.getInstance(ProxyMethodHandler.java:125)
[JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
[JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
[JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
[JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
[JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
[JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
[JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
[JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
[JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
[JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
[JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
[JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
[JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
[JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
[JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:198)
[JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:784)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
[JBossINF] Caused by: java.lang.NullPointerException
[JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionComponent.createSession(StatefulSessionComponent.java:238)
[JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.<init>(StatefulSessionObjectReferenceImpl.java:70)
[JBossINF] at org.jboss.as.weld.services.bootstrap.WeldEjbServices.resolveEjb(WeldEjbServices.java:53)
[JBossINF] at org.jboss.weld.bean.SessionBean.createReference(SessionBean.java:230)
[JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.<init>(EnterpriseBeanProxyMethodHandler.java:83)
[JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.<init>(EnterpriseBeanProxyMethodHandler.java:63)
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.createEnterpriseTargetBeanInstance(SessionBeanProxyInstantiator.java:80)
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.newInstance(SessionBeanProxyInstantiator.java:61)
[JBossINF] ... 40 more
{code}
Server log:
http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
Can you confirm it is the same case as the other NPE (meaning it is related and the recommended solution is to use suspend/resume or graceful shutdown)?
was (Author: mvinkler):
In EAP 7.0.0.DR11 run, I saw this exception which appeared near the Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133)
{code}
[JBossINF] [0m[31m16:48:34,010 ERROR [io.undertow.request] (default task-8) UT005023: Exception handling request to /clusterbench/ejbservlet: org.jboss.weld.exceptions.CreationException: WELD-000079: Could not instantiate a proxy for a session bean: Session bean [class org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB with qualifiers [@Any @Default]; local interfaces are [LocalStatefulSB]
[JBossINF] Proxy: class org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.newInstance(SessionBeanProxyInstantiator.java:72)
[JBossINF] at org.jboss.weld.bean.SessionBean.create(SessionBean.java:149)
[JBossINF] at org.jboss.weld.util.bean.IsolatedForwardingBean.create(IsolatedForwardingBean.java:45)
[JBossINF] at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:96)
[JBossINF] at org.jboss.weld.context.PassivatingContextWrapper$AbstractPassivatingContextWrapper.get(PassivatingContextWrapper.java:76)
[JBossINF] at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:101)
[JBossINF] at org.jboss.weld.bean.ContextualInstanceStrategy$CachingContextualInstanceStrategy.get(ContextualInstanceStrategy.java:178)
[JBossINF] at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50)
[JBossINF] at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:99)
[JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.getInstance(ProxyMethodHandler.java:125)
[JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
[JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
[JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
[JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
[JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
[JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
[JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
[JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
[JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
[JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
[JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
[JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
[JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
[JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
[JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:198)
[JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:784)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
[JBossINF] Caused by: java.lang.NullPointerException
[JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionComponent.createSession(StatefulSessionComponent.java:238)
[JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.<init>(StatefulSessionObjectReferenceImpl.java:70)
[JBossINF] at org.jboss.as.weld.services.bootstrap.WeldEjbServices.resolveEjb(WeldEjbServices.java:53)
[JBossINF] at org.jboss.weld.bean.SessionBean.createReference(SessionBean.java:230)
[JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.<init>(EnterpriseBeanProxyMethodHandler.java:83)
[JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.<init>(EnterpriseBeanProxyMethodHandler.java:63)
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.createEnterpriseTargetBeanInstance(SessionBeanProxyInstantiator.java:80)
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.newInstance(SessionBeanProxyInstantiator.java:61)
[JBossINF] ... 40 more
{code}
Server log:
http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
Can you confirm it is the same case as the other NPE (meaning the recommended solution is to use suspend/resume or graceful shutdown)?
> UT005023: Exception handling request: NullPointerException
> ----------------------------------------------------------
>
> Key: WFLY-5208
> URL: https://issues.jboss.org/browse/WFLY-5208
> Project: WildFly
> Issue Type: Bug
> Components: CDI / Weld
> Reporter: Michal Vinkler
> Assignee: Stuart Douglas
>
> Setup: 4-node cluster, one node at time either shuts down or undeploys application (clusterbench-ee7.ear), and then recovers after period of time, while standalone clients keep calling the application.
> This error was seen during clustering failover testing in the following scenarios:
> ejb-ejbservlet-shutdown-repl-async (failover of HTTP traffic accessing a servlet that locally invokes a stateful clustered EJB, failover type: server shutdown, with asynchronously replicated cache)
> ejb-ejbservlet-shutdown-repl-sync (the same as above, with synchronously replicated cache)
> ejb-ejbservlet-undeploy-dist-sync (failover type: application undeploy, with distributed cache which is replicated synchronously)
> It occurs in these two situations (intermittently):
> 1. server startup after previous shutdown
> {code}
> [JBossINF] [0m[0m17:23:14,240 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started repl cache from web container
> [JBossINF] [0m[0m17:23:14,247 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 67) WFLYCLINF0002: Started clusterbench-ee7.ear.clusterbench-ee7-web-default.war cache from web container
> [JBossINF] [0m[0m17:23:14,374 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 66) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [JBossINF] [0m[0m17:23:14,375 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 66) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [JBossINF] [0m[0m17:23:15,708 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 65) Initializing Mojarra 2.2.12-jbossorg-2 20150729-1131 for context '/clusterbench'
> [JBossINF] [0m[0m17:23:15,708 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 71) Initializing Mojarra 2.2.12-jbossorg-2 20150729-1131 for context '/clusterbench-passivating'
> [JBossINF] [0m[0m17:23:16,784 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 71) WFLYUT0021: Registered web context: /clusterbench-passivating
> [JBossINF] [0m[0m17:23:17,198 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 65) WFLYUT0021: Registered web context: /clusterbench
> [JBossINF] [0m[31m17:23:18,931 ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException
> [JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:122)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)
> [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
> [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
> 2. Server shutdown/application undeploy
> {code}
> [JBossINF] [0m[0m18:04:35,977 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0211: Suspending server
> [JBossINF] [0m[0m18:04:35,981 INFO [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested.
> 2015/08/18 18:04:36:037 EDT [DEBUG][RMI TCP Connection(14)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - [SHUTDOWN] JBossShutdown command executed successfully.
> 2015/08/18 18:04:36:037 EDT [DEBUG][RMI TCP Connection(14)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Waiting for server to shutdown.
> [JBossINF] [0m[0m18:04:36,048 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 77) WFLYUT0022: Unregistered web context: /clusterbench-passivating
> [JBossINF] [0m[0m18:04:36,063 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 81) WFLYUT0022: Unregistered web context: /clusterbench
> [JBossINF] [0m[0m18:04:36,072 INFO [org.infinispan.eviction.impl.PassivationManagerImpl] (ServerService Thread Pool -- 82) ISPN000029: Passivating all entries to disk
> [JBossINF] [0m[31m18:04:36,106 ERROR [io.undertow.request] (default task-63) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException
> [JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:122)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)
> [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
> [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
> During server shutdown, it occurred also with different stacktrace:
> {code}
> [JBossINF] [0m[31m17:19:11,728 ERROR [io.undertow.request] (default task-128) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException
> [JBossINF] at org.jboss.weld.servlet.WeldInitialListener.requestInitialized(WeldInitialListener.java:143)
> [JBossINF] at io.undertow.servlet.core.ApplicationListeners.requestInitialized(ApplicationListeners.java:216)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:281)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> [JBossINF]
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 2 months
- 1
- 0
[JBoss JIRA] (WFLY-5186) TimeoutException: Unable to acquire lock after 15 seconds
by Michal Vinkler (JIRA)
[ https://issues.jboss.org/browse/WFLY-5186?page=com.atlassian.jira.plugin.... ]
Michal Vinkler commented on WFLY-5186:
--------------------------------------
Still present in 7.0.0.DR11:
http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
In affected scenarios, the error is logged 1000+ times.
Also, in the same server log, this error occured near the ISPN000299: Unable to acquire lock after 15 seconds error:
{code}
[JBossINF] [0m[31m16:43:38,282 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (remote-thread--p4-t23) ISPN000136: Execution error: java.lang.IllegalStateException: Lock already released!
[JBossINF] at org.infinispan.util.concurrent.locks.impl.InfinispanLock$LockPlaceHolder.lock(InfinispanLock.java:324)
[JBossINF] at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:234)
[JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:198)
[JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:199)
[JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:165)
[JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:184)
[JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:163)
[JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:157)
[JBossINF] at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:215)
[JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:163)
[JBossINF] at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitLockControlCommand(TransactionSynchronizerInterceptor.java:75)
[JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:239)
[JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.visitLockControlCommand(StateTransferInterceptor.java:103)
[JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:113)
[JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:163)
[JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
[JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
[JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
[JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.visitLockControlCommand(InvocationContextInterceptor.java:76)
[JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
[JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
[JBossINF] at org.infinispan.commands.control.LockControlCommand.perform(LockControlCommand.java:122)
[JBossINF] at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
[JBossINF] at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
{code}
Can you confirm it is related?
> TimeoutException: Unable to acquire lock after 15 seconds
> ---------------------------------------------------------
>
> Key: WFLY-5186
> URL: https://issues.jboss.org/browse/WFLY-5186
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 10.0.0.Beta1
> Reporter: Michal Vinkler
> Assignee: Paul Ferraro
>
> Seen in these scenarios:
> ejb-ejbservlet-shutdown-repl-async
> ejb-ejbservlet-jvmkill-repl-async
> http-session-shutdown-repl-async-tcpStack
> http-session-undeploy-dist-sync
> With no obvious connection to failover of the nodes, usually all the 4 servers log this error few times during the testrun (each server different time):
> {code}
> [JBossINF] [0m[31m17:21:36,562 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (remote-thread--p4-t1) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key D89rThGoUJDT6kuJ-MV7w_4TeNsSpQvBoL2wKi0l and requestor GlobalTransaction:<perf20>:56262:remote. Lock is held by GlobalTransaction:<perf20>:51723:remote, while request came from perf20
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:180)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:118)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:200)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:141)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:199)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitLockControlCommand(TransactionSynchronizerInterceptor.java:75)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:196)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.visitLockControlCommand(StateTransferInterceptor.java:99)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.visitLockControlCommand(InvocationContextInterceptor.java:76)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.perform(LockControlCommand.java:129)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:85)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> The error is usually followed by a warning:
> {code}
> [JBossINF] [0m[33m17:21:36,563 WARN [org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler] (remote-thread--p4-t1) ISPN000071: Caught exception when handling command LockControlCommand{cache=clusterbench-ee7.ear.clusterbench-ee7-web-default.war, keys=[D89rThGoUJDT6kuJ-MV7w_4TeNsSpQvBoL2wKi0l], flags=[IGNORE_RETURN_VALUES], unlock=false, gtx=GlobalTransaction:<perf20>:56262:remote}: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key D89rThGoUJDT6kuJ-MV7w_4TeNsSpQvBoL2wKi0l and requestor GlobalTransaction:<perf20>:56262:remote. Lock is held by GlobalTransaction:<perf20>:51723:remote, while request came from perf20
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:180)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:118)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:200)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:141)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:199)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitLockControlCommand(TransactionSynchronizerInterceptor.java:75)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:196)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.visitLockControlCommand(StateTransferInterceptor.java:99)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:174)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.visitLockControlCommand(InvocationContextInterceptor.java:76)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:110)
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> [JBossINF] at org.infinispan.commands.control.LockControlCommand.perform(LockControlCommand.java:129)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:85)
> [JBossINF] at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> While the error was logged 15 times by all servers in this particular run, the warning message was logged 505 times.
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
> The same error with different stacktrace:
> {code}
> [JBossINF] [0m[31m17:21:16,585 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-45) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key UnknownSessionID [5365576765655250685155485253536866686865666767514850676951495057] and requestor GlobalTransaction:<perf20>:52228:local. Lock is held by GlobalTransaction:<perf20>:49990:local, while request came from local
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
> [JBossINF] at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:180)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.lockAndRegisterBackupLock(PessimisticLockingInterceptor.java:261)
> [JBossINF] at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataWriteCommand(PessimisticLockingInterceptor.java:131)
> [JBossINF] at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitReplaceCommand(AbstractLockingInterceptor.java:60)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:54)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:308)
> [JBossINF] at org.infinispan.interceptors.TxInterceptor.visitReplaceCommand(TxInterceptor.java:219)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:54)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleTxWriteCommand(StateTransferInterceptor.java:261)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:243)
> [JBossINF] at org.infinispan.statetransfer.StateTransferInterceptor.visitReplaceCommand(StateTransferInterceptor.java:124)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
> [JBossINF] at org.infinispan.interceptors.CacheMgmtInterceptor.visitReplaceCommand(CacheMgmtInterceptor.java:182)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
> [JBossINF] at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> [JBossINF] at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:54)
> [JBossINF] at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)
> [JBossINF] at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1613)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.replaceInternal(CacheImpl.java:1158)
> [JBossINF] at org.infinispan.cache.impl.CacheImpl.replace(CacheImpl.java:1151)
> [JBossINF] at org.infinispan.cache.impl.DecoratedCache.replace(DecoratedCache.java:502)
> [JBossINF] at org.infinispan.cache.impl.AbstractDelegatingCache.replace(AbstractDelegatingCache.java:262)
> [JBossINF] at org.wildfly.clustering.ee.infinispan.CacheEntryMutator.mutate(CacheEntryMutator.java:52)
> [JBossINF] at org.wildfly.clustering.ejb.infinispan.bean.InfinispanBean.close(InfinispanBean.java:113)
> [JBossINF] at org.wildfly.clustering.ejb.infinispan.InfinispanBeanManager$SchedulableBean.close(InfinispanBeanManager.java:406)
> [JBossINF] at org.jboss.as.ejb3.cache.distributable.DistributableCache.release(DistributableCache.java:136)
> [JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor.releaseInstance(StatefulSessionSynchronizationInterceptor.java:168)
> [JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionSynchronizationInterceptor$StatefulSessionSynchronization.afterCompletion(StatefulSessionSynchronizationInterceptor.java:250)
> [JBossINF] at org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList.afterCompletion(JCAOrderedLastSynchronizationList.java:147)
> [JBossINF] at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96)
> [JBossINF] at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:545)
> [JBossINF] at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:101)
> [JBossINF] at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> [JBossINF] at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
> [JBossINF] at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> [JBossINF] at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91)
> [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279)
> [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327)
> [JBossINF] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:66)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
> [JBossINF] at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)
> [JBossINF] at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
> [JBossINF] at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
> [JBossINF] at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:195)
> [JBossINF] at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
> [JBossINF] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> [JBossINF] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
> [JBossINF] at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$$$view3.getSerialAndIncrement(Unknown Source)
> [JBossINF] at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
> [JBossINF] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [JBossINF] at java.lang.reflect.Method.invoke(Method.java:497)
> [JBossINF] at org.jboss.weld.util.reflection.Reflections.invokeAndUnwrap(Reflections.java:434)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:127)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)
> [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
> [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
> The timeout stated in the exception is sometimes not exactly 15 seconds, so far I have seen these variants: 14.99 seconds, 14.97 seconds, and also only few milliseconds (!), see this server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-http-...
>
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 2 months
- 1
- 0
[JBoss JIRA] (WFLY-5208) UT005023: Exception handling request: NullPointerException
by Michal Vinkler (JIRA)
[ https://issues.jboss.org/browse/WFLY-5208?page=com.atlassian.jira.plugin.... ]
Michal Vinkler commented on WFLY-5208:
--------------------------------------
In EAP 7.0.0.DR11 run, I saw this exception which appeared near the Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133)
{code}
[JBossINF] [0m[31m16:48:34,010 ERROR [io.undertow.request] (default task-8) UT005023: Exception handling request to /clusterbench/ejbservlet: org.jboss.weld.exceptions.CreationException: WELD-000079: Could not instantiate a proxy for a session bean: Session bean [class org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB with qualifiers [@Any @Default]; local interfaces are [LocalStatefulSB]
[JBossINF] Proxy: class org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.newInstance(SessionBeanProxyInstantiator.java:72)
[JBossINF] at org.jboss.weld.bean.SessionBean.create(SessionBean.java:149)
[JBossINF] at org.jboss.weld.util.bean.IsolatedForwardingBean.create(IsolatedForwardingBean.java:45)
[JBossINF] at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:96)
[JBossINF] at org.jboss.weld.context.PassivatingContextWrapper$AbstractPassivatingContextWrapper.get(PassivatingContextWrapper.java:76)
[JBossINF] at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:101)
[JBossINF] at org.jboss.weld.bean.ContextualInstanceStrategy$CachingContextualInstanceStrategy.get(ContextualInstanceStrategy.java:178)
[JBossINF] at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50)
[JBossINF] at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:99)
[JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.getInstance(ProxyMethodHandler.java:125)
[JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
[JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
[JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
[JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
[JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
[JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
[JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
[JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
[JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
[JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
[JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
[JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
[JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
[JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:284)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
[JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
[JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:198)
[JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:784)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[JBossINF] at java.lang.Thread.run(Thread.java:745)
[JBossINF] Caused by: java.lang.NullPointerException
[JBossINF] at org.jboss.as.ejb3.component.stateful.StatefulSessionComponent.createSession(StatefulSessionComponent.java:238)
[JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.<init>(StatefulSessionObjectReferenceImpl.java:70)
[JBossINF] at org.jboss.as.weld.services.bootstrap.WeldEjbServices.resolveEjb(WeldEjbServices.java:53)
[JBossINF] at org.jboss.weld.bean.SessionBean.createReference(SessionBean.java:230)
[JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.<init>(EnterpriseBeanProxyMethodHandler.java:83)
[JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.<init>(EnterpriseBeanProxyMethodHandler.java:63)
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.createEnterpriseTargetBeanInstance(SessionBeanProxyInstantiator.java:80)
[JBossINF] at org.jboss.weld.injection.producer.ejb.SessionBeanProxyInstantiator.newInstance(SessionBeanProxyInstantiator.java:61)
[JBossINF] ... 40 more
{code}
Server log:
http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
Can you confirm it is the same case as the other NPE (meaning the recommended solution is to use suspend/resume or graceful shutdown)?
> UT005023: Exception handling request: NullPointerException
> ----------------------------------------------------------
>
> Key: WFLY-5208
> URL: https://issues.jboss.org/browse/WFLY-5208
> Project: WildFly
> Issue Type: Bug
> Components: CDI / Weld
> Reporter: Michal Vinkler
> Assignee: Stuart Douglas
>
> Setup: 4-node cluster, one node at time either shuts down or undeploys application (clusterbench-ee7.ear), and then recovers after period of time, while standalone clients keep calling the application.
> This error was seen during clustering failover testing in the following scenarios:
> ejb-ejbservlet-shutdown-repl-async (failover of HTTP traffic accessing a servlet that locally invokes a stateful clustered EJB, failover type: server shutdown, with asynchronously replicated cache)
> ejb-ejbservlet-shutdown-repl-sync (the same as above, with synchronously replicated cache)
> ejb-ejbservlet-undeploy-dist-sync (failover type: application undeploy, with distributed cache which is replicated synchronously)
> It occurs in these two situations (intermittently):
> 1. server startup after previous shutdown
> {code}
> [JBossINF] [0m[0m17:23:14,240 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started repl cache from web container
> [JBossINF] [0m[0m17:23:14,247 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 67) WFLYCLINF0002: Started clusterbench-ee7.ear.clusterbench-ee7-web-default.war cache from web container
> [JBossINF] [0m[0m17:23:14,374 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 66) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [JBossINF] [0m[0m17:23:14,375 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 66) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
> [JBossINF] [0m[0m17:23:15,708 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 65) Initializing Mojarra 2.2.12-jbossorg-2 20150729-1131 for context '/clusterbench'
> [JBossINF] [0m[0m17:23:15,708 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 71) Initializing Mojarra 2.2.12-jbossorg-2 20150729-1131 for context '/clusterbench-passivating'
> [JBossINF] [0m[0m17:23:16,784 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 71) WFLYUT0021: Registered web context: /clusterbench-passivating
> [JBossINF] [0m[0m17:23:17,198 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 65) WFLYUT0021: Registered web context: /clusterbench
> [JBossINF] [0m[31m17:23:18,931 ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException
> [JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:122)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)
> [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
> [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
> 2. Server shutdown/application undeploy
> {code}
> [JBossINF] [0m[0m18:04:35,977 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0211: Suspending server
> [JBossINF] [0m[0m18:04:35,981 INFO [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested.
> 2015/08/18 18:04:36:037 EDT [DEBUG][RMI TCP Connection(14)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - [SHUTDOWN] JBossShutdown command executed successfully.
> 2015/08/18 18:04:36:037 EDT [DEBUG][RMI TCP Connection(14)-10.16.90.52] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Waiting for server to shutdown.
> [JBossINF] [0m[0m18:04:36,048 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 77) WFLYUT0022: Unregistered web context: /clusterbench-passivating
> [JBossINF] [0m[0m18:04:36,063 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 81) WFLYUT0022: Unregistered web context: /clusterbench
> [JBossINF] [0m[0m18:04:36,072 INFO [org.infinispan.eviction.impl.PassivationManagerImpl] (ServerService Thread Pool -- 82) ISPN000029: Passivating all entries to disk
> [JBossINF] [0m[31m18:04:36,106 ERROR [io.undertow.request] (default task-63) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException
> [JBossINF] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.isRemoved(StatefulSessionObjectReferenceImpl.java:133)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:122)
> [JBossINF] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)
> [JBossINF] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_Weld$EnterpriseProxy$.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.ejb.stateful.LocalStatefulSB$Proxy$_$$_WeldClientProxy.getSerialAndIncrement(Unknown Source)
> [JBossINF] at org.jboss.test.clusterbench.web.ejb.LocalEjbServlet.doGet(LocalEjbServlet.java:38)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
> [JBossINF] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> [JBossINF] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
> [JBossINF] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
> [JBossINF] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
> [JBossINF] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
> [JBossINF] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
> [JBossINF] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
> [JBossINF] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
> [JBossINF] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
> During server shutdown, it occurred also with different stacktrace:
> {code}
> [JBossINF] [0m[31m17:19:11,728 ERROR [io.undertow.request] (default task-128) UT005023: Exception handling request to /clusterbench/ejbservlet: java.lang.NullPointerException
> [JBossINF] at org.jboss.weld.servlet.WeldInitialListener.requestInitialized(WeldInitialListener.java:143)
> [JBossINF] at io.undertow.servlet.core.ApplicationListeners.requestInitialized(ApplicationListeners.java:216)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:281)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
> [JBossINF] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
> [JBossINF] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
> [JBossINF] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:778)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> [JBossINF]
> {code}
> Server log:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-ejb-e...
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 2 months
- 1
- 0
[JBoss JIRA] (WFCORE-1031) PS1 scripts invocation ends with "cannot be loaded because running scripts is disabled on thi s system", affect MS Windows desktop series
by Rostislav Svoboda (JIRA)
Rostislav Svoboda created WFCORE-1031:
-----------------------------------------
Summary: PS1 scripts invocation ends with "cannot be loaded because running scripts is disabled on thi s system", affect MS Windows desktop series
Key: WFCORE-1031
URL: https://issues.jboss.org/browse/WFCORE-1031
Project: WildFly Core
Issue Type: Bug
Components: Scripts
Reporter: Rostislav Svoboda
Assignee: Tomaz Cerar
PS1 scripts invocation ends with "cannot be loaded because running scripts is disabled on thi s system".
This was spotted on Windows 10 but it seems it affect more MS Windows desktop series (at leas 8.x too) - based on HiChat discussion.
When I open PowerShel, enter the bin folder and run .\standalone.ps1 I get this:
{code}
.\standalone.ps1 : File E:\jboss-eap-7.0\bin\standalone.ps1 cannot be loaded because running scripts is disabled on this s system.
For more information, see about_Execution_Policies at http://go.microsoft.com/fwlink/?LinkID=135170.
At line:1 char:1
+ .\standalone.ps1
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : SecurityError: (:) [], PSSecurityException
+ FullyQualifiedErrorId : UnauthorizedAccess
{code}
what "helped" is this:
Set-ExecutionPolicy RemoteSigned
but I had to run powershell as administrator because it's changing registry key ... so well, not straightforward usage of ps1 scripts ... Not everybody is able to run in admin mode.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 2 months
- 1
- 0