[infinispan-issues] [JBoss JIRA] (ISPN-2153) Distributed ejb cache container on AS7 - Server hangs when invoking an SFSB
Dan Berindei (JIRA)
jira-events at lists.jboss.org
Wed Jul 11 18:11:12 EDT 2012
[ https://issues.jboss.org/browse/ISPN-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705563#comment-12705563 ]
Dan Berindei commented on ISPN-2153:
------------------------------------
There was a comment on the forum about this being caused by our unsafe usage of HashMap in a multithreaded environment. I don't think this is caused by unsafe usage of HashMap for two reasons:
1. The HashSet that appears in the stack trace is never modified (after it has been initialized). The concurrency issues in HashMap appear only with concurrent modifications.
2. The profiler screenshot shows KeyAffinityServiceImpl.getKeyForAddress's to have a very significant self time, at 11%. If there was a hang inside the HashMap I would have expected everything but the HashMap methods (or maybe KeyAffinityServiceImpl.isNodeInConsistentHash, assuming everything was inlined) to take > 90% of the time.
Based on this, I'm pretty sure the problem is that the key generator is not actually generating any keys for that node and KeyAffinityServiceImpl.getKeyForAddress is stuck in the while loop.
> Distributed ejb cache container on AS7 - Server hangs when invoking an SFSB
> ---------------------------------------------------------------------------
>
> Key: ISPN-2153
> URL: https://issues.jboss.org/browse/ISPN-2153
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 5.1.5.FINAL
> Environment: JBoss AS 7.2.0.Alpha1-SNAPSHOT, EAP6
> Reporter: Heinz Wilming
> Assignee: Manik Surtani
> Priority: Critical
> Fix For: 5.2.0.ALPHA2, 5.1.6.CR1
>
>
> I'm working on scalable ha-clusters and I have a domain-mode cluster in mind which uses Infinispan-distribution to be scalable. So I started to make the cache-containers use distributed-caches instead of replicated ones.
> It starts up well and seems like everything is ok. But when I try to access a stateful-session-bean the server hangs. It only hangs when I try to use a distributed cache for the container ejb. If I use a replicated cache for ejb but still use distributed caches for cluster and web everything works fine. The problem does appear independent from the number of cluster nodes.
> By hanging I mean that the server won't complete the request. It will start to shutdown on CTRL+C but will not finish to. The only way to kill the server is kill -9. Attached to this post there is a file containing the stdout of the server with a thread-dump (kill -3) appended at the point where I requested a SFSB and nothing happened.
> Threaddump:
> [Server:server-one] "ajp-/192.168.122.165:8009-2" daemon prio=10 tid=0x08708800 nid=0x2583 runnable [0xb0882000]
> [Server:server-one] java.lang.Thread.State: RUNNABLE
> [Server:server-one] at java.util.HashMap.containsKey(HashMap.java:352)
> [Server:server-one] at java.util.HashSet.contains(HashSet.java:201)
> [Server:server-one] at org.infinispan.affinity.KeyAffinityServiceImpl.isNodeInConsistentHash(KeyAffinityServiceImpl.java:351)
> [Server:server-one] at org.infinispan.affinity.KeyAffinityServiceImpl.getKeyForAddress(KeyAffinityServiceImpl.java:145)
> [Server:server-one] at org.jboss.as.clustering.ejb3.cache.backing.infinispan.InfinispanBackingCacheEntryStore.createIdentifier(InfinispanBackingCacheEntryStore.java:123)
> [Server:server-one] at org.jboss.as.clustering.ejb3.cache.backing.infinispan.InfinispanBackingCacheEntryStore.createIdentifier(InfinispanBackingCacheEntryStore.java:71)
> [Server:server-one] at org.jboss.as.ejb3.cache.impl.backing.SerializationGroupMemberContainer.createIdentifier(SerializationGroupMemberContainer.java:83)
> [Server:server-one] at org.jboss.as.ejb3.cache.impl.backing.SerializationGroupMemberContainer.createIdentifier(SerializationGroupMemberContainer.java:51)
> [Server:server-one] at org.jboss.as.ejb3.cache.impl.backing.PassivatingBackingCacheImpl.createIdentifier(PassivatingBackingCacheImpl.java:96)
> [Server:server-one] at org.jboss.as.ejb3.cache.impl.backing.PassivatingBackingCacheImpl.createIdentifier(PassivatingBackingCacheImpl.java:60)
> [Server:server-one] at org.jboss.as.ejb3.cache.spi.impl.AbstractCache.createIdentifier(AbstractCache.java:48)
> [Server:server-one] at org.jboss.as.ejb3.cache.spi.impl.AbstractCache.createIdentifier(AbstractCache.java:39)
> [Server:server-one] at org.jboss.as.ejb3.component.stateful.StatefulSessionComponentInstance.<init>(StatefulSessionComponentInstance.java:70)
> [Server:server-one] at org.jboss.as.ejb3.component.stateful.StatefulSessionComponent.instantiateComponentInstance(StatefulSessionComponent.java:259)
> [Server:server-one] at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:150)
> [Server:server-one] at org.jboss.as.ejb3.component.stateful.StatefulSessionComponent.constructComponentInstance(StatefulSessionComponent.java:145)
> [Server:server-one] at org.jboss.as.ejb3.component.stateful.StatefulSessionComponent.constructComponentInstance(StatefulSessionComponent.java:76)
> [Server:server-one] at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85)
> [Server:server-one] at org.jboss.as.ejb3.component.stateful.StatefulSessionComponent.createInstance(StatefulSessionComponent.java:135)
> [Server:server-one] at org.jboss.as.ejb3.component.stateful.StatefulSessionComponent.createInstance(StatefulSessionComponent.java:76)
> [Server:server-one] at org.jboss.as.ejb3.cache.TransactionAwareObjectFactory.createInstance(TransactionAwareObjectFactory.java:53)
> [Server:server-one] at org.jboss.as.ejb3.cache.impl.backing.PassivatingBackingCacheImpl.create(PassivatingBackingCacheImpl.java:121)
> [Server:server-one] at org.jboss.as.ejb3.cache.impl.GroupAwareCache.create(GroupAwareCache.java:67)
> [Server:server-one] at org.jboss.as.ejb3.cache.impl.GroupAwareCache.create(GroupAwareCache.java:41)
> [Server:server-one] at org.jboss.as.ejb3.component.stateful.StatefulSessionComponent.createSession(StatefulSessionComponent.java:241)
> [Server:server-one] at org.jboss.as.weld.ejb.StatefulSessionObjectReferenceImpl.<init>(StatefulSessionObjectReferenceImpl.java:72)
> [Server:server-one] at org.jboss.as.weld.services.bootstrap.WeldEjbServices.resolveEjb(WeldEjbServices.java:60)
> [Server:server-one] at org.jboss.weld.bean.SessionBean.createReference(SessionBean.java:412)
> [Server:server-one] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.<init>(EnterpriseBeanProxyMethodHandler.java:69)
> [Server:server-one] at org.jboss.weld.bean.SessionBean.create(SessionBean.java:297)
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
More information about the infinispan-issues
mailing list