10:24 p.m.
[
https://issues.jboss.org/browse/ISPN-1255?page=com.atlassian.jira.plugin....
]
Erik Salter updated ISPN-1255:
------------------------------
Attachment: join_failure_thread_dump
I reproduced the join() failure.
--- MBeans waiting for other MBeans ---
ObjectName: net.beaumaris:InfinispanCacheDeployer=InfinispanCacheDeployer
State: FAILED
Reason: org.infinispan.CacheException: Unable to invoke method private void
org.infinispan.distribution.DistributionManagerImpl.join() throws java.lang.Exception on
object
I Depend On:
jboss:name=SystemProperties,type=Service
Depends On Me:
jboss.web.deployment:war=dht.billing-1.1.1.war,id=-961963574
jboss.web.deployment:war=dht.poilist-1.1.1.war,id=568459511
jboss.web.deployment:war=dht.session-1.1.1.war,id=1470944293
jboss.web.deployment:war=odrm-resource-1.1.1.war,id=-986086344
ObjectName: jboss.web.deployment:war=dht.billing-1.1.1.war,id=-961963574
State: NOTYETINSTALLED
I Depend On:
jboss.esb:deployment=spring.esb
net.beaumaris:InfinispanCacheDeployer=InfinispanCacheDeployer
ObjectName: jboss.web.deployment:war=dht.poilist-1.1.1.war,id=568459511
State: NOTYETINSTALLED
I Depend On:
jboss.esb:deployment=spring.esb
net.beaumaris:InfinispanCacheDeployer=InfinispanCacheDeployer
ObjectName: jboss.web.deployment:war=dht.session-1.1.1.war,id=1470944293
State: NOTYETINSTALLED
I Depend On:
jboss.esb:deployment=spring.esb
net.beaumaris:InfinispanCacheDeployer=InfinispanCacheDeployer
ObjectName: jboss.web.deployment:war=odrm-resource-1.1.1.war,id=-986086344
State: NOTYETINSTALLED
I Depend On:
jboss.esb:deployment=spring.esb
net.beaumaris:InfinispanCacheDeployer=InfinispanCacheDeployer
--- MBEANS THAT ARE THE ROOT CAUSE OF THE PROBLEM ---
ObjectName: net.beaumaris:InfinispanCacheDeployer=InfinispanCacheDeployer
State: FAILED
Reason: org.infinispan.CacheException: Unable to invoke method private void
org.infinispan.distribution.DistributionManagerImpl.join() throws java.lang.Exception on
object
I Depend On:
jboss:name=SystemProperties,type=Service
Depends On Me:
jboss.web.deployment:war=dht.billing-1.1.1.war,id=-961963574
jboss.web.deployment:war=dht.poilist-1.1.1.war,id=568459511
jboss.web.deployment:war=dht.session-1.1.1.war,id=1470944293
jboss.web.deployment:war=odrm-resource-1.1.1.war,id=-986086344
RequestIgnoredException on rehash using the Distributed Executor
Service
------------------------------------------------------------------------
Key: ISPN-1255
URL: https://issues.jboss.org/browse/ISPN-1255
Project: Infinispan
Issue Type: Bug
Affects Versions: 5.0.0.CR7
Reporter: Erik Salter
Assignee: Dan Berindei
Fix For: 5.0.0.FINAL
Attachments: cacheTest.zip, ispn1255.log, join_failure_thread_dump,
server_node1.log, server_node2.log
My application exposes its distributed operations via a REST-based infrastructure. To
minimize the delta between JBoss starting and the cache starting, I used the new
Distributed Executor to "sticky" a task to the data owner of a set of keys (with
the same hash code).
NOTE: Rehash still causes problems seen in ISPN-1106. (Attached new logs)
I see a lot of the following error from the DistributedExecutorService when the new
node's cache doesn't start in a timely manner:
Reason: java.lang.IllegalStateException: Invalid response
{Satriani-52149(PHL)=RequestIgnoredResponse}
In addition, I see:
org.infinispan.util.concurrent.TimeoutException: Timed out waiting for valid responses!
It takes the cache about 2+ minutes at low throughput rate (30 tx/s) to recover. For
high throughput rate, the cluster doesn't recover.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira