Data gravitation RspFilter makes cache misses very slow ------------------------------------------------------- Key: JBCACHE-1227 URL: http://jira.jboss.com/jira/browse/JBCACHE-1227 Project: JBoss Cache Issue Type: Bug Security Level: Public (Everyone can see) Affects Versions: 2.1.0.CR2 Reporter: Brian Stansberry Assigned To: Brian Stansberry Priority: Critical Fix For: 2.1.0.GA DataGravitatorInterceptor.ResponseValidityFilter will make all gravitation attempts take buddyCommunicationTimeout ms if the data is not available (i.e. if its a cluster-wide cache miss). Logging showing what happens with a 2 second buddyCommunicationTimeout : 2007-11-28 14:24:49,634 DEBUG [org.jboss.cache.interceptors.DataGravitatorInterceptor] cache=192.168.1.164:1621; requesting data gravitation for Fqn /JSESSION/localhost/http-field-pass/SfrkE9Utsz-EfMjYpItWEA__/ATTRIBUTE/TEST_ID 2007-11-28 14:24:49,743 DEBUG [org.jboss.cache.CacheImpl.DefaultPartition-TomcatCache] callRemoteMethods(): valid members are [192.168.1.165:1674] methods: /JSESSION/localhost/http-field-pass/SfrkE9Utsz-EfMjYpItWEA__/ATTRIBUTE/TEST_ID 2007-11-28 14:24:49,743 DEBUG [org.jboss.cache.marshall.InactiveRegionAwareRpcDispatcher] dests=[192.168.1.165:1674], method_call=MethodName: gravitateData; MethodIdInteger: 35; Args: (/JSESSION/localhost/http-field-pass/SfrkE9Utsz-EfMjYpItWEA__/ATTRIBUTE/TEST_ID, true), mode=1, timeout=2000 2007-11-28 14:24:49,743 DEBUG [org.jboss.cache.marshall.VersionAwareMarshaller] Wrote version 21 2007-11-28 14:24:49,743 DEBUG [org.jboss.cache.marshall.AbstractMarshaller] extract(): received MethodName: gravitateData; MethodIdInteger: 35; Args: (/JSESSION/localhost/http-field-pass/SfrkE9Utsz-EfMjYpItWEA__/ATTRIBUTE/TEST_ID, true)extracted fqn: /JSESSION/localhost/http-field-pass/SfrkE9Utsz-EfMjYpItWEA__/ATTRIBUTE/TEST_ID 2007-11-28 14:24:49,743 DEBUG [org.jboss.cache.marshall.CacheMarshaller200] Region based call. Using region /JSESSION/localhost/http-field-pass 2007-11-28 14:24:49,743 DEBUG [org.jboss.cache.marshall.CacheMarshaller200] Marshalling object MethodName: gravitateData; MethodIdInteger: 35; Args: (/JSESSION/localhost/http-field-pass/SfrkE9Utsz-EfMjYpItWEA__/ATTRIBUTE/TEST_ID, true) 2007-11-28 14:24:49,743 DEBUG [org.jboss.cache.marshall.CacheMarshaller200] Writing region /JSESSION/localhost/http-field-pass to stream 2007-11-28 14:24:49,743 DEBUG [org.jboss.cache.marshall.InactiveRegionAwareRpcDispatcher] real_dests=[192.168.1.165:1674] 2007-11-28 14:24:49,743 DEBUG [org.jboss.cache.marshall.VersionAwareMarshaller] Read version 21 2007-11-28 14:24:49,759 DEBUG [org.jboss.cache.marshall.CacheMarshaller200] Unmarshalled regionFqn /JSESSION/localhost/http-field-pass from stream 2007-11-28 14:24:49,759 DEBUG [org.jboss.cache.marshall.CacheMarshaller200] Unmarshalled object Result dataFound=false nodeData=null fqn=null 2007-11-28 14:24:51,759 DEBUG [org.jboss.cache.marshall.InactiveRegionAwareRpcDispatcher] responses: [sender=192.168.1.165:1674, retval=null, received=false, suspected=false] The 2nd to last line shows the response being unmarshalled, yet 2 secs later the RpcDispatcher reports no response was received from the node that sent the response. Cause is DataGravitatorInterceptor.ResponseValidityFilter.needMoreResponses() which will always return true until it gets a response where GravitateResult.isDataFound() is true. That response will never arrive if the data doesn't exist. So RpcDispatcher will keep waiting for more responses until it times out. I think the solution is to pass the vector of members being called to ResponseValidityFilter, which keeps a copy. In isAcceptable, remove the member that sent the response. In needMoreResponses, return false if all members have sent responses. I'll attach a patch implementing the above in a minute. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira