4:16 p.m.
Well, maybe not CR5, since I haven't tested this. But, as briefly discussed on the dev
list I'm seeing a failure in a AS testsuite test w/ a snapshot built earlier in the
week against the head of 2.2.X branch. I assume it's still there in CR5
Problem is remote call comes in to delete a node that's previously been passivated to
disk. As part of handling that call the node is read from disk; this fails due to not
using the correct classloader. Region-based marshalling is used, buddy replication,
FileCacheLoader.
Following is some analysis; not complete but as far as I can go for now. To reproduce,
checkout the head of the AS, change the JBC version in component-matrix/pom.xml to CR5,
build AS and testsuite, and from testsuite dir
./build.sh tests-clustering-field-br
OK, what I' seeing. First the failure:
| 2008-06-27 12:53:50,846 TRACE [org.jboss.cache.interceptors.ActivationInterceptor]
(Incoming-1,192.168.1.145:43241) loadNode
/_BUDDY_BACKUP_/127.0.0.1_43238/JSESSION/localhost/http-field-pass/giGN4tImzJAMCoFaHg-gZw__/__JBossInternal__/ATTRIBUTE/TEST_DESER/_ID_/5c4o141-i7090w-fhz2z6jb-1-fhz323cn-2k
| 2008-06-27 12:53:50,846 TRACE [org.jboss.cache.RegionManager]
(Incoming-1,192.168.1.145:43241) Contents of RegionsRegistry:
{/_BUDDY_BACKUP_/127.0.0.1_43238=RegionImpl{fqn=/_BUDDY_BACKUP_/127.0.0.1_43238;
classloader=null; status=INACTIVE; eviction=true; timerThreadRegistered=true},
/=RegionImpl{fqn=/; classloader=null; status=INACTIVE; eviction=true;
timerThreadRegistered=true},
/JSESSION/localhost/http-field-pass=RegionImpl{fqn=/JSESSION/localhost/http-field-pass;
classloader=org.jboss.web.tomcat.service.WebCtxLoader$ENCLoader@41fc8c; status=ACTIVE;
eviction=false; timerThreadRegistered=false}}
| 2008-06-27 12:53:50,847 TRACE [org.jboss.cache.RegionManager]
(Incoming-1,192.168.1.145:43241) Trying next region /_BUDDY_BACKUP_/127.0.0.1_43238 and
got RegionImpl{fqn=/_BUDDY_BACKUP_/127.0.0.1_43238; classloader=null; status=INACTIVE;
eviction=true; timerThreadRegistered=true}
| 2008-06-27 12:53:50,847 TRACE [org.jboss.cache.RegionManager]
(Incoming-1,192.168.1.145:43241) Trying next region / and got RegionImpl{fqn=/;
classloader=null; status=INACTIVE; eviction=true; timerThreadRegistered=true}
| 2008-06-27 12:53:50,847 TRACE [org.jboss.cache.marshall.CacheMarshaller210]
(Incoming-1,192.168.1.145:43241) Unmarshalled region as null. Not using a context class
loader to unmarshall.
| 2008-06-27 12:53:50,847 TRACE [org.jboss.cache.marshall.CacheMarshaller210]
(Incoming-1,192.168.1.145:43241) Unmarshalled regionFqn null from stream
| 2008-06-27 12:53:50,847 DEBUG [org.jboss.cache.marshall.CacheMarshaller210]
(Incoming-1,192.168.1.145:43241) Region does not exist for Fqn null - not using a context
classloader.
| 2008-06-27 12:53:50,851 ERROR [org.jboss.cache.interceptors.TxInterceptor]
(Incoming-1,192.168.1.145:43241) prepare failed!
| java.lang.ClassNotFoundException: org.jboss.test.cluster.web.DeserializationSensor
| at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
| at java.security.AccessController.doPrivileged(Native Method)
| at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
| at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
| at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
| at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
| at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
| at java.lang.Class.forName0(Native Method)
| at java.lang.Class.forName(Class.java:242)
| at
org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:207)
| at
org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:1009)
| at
org.jboss.classloader.spi.base.BaseClassLoader.loadClassFromDomain(BaseClassLoader.java:728)
| at
org.jboss.classloader.spi.base.BaseClassLoader.loadClass(BaseClassLoader.java:372)
| at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
| at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
| at java.lang.Class.forName0(Native Method)
| at java.lang.Class.forName(Class.java:242)
| at
org.jboss.util.stream.MarshalledValueInputStream.resolveClass(MarshalledValueInputStream.java:62)
| at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1544)
| at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1466)
| at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1699)
| at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1305)
| at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348)
| at
org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:540)
| at
org.jboss.cache.marshall.CacheMarshaller200.populateFromStream(CacheMarshaller200.java:750)
| at
org.jboss.cache.marshall.CacheMarshaller200.unmarshallHashMap(CacheMarshaller200.java:713)
| at
org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:571)
| at
org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:513)
| at
org.jboss.cache.marshall.CacheMarshaller200.objectFromObjectStreamRegionBased(CacheMarshaller200.java:202)
| at
org.jboss.cache.marshall.CacheMarshaller200.objectFromObjectStream(CacheMarshaller200.java:126)
| at
org.jboss.cache.marshall.VersionAwareMarshaller.objectFromObjectStream(VersionAwareMarshaller.java:326)
| at org.jboss.cache.loader.FileCacheLoader.doUnmarshall(FileCacheLoader.java:481)
| at
org.jboss.cache.loader.AbstractCacheLoader.regionAwareUnmarshall(AbstractCacheLoader.java:214)
| at org.jboss.cache.loader.FileCacheLoader.loadAttributes(FileCacheLoader.java:434)
| at org.jboss.cache.loader.FileCacheLoader.get(FileCacheLoader.java:201)
| at
org.jboss.cache.interceptors.CacheLoaderInterceptor.loadData(CacheLoaderInterceptor.java:564)
| at
org.jboss.cache.interceptors.CacheLoaderInterceptor.loadNode(CacheLoaderInterceptor.java:491)
| at
org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:305)
| at
org.jboss.cache.interceptors.CacheLoaderInterceptor.visitPutKeyValueCommand(CacheLoaderInterceptor.java:106)
| at
org.jboss.cache.interceptors.ActivationInterceptor.visitPutKeyValueCommand(ActivationInterceptor.java:135)
| at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:92)
| at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:142)
| at
org.jboss.cache.interceptors.PessimisticLockInterceptor.handlePutCommand(PessimisticLockInterceptor.java:118)
| at
org.jboss.cache.interceptors.PessimisticLockInterceptor.handlePutKeyValueCommand(PessimisticLockInterceptor.java:86)
| at
org.jboss.cache.interceptors.base.PostProcessingCommandInterceptor.visitPutKeyValueCommand(PostProcessingCommandInterceptor.java:56)
| at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:92)
| at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:142)
| at
org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:129)
| at
org.jboss.cache.interceptors.ReplicationInterceptor.visitPutKeyValueCommand(ReplicationInterceptor.java:86)
| at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:92)
| at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:142)
| at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:157)
| at
org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:35)
| at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:92)
| at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:142)
| at
org.jboss.cache.interceptors.TxInterceptor.replayModifications(TxInterceptor.java:502)
| at
org.jboss.cache.interceptors.TxInterceptor.handleRemotePrepare(TxInterceptor.java:382)
| at
org.jboss.cache.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:105)
| at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:46)
| at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:142)
| at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:157)
| at
org.jboss.cache.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:110)
| at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:46)
| at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:142)
| at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:158)
| at
org.jboss.cache.interceptors.InvocationContextInterceptor.visitPrepareCommand(InvocationContextInterceptor.java:84)
| at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:46)
| at
org.jboss.cache.interceptors.InterceptorChain.invokeRemote(InterceptorChain.java:239)
| at
org.jboss.cache.commands.remote.ReplicateCommand.processSingleCommand(ReplicateCommand.java:116)
| at
org.jboss.cache.commands.remote.ReplicateCommand.perform(ReplicateCommand.java:92)
| at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:173)
| at
org.jboss.cache.marshall.InactiveRegionAwareRpcDispatcher.handle(InactiveRegionAwareRpcDispatcher.java:73)
| at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:624)
| at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:533)
| at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:365)
| at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:764)
| at org.jgroups.JChannel.up(JChannel.java:1265)
| at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:441)
| at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:373)
| at org.jgroups.protocols.pbcast.STATE_TRANSFER.up(STATE_TRANSFER.java:144)
| at org.jgroups.protocols.FRAG2.up(FRAG2.java:205)
| at org.jgroups.protocols.FC.up(FC.java:411)
| at org.jgroups.protocols.pbcast.GMS.up(GMS.java:772)
| at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
| at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:627)
| at org.jgroups.protocols.UNICAST.up(UNICAST.java:286)
| at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:729)
| at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
| at org.jgroups.protocols.FD.up(FD.java:284)
| at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:309)
| at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
| at org.jgroups.protocols.Discovery.up(Discovery.java:244)
| at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2191)
| at org.jgroups.protocols.TP.passMessageUp(TP.java:1235)
| at org.jgroups.protocols.TP.access$100(TP.java:49)
| at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1802)
| at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1776)
| at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
| at java.lang.Thread.run(Thread.java:595)
|
The earlier logging from RegionManager is due to calls from
AbstractCacheLoader.regionAwareUnmarshall(), which is trying to find an appropriate
classloader to set and fails. I think this comes down to a problem in
AbstractCacheLoader.regionAwareMarshall() and regionAwareUnmarshall():
Region r = regionManager == null ? null : regionManager.getRegion(fqn,
Region.Type.MARSHALLING, false);
That call won't convert from a BR FQN to a main tree FQN, so it doesn't find the
region. Better would be to call a method that does that, a la
AbstractMarshaller.getRegion(Fqn):
| protected Region getRegion(Fqn fqn)
| {
| if (fqn == null) return null;
| if (buddyFqnTransformer.isBackupFqn(fqn))
| {
| // Strip out the buddy group portion
| fqn = buddyFqnTransformer.getActualFqn(fqn);
| }
| return regionManager.getRegion(fqn, Region.Type.MARSHALLING, false);
| }
|
That should wrap it up, yes? Problem is I look at the code in 2.1.1.GA and it looks the
same, so I don't understand why the test is failing now and not w/ 2.1.1.GA. So
I'm missing something important.
Could it be that 2.1.1.GA wouldn't try to load the node from disk when doing the
remove, i.e. the problem's been there all along but a behavior change has exposed it?
View the original post :
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4161306#...
Reply to the post :
http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...