[jbosscache-issues] [JBoss JIRA] Created: (JBCACHE-1580) Type safety problem with dead member buddy backup region

[Brian Stansberry (JIRA)]

Type safety problem with dead member buddy backup region
--------------------------------------------------------

                 Key: JBCACHE-1580
                 URL: https://jira.jboss.org/jira/browse/JBCACHE-1580
             Project: JBoss Cache
          Issue Type: Bug
      Security Level: Public (Everyone can see)
          Components: Buddy Replication, Cache loaders
    Affects Versions: 3.2.4.GA
            Reporter: Brian Stansberry
            Assignee: Manik Surtani
            Priority: Critical


I'm seeing problems with data gravitation when testing JBC 3.2.4 against JBoss AS trunk and the current EAP branch. I think the problem I describe here goes back a long way; nothing new in 3.2.4, but for some reason it's popping up now.[1]

The problem is that when a :DEAD member region is created, it gets a child node whose name is of type Integer.  There is code in JBC that assumes names for all children of these :DEAD nodes will be of type Integer. But CacheLoaders, particularly FileCacheLoader, cannot make such a guarantee. If the attempt to determine the children of the :DEAD node results in a call to CacheLoader.getChildrenNames(), there are type safety problems, e.g. this failure when a cache get() for a non-existent Fqn is invoked:

java.lang.ArrayStoreException: java.lang.String
	at java.util.AbstractCollection.toArray(AbstractCollection.java:171)
	at org.jboss.cache.commands.read.GravitateDataCommand.perform(GravitateDataCommand.java:135)
	at org.jboss.cache.interceptors.CallInterceptor.invokeCommand(CallInterceptor.java:108)
	at org.jboss.cache.interceptors.CallInterceptor.handleDefault(CallInterceptor.java:99)
	at org.jboss.cache.commands.AbstractVisitor.visitGravitateDataCommand(AbstractVisitor.java:135)

You can see the problem in this snippet from GravitateDataCommand.perform():

                  if (buddyFqnTransformer.isDeadBackupRoot(backupRoot))
                  {
                     Set<Integer> deadChildNames = new TreeSet<Integer>(spi.getChildrenNames(backupRoot));
                     Integer[] elems = deadChildNames.toArray(new Integer[deadChildNames.size()]);

spi.getChildrenNames() is not guaranteed to return a Set<Integer>, and definitely will not if it calls FileCacheLoader.getChildrenNames().

I've also seen ClassCastException in that same area in a compareTo() method being invoked by TreeSet.

I'll attach a patch that makes the problem go away (i.e. allows the failing AS test to pass.) Basically adds a utility method to BuddyFqnTransformer that does a conversion of the return value from CacheLoader.getChildrenNames() to Set<Integer>.  Then changes the 3 places that use the contents of the CacheLoader.getChildrenNames() to have them use this utility method.

Another approach would be to change the return type of FileCacheLoader.getChildrenNames() to Set<?> (same as the interface) and call the new BuddyFqnTransformer utility method from there. But that changes the API of FileCacheLoader, plus requires injecting BuddyFqnTransformer into FileCacheLoader.

Sorry; I don't have a proper unit test for this. :-(  I thought this was an AS bug (there was a related one -- JBCLUSTER-269) and used all my time trying to fix it at the wrong level.

[1] I only saw the problem when I started testing AS + 3.2.4 + JGroups 2.6.15. With 2.6.13 it didn't happen. I think the JGroups change must have for some reason resulted in :DEAD member regions being created that weren't before; the test that's failing involves stopping and starting caches. Perhaps a timing thing. There was a flaw in an AS class that should have resulted in the test failing if :DEAD member regions were being created before, so the fact that the test has been passing tells me they weren't.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira