I am doing a POC to test out some things with Infinispan.  I have two nodes with the same deployment on each one.  If I go to a node, doesn't matter which one, and create a cache there, I can access that cache from the second node, but when I go to create another cache on the second node, I get the following exception:

15:05:23,316 INFO  [STDOUT] 15:05:23,314 ERROR [RpcManagerImpl] unexpected error while replicating
org.infinispan.manager.NamedCacheNotFoundException: Cache: cacheB
at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:72)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:177)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:149)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:578)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:489)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:365)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:771)
at org.jgroups.JChannel.up(JChannel.java:1465)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:954)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:430)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:265)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:190)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:419)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:419)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:888)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:577)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:295)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:707)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:120)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:169)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:269)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:210)
at org.jgroups.protocols.Discovery.up(Discovery.java:292)
at org.jgroups.protocols.PING.up(PING.java:67)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1093)
at org.jgroups.protocols.TP.access$100(TP.java:56)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1633)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1615)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
15:05:23,318 INFO  [STDOUT] 15:05:23,317 ERROR [JoinTask] Caught exception!
org.infinispan.CacheException: org.infinispan.manager.NamedCacheNotFoundException: Cache: cacheB
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:114)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:125)
at org.infinispan.distribution.JoinTask.retrieveOldCH(JoinTask.java:192)
at org.infinispan.distribution.JoinTask.performRehash(JoinTask.java:87)
at org.infinispan.distribution.RehashTask.call(RehashTask.java:53)
at org.infinispan.distribution.RehashTask.call(RehashTask.java:33)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: org.infinispan.manager.NamedCacheNotFoundException: Cache: cacheB
at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:72)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:177)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:149)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:578)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:489)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:365)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:771)
at org.jgroups.JChannel.up(JChannel.java:1465)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:954)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:430)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:265)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:190)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:419)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:419)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:888)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:577)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:295)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:707)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:120)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:169)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:269)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:210)
at org.jgroups.protocols.Discovery.up(Discovery.java:292)
at org.jgroups.protocols.PING.up(PING.java:67)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1093)
at org.jgroups.protocols.TP.access$100(TP.java:56)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1633)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1615)
... 3 more


I'm accessing this with the following configuration:
<?xml version="1.0" encoding="UTF-8"?>
<infinispan
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="urn:infinispan:config:4.2 http://www.infinispan.org/schemas/infinispan-config-4.2.xsd"
      xmlns="urn:infinispan:config:4.2">
   
   <global>
      <transport 
         clusterName="myCluster" 
         transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport"/>
      <globalJmxStatistics enabled="true"/>
   </global>

   <default>
      <jmxStatistics enabled="true"/>
      <clustering mode="distribution">

         <l1 enabled="true" lifespan="60000"/>
         <hash numOwners="2" rehashRpcTimeout="120000"/>
         <async/>
      </clustering>
   </default>
</infinispan>

And this is the code I'm executing:
            DefaultCacheManager cacheContainer = (DefaultCacheManager) ctx.lookup("java:comp/env/CacheContainer");
            Cache cache;
            if (cacheContainer.getCacheNames() != null && cacheContainer.getCacheNames().contains(name)) {
                cache = cacheContainer.getCache(name);
            } else {
                Configuration conf = new Configuration();
                if (name.endsWith("A")) {
                    conf.setExpirationLifespan(5000);
                } else if (name.endsWith("B")) {
                    conf.setExpirationLifespan(30000);
                } else {
                    conf.setExpirationLifespan(60000);
                }

                cacheContainer.defineConfiguration(name, conf);

                cache = cacheContainer.getCache(name);
            }
            Integer counter = (Integer) cache.get("counter");
            if (counter == null)
                counter = new Integer(0);
            counter++;
            response.getOutputStream().println("Counter : " + counter);
            cache.put("counter", counter);


This seems like a pretty straightforward use-case.  I'm going to grab the git repo and start to look into it, but any help or hints would be great.

Thanks,
Brian Ashburn