<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<body link="#355491" alink="#4262a1" vlink="#355491" style="background: #e2e2e2; margin: 0; padding: 20px;">
<div>
<table cellpadding="0" bgcolor="#FFFFFF" border="0" cellspacing="0" style="border: 1px solid #dadada; margin-bottom: 30px; width: 100%; -moz-border-radius: 6px; -webkit-border-radius: 6px;">
<tbody>
<tr>
<td>
<table border="0" cellpadding="0" cellspacing="0" bgcolor="#FFFFFF" style="border: solid 2px #ccc; background: #dadada; width: 100%; -moz-border-radius: 6px; -webkit-border-radius: 6px;">
<tbody>
<tr>
<td bgcolor="#000000" valign="middle" height="58px" style="border-bottom: 1px solid #ccc; padding: 20px; -moz-border-radius-topleft: 3px; -moz-border-radius-topright: 3px; -webkit-border-top-right-radius: 5px; -webkit-border-top-left-radius: 5px;">
<h1 style="color: #333333; font: bold 22px Arial, Helvetica, sans-serif; margin: 0; display: block !important;">
<!-- To have a header image/logo replace the name below with your img tag -->
<!-- Email clients will render the images when the message is read so any image -->
<!-- must be made available on a public server, so that all recipients can load the image. -->
<a href="http://community.jboss.org/index.jspa" style="text-decoration: none; color: #E1E1E1">Community</a></h1>
</td>
</tr>
<tr>
<td bgcolor="#FFFFFF" style="font: normal 12px Arial, Helvetica, sans-serif; color:#333333; padding: 20px; -moz-border-radius-bottomleft: 4px; -moz-border-radius-bottomright: 4px; -webkit-border-bottom-right-radius: 5px; -webkit-border-bottom-left-radius: 5px;"><h3 style="margin: 10px 0 5px; font-size: 17px; font-weight: normal;">
JbossCache and Websphere 6.1 Cluster
</h3>
<span style="margin-bottom: 10px;">
created by <a href="http://community.jboss.org/people/lorod%40cadit.it">DIEGO LORO</a> in <i>JBoss Cache</i> - <a href="http://community.jboss.org/message/539442#539442">View the full discussion</a>
</span>
<hr style="margin: 20px 0; border: none; background-color: #dadada; height: 1px;">
<div class="jive-rendered-content"><p>Hello all,</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>I'm using a Websphere 6.1 Cluster made up of 2 nodes, each node on a different server (node1 on server1, node2 on server2).</p><p>I'm using JbossCache Api 3.0.3GA as caching instrument of my WebApplication.</p><p>I have two different cache instances: the first for user layouts on file system, the second for user data in memory.</p><p>All works fine when I start up the cluster, data sharing between nodes of the two instances is great (in UDP and olso in TCP mode).</p><p>Layout Cache is binded on port 7800 on the two servers/nodes, the user data cache is binded on 7801 port. </p><p>That's my TCP configuration for only one of the two cache instances on the two nodes/servers of my cluster (all the files in jbc.zip file attached).</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p><table border="1" cellpadding="3" cellspacing="0" style="width: 100%; border: #000000 1px solid;"><tbody><tr><th align="center" style="background-color:#6690BC;" valign="middle"><span style="color: #ffffff;"><strong style="color: #ffffff; "><strong>Layout Cache on Server1/Node1</strong></strong></span></th><th align="center" style="background-color:#6690BC;" valign="middle"><span style="color: #ffffff;"><strong>Layout Cache on Server2/Node2</strong></span></th></tr><tr><td><?xml version="1.0" encoding="utf-8" ?><br/><jbosscache xmlns:xsi="<a class="jive-link-external-small" href="http://www.w3.org/2001/XMLSchema-instance">http://www.w3.org/2001/XMLSchema-instance</a>"<br/>            xmlns="urn:jboss:jbosscache-core:config:3.0"><br/> <br/>     <!-- CLUSTERING: sharing via TCPIP --><br/> <clustering mode="replication" clusterName="LAYOUT"><br/>  <stateRetrieval timeout="600000" fetchInMemoryState="true"/><br/>  <async/><br/>  <jgroupsConfig><br/>   <TCP start_port="7800" bind_addr="172.16.15.170" loopback="false"/><br/>   <TCPPING timeout="3000" <br/>   initial_hosts="172.16.15.17[7800],172.16.15.170[7800]"<br/>     port_range="0" num_initial_members="2" /><br/>   <MERGE2 max_interval="30000" min_interval="10000"/><br/>   <FD max_tries="5" shun="true" timeout="10000"/><br/>   <VERIFY_SUSPECT timeout="1500"/><br/>   <pbcast.NAKACK use_mcast_xmit="false" gc_lag="0" <br/>    retransmit_timeout="300,600,1200,2400,4800" discard_delivered_msgs="true" /> <br/>   <pbcast.STABLE desired_avg_gossip="50000" max_bytes="400000" stability_delay="1000"/><br/>   <pbcast.GMS join_timeout="5000" print_local_addr="true" shun="false"<br/>     view_ack_collection_timeout="5000" view_bundling="true"/><br/>   <FC max_credits="1000000" min_threshold="0.20"/><br/>   <FRAG2 frag_size="60000"/><br/>   <pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/><br/>   <!--pbcast.STREAMING_STATE_TRANSFER <br/>    socket_buffer_size="1048576" /--> <br/>   <pbcast.FLUSH timeout="0"/><br/>  </jgroupsConfig><br/> </clustering><br/> <br/> <!-- FS: persist on FS --><br/>    <loaders><br/>        <loader<br/>                async="false"<br/>                purgeOnStartup="false"<br/>    fetchPersistentState="true"><br/>            <properties><br/>                location=/opt/wax-jbosscache/#MyLayoutDB<br/>            </properties><br/>        </loader><br/>    </loaders><br/>    <br/></jbosscache></td><td><?xml version="1.0" encoding="utf-8" ?><br/><jbosscache xmlns:xsi="<a class="jive-link-external-small" href="http://www.w3.org/2001/XMLSchema-instance">http://www.w3.org/2001/XMLSchema-instance</a>"<br/>            xmlns="urn:jboss:jbosscache-core:config:3.0"><br/> <br/>     <!-- CLUSTERING: sharing via TCPIP --><br/> <clustering mode="replication" clusterName="LAYOUT"><br/>  <stateRetrieval timeout="600000" fetchInMemoryState="true"/><br/>  <async/><br/>  <jgroupsConfig><br/>   <TCP start_port="7800" bind_addr="172.16.15.17" loopback="false"/><br/>   <TCPPING timeout="3000" <br/>   initial_hosts="172.16.15.17[7800],172.16.15.170[7800]"<br/>     port_range="0" num_initial_members="2" /><br/>   <MERGE2 max_interval="30000" min_interval="10000"/><br/>   <FD max_tries="5" shun="true" timeout="10000"/><br/>   <VERIFY_SUSPECT timeout="1500"/><br/>   <pbcast.NAKACK use_mcast_xmit="false" gc_lag="0" <br/>    retransmit_timeout="300,600,1200,2400,4800" discard_delivered_msgs="true" /> <br/>   <pbcast.STABLE desired_avg_gossip="50000" max_bytes="400000" stability_delay="1000"/><br/>   <pbcast.GMS join_timeout="5000" print_local_addr="true" shun="false"<br/>     view_ack_collection_timeout="5000" view_bundling="true"/><br/>   <FC max_credits="1000000" min_threshold="0.20"/><br/>   <FRAG2 frag_size="60000"/><br/>   <pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/><br/>   <!--pbcast.STREAMING_STATE_TRANSFER <br/>    socket_buffer_size="1048576" /--> <br/>   <pbcast.FLUSH timeout="0"/><br/>  </jgroupsConfig><br/> </clustering><br/> <br/> <!-- FS: persist on FS --><br/>    <loaders><br/>        <loader<br/>                async="false"<br/>                purgeOnStartup="false"<br/>    fetchPersistentState="true"><br/>            <properties><br/>                location=/opt/wax-jbosscache/#MyLayoutDB<br/>            </properties><br/>        </loader><br/>    </loaders><br/>    <br/></jbosscache></td></tr></tbody></table></p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p><span style="text-decoration: underline;">Problems start when I shut down one of the two nodes and then I restart it, because this node is unable to retreive the cache state from the other node of the cluster (I can see in the server logs the cluster configuration beeing updated with the new node but nothing works fine).</span></p><p><span style="text-decoration: underline;">That is the error I found calling the restarted node:</span></p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p><em>at org.jboss.cache.remoting.jgroups.ChannelMessageListener.stateReceivingFailed(ChannelMessageListener.java:130) <br/>     at org.jboss.cache.remoting.jgroups.ChannelMessageListener.setState(ChannelMessageListener.java:299) <br/>     at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:714) <br/>     at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:776) <br/>     at org.jgroups.JChannel.up(JChannel.java:1226) <br/>     at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:462) <br/>     at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:443) <br/>     at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:488) <br/>     at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:453) <br/>     at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:211) <br/>     at org.jgroups.protocols.FRAG2.up(FRAG2.java:192) <br/>     at org.jgroups.protocols.FC.up(FC.java:468) <br/>     at org.jgroups.protocols.pbcast.GMS.up(GMS.java:796) <br/>     at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233) <br/>     at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:616) <br/>     at org.jgroups.protocols.UNICAST.up(UNICAST.java:282) <br/>     at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:747) <br/>     at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167) <br/>     at org.jgroups.protocols.FD.up(FD.java:284) <br/>     at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:308) <br/>     at org.jgroups.protocols.MERGE2.up(MERGE2.java:144) <br/>     at org.jgroups.protocols.Discovery.up(Discovery.java:263) <br/>     at org.jgroups.protocols.PING.up(PING.java:270) <br/>     at org.jgroups.protocols.TP.passMessageUp(TP.java:1277) <br/>     at org.jgroups.protocols.TP.access$100(TP.java:49) <br/>     at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1830) <br/>     at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1804) <br/>     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665) <br/>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690) <br/>     at java.lang.Thread.run(Thread.java:797) <br/>     Caused by: java.lang.NoClassDefFoundError: com.cadit.wax.ws.classes.Session (initialization failure) <br/>     at java.lang.J9VMInternals.initialize(J9VMInternals.java:123) <br/>     at java.io.ObjectStreamClass.hasStaticInitializer(Native Method) <br/>     at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1723) <br/>     at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:103) <br/>     at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:235) <br/>     at java.security.AccessController.doPrivileged(AccessController.java:192) <br/>     at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:232) <br/>     at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:597) <br/>     at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1561) <br/>     at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1475) <br/>     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1708) <br/>     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314) <br/>     at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1927) <br/>     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1851) <br/>     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1728) <br/>     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314) <br/>     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:354) <br/>     at java.util.HashMap.readObject(HashMap.java:1068) <br/>     at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) <br/>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) <br/>     at java.lang.reflect.Method.invoke(Method.java:615) <br/>     at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1001) <br/>     at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1828) <br/>     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1728) <br/>     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314) <br/>     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:354) <br/>     at org.jboss.cache.marshall.NodeData.readExternal(NodeData.java:112) <br/>     at org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:522) <br/>     at org.jboss.cache.marshall.CacheMarshaller200.populateFromStream(CacheMarshaller200.java:660) <br/>     at org.jboss.cache.marshall.CacheMarshaller200.unmarshallLinkedList(CacheMarshaller200.java:611) <br/>     at org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:486) <br/>     at org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:433) <br/>     at org.jboss.cache.marshall.CacheMarshaller200.objectFromObjectStream(CacheMarshaller200.java:141) <br/>     at org.jboss.cache.marshall.VersionAwareMarshaller.objectFromObjectStream(VersionAwareMarshaller.java:360) <br/>     at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.readNodesAsList(DefaultStateTransferIntegrator.java:249) <br/>     at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateTransientState(DefaultStateTransferIntegrator.java:210) <br/>     at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateTransientState(DefaultStateTransferIntegrator.java:108) <br/>     at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateState(DefaultStateTransferIntegrator.java:81) <br/>     at org.jboss.cache.statetransfer.DefaultStateTransferManager.setState(DefaultStateTransferManager.java:191) <br/>     at org.jboss.cache.statetransfer.DefaultStateTransferManager.setState(DefaultStateTransferManager.java:155) <br/>     at org.jboss.cache.remoting.jgroups.ChannelMessageListener.setState(ChannelMessageListener.java:294) <br/>     ... 28 more <br/>     Caused by: java.lang.Throwable: org.jboss.cache.CacheException: java.lang.reflect.InvocationTargetException <br/>     at org.jboss.cache.util.reflect.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:148) <br/>     at org.jboss.cache.factories.ComponentRegistry$PrioritizedMethod.invoke(ComponentRegistry.java:883) <br/>     at org.jboss.cache.factories.ComponentRegistry.internalStart(ComponentRegistry.java:680) <br/>     at org.jboss.cache.factories.ComponentRegistry.start(ComponentRegistry.java:561) <br/>     at org.jboss.cache.invocation.CacheInvocationDelegate.start(CacheInvocationDelegate.java:301) <br/>     at org.jboss.cache.DefaultCacheFactory.createCache(DefaultCacheFactory.java:119) <br/>     at org.jboss.cache.DefaultCacheFactory.createCache(DefaultCacheFactory.java:94) <br/>     at org.jboss.cache.DefaultCacheFactory.createCache(DefaultCacheFactory.java:71) <br/>     at it.cadit.wax.cache.jbosscache.WAXCacheFactory.<init>(WAXCacheFactory.java:35) <br/>     at it.cadit.wax.cache.common.WAXCacheFactoryType$1.getCacheFactory(WAXCacheFactoryType.java:18) <br/>     at com.cadit.wax.util.Utility.initializeCache(Utility.java:959) <br/>     at com.cadit.wax.cache.Repository.initializeCache(Repository.java:26) <br/>     at com.cadit.wax.ws.classes.Session.<clinit>(Session.java:30) <br/>     at java.lang.J9VMInternals.initializeImpl(Native Method) <br/>     at java.lang.J9VMInternals.initialize(J9VMInternals.java:177) <br/>     at com.cadit.wax.ws.city.authws.AuthWSSoapBindingImpl.<clinit>(AuthWSSoapBindingImpl.java:292) <br/>     at java.lang.J9VMInternals.initializeImpl(Native Method) <br/>     at java.lang.J9VMInternals.initialize(J9VMInternals.java:177) <br/>     at java.lang.Class.forNameImpl(Native Method) <br/>     at java.lang.Class.forName(Class.java:131) <br/>     at com.cadit.wax.ResourceManagerListener$Configuration.createObject(ResourceManagerListener.java:441) <br/>     at com.cadit.wax.ResourceManagerListener$Configuration.createAuthService(ResourceManagerListener.java:397) <br/>     at com.cadit.wax.ResourceManagerListener$Configuration.load(ResourceManagerListener.java:276) <br/>     at com.cadit.wax.ResourceManagerListener.loadConfiguration(ResourceManagerListener.java:698) <br/>     at com.cadit.wax.ResourceManagerListener.configure(ResourceManagerListener.java:690) <br/>     at com.cadit.wax.ResourceManagerListener.contextInitialized(ResourceManagerListener.java:722) <br/>     at com.ibm.ws.wswebcontainer.webapp.WebApp.notifyServletContextCreated(WebApp.java:605) <br/>     at com.ibm.ws.webcontainer.webapp.WebApp.commonInitializationFinish(WebApp.java:265) <br/>     at com.ibm.ws.wswebcontainer.webapp.WebApp.initialize(WebApp.java:271) <br/>     at com.ibm.ws.wswebcontainer.webapp.WebGroup.addWebApplication(WebGroup.java:88) <br/>     at com.ibm.ws.wswebcontainer.VirtualHost.addWebApplication(VirtualHost.java:157) <br/>     at com.ibm.ws.wswebcontainer.WebContainer.addWebApp(WebContainer.java:653) <br/>     at com.ibm.ws.wswebcontainer.WebContainer.addWebApplication(WebContainer.java:606) <br/>     at com.ibm.ws.webcontainer.component.WebContainerImpl.install(WebContainerImpl.java:333) <br/>     at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:549) <br/>     at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1295) <br/>     at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1129) <br/>     at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:567) <br/>     at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:814) <br/>     at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:948) <br/>     at com.ibm.ws.runtime.component.ApplicationMgrImpl$AppInitializer.run(ApplicationMgrImpl.java:2114) <br/>     at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:340) <br/>     at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1498) <br/>     Caused by: java.lang.Throwable: java.lang.reflect.InvocationTargetException <br/>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) <br/>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) <br/>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) <br/>     at java.lang.reflect.Method.invoke(Method.java:615) <br/>     at org.jboss.cache.util.reflect.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:144) <br/>     ... 42 more <br/>     Caused by: java.lang.Throwable: org.jboss.cache.CacheException: Unable to connect to JGroups channel <br/>     at org.jboss.cache.RPCManagerImpl.start(RPCManagerImpl.java:252) <br/>     ... 47 more <br/>     Caused by: java.lang.Throwable: org.jgroups.StateTransferException: 172.16.15.170:51651 could not fetch state null from null <br/>     at org.jgroups.JChannel.connect(JChannel.java:466) <br/>     at org.jboss.cache.RPCManagerImpl.start(RPCManagerImpl.java:242) <br/>     ... 47 more <br/>     Caused by: java.lang.Throwable: org.jgroups.StateTransferException: 172.16.15.170:51651 could not fetch state null from null <br/>     at org.jgroups.JChannel.connect(JChannel.java:459) <br/>     ... 48 more <br/></em></p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>I don't thing the problem is the timeout because is already big:</p><p>Any suggestion?</p><p>Thank's to all  !!!</p></div>
<div style="background-color: #f4f4f4; padding: 10px; margin-top: 20px;">
<p style="margin: 0;">Reply to this message by <a href="http://community.jboss.org/message/539442#539442">going to Community</a></p>
<p style="margin: 0;">Start a new discussion in JBoss Cache at <a href="http://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2052">Community</a></p>
</div></td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</div>
</body>
</html>