[jboss-user] [JBoss Cache] - JbossCache and Websphere 6.1 Cluster
DIEGO LORO
do-not-reply at jboss.com
Mon Apr 26 04:00:38 EDT 2010
DIEGO LORO [http://community.jboss.org/people/lorod%40cadit.it] created the discussion
"JbossCache and Websphere 6.1 Cluster"
To view the discussion, visit: http://community.jboss.org/message/539442#539442
--------------------------------------------------------------
Hello all,
I'm using a Websphere 6.1 Cluster made up of 2 nodes, each node on a different server (node1 on server1, node2 on server2).
I'm using JbossCache Api 3.0.3GA as caching instrument of my WebApplication.
I have two different cache instances: the first for user layouts on file system, the second for user data in memory.
All works fine when I start up the cluster, data sharing between nodes of the two instances is great (in UDP and olso in TCP mode).
Layout Cache is binded on port 7800 on the two servers/nodes, the user data cache is binded on 7801 port.
That's my TCP configuration for only one of the two cache instances on the two nodes/servers of my cluster (all the files in jbc.zip file attached).
|| **Layout Cache on Server1/Node1** || *Layout Cache on Server2/Node2* ||
| <?xml version="1.0" encoding="utf-8" ?>
<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:jboss:jbosscache-core:config:3.0">
<!-- CLUSTERING: sharing via TCPIP -->
<clustering mode="replication" clusterName="LAYOUT">
<stateRetrieval timeout="600000" fetchInMemoryState="true"/>
<async/>
<jgroupsConfig>
<TCP start_port="7800" bind_addr="172.16.15.170" loopback="false"/>
<TCPPING timeout="3000"
initial_hosts="172.16.15.17[7800],172.16.15.170[7800]"
port_range="0" num_initial_members="2" />
<MERGE2 max_interval="30000" min_interval="10000"/>
<FD max_tries="5" shun="true" timeout="10000"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK use_mcast_xmit="false" gc_lag="0"
retransmit_timeout="300,600,1200,2400,4800" discard_delivered_msgs="true" />
<pbcast.STABLE desired_avg_gossip="50000" max_bytes="400000" stability_delay="1000"/>
<pbcast.GMS join_timeout="5000" print_local_addr="true" shun="false"
view_ack_collection_timeout="5000" view_bundling="true"/>
<FC max_credits="1000000" min_threshold="0.20"/>
<FRAG2 frag_size="60000"/>
<pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/>
<!--pbcast.STREAMING_STATE_TRANSFER
socket_buffer_size="1048576" /-->
<pbcast.FLUSH timeout="0"/>
</jgroupsConfig>
</clustering>
<!-- FS: persist on FS -->
<loaders>
<loader
async="false"
purgeOnStartup="false"
fetchPersistentState="true">
<properties>
location=/opt/wax-jbosscache/#MyLayoutDB
</properties>
</loader>
</loaders>
</jbosscache> | <?xml version="1.0" encoding="utf-8" ?>
<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:jboss:jbosscache-core:config:3.0">
<!-- CLUSTERING: sharing via TCPIP -->
<clustering mode="replication" clusterName="LAYOUT">
<stateRetrieval timeout="600000" fetchInMemoryState="true"/>
<async/>
<jgroupsConfig>
<TCP start_port="7800" bind_addr="172.16.15.17" loopback="false"/>
<TCPPING timeout="3000"
initial_hosts="172.16.15.17[7800],172.16.15.170[7800]"
port_range="0" num_initial_members="2" />
<MERGE2 max_interval="30000" min_interval="10000"/>
<FD max_tries="5" shun="true" timeout="10000"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK use_mcast_xmit="false" gc_lag="0"
retransmit_timeout="300,600,1200,2400,4800" discard_delivered_msgs="true" />
<pbcast.STABLE desired_avg_gossip="50000" max_bytes="400000" stability_delay="1000"/>
<pbcast.GMS join_timeout="5000" print_local_addr="true" shun="false"
view_ack_collection_timeout="5000" view_bundling="true"/>
<FC max_credits="1000000" min_threshold="0.20"/>
<FRAG2 frag_size="60000"/>
<pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/>
<!--pbcast.STREAMING_STATE_TRANSFER
socket_buffer_size="1048576" /-->
<pbcast.FLUSH timeout="0"/>
</jgroupsConfig>
</clustering>
<!-- FS: persist on FS -->
<loaders>
<loader
async="false"
purgeOnStartup="false"
fetchPersistentState="true">
<properties>
location=/opt/wax-jbosscache/#MyLayoutDB
</properties>
</loader>
</loaders>
</jbosscache> |
Problems start when I shut down one of the two nodes and then I restart it, because this node is unable to retreive the cache state from the other node of the cluster (I can see in the server logs the cluster configuration beeing updated with the new node but nothing works fine).
That is the error I found calling the restarted node:
+at org.jboss.cache.remoting.jgroups.ChannelMessageListener.stateReceivingFailed(ChannelMessageListener.java:130)
at org.jboss.cache.remoting.jgroups.ChannelMessageListener.setState(ChannelMessageListener.java:299)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:714)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:776)
at org.jgroups.JChannel.up(JChannel.java:1226)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:462)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:443)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:488)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:453)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:211)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:192)
at org.jgroups.protocols.FC.up(FC.java:468)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:796)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:616)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:747)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
at org.jgroups.protocols.FD.up(FD.java:284)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:308)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
at org.jgroups.protocols.Discovery.up(Discovery.java:263)
at org.jgroups.protocols.PING.up(PING.java:270)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1277)
at org.jgroups.protocols.TP.access$100(TP.java:49)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1830)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1804)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)
at java.lang.Thread.run(Thread.java:797)
Caused by: java.lang.NoClassDefFoundError: com.cadit.wax.ws.classes.Session (initialization failure)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:123)
at java.io.ObjectStreamClass.hasStaticInitializer(Native Method)
at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1723)
at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:103)
at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:235)
at java.security.AccessController.doPrivileged(AccessController.java:192)
at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:232)
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:597)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1561)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1475)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1708)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1927)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1851)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1728)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:354)
at java.util.HashMap.readObject(HashMap.java:1068)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:615)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1001)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1828)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1728)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:354)
at org.jboss.cache.marshall.NodeData.readExternal(NodeData.java:112)
at org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:522)
at org.jboss.cache.marshall.CacheMarshaller200.populateFromStream(CacheMarshaller200.java:660)
at org.jboss.cache.marshall.CacheMarshaller200.unmarshallLinkedList(CacheMarshaller200.java:611)
at org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:486)
at org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:433)
at org.jboss.cache.marshall.CacheMarshaller200.objectFromObjectStream(CacheMarshaller200.java:141)
at org.jboss.cache.marshall.VersionAwareMarshaller.objectFromObjectStream(VersionAwareMarshaller.java:360)
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.readNodesAsList(DefaultStateTransferIntegrator.java:249)
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateTransientState(DefaultStateTransferIntegrator.java:210)
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateTransientState(DefaultStateTransferIntegrator.java:108)
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateState(DefaultStateTransferIntegrator.java:81)
at org.jboss.cache.statetransfer.DefaultStateTransferManager.setState(DefaultStateTransferManager.java:191)
at org.jboss.cache.statetransfer.DefaultStateTransferManager.setState(DefaultStateTransferManager.java:155)
at org.jboss.cache.remoting.jgroups.ChannelMessageListener.setState(ChannelMessageListener.java:294)
... 28 more
Caused by: java.lang.Throwable: org.jboss.cache.CacheException: java.lang.reflect.InvocationTargetException
at org.jboss.cache.util.reflect.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:148)
at org.jboss.cache.factories.ComponentRegistry$PrioritizedMethod.invoke(ComponentRegistry.java:883)
at org.jboss.cache.factories.ComponentRegistry.internalStart(ComponentRegistry.java:680)
at org.jboss.cache.factories.ComponentRegistry.start(ComponentRegistry.java:561)
at org.jboss.cache.invocation.CacheInvocationDelegate.start(CacheInvocationDelegate.java:301)
at org.jboss.cache.DefaultCacheFactory.createCache(DefaultCacheFactory.java:119)
at org.jboss.cache.DefaultCacheFactory.createCache(DefaultCacheFactory.java:94)
at org.jboss.cache.DefaultCacheFactory.createCache(DefaultCacheFactory.java:71)
at it.cadit.wax.cache.jbosscache.WAXCacheFactory.<init>(WAXCacheFactory.java:35)
at it.cadit.wax.cache.common.WAXCacheFactoryType$1.getCacheFactory(WAXCacheFactoryType.java:18)
at com.cadit.wax.util.Utility.initializeCache(Utility.java:959)
at com.cadit.wax.cache.Repository.initializeCache(Repository.java:26)
at com.cadit.wax.ws.classes.Session.<clinit>(Session.java:30)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:177)
at com.cadit.wax.ws.city.authws.AuthWSSoapBindingImpl.<clinit>(AuthWSSoapBindingImpl.java:292)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:177)
at java.lang.Class.forNameImpl(Native Method)
at java.lang.Class.forName(Class.java:131)
at com.cadit.wax.ResourceManagerListener$Configuration.createObject(ResourceManagerListener.java:441)
at com.cadit.wax.ResourceManagerListener$Configuration.createAuthService(ResourceManagerListener.java:397)
at com.cadit.wax.ResourceManagerListener$Configuration.load(ResourceManagerListener.java:276)
at com.cadit.wax.ResourceManagerListener.loadConfiguration(ResourceManagerListener.java:698)
at com.cadit.wax.ResourceManagerListener.configure(ResourceManagerListener.java:690)
at com.cadit.wax.ResourceManagerListener.contextInitialized(ResourceManagerListener.java:722)
at com.ibm.ws.wswebcontainer.webapp.WebApp.notifyServletContextCreated(WebApp.java:605)
at com.ibm.ws.webcontainer.webapp.WebApp.commonInitializationFinish(WebApp.java:265)
at com.ibm.ws.wswebcontainer.webapp.WebApp.initialize(WebApp.java:271)
at com.ibm.ws.wswebcontainer.webapp.WebGroup.addWebApplication(WebGroup.java:88)
at com.ibm.ws.wswebcontainer.VirtualHost.addWebApplication(VirtualHost.java:157)
at com.ibm.ws.wswebcontainer.WebContainer.addWebApp(WebContainer.java:653)
at com.ibm.ws.wswebcontainer.WebContainer.addWebApplication(WebContainer.java:606)
at com.ibm.ws.webcontainer.component.WebContainerImpl.install(WebContainerImpl.java:333)
at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:549)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1295)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1129)
at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:567)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:814)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:948)
at com.ibm.ws.runtime.component.ApplicationMgrImpl$AppInitializer.run(ApplicationMgrImpl.java:2114)
at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:340)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1498)
Caused by: java.lang.Throwable: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:615)
at org.jboss.cache.util.reflect.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:144)
... 42 more
Caused by: java.lang.Throwable: org.jboss.cache.CacheException: Unable to connect to JGroups channel
at org.jboss.cache.RPCManagerImpl.start(RPCManagerImpl.java:252)
... 47 more
Caused by: java.lang.Throwable: org.jgroups.StateTransferException: 172.16.15.170:51651 could not fetch state null from null
at org.jgroups.JChannel.connect(JChannel.java:466)
at org.jboss.cache.RPCManagerImpl.start(RPCManagerImpl.java:242)
... 47 more
Caused by: java.lang.Throwable: org.jgroups.StateTransferException: 172.16.15.170:51651 could not fetch state null from null
at org.jgroups.JChannel.connect(JChannel.java:459)
... 48 more
+
I don't thing the problem is the timeout because is already big:
Any suggestion?
Thank's to all !!!
--------------------------------------------------------------
Reply to this message by going to Community
[http://community.jboss.org/message/539442#539442]
Start a new discussion in JBoss Cache at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2052]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/jboss-user/attachments/20100426/8ca7ad94/attachment.html
More information about the jboss-user
mailing list