<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<body link="#355491" alink="#4262a1" vlink="#355491" style="background: #e2e2e2; margin: 0; padding: 20px;">
<div>
<table cellpadding="0" bgcolor="#FFFFFF" border="0" cellspacing="0" style="border: 1px solid #dadada; margin-bottom: 30px; width: 100%; -moz-border-radius: 6px; -webkit-border-radius: 6px;">
<tbody>
<tr>
<td>
<table border="0" cellpadding="0" cellspacing="0" bgcolor="#FFFFFF" style="border: solid 2px #ccc; background: #dadada; width: 100%; -moz-border-radius: 6px; -webkit-border-radius: 6px;">
<tbody>
<tr>
<td bgcolor="#000000" valign="middle" height="58px" style="border-bottom: 1px solid #ccc; padding: 20px; -moz-border-radius-topleft: 3px; -moz-border-radius-topright: 3px; -webkit-border-top-right-radius: 5px; -webkit-border-top-left-radius: 5px;">
<h1 style="color: #333333; font: bold 22px Arial, Helvetica, sans-serif; margin: 0; display: block !important;">
<!-- To have a header image/logo replace the name below with your img tag -->
<!-- Email clients will render the images when the message is read so any image -->
<!-- must be made available on a public server, so that all recipients can load the image. -->
<a href="http://community.jboss.org/index.jspa" style="text-decoration: none; color: #E1E1E1">JBoss Community</a></h1>
</td>
</tr>
<tr>
<td bgcolor="#FFFFFF" style="font: normal 12px Arial, Helvetica, sans-serif; color:#333333; padding: 20px; -moz-border-radius-bottomleft: 4px; -moz-border-radius-bottomright: 4px; -webkit-border-bottom-right-radius: 5px; -webkit-border-bottom-left-radius: 5px;"><h3 style="margin: 10px 0 5px; font-size: 17px; font-weight: normal;">
SuspectException seen by one node when the other node in the cluster goes down
</h3>
<span style="margin-bottom: 10px;">
created by <a href="http://community.jboss.org/people/aditi.andhare">Aditi Andhare</a> in <i>JBoss Cache</i> - <a href="http://community.jboss.org/message/570224#570224">View the full discussion</a>
</span>
<hr style="margin: 20px 0; border: none; background-color: #dadada; height: 1px;">
<div class="jive-rendered-content"><p>Hi all,</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>We are using the following configuration:</p><p>jGroups 3.2.0GA</p><p>jboss cache 3.2.0 GA</p><p>jboss AS 5.1.0 GA</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>I have two nodes in my clustered setup. Consider the case when there is continous load on both of these nodes. Now I stop one of the nodes in the cluster. The other node which is still active sees the SuspectException for a very few transactions hence affecting transactions performed by the active node. Here is the stack trace:</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>org.jboss.cache.SuspectException: Suspected member: 10.17.221.19:59378<br/>        at org.jboss.cache.RPCManagerImpl.callRemoteMethods(RPCManagerImpl.java:764)<br/>        at org.jboss.cache.RPCManagerImpl.callRemoteMethods(RPCManagerImpl.java:716)<br/>        at org.jboss.cache.RPCManagerImpl.callRemoteMethods(RPCManagerImpl.java:721)<br/>        at org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:161)<br/>        at org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:135)<br/>        at org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:107)<br/>        at org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:160)<br/>        at org.jboss.cache.interceptors.ReplicationInterceptor.visitPutDataMapCommand(ReplicationInterceptor.java:113)<br/>        at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)<br/>        at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)<br/>        at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)<br/>        at org.jboss.cache.commands.AbstractVisitor.visitPutDataMapCommand(AbstractVisitor.java:60)<br/>        at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)<br/>        at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)<br/>        at org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInterceptor.java:301)<br/>        at org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.java:283)<br/>        at org.jboss.cache.commands.AbstractVisitor.visitPutDataMapCommand(AbstractVisitor.java:60)<br/>        at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)<br/>        at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)<br/>        at org.jboss.cache.interceptors.CacheMgmtInterceptor.visitPutDataMapCommand(CacheMgmtInterceptor.java:97)<br/>        at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)<br/>        at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)<br/>        at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)<br/>        at org.jboss.cache.interceptors.InvocationContextInterceptor.visitPutDataMapCommand(InvocationContextInterceptor.java:64)<br/>        at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)<br/>        at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:287)<br/>        at org.jboss.cache.invocation.CacheInvocationDelegate.invokePut(CacheInvocationDelegate.java:705)<br/>        at org.jboss.cache.invocation.CacheInvocationDelegate.put(CacheInvocationDelegate.java:519)<br/>        at org.jboss.cache.invocation.NodeInvocationDelegate.addChild(NodeInvocationDelegate.java:337)<br/>        at com.openwave.servicebroker.util.CacheUtil.createNode(CacheUtil.java:204)<br/>        at com.openwave.servicebroker.ServiceBrokerImpl.serviceRequest(ServiceBrokerImpl.java:838)<br/>        at servicebroker.ServiceBroker$Processor$serviceRequest.process(ServiceBroker.java:809)<br/>        at servicebroker.ServiceBroker$Processor.process(ServiceBroker.java:626)<br/>        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)<br/>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)<br/>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)<br/>        at java.lang.Thread.run(Thread.java:619)</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>The code statement which causes the above exception is not in a transaction. Here is the FD configurations in the conf files:</p><p>         <FD_SOCK/><br/>         <FD max_tries="20" shun="false" timeout="60000"/><br/>         <VERIFY_SUSPECT timeout="1500"/></p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>Upon analysis of logs I see that this exception is always caused during the short time gap when one node is suspected and when the cluster recieves a new view.</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>2010-10-29 <strong>03:04:12,480</strong> INFO  [org.jboss.ha.framework.interfaces.HAPartition.lifecycle.ClusterOne] (VERIFY_SUSPECT.TimerThread,ClusterOne,10.17.221.18:48782) Suspected member: 10.17.221.19:34650<br/>2010-10-29 <strong>03:04:12,534</strong> INFO  [org.jboss.ha.framework.interfaces.HAPartition.lifecycle.ClusterOne] (Incoming-17,10.17.221.18:48782) New cluster view for partition ClusterOne (id: 4, delta: -1) : [10.17.221.18:1099]</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>So my question is:</p><p>1. Is this an expected behaviour?</p><p>2. Whenever a member in a cluster goes down, will all the active transactions seen by the other active members in the cluster fail due to the Suspect Exception?</p><p>3. Or are there any configuration settings that I am missing out here?</p><p style="min-height: 8pt; height: 8pt; padding: 0px;"> </p><p>Thanks in advance for all your help.</p><p>Aditi</p></div>
<div style="background-color: #f4f4f4; padding: 10px; margin-top: 20px;">
<p style="margin: 0;">Reply to this message by <a href="http://community.jboss.org/message/570224#570224">going to Community</a></p>
<p style="margin: 0;">Start a new discussion in JBoss Cache at <a href="http://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2052">Community</a></p>
</div></td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</div>
</body>
</html>