9:50 p.m.
JMS feature does not work after cluster failover
------------------------------------------------
Key: JBSEAM-4212
URL: https://jira.jboss.org/jira/browse/JBSEAM-4212
Project: Seam
Issue Type: Bug
Components: Remoting
Affects Versions: 2.0.2.SP1
Environment: JBoss 4.2.3.GA (clustered)
Reporter: Scott McNab
Assignee: Shane Bryzak
We have configured Seam to work in a clustered environment by using a custom
JMSConnectionProvider that fetches the UIL2ConnectionFactory from HA-JNDI. This
arrangements works perfectly fine while both servers are up - i.e. a web client connected
to the slave server will correctly receive Asynchronous Seam notifications from a JMS
topic hosted on the master server.
However, when the master server fails over to the slave, the Seam JMS support on the slave
stops working. We get the following exception:
06-02 12:52:45.405 ERROR [seam.remoting.messaging.SubscriptionRegistry]
(http-192.168.100.140-80-2:) org.jboss.mq.SpyJMSException: Cannot get the Topic from the
provider; - nested throwable: (java.io.IOE
xception: Client is not connected)
06-02 12:52:45.406 ERROR [jboss.seam.remoting.Remoting] (http-192.168.100.140-80-2:)
Error
java.lang.NullPointerException
at
org.jboss.seam.remoting.messaging.SubscriptionRequest.marshal(SubscriptionRequest.java:31)
at
org.jboss.seam.remoting.SubscriptionHandler.marshalResponse(SubscriptionHandler.java:107)
at
org.jboss.seam.remoting.SubscriptionHandler.handle(SubscriptionHandler.java:96)
at org.jboss.seam.remoting.Remoting.getResource(Remoting.java:111)
at org.jboss.seam.servlet.SeamResourceServlet.doGet(SeamResourceServlet.java:75)
at org.jboss.seam.servlet.SeamResourceServlet.doPost(SeamResourceServlet.java:92)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:83)
at org.jboss.seam.debug.hot.HotDeployFilter.doFilter(HotDeployFilter.java:68)
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
at org.jboss.seam.web.LoggingFilter.doFilter(LoggingFilter.java:58)
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
at org.jboss.seam.web.MultipartFilter.doFilter(MultipartFilter.java:85)
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
at org.jboss.seam.web.ExceptionFilter.doFilter(ExceptionFilter.java:64)
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
at org.jboss.seam.web.RedirectFilter.doFilter(RedirectFilter.java:45)
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
at org.jboss.seam.servlet.SeamFilter.doFilter(SeamFilter.java:158)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
at
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
at
org.jboss.web.tomcat.service.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:97)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:432)
at
org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
at
org.jboss.web.tomcat.service.sso.ClusteredSingleSignOn.invoke(ClusteredSingleSignOn.java:677)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
at java.lang.Thread.run(Thread.java:595)
We have diagnosed this issue and have developed a fix. The problem is as follows:
- once failover occurs, the topicConnection field in the SubscriptionRegistry object
becomes invalid, as the connection to the master server has died.
- every subsequent call to SubscriptionRegistry.subscribe() will fail with
SpyJMSException: Cannot get the Topic from the provider; Client is not connected.
- even if the master server recovers, this connection is dead and all Seam JMS
subscriptions will no longer work.
The fix is to catch the Exception and retry with a new topicConnection if a subscribe
request fails. The following patch on
jboss-seam-2.0.2.SP1/src/remoting/org/jboss/seam/remoting/messaging/SubscriptionRegistry.java
works for us:
Index: SubscriptionRegistry.java
===================================================================
--- SubscriptionRegistry.java (revision 5528)
+++ SubscriptionRegistry.java (working copy)
@@ -139,7 +139,21 @@
RemoteSubscriber sub = new RemoteSubscriber(UUID.randomUUID().toString(),
topicName);
try {
- sub.subscribe(getTopicConnection());
+ try {
+ sub.subscribe(getTopicConnection());
+ } catch (Exception e) {
+ // If we failed during the first attempt, we might have a dead JMS
connection.
+ // Clear the topic connection and try again.
+ if (topicConnection != null) {
+ try {
+ topicConnection.close();
+ } catch (Exception e1) {
+ // Ignore any exception during the close, since this
connection is probably dead anyway
+ }
+ }
+ topicConnection = null;
+ sub.subscribe(getTopicConnection());
+ }
subscriptions.put(sub.getToken(), sub);
// Save the client's token in their session context
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira