[Clustering/JBoss] - clustering/JMS problem among machines not using same partiti - jboss-user

Thursday, 5 February 2009

Hello, all.  We are using jboss-4.2.3.GA on some machines with CentOS 4.4 and some with
CentOS 5.  For development purposes we do not like to cluster so we all change our
mcast_port in cluster-service.xml to something unique.  That was fine for a time.  Then we
realized that even though JBoss was not clustering, JMS was.  I read
http://jboss.org/community/docs/DOC-12460 and consequently started using unique partition
names and jboss.partition.udpGroup addresses.  This LOOKS like it is working, but does
not.

Here is the start of my JBoss log on a machine called "cars" at 150.102.65.10:

  | ================================================================================
  | JBoss Bootstrap Environment
  | 
  | JBOSS_HOME: /usr/jboss-4.2.3.GA
  | 
  | JAVA:
  | 
  | JAVA_OPTS:  -server -Xmx512m -XX:PermSize=96m -XX:MaxPermSize=128m
-Dprogram.name=jboss
  | 
  | CLASSPATH:  /usr/jboss-4.2.3.GA/bin/run.jar:/usr/java/jdk1.5.0_02/lib/tools.jar
  | 
  | CMD_START:  /usr/java/jdk1.5.0_02/bin/java -server -Xmx512m -XX:PermSize=96m
-XX:MaxPermSize=128m -Dprogram.name=jboss -classpath
/usr/jboss-4.2.3.GA/bin/run.jar:/usr/java/jdk1.5.0_02/lib/tools.jar org.jboss.Main -c
appServer -b cars -Djava.security.manager=java.lang.SecurityManager
-Djava.security.policy=/usr/jboss-4.2.3.GA/server/appServer/conf/server.policy
-Djava.rmi.server.codebase="file:/usr/jboss-4.2.3.GA/server/appServer/lib/KDCSservices.jar
file:/usr/jboss-4.2.3.GA/server/appServer/lib/kdcsclient.jar"
-Djava.rmi.server.hostname=cars -Dkdcs.hostname=cars
-Djboss.partition.name=workstation_32_development_Rochester
  |  -Djboss.partition.udpGroup=228.1.2.10 -Djboss.platform.mbeanserver
-Djava.endorsed.dirs=/usr/jboss-4.2.3.GA/lib/endorsed
  | ================================================================================
  | 
  | 10:11:13,954 INFO  [JChannel] JGroups version: 2.4.1 SP-4
  | 10:11:14,661 INFO  [STDOUT]
  | -------------------------------------------------------
  | GMS: address is 150.102.65.10:52299
  | -------------------------------------------------------
  | 10:11:16,684 INFO  [TreeCache] viewAccepted(): [150.102.65.10:52299|0]
[150.102.65.10:52299]
  | 10:11:16,709 INFO  [TreeCache] TreeCache local address is 150.102.65.10:52299
  | 10:11:16,709 INFO  [TreeCache] State could not be retrieved (we are the first member
in group)
  | 10:11:16,709 INFO  [TreeCache] parseConfig(): PojoCacheConfig is empty
  | 10:11:20,919 INFO  [NativeServerConfig] JBoss Web Services - Native
  | 10:11:20,919 INFO  [NativeServerConfig] jbossws-3.0.1-native-2.0.4.GA
(build=200803312044)
  | 10:11:22,094 INFO  [SnmpAgentService] SNMP agent going active
  | 10:11:22,610 INFO  [JChannel] JGroups version: 2.4.1 SP-4
  | 10:11:22,759 INFO  [workstation_32_development_Rochester] Initializing
  | 10:11:22,809 INFO  [STDOUT]
  | -------------------------------------------------------
  | GMS: address is 150.102.65.10:52303
  | -------------------------------------------------------
  | 10:11:24,817 INFO  [workstation_32_development_Rochester] Number of cluster members:
1
  | 10:11:24,818 INFO  [workstation_32_development_Rochester] Other members: 0
  | 10:11:24,818 INFO  [workstation_32_development_Rochester] Fetching state (will wait
for 30000 milliseconds):
  | 10:11:24,818 INFO  [workstation_32_development_Rochester] State could not be retrieved
(we are the first member in group)
  | 10:11:24,862 INFO  [HANamingService] Started ha-jndi bootstrap jnpPort=1100,
backlog=50, bindAddress=cars/150.102.65.10
  | 10:11:24,868 INFO  [DetachedHANamingService$AutomaticDiscovery] Listening on
cars/150.102.65.10:1102, group=228.1.2.10, HA-JNDI address=150.102.65.10:1100
  | 10:11:26,171 INFO  [TreeCache] No transaction manager lookup class has been defined.
Transactions cannot be used
  | 10:11:26,336 INFO  [JChannel] JGroups version: 2.4.1 SP-4
  | 10:11:26,481 INFO  [STDOUT]
  | -------------------------------------------------------
  | GMS: address is 150.102.65.10:52309
  | -------------------------------------------------------
  | 10:11:28,485 INFO  [TreeCache] viewAccepted(): [150.102.65.10:52309|0]
[150.102.65.10:52309]
  | 10:11:28,486 INFO  [TreeCache] TreeCache local address is 150.102.65.10:52309
  | 10:11:28,640 INFO  [JChannel] JGroups version: 2.4.1 SP-4
  | 10:11:28,790 INFO  [STDOUT]
  | -------------------------------------------------------
  | GMS: address is 150.102.65.10:52312
  | -------------------------------------------------------
  | 10:11:30,794 INFO  [TreeCache] viewAccepted(): [150.102.65.10:52312|0]
[150.102.65.10:52312]
  | 10:11:30,796 INFO  [TreeCache] TreeCache local address is 150.102.65.10:52312
  | 

Looks great, right?  cars is the only machine in the cluster with the partition name
workstation_32_development_Rochester.  Before when we all were using DefaultPartition, and
only had unique mcast_ports, many other ip addresses showed up in the list following
"[TreeCache] viewAccepted():"

Then, for unknown reasons it tries to access another machine called "wizard":

  | 10:12:52,894 INFO  [QuartzJob] startSingleton(): Start QuartzJob Singleton
  | 10:12:57,500 ERROR [QuartzJob] *********************Exception
Thrown*********************
  | 10:12:57,501 ERROR [QuartzJob] createJobAndTrigger(): null
  | 10:12:57,501 ERROR [QuartzJob] *********************End of
Exception*********************
  | 10:12:57,501 ERROR [STDERR] javax.naming.CommunicationException [Root exception is
java.rmi.RemoteException: Service unavailable.; nested exception is:
  |         java.rmi.ConnectIOException: Exception creating connection to: wizard; nested
exception is:
  |         java.net.NoRouteToHostException: No route to host]
  | 10:12:57,502 ERROR [STDERR]     at
org.jnp.interfaces.NamingContext.lookup(NamingContext.java:780)
  | 10:12:57,502 ERROR [STDERR]     at
org.jnp.interfaces.NamingContext.lookup(NamingContext.java:627)
  | 10:12:57,502 ERROR [STDERR]     at
javax.naming.InitialContext.lookup(InitialContext.java:351)
  | 10:12:57,502 ERROR [STDERR]     at
kdcs.timers.QuartzJob.jobInvoiceHandlerJob(QuartzJob.java:797)
  | 10:12:57,502 ERROR [STDERR]     at
kdcs.timers.QuartzJob.createJobAndTrigger(QuartzJob.java:849)
  | 10:12:57,502 ERROR [STDERR]     at
kdcs.timers.QuartzJob.startSingleton(QuartzJob.java:390)
  | 10:12:57,502 ERROR [STDERR]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
  | 10:12:57,502 ERROR [STDERR]     at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  | 10:12:57,503 ERROR [STDERR]     at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 10:12:57,503 ERROR [STDERR]     at java.lang.reflect.Method.invoke(Method.java:585)
  | 10:12:57,503 ERROR [STDERR]     at
org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
  | 10:12:57,503 ERROR [STDERR]     at
org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
  | 10:12:57,503 ERROR [STDERR]     at
org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
  | 10:12:57,503 ERROR [STDERR]     at
org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
  | 10:12:57,503 ERROR [STDERR]     at
org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
  | 10:12:57,503 ERROR [STDERR]     at
org.jboss.ha.singleton.HASingletonController.invokeSingletonMBeanMethod(HASingletonController.java:207)
  | 10:12:57,503 ERROR [STDERR]     at
org.jboss.ha.singleton.HASingletonController.startSingleton(HASingletonController.java:144)
  | 10:12:57,503 ERROR [STDERR]     at
org.jboss.ha.singleton.HASingletonSupport.startNewMaster(HASingletonSupport.java:272)
  | 10:12:57,503 ERROR [STDERR]     at
org.jboss.ha.singleton.HASingletonSupport.makeThisNodeMaster(HASingletonSupport.java:254)
  | 10:12:57,503 ERROR [STDERR]     at
org.jboss.ha.singleton.HASingletonSupport.partitionTopologyChanged(HASingletonSupport.java:196)
  | 10:12:57,503 ERROR [STDERR]     at
org.jboss.ha.jmx.HAServiceMBeanSupport$1.replicantsChanged(HAServiceMBeanSupport.java:247)
  | 10:12:57,503 ERROR [STDERR]     at
org.jboss.ha.framework.server.DistributedReplicantManagerImpl.notifyKeyListeners(DistributedReplicantManagerImpl.java:846)
  | 10:12:57,504 ERROR [STDERR]     at
org.jboss.ha.framework.server.DistributedReplicantManagerImpl.add(DistributedReplicantManagerImpl.java:409)
  | 10:12:57,504 ERROR [STDERR]     at
org.jboss.ha.jmx.HAServiceMBeanSupport.registerDRMListener(HAServiceMBeanSupport.java:255)
  | 10:12:57,504 ERROR [STDERR]     at
org.jboss.ha.jmx.HAServiceMBeanSupport.startService(HAServiceMBeanSupport.java:177)
  | 10:12:57,504 ERROR [STDERR]     at
org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:289)
  | 10:12:57,504 ERROR [STDERR]     at
org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMBeanSupport.java:245)
  | 10:12:57,504 ERROR [STDERR]     at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown
Source)
  | 10:12:57,504 ERROR [STDERR]     at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  | 10:12:57,504 ERROR [STDERR]     at java.lang.reflect.Method.invoke(Method.java:585)
  | 10:12:57,504 ERROR [STDERR]     at
org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
  | 10:12:57,504 ERROR [STDERR]     at
org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
  | 10:12:57,504 ERROR [STDERR]     at
org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
  | 10:12:57,504 ERROR [STDERR]     at
org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
  | 

wizard used to be in cars' /etc/hosts/ file, but I commented it out, so it makes sense
that it cannot create a connection.

What I do not understand is, why is it trying to connect to wizard?????

I have searched and searched again and there is no reference to wizard in any config files
nor is cars still trying to use DefaultPartition!

Any help would be GREATLY appreciated!!!

Thanks,
Josephine

View the original post :
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4207389#...

Reply to the post :
http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

[Clustering/JBoss] - clustering/JMS problem among machines not using same partiti