[infinispan-dev] Clustering standalone Infinispan w/ WF running Infinispan

Tue Sep 30 03:02:27 EDT 2014

I don't know what Kurt is doing, but Stelios is attempting to cluster an 
application using embedded Infinispan deployed within WF together with 
an Infinispan Server instance.
The application is managing its own caches, and therefore it is not 
interacting with the underlying Infinispan and JGroups subsystems in WF.
Infinispan Server uses its Infinispan and JGroups subsystems (which are 
forked from WF's) and therefore are using MuxChannels.

I told Stelios to use a MuxChannel-wrapped Channel in his application 
and it solved part of the issue (he was initially importing the one 
included in the WF's jgroups subsystem, but now he's using his local 
copy), but now he has run into further problems and I believe what Paul 
& Dennis have written might be correct.

The code that configures this is in 
EmbeddedCacheManagerConfigurationService:

GlobalConfigurationBuilder builder = new GlobalConfigurationBuilder();
ModuleLoader moduleLoader = this.dependencies.getModuleLoader();
builder.serialization().classResolver(ModularClassResolver.getInstance(moduleLoader));

I don't know how you'd get a ModuleLoader from within a WF deployment, 
but I'm sure it can be done.

Tristan

On 29/09/14 18:57, Paul Ferraro wrote:
> You should not need to use a MuxChannel.  This would only be necessary if there are other EAP services sharing the channel.  Using a MuxChannel allows your standalone Infinispan instance to filter these irrelevant messages.  However, in JDG, there should be no other services other than Infinispan using the channel - hence the MuxChannel stuff is unnecessary.
>
> I think Dennis earlier response was spot on.  EAP/JDG configures it's cache managers using a ModularClassResolver (which includes a module name along with the class name when marshalling).  Your standalone Infinispan instances do not use this and therefore cannot make sense of the message body.
>
> Paul
>
> ----- Original Message -----
>> From: "Kurt T Stam" <kurt.stam at jboss.com>
>> To: "Stelios Koussouris" <stkousso at redhat.com>, "Radoslav Husar" <rhusar at redhat.com>
>> Cc: "Galder Zamarreño" <galder at redhat.com>, "Paul Ferraro" <paul.ferraro at redhat.com>, "Richard Achmatowicz"
>> <rachmato at redhat.com>, "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>> Sent: Monday, September 29, 2014 11:39:59 AM
>> Subject: Re: Clustering standalone Infinispan w/ WF running Infinispan
>>
>> Thanks for following up Stelios, I think Galder is traveling the next 2
>> weeks.
>>
>> So - do we need fixes on both ends then so that the boot order does not
>> matter? In which project(s) would we apply
>> there changes? Or can they be applied in the end-user's code?
>>
>> Thx,
>>
>> --Kurt
>>
>>
>>
>> On 9/26/14, 11:19 AM, Stelios Koussouris wrote:
>>> Hi,
>>>
>>> Rado: It is both ways. ie. if I start first the JDG Server I get the issue
>>> on the library mode side when I start that one. If reverse the order of
>>> startup I get it in the JDG Server side.
>>>
>>> Question:
>>> -----------------------------------------------------------------------------------------------------------------------
>>> ...IMO the channel needs to be wrapped as
>>> org.jboss.as.clustering.jgroups.MuxChannel before passing to infinispan.
>>> ...
>>> -----------------------------------------------------------------------------------------------------------------------
>>> For now that this is not being done. If I wanted to do it manually on the
>>> library side where I can create the protocol programmatically we are
>>> talking about something like this?
>>>
>>> ProtocolStackConfigurator configurator =
>>> ConfiguratorFactory.getStackConfigurator("jgroups-udp.xml");
>>> MuxChannel channel = new MuxChannel(configurator);
>>> org.infinispan.remoting.transport.Transport transport = new
>>> org.infinispan.remoting.transport.jgroups.JGroupsTransport(channel);
>>>
>>> ....
>>> then replace the below
>>> new
>>> GlobalConfigurationBuilder().clusteredDefault().globalJmxStatistics().cacheManagerName("RDSCacheManager").allowDuplicateDomains(true).enable().transport().clusterName("UDM-CLUSTER").addProperty("configurationFile",
>>> "jgroups-udp.xml")
>>>    
>>> WITH
>>> new
>>> GlobalConfigurationBuilder().clusteredDefault().globalJmxStatistics().cacheManagerName("RDSCacheManager").allowDuplicateDomains(true).enable().transport(Transport).clusterName("UDM-CLUSTER")
>>>
>>> Btw, someone mentioned that if I follow this method I need to to know the
>>> assigned mux ids, but that is not quite clear what it means with regards
>>> to the JGroupsTransport configuration
>>>
>>> Thanks,
>>>
>>> Stylianos Kousouris
>>> Red Hat Middleware Consultant
>>>
>>> ----- Original Message -----
>>> From: "Radoslav Husar" <rhusar at redhat.com>
>>> To: "Galder Zamarreño" <galder at redhat.com>, "Paul Ferraro"
>>> <paul.ferraro at redhat.com>
>>> Cc: "Richard Achmatowicz" <rachmato at redhat.com>, "infinispan -Dev List"
>>> <infinispan-dev at lists.jboss.org>, "Stelios Koussouris"
>>> <stkousso at redhat.com>, "Kurt T Stam" <kurt.stam at jboss.com>
>>> Sent: Friday, 26 September, 2014 3:47:16 PM
>>> Subject: Re: Clustering standalone Infinispan w/ WF running Infinispan
>>>
>>>    From what Stelios is telling me the question is a little bit other way
>>> round: he is using library mode infinispan and jgroups in EAP and
>>> connecting to JDG. So the question is what JDG is doing with the stack,
>>> not AS/WF as its infinispan/jgroups subsystem is not used.
>>>
>>> Unfortunately I don't have access to the JDG repo so I don't know what
>>> changes have been made there but if you are using the same jgroups
>>> logic, IMO the channel needs to be wrapped as
>>> org.jboss.as.clustering.jgroups.MuxChannel before passing to infinispan.
>>>
>>> Rado
>>>
>>> On 26/09/14 15:03, Galder Zamarreño wrote:
>>>> Hey Paul,
>>>>
>>>> In the last couple of days, a couple of people have encountered the
>>>> exception in [1] when trying to cluster a standalone Infinispan app with
>>>> its own JGroups configuration file with a AS/WF running Infinispan cache.
>>>>
>>>>    From my POV, 3 possible causes:
>>>>
>>>> 1. Dependency mismatches between AS/WF and the standalone app. Having done
>>>> some quick study of Kurt’s case, apart from micro version changes, all
>>>> looks good.
>>>>
>>>> 2. Mismatch in the Infinispan and/or JGroups configuration file.
>>>>
>>>> 3. AS/WF puts something on the clustered wire that standalone Infinispan
>>>> does not expect. Are you still doing multiplexing? Could you be adding
>>>> extra info to the wire?
>>>>
>>>> With this email, I’m trying to get some clarification from you if the
>>>> issue could be due to 3rd option. If it’s either of the first two, it’s a
>>>> matter of digging and finding the difference, but if it’s 3rd one, it’s
>>>> more problematic.
>>>>
>>>> Any ideas?
>>>>
>>>> [1] https://gist.github.com/skoussou/92f062f2d0bd17168e01
>>>> --
>>>> Galder Zamarreño
>>>> galder at redhat.com
>>>> twitter.com/galderz
>>>>
>>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev