[jboss-cluster-dev] Consistent Hashing in mod_cluster

Tue Apr 13 05:43:49 EDT 2010

On 9 Apr 2010, at 16:00, Brian Stansberry wrote:

> I'm putting this on the cluster-dev list as there's no reason for it to 
> continue in private.
> 
> Manik, I agree with your comment about avoiding tight coupling between 
> the ISPN hash algorithm and something inside mod_cluster. The approach 
> we decided to take was to change the jvmRoute -- see 
> https://jira.jboss.org/jira/browse/JBAS-7853. The Mobicents folks are 
> already familiar with that approach since the AS has long done that on 
> failover. They've added a browser redirect technique that would even 
> avoid the "1 'inefficient' dispatch" issue you noted.
> 
> What's different about what Vladimir's raising here is the need to try 
> to group sessions together; i.e. use something other than the cache key 
> (i.e. jsessionid) as the factor in the hash algorithm. He mentioned a 
> use case for multiple users in a chat where the goal would be to cache 
> sessions locally with chat state. AIUI there's also a use case where a 
> user has a SIP session and an HTTP session, so the goal would be to have 
> both session cached together.
> 
> Doing that will clearly require quite a bit of AS work -- some pluggable 
> policy for determining what the "affinity key" is based on the call 
> context (null == use the cache key for hashing). Then use the 
> affinityKey can calling into ISPN (i.e. ISPN-359) and when determining 
> where to fail over to (JBAS-7853).

I presume you're following the ISPN-359 thread on infinispan-dev... 

> This will also be useful for the HTTPSession + SFSB in same request use 
> case, where colocating the SFSB and the web session is optimal.

Right.

> I hadn't opened a JIRA for that yet; just tried to but JIRA seems to not 
> be working?
> 
> Hmm, an issue is if the caches involved in the overall picture (e.g. 
> httpsession cache and "chat state" cache) end up using different 
> CacheManagers/JGroups Channels. This would quite likely happen (see 
> http://community.jboss.org/thread/149868 for discussion why). If that's 
> the case, the views between the channels could end up slightly different 
> -- e.g. servers A and B are booting at the same time, so due to slight 
> differences in the boot, Channel for HTTPSession cache ends up with view 
> {A, B} and Channel for "chat state" cache ends up with {B, A}. In that 
> case I wouldn't expect there to be a reliable, performant way to ensure 
> affinity, since the view is a major factor in the hashing algorithm.

Yes, it would not be useful at all if the 2 components end up with different views.  It would only make sense if the web session cache and the SFSB session cache share the same CacheManager/Channel.

> 
> On 04/09/2010 04:53 AM, Manik Surtani wrote:
>> I made a few comments on the JIRA to respond to your comment; Brian, correct me if I have missed something, but this is pretty much what we spoke about in Brno?  Do you have a JIRA in JBAS for this as well?
>> 
>> 
>> On 8 Apr 2010, at 19:39, Vladimir Ralev wrote:
>> 
>>> Thanks, this will work great for us. However, this issue seems to be scoped purely in Infinispan https://jira.jboss.org/jira/browse/ISPN-359
>>> 
>>> A reciprocal feature is needed in mod_cluster. Would it be possible mod_cluster to use a URI parameter, a cookie as affinity key. AFAIK from mod_jk, it only supports using the jsessionid cookie and uri param (hardcoded). Ultimately we'd like to have it use a customer GET param or cookie for affinity. I couldnt find any doc related to that. Would this be possible?
>>> 
>>> ----- Original Message -----
>>> From: "Manik Surtani"<msurtani at redhat.com>
>>> To: "Brian Stansberry"<brian.stansberry at redhat.com>
>>> Cc: "Vladimir Ralev"<vralev at redhat.com>, "Brian Stansberry"<bstansbe at redhat.com>, "Jean Deruelle"<jderuell at redhat.com>, "Paul Ferraro"<paul.ferraro at redhat.com>
>>> Sent: Thursday, April 8, 2010 1:43:00 PM GMT +02:00 Athens, Bucharest, Istanbul
>>> Subject: Re: Consistent Hashing in mod_cluster
>>> 
>>> 
>>> On 6 Apr 2010, at 20:02, Brian Stansberry wrote:
>>> 
>>>> Hi Vladimir,
>>>> 
>>>> I've been on vacation. I replied to your question on the wiki page, but what you're talking about here is a bigger picture thing.
>>>> 
>>>> Sounds like your situation is a good use case for https://jira.jboss.org/jira/browse/ISPN-359 which is an Infinispan JIRA. (At least I believe that's the JIRA that gets into allowing the caller to provide hints such that separate keys hash to the same storage nodes -- Manik, please correct me if I'm wrong.)
>>> 
>>> Yup, this is correct.
>>> 
>>>> 
>>>> On 03/31/2010 01:58 PM, Vladimir Ralev wrote:
>>>>> Hello Brian,
>>>>> 
>>>>> Not sure if you are following http://community.jboss.org/wiki/Consistent-hashingbasedJBCdatapartitioningwmodcluster so decided to email you.
>>>>> 
>>>>> We are very interested in the consistent hashing feature in the load balancer working cooperatively with the cache and would like to follow the development and contribute if needed. Particularly, we in Moicents already have a number of use-cases that concern bothtelecom and pure web applications. For example - a web based chat room will have a number of participants with different HTTP sessions and they share the same state somewhere in another cache at the same node. It would be desireable to failover such groups of users together to avoid having them to lookup the chat state in remote caches over the network. That is why, if possible, we are interested in using not just the HTTP session ID for lookups, but have a configurable affinity key/mask that is taken from the messages, so that you could route all requests with http://redhat.com/?chatroom=1 always to the same node, and "chatroom" as an URL param is the affinity key with value "1".
>>>>> 
>>>>> Sailfin/Glassfish have something similar with their CLB.
>>>>> 
>>>>> In Mobicents we would be using it to coordinate not only mod_cluster and cache, but the SIP load balancer and other protocol load balancers as well. In fact, our load balancer already supports HTTP consistent hashing, however it doesn't have the high performance of mod_cluster (no AJP either) and is only recommended when SIP+HTTP coordination is a requirement for our telco users.
>>>>> 
>>>>> http://docs.google.com/present/view?id=dc5jp5vx_89cxdvtxcm (Slide 17 has a helpful diagram)
>>>>> 
>>>>> The other technique we are using to coordinate multiprotocol load balancers with local cache is this
>>>>> 
>>>>> http://docs.google.com/present/edit?id=0AdgrvwaZgnJ1ZGM1anA1dnhfOTZnZjYyc2ZwOQ&hl=en
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> Brian Stansberry
>>>> Lead, AS Clustering
>>>> JBoss by Red Hat
>>> 
>> 
> 
> 
> -- 
> Brian Stansberry
> Lead, AS Clustering
> JBoss by Red Hat
> _______________________________________________
> jboss-cluster-dev mailing list
> jboss-cluster-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-cluster-dev

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/jboss-cluster-dev/attachments/20100413/fcce76a1/attachment-0001.html 