[infinispan-dev] Hot Rod - pt2

Wed Dec 16 08:49:12 EST 2009

On 12/16/2009 12:54 PM, Mircea Markus wrote:
>
> On Dec 16, 2009, at 9:26 AM, Galder Zamarreno wrote:
>
>>
>>
>> On 12/15/2009 01:13 PM, Manik Surtani wrote:
>>> A few comments:
>>>
>>> - Why do you have OpCode in your response header?  Surely this is redundant?  If the client is sync, it knows what it sent.  If it is async, it has a message ID.
>>
>> True, I fixed that.
>>
>>> - 'Not So Dumb' and 'Clever' response headers should be optional?  Surely this stuff is only sent when there is a topology change?  We also may need some extra info here - how does the back-end know to send this info?  If a client hits different back-end nodes, and there is a topology change, how does Node A decide that it should not bother with topology info since the client already hit Node B after the topo change and has the new topology map?  Perhaps a TopologyVersion (== JGroups View ID) should be sent back with any topo map, and the client would send it's current TopologyVersion with every request (non-dumb clients only)?  Could be a vlong...
>>
>> Yeah, as you and Mircea suggest, clients sending the view id could help
>> better manage size of responses.
>>
>> Something to note here is that not all cluster view changes necessarily
>> involve a client side topology view change, cos if you have N infinispan
>> nodes in the cluster, you could have M running hot rod server where M<=
>> N. So, Hot rod will need to figure out when there's been a view change
>> in the cluster that involves the addition or removal of a hot rod
>> running Infinispan instance (similar thing to what happens when a
>> clustered EJB is deployed, the target list for that clustered EJB gets
>> updated).
> yep, that's a good point. Every Hotrod server has to know exactly what other hotrod server are there running. I assume that because one way or the other each hotrod node will have to piggyback the list of active hotrod servers to the clients. Can't the hotrod server track the topology changes (topology of hot rod servers) and increase the view id number at that time only? Actually I think it can also rely on the jgroups view id, but only update the hotrod-view-id when jgroups cluster change overlaps with hotrod cluster change (i.e. a infinispan instance that has a hotrod serv started is added or removed).

Indeed, that's how I was planning to do it. You can still use the 
JGroups view id, but changing it only when a new hot rod server has 
joined or left. We'll need a remote command that registers/deregisters 
themselves cluster wide in this list (hot rod server subset cluster list).

>>
>> So, bearing in mind this, could we just use the JGroups view id for
>> this? AFAIK, it's 0 based long but shouldn't cause problems with
>> complete cluster restarts. If the whole cluster gets restarted, existing
>> connections will be closed and at that point, clients could revert back
>> to trying to connect one of their known hot rod servers and pass -1
> wouldn't they pass the old view Id at this time rather than -1? Guess the client will realize that the Socket is closed, and at that time will send an "I don't have view" request, i.e. -1?

Hmmmm, those two sentences seem to contradict each other. When there's a 
full cluster restart, and your sockets gets "closed by peer" and you 
can't contact any of the remaining servers, next time I think it's 
better if you send -1 rather than trying to send the old view. 
Otherwise, you might have issues like this:
- Client had vClient=2.
- Cluster restarts and after two hot rod cluster nodes started up, 
vServer is 2.
- Client figured out that the cluster restarted and restablishes the 
contact and sends vClient=2.
- The server does not send an updated view.

Now, this would be a problem for example if while the server was down, 
you had changed the TCP port where the hot rod server was listening, or 
there was some other change that might have affected the calculation of 
the cluster node hashcode.

If the client sends -1, you avoid this issues, it's like a local reset.

>> as
>> view id which means that I have no view, so the responding server would
>> send back the hot rod cluster view.
>>
>> I'll add this to the wiki.
>
>>
>>>
>>> Cheers
>>> Manik
>>>
>>>
>>> On 14 Dec 2009, at 20:08, Galder Zamarreno wrote:
>>>
>>>> Hi all,
>>>>
>>>> Re: http://community.jboss.org/wiki/HotRodProtocol
>>>>
>>>> I've updated the wiki with the following stuff:
>>>> - Renamed replaceIfEquals to replaceIfUnmodified
>>>> - Added remove and removeIfUnmodified.
>>>> - Added containsKey command.
>>>> - Added getWithCas command so that cas value can be returned. I decided
>>>> for a separate command rather than adding cas to get return because you
>>>> don't always want cas to be returned. Having a separate command makes
>>>> better use of network bandwith.
>>>> - Added stats command. JMX attributes are basically accessible through
>>>> this, including cache size.
>>>> - Added error handling section and updated status codes.
>>>>
>>>> Note that Mircea added some interesting comments and I replied to them
>>>> directly in the wiki.
>>>>
>>>> Still remaining to add:
>>>> - Commands: putForExternalRead evict, clear, version, name and quite
>>>> commands.
>>>> - Passing flags.
>>>>
>>>> Regards,
>>>>
>>>> p.s. Updating this has been quite a struggle due to F12 + FF 3.5.5
>>>> crashing at least 5 times, plus parts of the wiki dissapearing after
>>>> publishing them!
>>>> --
>>>> Galder Zamarreño
>>>> Sr. Software Engineer
>>>> Infinispan, JBoss Cache
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> --
>>> Manik Surtani
>>> manik at jboss.org
>>> Lead, Infinispan
>>> Lead, JBoss Cache
>>> http://www.infinispan.org
>>> http://www.jbosscache.org
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Galder Zamarreño
>> Sr. Software Engineer
>> Infinispan, JBoss Cache
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache