[infinispan-dev] Fwd: Backward compatibility and rolling upgrades

[Dan Berindei]

Forgot to send to the list :)


---------- Forwarded message ----------
From: Dan Berindei <dan.berindei at gmail.com>
Date: Tue, May 15, 2012 at 7:41 PM
Subject: Re: Backward compatibility and rolling upgrades
To: Manik Surtani <manik at jboss.org>


On Tue, May 15, 2012 at 4:31 PM, Manik Surtani <manik at jboss.org> wrote:
>
> On 15 May 2012, at 10:53, Sanne Grinovero wrote:
>
>> Since I've been long advocating this (thanks Tristan to point it out
>> first) I appreciate the strong push from the project lead.
>> Still as you say in the first half of the mail this wasn't supposed to
>> work in previous releases, so why are your 3 points in the end of the
>> mail asking specific details to migrate from 4.2 ?
>
> Because we (Red Hat services) would need to provide a ramp-up to getting
people already using 4.2 onto JDG (based on 5.1) without any downtime.
 Granted, Red Hat services will charge for this, we still need to be able
to help them out with building such a tool.
>
>> It's too late to bridge for existing versions, you can't even patch
>> them as people running them are already running them.. unless you
>> intend to make intermediate releases, such as a "4.2/5.1" specific
>> release.
>
> I was thinking perhaps a 5.1.Compat release which can parse and
participate in a 4.2-style state transfer, etc.  It would be a case of then
upgrading half the cluster to 5.1.Compat, then the other half to full 5.1,
and then the first half again to full 5.1.
>

Manik, I doubt that stock JGroups 3.0.x is binary compatible with
JGroups 2.12.x, but we could perhaps copy the protocols from 2.12 to
create a 3.0.compat version. When the entire cluster is running
5.1.compat, we could close the compat-mode channels all at once and
start new JGroups 3.0 channels. We'd need some kind of "disable state
transfer" switch, but it sounds doable.

OTOH, 4.2 state transfer requires partial state transfer support from
JGroups (at least in replicated mode). I'm not sure if Bela can hack
partial state transfer back into 3.0 to support state transfer from
4.2 -> 5.1 in replicated mode. For distributed caches I think it may
be easier, the 4.2 rehashing code should still work with JGroups 3.0.

Even if we do get state transfer working, the nodes then have to
interact: 5.1 nodes have to respond to 4.2 commands, and 4.2 nodes
have to respond to 5.1 commands. I'm pretty sure we have some
incompatibilities, with all the optimization rounds that we had going
on...

>> Did you see the issues we opened back in Lisbon on JIRA? they are
>> supposed to make it possible to perform a rolling upgrade, but need to
>> be included in the project *before* as I had already pointed out, to
>> make it possible to migrate to future versions.
>>
>> All issues are still open, apparently not considered important enough
>> :-( see https://issues.jboss.org/browse/ISPN-1410 and all it's
>> dependencies.
>
> I saw this, but I think there are a few bits missing.  From your umbrella
JIRA:
>
> ISPN-1409: why is this necessary?  CacheLoaders use
VersionAwareMarshallers which create versioned streams as it is.  So from a
serialisation perspective, we should be protected.
>

Looking at the code of VersionAwareMarshaller, it reads the version id
in startObjectInput but it doesn't do anything with it...
Also, how do you register an Externalizer to handle a particular version?

> ISPN-1406: Not sure I understand the purpose of this.  If we're writing
through to a cache store then this happens anyway.
>

Assuming VersionAwareMarshaller does pick the right Externalizer based
on the version id...

>> As I see it, even JDG customers won't be able to migrate to newer
>> versions since this wasn't tested and for sure there will be more
>> changes needed: I certainly won't presume our theoretical draft will
>> work flawlessly in practice.
>
> Yes, this definitely needs work, and more than that, as you say, some
real-world migration tests.
>
>> Also the thread we opened on Flags is relevant (active even today)
>
> +1.
>
> But I'm still very curious about the 3 points I raised in my original
email.  :)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20120515/fbe92d5d/attachment.html