6:43 p.m.
Forgot to send to the list :)
---------- Forwarded message ----------
From: Dan Berindei <dan.berindei(a)gmail.com Date:
Tue, May 15, 2012 at 7:41 PM
Subject: Re: Backward compatibility and rolling upgrades
To: Manik Surtani <manik(a)jboss.org
On Tue, May 15, 2012 at 4:31 PM, Manik Surtani <manik(a)jboss.org> wrote:
Manik, I doubt that stock JGroups 3.0.x is binary compatible with
JGroups 2.12.x, but we could perhaps copy the protocols from 2.12 to
create a 3.0.compat version. When the entire cluster is running
5.1.compat, we could close the compat-mode channels all at once and
start new JGroups 3.0 channels. We'd need some kind of "disable state
transfer" switch, but it sounds doable.
OTOH, 4.2 state transfer requires partial state transfer support from
JGroups (at least in replicated mode). I'm not sure if Bela can hack
partial state transfer back into 3.0 to support state transfer from
4.2 -> 5.1 in replicated mode. For distributed caches I think it may
be easier, the 4.2 rehashing code should still work with JGroups 3.0.
Even if we do get state transfer working, the nodes then have to
interact: 5.1 nodes have to respond to 4.2 commands, and 4.2 nodes
have to respond to 5.1 commands. I'm pretty sure we have some
incompatibilities, with all the optimization rounds that we had going
on...
>> Did you see the issues we opened back in Lisbon on JIRA? they are
>> supposed to make it possible to perform a rolling upgrade, but need to
>> be included in the project *before* as I had already pointed out, to
>> make it possible to migrate to future versions.
> >> All issues are still open, apparently not considered
important enough
>> :-( see https://issues.jboss.org/browse/ISPN-1410 and all it's
>> dependencies.
> I saw this, but I think there are a few bits missing.
From your umbrella
JIRA:
> ISPN-1409: why is this necessary? CacheLoaders use
VersionAwareMarshallers which create versioned streams as it is. So from a
serialisation perspective, we should be protected.
Looking at the code of VersionAwareMarshaller, it reads the version id
in startObjectInput but it doesn't do anything with it...
Also, how do you register an Externalizer to handle a particular version?
Assuming VersionAwareMarshaller does pick the right Externalizer based
on the version id...
>> As I see it, even JDG customers won't be able to migrate to newer
>> versions since this wasn't tested and for sure there will be more
>> changes needed: I certainly won't presume our theoretical draft will
>> work flawlessly in practice.
> Yes, this definitely needs work, and more than that, as
you say, some
real-world migration tests.
>> Also the thread we opened on Flags is relevant (active
even today)
> +1.
> But I'm still very curious about the 3 points I raised
in my original
email. :)
On 15 May 2012, at 10:53, Sanne Grinovero wrote:
> Since I've been long advocating this (thanks Tristan to point it out
> first) I appreciate the strong push from the project lead.
> Still as you say in the first half of the mail this wasn't supposed to
> work in previous releases, so why are your 3 points in the end of the
> mail asking specific details to migrate from 4.2 ?
Because we (Red Hat services) would need to provide a ramp-up to getting
people
already using 4.2 onto JDG (based on 5.1) without any downtime.
Granted, Red Hat services will charge for this, we still need to be able
to help them out with building such a tool.
> It's too late to bridge for existing versions, you can't even patch
> them as people running them are already running them.. unless you
> intend to make intermediate releases, such as a "4.2/5.1" specific
> release.
I was thinking perhaps a 5.1.Compat release which can parse and
participate in a
4.2-style state transfer, etc. It would be a case of then
upgrading half the cluster to 5.1.Compat, then the other half to full 5.1,
and then the first half again to full 5.1.
ISPN-1406: Not sure I understand the purpose of this. If we're
writing
through to a cache store then this happens anyway.
12:02 p.m.
New subject: Backward compatibility and rolling upgrades
On May 15, 2012, at 6:43 PM, Dan Berindei wrote:
Forgot to send to the list :)
---------- Forwarded message ----------
From: Dan Berindei <dan.berindei(a)gmail.com>
Date: Tue, May 15, 2012 at 7:41 PM
Subject: Re: Backward compatibility and rolling upgrades
To: Manik Surtani <manik(a)jboss.org>
On Tue, May 15, 2012 at 4:31 PM, Manik Surtani <manik(a)jboss.org> wrote:
>
> On 15 May 2012, at 10:53, Sanne Grinovero wrote:
>
>> Since I've been long advocating this (thanks Tristan to point it out
>> first) I appreciate the strong push from the project lead.
>> Still as you say in the first half of the mail this wasn't supposed to
>> work in previous releases, so why are your 3 points in the end of the
>> mail asking specific details to migrate from 4.2 ?
>
> Because we (Red Hat services) would need to provide a ramp-up to getting people
already using 4.2 onto JDG (based on 5.1) without any downtime. Granted, Red Hat services
will charge for this, we still need to be able to help them out with building such a
tool.
>
>> It's too late to bridge for existing versions, you can't even patch
>> them as people running them are already running them.. unless you
>> intend to make intermediate releases, such as a "4.2/5.1" specific
>> release.
>
> I was thinking perhaps a 5.1.Compat release which can parse and participate in a
4.2-style state transfer, etc. It would be a case of then upgrading half the cluster to
5.1.Compat, then the other half to full 5.1, and then the first half again to full 5.1.
>
Manik, I doubt that stock JGroups 3.0.x is binary compatible with
JGroups 2.12.x, but we could perhaps copy the protocols from 2.12 to
create a 3.0.compat version. When the entire cluster is running
5.1.compat, we could close the compat-mode channels all at once and
start new JGroups 3.0 channels. We'd need some kind of "disable state
transfer" switch, but it sounds doable.
OTOH, 4.2 state transfer requires partial state transfer support from
JGroups (at least in replicated mode). I'm not sure if Bela can hack
partial state transfer back into 3.0 to support state transfer from
4.2 -> 5.1 in replicated mode. For distributed caches I think it may
be easier, the 4.2 rehashing code should still work with JGroups 3.0.
Even if we do get state transfer working, the nodes then have to
interact: 5.1 nodes have to respond to 4.2 commands, and 4.2 nodes
have to respond to 5.1 commands. I'm pretty sure we have some
incompatibilities, with all the optimization rounds that we had going
on...
>> Did you see the issues we opened back in Lisbon on JIRA? they are
>> supposed to make it possible to perform a rolling upgrade, but need to
>> be included in the project *before* as I had already pointed out, to
>> make it possible to migrate to future versions.
>>
>> All issues are still open, apparently not considered important enough
>> :-( see https://issues.jboss.org/browse/ISPN-1410 and all it's
>> dependencies.
>
> I saw this, but I think there are a few bits missing. From your umbrella JIRA:
>
> ISPN-1409: why is this necessary? CacheLoaders use VersionAwareMarshallers which
create versioned streams as it is. So from a serialisation perspective, we should be
protected.
>
Looking at the code of VersionAwareMarshaller, it reads the version id
in startObjectInput but it doesn't do anything with it…
^ Indeed, this part is really half baked.
Also, how do you register an Externalizer to handle a particular
version?
Not there yet.
This is one of the things I'm gonna try to tighten up as part of ISPN-2034
> ISPN-1406: Not sure I understand the purpose of this. If we're writing through
to a cache store then this happens anyway.
>
Assuming VersionAwareMarshaller does pick the right Externalizer based
on the version id...
>> As I see it, even JDG customers won't be able to migrate to newer
>> versions since this wasn't tested and for sure there will be more
>> changes needed: I certainly won't presume our theoretical draft will
>> work flawlessly in practice.
>
> Yes, this definitely needs work, and more than that, as you say, some real-world
migration tests.
>
>> Also the thread we opened on Flags is relevant (active even today)
>
> +1.
>
> But I'm still very curious about the 3 points I raised in my original email.
:)
>
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
3:11 p.m.
Configuration changes within the *same* version, such as going from
NAKACK to NAKACK2, would also have to go through the mechanism you
suggested.
Handling class loader issues alone will be tricky...
I thought about a simple&stupid approach: at the end of the day, all we
care about is the data, in-memory and on-disk (cacheloaders). So how
about the following:
- Cluster A is running on version 4
- Start a cluster B, running on version 5
- Slowly transfer the state of cluster A into B
- When done, switch the clients over to B. If the clients run inside the
same JVM, activate them in cluster B
- Shut down cluster A
The state transfer from A to B would be similar to the non-blocking
state transfer:
- Perhaps insert a new interceptor (at runtime) into all nodes in A
- Copy all modification in A to a queue
- Iterate over the data and send it to cluster B. This could potentially
be done via Hot Rod.
- When done, send the queued data to B and clear the queue (short period
of synchrionization here)
We would have to make sure that state on-disk (e.g. persisted via a
cacheloader to a DB) would also get transferred. Hmm, maybe not, if
Infinispan can handle data stored in version A from version B...
WDYT ?
--
Bela Ban, JGroups lead (http://www.jgroups.org)
4425
days inactive
4426
days old
infinispan-dev@lists.jboss.org
2 comments
3 participants
participants (3)
-
Bela Ban -
Dan Berindei -
Galder Zamarreño