[Hawkular-dev] Rolling upgrades (on Kubernetes)

[Heiko W.Rupp]

On 3 Mar 2017, at 16:54, Matt Wringe wrote:

> But how do we handle the situations where we want to go from Cassandra 
> 3.x to Cassandra 4.x? Or what happens when we need to update 
> components when those components themselves don't support this type of 
> rollback?

I did not say this is easy :)

Let me construct a case to illustrate this.

In version N we have a column that stores a string in the form
"tenantId:metricid". Now we decide for N+1 that this is not
good and we need to split that in two separate columns.

In a traditional way we would do the split and also remove the
old single column. Here a rollback is not possible.

If we change this to keep the old column around and the two
new ones we are safe for version N+1, but as there may be
still version N containers around we may end up with inconsistencies
when version N updates an item and N+1 already has split that row.

Here version N+1 would potentially need to read/write from both
scenarios and also update both (if the old one still exists, see below).
This allows rolling back.

Version N+2 could then drop the old "tenantId:metricid" column
and only operate on the split columns.

Heavier updates like C* version 3.x to 4.x can probably really not
be done without downtime - especially when they require massive
migrations of Gigabytes of data.
But still in this case the Metrics frontend code could work against
both versions of C* and redundantly store data in both while slowly
migrating data over. Reads could then still go to the old code by
default until the migration is complete.