[Apiman-user] APIMAN, clustering, and versioning
Eric Wittmann
eric.wittmann at redhat.com
Tue Jun 7 14:55:16 EDT 2016
The way we handle upgrades from older versions (version X) to newer ones
that have breaking schema changes (version Y) is as follows:
1) export all data from version X - the result will be a (potentially
large) JSON file - "version X" will be included in the meta-data for
this export file)
2) install version Y (new instances, new+empty DB instance)
3) import the file from step #1 into version Y
4) profit!
The important point here is that step #3 will automatically detect that
it is importing an older file and will "upgrade" the data to the latest
version, if necessary.
This approach allows us to support the following:
- Upgrade from arbitrary older version of apiman to an arbitrary newer
version
- Upgrade apiman from one storage mechanism to another (e.g. switch from
mysql to postgresql, or even switch from mysql to elasticsearch)
- Automatic, periodic backup of all relevant data for later restoration
It's worth noting that metrics information is currently not included in
this process - we'll need a solution to that in the future. The current
assumption is that the metrics data is stored in elasticsearch in a
format that will not change soon...
Of course, this isn't a good story for a cluster of nodes that require
rolling updates. Clearly to support that we would need to prevent any
breaking DB schema changes so...
...to answer your other question - I don't anticipate any *breaking*
changes to the schema in the foreseeable future. But as you say, I
can't guarantee it. :) I would expect any schema changes to be
additive (new tables or columns).
That said, we will not likely be providing DDL patch scripts in
community - so any in-place DB upgrades will be more manual than you
might like (e.g. diff the full DDLs to create your own patch script).
-Eric
On 6/7/2016 1:44 PM, Christopher Stolte wrote:
> Hi Eric,
>
>
> Thanks for your response. In the event of a schema change, would your
> team be providing anything in the way of migration/validation scripts or
> tools? I assume it would be mentioned in the release notes, right? I
> know you can't predict the future, but could you characterize the
> frequency of such changes at this point? (i.e. known schema changes
> coming soon and often / probably something in the next 6 months /
> unlikely to rare...)
>
>
> Thanks,
>
> Chris
>
> ------------------------------------------------------------------------
> *From:* Eric Wittmann <eric.wittmann at redhat.com>
> *Sent:* Monday, June 6, 2016 3:02:48 PM
> *To:* Christopher Stolte; apiman-user at lists.jboss.org
> *Subject:* Re: [Apiman-user] APIMAN, clustering, and versioning
>
> Hi Chris. Thanks for the question.
>
> Some responses below:
>
> On 6/3/2016 1:02 PM, Christopher Stolte wrote:
>> Our team is hoping to use Apiman in a production environment that
>> includes a load-balanced cluster of Apiman gateway instances. Those
>> instances would share Elasticsearch storage for metrics etc., as per
>> this architecture summary:
>
> Note that if you simply have multiple instances of the gateway running,
> all pointed to the same ES storage, you do not need those gateway
> instances to be "clustered". It's a subtle point, but it might be
> important. Note that this point may change in the future as we create
> better implementations of certain components (like rate limiting).
>
>> As we imagine this scenario, some questions pop up that we don't have
>> answers to. For example, what assumptions can we make about data
>> integrity when Apiman releases a new version? Are major releases the
>> only ones that might break a schema (and thus not coexist happily in a
>> cluster with other instances of a different version) ? I apologize if
>> this is documented somewhere - I took a look around but didn't find
>> anything related to versioning semantics. Basically we'd like to
>> understand the upgrade path for a given instance within a cluster.
>
> This is a great question, and something that isn't yet very mature
> within apiman. The short answer is that we don't make any guarantees
> (currently) in community with regard to when the schema may change.
> Rolling upgrades is not yet something we've explicitly planned for. The
> current (easy) approach is to stand up a new environment for the new
> version of apiman, then migrate your data from old to new. Once done,
> you can do a cutover. If you have a large number of instances this may
> not be possible.
>
>> Apiman team: what have you tested or consciously targeted as far as
>> clustered environments? Has anyone out there in the community tried to
>> use a cluster of gateways? Has there been any work done related to this
>> ticket?
>
> We have just begun our internal clustering testing, so we be getting
> this sort of experience very soon. We'll be testing for HA and
> scalability first and foremost. That said, the software is designed to
> easily run in a "cluster" (quotes used because as previously mentioned,
> the nodes do not actually need to be running in a proper cluster).
>
> -Eric
More information about the Apiman-user
mailing list