[keycloak-dev] deprecated removals in future

Tue Jul 21 02:57:07 EDT 2015

On 20.7.2015 18:08, Stian Thorgersen wrote:
>
> ----- Original Message -----
>> From: "Bill Burke" <bburke at redhat.com>
>> To: "Stian Thorgersen" <stian at redhat.com>, "Marek Posolda" <mposolda at redhat.com>
>> Cc: keycloak-dev at lists.jboss.org
>> Sent: Monday, 20 July, 2015 5:21:56 PM
>> Subject: Re: [keycloak-dev] deprecated removals in future
>>
>> They would upgrade to 1.5 first, then upgrade to 2.x.
> In the example I gave they are not able to upgrade to 1.5 as there was a bug in the migration that made it not work the db they use. So we'd have to release a 1.5.1 for them.
>
> I seriously don't think we should be putting these kinda off restrictions in place. It's going to come back and bite us.
Not sure which way is better... It looks easier to me to fix potential 
migration bug and release 1.5.1 then supporting migration from 1.0 to 
any version.

Also note that there are some RDBMS, which don't work right now but we 
need to support them . Right now some of them ( at least IBM DB2 and 
Postgres PLUS ) are broken even in the first jpa-changelog-1.0.0.Final . 
So supporting them might require to do changes in the old liquibase 
scripts (like jpa-changelog-1.0.0.Final ) but that's not good as it 
could break migration for other databases. Doing "cut off" at some point 
and start from scratch allows us to easily add support for those RDBMS 
without breaking migration for other dbs.

BTW. Are we only JBoss project having similar issues? Maybe we can 
inspire from some others? I know JPP had IDM and JCR databases to deal 
with, where migration wasn't so bad due to the most of the stuff was 
saved as key/value pairs. Not sure about other projects. There is also 
project Windup - http://windup.jboss.org/ but that's probably not for DB 
migration...

Marek
>
>> On 7/20/2015 12:37 AM, Stian Thorgersen wrote:
>>> -1000 A user should be allowed to upgrade from any version, not only
>>> because of convenience, but also because there may be no other option. For
>>> example we've had users that have been unable to upgrade to certain
>>> versions in the past due to bugs in the migration stuff. What if someone
>>> is stuck on an old version 1.2, while version 1.5 is required to upgrade
>>> to version 2.0 and the user can't upgrade to 1.3, 1.4 or 1.5? We'd have to
>>> do a 1.2.1 release for the user just to allow them to upgrade.
>>>
>>> ----- Original Message -----
>>>> From: "Marek Posolda" <mposolda at redhat.com>
>>>> To: "Bill Burke" <bburke at redhat.com>, "Stian Thorgersen"
>>>> <stian at redhat.com>
>>>> Cc: keycloak-dev at lists.jboss.org
>>>> Sent: Friday, 17 July, 2015 7:53:44 PM
>>>> Subject: Re: [keycloak-dev] deprecated removals in future
>>>>
>>>> The approach like MigrationModelManager or our current JSON migration is
>>>> cool and easy, but it stops working if you want to get rid of outdated
>>>> properties. For example if you want to remove RealmModel.getFoo() but
>>>> MigrateTo13.java is using getFoo property, you will see the errors with
>>>> new versions where property "foo" on RealmModel is not available
>>>> anymore...
>>>>
>>>> So what if we use something like "milestones" releases and support
>>>> migration just between those? For example milestone release can be 1.5 ,
>>>> 1.10 and 1.15 (or 2.0 or whatever). Then if someone wants to migrate
>>>> from 1.0 to 1.12 he needs to go by migration path like:
>>>> - First migrate from 1.0 to 1.5
>>>> - Then migrate from 1.5 to 1.10
>>>> - Finally migrate from 1.10 to 1.12
>>>>
>>>> For the 1.0 to 1.5 we can use the "easy" approach with keeping of all
>>>> the outdated properties on models and JSON representations. Then in 1.6
>>>> we will start from scratch and remove previous migrations and old
>>>> properties until 1.5. So migration to 1.6 is supported just from 1.5 etc.
>>>>
>>>> For the 1.6 we will start from scratch with everything including
>>>> liquibase DDL scripts - to avoid situation when there are more and more
>>>> liquibase commands executed at initial boot.
>>>>
>>>> Marek
>>>>
>>>> On 17.7.2015 15:45, Bill Burke wrote:
>>>>> You would still need to write a shit ton of raw SQL and JSON scripts
>>>>> to convert from old models (1.0, 1.1, 1.2, 1.3, 1.4, etc...) to the
>>>>> new Json realm model you want to instill.
>>>>>
>>>>> I really worry that we'll be writing a crappy version of Mongo on top
>>>>> of a relational store.
>>>>>
>>>>> On 7/17/2015 9:21 AM, Stian Thorgersen wrote:
>>>>>> Yeah, so let's drop the SQL stuff. We'll only have one single realm
>>>>>> provider which is backed by json representations. Then we store json
>>>>>> representations in Mongo or as blobs in the database.
>>>>>>
>>>>>> After that we only have to deal with one migration.
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Bill Burke" <bburke at redhat.com>
>>>>>>> To: "Stian Thorgersen" <stian at redhat.com>, "Marek Posolda"
>>>>>>> <mposolda at redhat.com>
>>>>>>> Cc: keycloak-dev at lists.jboss.org
>>>>>>> Sent: Friday, 17 July, 2015 2:33:31 PM
>>>>>>> Subject: Re: [keycloak-dev] deprecated removals in future
>>>>>>>
>>>>>>> I'm not spending a week+ writing error prone raw JDBC and generic JSON
>>>>>>> scripts.  There's a lot of model migration code that would be a shit
>>>>>>> ton
>>>>>>> of raw SQL if you wanted to convert it.  A new starting point would be
>>>>>>> half a days work.
>>>>>>>
>>>>>>> On 7/17/2015 2:28 AM, Stian Thorgersen wrote:
>>>>>>>> -1 It'll actually be more work to drop backwards compatibility for the
>>>>>>>> model as we'd have to write a new "starting" point. For JSON
>>>>>>>> representations it's just messy and has deprecated stuff in it
>>>>>>>> because I
>>>>>>>> did a crap job at implementing it. If we write a proper way to migrate
>>>>>>>> json representations we won't need this and it'll just be a
>>>>>>>> transformation
>>>>>>>> pipeline that transforms the json to match the representation classes.
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Marek Posolda" <mposolda at redhat.com>
>>>>>>>>> To: "Bill Burke" <bburke at redhat.com>, keycloak-dev at lists.jboss.org
>>>>>>>>> Sent: Friday, 17 July, 2015 8:24:50 AM
>>>>>>>>> Subject: Re: [keycloak-dev] deprecated removals in future
>>>>>>>>>
>>>>>>>>> +1
>>>>>>>>>
>>>>>>>>> Hopefully we can drop backwards compatibility for JSON reps
>>>>>>>>> between 1.X
>>>>>>>>> and 2.0 (ie. realm JSON representation exported in 1.X is not
>>>>>>>>> supposed
>>>>>>>>> to work to be imported in 2.0). That will allow us to drop old
>>>>>>>>> metadata
>>>>>>>>> from JSON and models and just create DB migration scripts for 1.X
>>>>>>>>> to 2.0
>>>>>>>>> migration.
>>>>>>>>>
>>>>>>>>> Marek
>>>>>>>>>
>>>>>>>>> On 17.7.2015 02:29, Bill Burke wrote:
>>>>>>>>>> We're accumulating deprecated metadata in models and data model.
>>>>>>>>>> Eventually we should draw a line in the sand and say we will only
>>>>>>>>>> support migration from a certain version to current.
>>>>>>>>>>
>>>>>>>>>> i.e. Keycloak 2.0 would only allow migration from the last 1.x
>>>>>>>>>> version.
>>>>>>>>>>
>>>>>>>>>> We need to clean all the old stuff out prior to product.
>>>>>>>>> _______________________________________________
>>>>>>>>> keycloak-dev mailing list
>>>>>>>>> keycloak-dev at lists.jboss.org
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>>>>
>>>>>>> --
>>>>>>> Bill Burke
>>>>>>> JBoss, a division of Red Hat
>>>>>>> http://bill.burkecentral.com
>>>>>>>
>>>>
>> --
>> Bill Burke
>> JBoss, a division of Red Hat
>> http://bill.burkecentral.com
>>