[keycloak-dev] infinispan as a storage mechanism

Tue Feb 27 15:08:21 EST 2018

It's not going to help with startup time, but perhaps one simple way to
make the testsuite run faster would be configuring the h2 dB as
in-mem/non-persisted.

On 27 Feb 2018 7:11 pm, "Stian Thorgersen" <sthorger at redhat.com> wrote:

>
>
> On 27 February 2018 at 15:02, Bill Burke <bburke at redhat.com> wrote:
>
>> This is something I've been doing off and on for awhile.  An hour here
>> or there.  Its a lot of monotonous work.
>>
>> Well worth it IMO if our build times drop drastically.  30 min build
>> is becoming burdensome.  If the console tests get turned on combined
>> with all the other tests that are added daily, I can see this turning
>> into 60 min by the end of the year.  Not to mention when I run a
>> single test from the IDE it takes like 10 seconds to start.  I'm just
>> sick and tired of it.
>>
>
> I can only see how it saves time on starting the Keycloak server, not for
> individual tests. That means you're only saving a few seconds each time the
> server is started, which is not many times for a full run. You'll probably
> end up shaving 1 min of the whole 30 min at best.
>
> Andy also mentioned that the Hibernate guys are working on performance.
> There is something weird about the Hibernate startup time in general. So if
> there's improvements there that 1 min you could shave would become even
> less.
>
> We're also working towards having the full testsuite ran on PRs, which
> means as a dev you would be better of picking and running only the tests
> you believe may be affected and let the CI run the complete set of jobs.
>
> I'd never bother running the admin console tests locally unless I've done
> some changes to the admin console. I also found that running the tests from
> the IDE run significantly quicker than through Maven. The difference there
> is down to something else than JPA as that's used in both cases. Personally
> I very rarely build the full distro or run the tests from Maven at all. I
> just let Travis do that, while I run tests through the IDE.
>
>
>>
>> For migration, if you base your stored objects on Maps, you only have
>> to worry about cases where you're modifying objects that are
>> serialized.  These would need to be versioned as per java
>> serialization.  New things
>>
>
> I really don't want us to maintain two different persistence layers. It's
> a lot of duplicated effort. Becomes even worse when you start thinking
> about things like migration, cross-dc, rolling upgrades, etc..
>
> If you are suggesting to completely drop our JPA store altogether and use
> Infinispan with a cache store instead that could be an interesting option,
> but I have loads and loads of concerns around that.
>
>
>>
>>
>> On Tue, Feb 27, 2018 at 1:53 AM, Stian Thorgersen <sthorger at redhat.com>
>> wrote:
>> > I'm really not convinced about this. Infinispan still needs to persist
>> the
>> > data. It still needs to handle migration whenever we change things. It's
>> > another layer to get working correctly, etc.. I think we have more
>> important
>> > work than to work on another data layer.
>> >
>> > On 26 February 2018 at 21:47, Bill Burke <bburke at redhat.com> wrote:
>> >>
>> >> I'm thinking of this mostly for running our testsuite.  If you're not
>> >> developing on DB, much nicer if your test startup times is
>> >> milliseconds rather than 5-10 secs.
>> >>
>> >> For production, I'm thinking more of when people need lightweight
>> >> keycloak instances and are doing a lot of identity federation.  This
>> >> is really long term thoughts though.
>> >>
>> >> On Mon, Feb 26, 2018 at 2:56 PM, Pedro Igor Silva <psilva at redhat.com>
>> >> wrote:
>> >> > I think MongoDB will start supporting transactions very soon on v4
>> ....
>> >> >
>> >> > I'm not sure about running both app and database in the same VM
>> though.
>> >> > For
>> >> > dev purposes that is fine, but in real world scenarios you probably
>> want
>> >> > to
>> >> > avoid sharing resources (mem, cpu) with your DB. In your case, people
>> >> > will
>> >> > probably need JBoss DataGrid in production.
>> >> >
>> >> > On Mon, Feb 26, 2018 at 4:30 PM, Bill Burke <bburke at redhat.com>
>> wrote:
>> >> >>
>> >> >> Other than MongoDB not supporting transactions or even sessions?
>> And
>> >> >> requiring a DB to be run in a separate VM?
>> >> >>
>> >> >> No not really :)
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Mon, Feb 26, 2018 at 12:54 PM, Pedro Igor Silva <
>> psilva at redhat.com>
>> >> >> wrote:
>> >> >> > Isn't this somewhat related with what we used to have with
>> MongoDB ?
>> >> >> >
>> >> >> > On Mon, Feb 26, 2018 at 2:35 PM, Bill Burke <bburke at redhat.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> If we had a built-in, clusterable storage mechanism for Keycloak
>> >> >> >> using
>> >> >> >> Infinispan we would:
>> >> >> >> * Shorten build times drastically.  30 minutes and growing for me
>> >> >> >> for
>> >> >> >> JPA builds.  Liquibase + JPA startup takes 5-7 seconds on my box.
>> >> >> >> * Simpler startup.  No need to start a DB.
>> >> >> >> * Reduce memory footprint?  I think JPA is responsible for a lot
>> of
>> >> >> >> classes loaded.
>> >> >> >>
>> >> >> >> I've started some work on this in spare time.  I'd say I'd be
>> done
>> >> >> >> in
>> >> >> >> like 2 months considering the other work I have in queue.
>> >> >> >>
>> >> >> >> Looking at FineGrainAtomicMap as an implementation.  Should make
>> DB
>> >> >> >> migration simple and replication quicker.
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Bill Burke
>> >> >> >> Red Hat
>> >> >> >> _______________________________________________
>> >> >> >> keycloak-dev mailing list
>> >> >> >> keycloak-dev at lists.jboss.org
>> >> >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>> >> >> >
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Bill Burke
>> >> >> Red Hat
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Bill Burke
>> >> Red Hat
>> >> _______________________________________________
>> >> keycloak-dev mailing list
>> >> keycloak-dev at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>> >
>> >
>>
>>
>>
>> --
>> Bill Burke
>> Red Hat
>>
>
>