Re: [keycloak-dev] infinispan as a storage mechanism

Another thing that might provide a better experience is run adapter tests using Undertow instead of booting a Wildfly app server. On Wed, Feb 28, 2018 at 6:23 AM, Marek Posolda <mposolda(a)redhat.com&gt; wrote: > I am also personally not concerned about running full testsuite through > "mvn clean install", however I agree will be good to improve startup > when you run single test from IDE. > > We already use in-memory H2 by default on embedded undertow. But maybe > it helps if we skip liquibase at all and let Hibernate to create the > schema when running testsuite with in-mem H2? The problem with Liquibase > is, that there are more and more changelogs and more and more changes > and we can't get rid old changelogs. So there are many things like the > table is created by jpa-changelog-1.0.0, but then immediately dropped by > jpa-changelog-1.5.0 etc. This is not so big issue for production > environments when schema is created just once, but sucks during > development IMO. > > Question is, how much time is spent on schema initialization and how > much on other initialization things (EG. importing master realm and > "test" realm etc)... Maybe to try some startup profiling will help... > > Marek > > > On 27/02/18 21:08, Stian Thorgersen wrote: >> It's not going to help with startup time, but perhaps one simple way to >> make the testsuite run faster would be configuring the h2 dB as >> in-mem/non-persisted. >> >> On 27 Feb 2018 7:11 pm, "Stian Thorgersen" <sthorger(a)redhat.com&gt; wrote: >> >>> On 27 February 2018 at 15:02, Bill Burke <bburke(a)redhat.com&gt; wrote: >>> >>>> This is something I've been doing off and on for awhile. An hour here >>>> or there. Its a lot of monotonous work. >>>> >>>> Well worth it IMO if our build times drop drastically. 30 min build >>>> is becoming burdensome. If the console tests get turned on combined >>>> with all the other tests that are added daily, I can see this turning >>>> into 60 min by the end of the year. Not to mention when I run a >>>> single test from the IDE it takes like 10 seconds to start. I'm just >>>> sick and tired of it. >>>> >>> I can only see how it saves time on starting the Keycloak server, not > for >>> individual tests. That means you're only saving a few seconds each time > the >>> server is started, which is not many times for a full run. You'll > probably >>> end up shaving 1 min of the whole 30 min at best. >>> >>> Andy also mentioned that the Hibernate guys are working on performance. >>> There is something weird about the Hibernate startup time in general. > So if >>> there's improvements there that 1 min you could shave would become even >>> less. >>> >>> We're also working towards having the full testsuite ran on PRs, which >>> means as a dev you would be better of picking and running only the tests >>> you believe may be affected and let the CI run the complete set of jobs. >>> >>> I'd never bother running the admin console tests locally unless I've > done >>> some changes to the admin console. I also found that running the tests > from >>> the IDE run significantly quicker than through Maven. The difference > there >>> is down to something else than JPA as that's used in both cases. > Personally >>> I very rarely build the full distro or run the tests from Maven at all. > I >>> just let Travis do that, while I run tests through the IDE. >>> >>> >>>> For migration, if you base your stored objects on Maps, you only have >>>> to worry about cases where you're modifying objects that are >>>> serialized. These would need to be versioned as per java >>>> serialization. New things >>>> >>> I really don't want us to maintain two different persistence layers. > It's >>> a lot of duplicated effort. Becomes even worse when you start thinking >>> about things like migration, cross-dc, rolling upgrades, etc.. >>> >>> If you are suggesting to completely drop our JPA store altogether and > use >>> Infinispan with a cache store instead that could be an interesting > option, >>> but I have loads and loads of concerns around that. >>> >>> >>>> On Tue, Feb 27, 2018 at 1:53 AM, Stian Thorgersen <sthorger(a)redhat.com >>>> wrote: >>>>> I'm really not convinced about this. Infinispan still needs to persist >>>> the >>>>> data. It still needs to handle migration whenever we change things. > It's >>>>> another layer to get working correctly, etc.. I think we have more >>>> important >>>>> work than to work on another data layer. >>>>> >>>>> On 26 February 2018 at 21:47, Bill Burke <bburke(a)redhat.com&gt; wrote: >>>>>> I'm thinking of this mostly for running our testsuite. If you're not >>>>>> developing on DB, much nicer if your test startup times is >>>>>> milliseconds rather than 5-10 secs. >>>>>> >>>>>> For production, I'm thinking more of when people need lightweight >>>>>> keycloak instances and are doing a lot of identity federation. This >>>>>> is really long term thoughts though. >>>>>> >>>>>> On Mon, Feb 26, 2018 at 2:56 PM, Pedro Igor Silva <psilva(a)redhat.com >>>>>> wrote: >>>>>>> I think MongoDB will start supporting transactions very soon on v4 >>>> .... >>>>>>> I'm not sure about running both app and database in the same VM >>>> though. >>>>>>> For >>>>>>> dev purposes that is fine, but in real world scenarios you probably >>>> want >>>>>>> to >>>>>>> avoid sharing resources (mem, cpu) with your DB. In your case, > people >>>>>>> will >>>>>>> probably need JBoss DataGrid in production. >>>>>>> >>>>>>> On Mon, Feb 26, 2018 at 4:30 PM, Bill Burke <bburke(a)redhat.com&gt; >>>> wrote: >>>>>>>> Other than MongoDB not supporting transactions or even sessions? >>>> And >>>>>>>> requiring a DB to be run in a separate VM? >>>>>>>> >>>>>>>> No not really :) >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Feb 26, 2018 at 12:54 PM, Pedro Igor Silva < >>>> psilva(a)redhat.com&gt; >>>>>>>> wrote: >>>>>>>>> Isn't this somewhat related with what we used to have with >>>> MongoDB ? >>>>>>>>> On Mon, Feb 26, 2018 at 2:35 PM, Bill Burke <bburke(a)redhat.com&gt; >>>>>>>>> wrote: >>>>>>>>>> If we had a built-in, clusterable storage mechanism for Keycloak >>>>>>>>>> using >>>>>>>>>> Infinispan we would: >>>>>>>>>> * Shorten build times drastically. 30 minutes and growing for me >>>>>>>>>> for >>>>>>>>>> JPA builds. Liquibase + JPA startup takes 5-7 seconds on my box. >>>>>>>>>> * Simpler startup. No need to start a DB. >>>>>>>>>> * Reduce memory footprint? I think JPA is responsible for a lot >>>> of >>>>>>>>>> classes loaded. >>>>>>>>>> >>>>>>>>>> I've started some work on this in spare time. I'd say I'd be >>>> done >>>>>>>>>> in >>>>>>>>>> like 2 months considering the other work I have in queue. >>>>>>>>>> >>>>>>>>>> Looking at FineGrainAtomicMap as an implementation. Should make >>>> DB >>>>>>>>>> migration simple and replication quicker. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Bill Burke >>>>>>>>>> Red Hat >>>>>>>>>> _______________________________________________ >>>>>>>>>> keycloak-dev mailing list >>>>>>>>>> keycloak-dev(a)lists.jboss.org >>>>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>> >>>>>>>> -- >>>>>>>> Bill Burke >>>>>>>> Red Hat >>>>>> >>>>>> -- >>>>>> Bill Burke >>>>>> Red Hat >>>>>> _______________________________________________ >>>>>> keycloak-dev mailing list >>>>>> keycloak-dev(a)lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>>> -- >>>> Bill Burke >>>> Red Hat >>>> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev