Re: [keycloak-dev] infinispan as a storage mechanism

It's not going to help with startup time, but perhaps one simple way to make the testsuite run faster would be configuring the h2 dB as in-mem/non-persisted. On 27 Feb 2018 7:11 pm, "Stian Thorgersen" <sthorger(a)redhat.com&gt; wrote: > > On 27 February 2018 at 15:02, Bill Burke <bburke(a)redhat.com&gt; wrote: > >> This is something I've been doing off and on for awhile. An hour here >> or there. Its a lot of monotonous work. >> >> Well worth it IMO if our build times drop drastically. 30 min build >> is becoming burdensome. If the console tests get turned on combined >> with all the other tests that are added daily, I can see this turning >> into 60 min by the end of the year. Not to mention when I run a >> single test from the IDE it takes like 10 seconds to start. I'm just >> sick and tired of it. >> > I can only see how it saves time on starting the Keycloak server, not for > individual tests. That means you're only saving a few seconds each time the > server is started, which is not many times for a full run. You'll probably > end up shaving 1 min of the whole 30 min at best. > > Andy also mentioned that the Hibernate guys are working on performance. > There is something weird about the Hibernate startup time in general. So if > there's improvements there that 1 min you could shave would become even > less. > > We're also working towards having the full testsuite ran on PRs, which > means as a dev you would be better of picking and running only the tests > you believe may be affected and let the CI run the complete set of jobs. > > I'd never bother running the admin console tests locally unless I've done > some changes to the admin console. I also found that running the tests from > the IDE run significantly quicker than through Maven. The difference there > is down to something else than JPA as that's used in both cases. Personally > I very rarely build the full distro or run the tests from Maven at all. I > just let Travis do that, while I run tests through the IDE. > > >> For migration, if you base your stored objects on Maps, you only have >> to worry about cases where you're modifying objects that are >> serialized. These would need to be versioned as per java >> serialization. New things >> > I really don't want us to maintain two different persistence layers. It's > a lot of duplicated effort. Becomes even worse when you start thinking > about things like migration, cross-dc, rolling upgrades, etc.. > > If you are suggesting to completely drop our JPA store altogether and use > Infinispan with a cache store instead that could be an interesting option, > but I have loads and loads of concerns around that. > > >> >> On Tue, Feb 27, 2018 at 1:53 AM, Stian Thorgersen <sthorger(a)redhat.com&gt; >> wrote: >>> I'm really not convinced about this. Infinispan still needs to persist >> the >>> data. It still needs to handle migration whenever we change things. It's >>> another layer to get working correctly, etc.. I think we have more >> important >>> work than to work on another data layer. >>> >>> On 26 February 2018 at 21:47, Bill Burke <bburke(a)redhat.com&gt; wrote: >>>> I'm thinking of this mostly for running our testsuite. If you're not >>>> developing on DB, much nicer if your test startup times is >>>> milliseconds rather than 5-10 secs. >>>> >>>> For production, I'm thinking more of when people need lightweight >>>> keycloak instances and are doing a lot of identity federation. This >>>> is really long term thoughts though. >>>> >>>> On Mon, Feb 26, 2018 at 2:56 PM, Pedro Igor Silva <psilva(a)redhat.com&gt; >>>> wrote: >>>>> I think MongoDB will start supporting transactions very soon on v4 >> .... >>>>> I'm not sure about running both app and database in the same VM >> though. >>>>> For >>>>> dev purposes that is fine, but in real world scenarios you probably >> want >>>>> to >>>>> avoid sharing resources (mem, cpu) with your DB. In your case, people >>>>> will >>>>> probably need JBoss DataGrid in production. >>>>> >>>>> On Mon, Feb 26, 2018 at 4:30 PM, Bill Burke <bburke(a)redhat.com&gt; >> wrote: >>>>>> Other than MongoDB not supporting transactions or even sessions? >> And >>>>>> requiring a DB to be run in a separate VM? >>>>>> >>>>>> No not really :) >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Feb 26, 2018 at 12:54 PM, Pedro Igor Silva < >> psilva(a)redhat.com&gt; >>>>>> wrote: >>>>>>> Isn't this somewhat related with what we used to have with >> MongoDB ? >>>>>>> On Mon, Feb 26, 2018 at 2:35 PM, Bill Burke <bburke(a)redhat.com&gt; >>>>>>> wrote: >>>>>>>> If we had a built-in, clusterable storage mechanism for Keycloak >>>>>>>> using >>>>>>>> Infinispan we would: >>>>>>>> * Shorten build times drastically. 30 minutes and growing for me >>>>>>>> for >>>>>>>> JPA builds. Liquibase + JPA startup takes 5-7 seconds on my box. >>>>>>>> * Simpler startup. No need to start a DB. >>>>>>>> * Reduce memory footprint? I think JPA is responsible for a lot >> of >>>>>>>> classes loaded. >>>>>>>> >>>>>>>> I've started some work on this in spare time. I'd say I'd be >> done >>>>>>>> in >>>>>>>> like 2 months considering the other work I have in queue. >>>>>>>> >>>>>>>> Looking at FineGrainAtomicMap as an implementation. Should make >> DB >>>>>>>> migration simple and replication quicker. >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Bill Burke >>>>>>>> Red Hat >>>>>>>> _______________________________________________ >>>>>>>> keycloak-dev mailing list >>>>>>>> keycloak-dev(a)lists.jboss.org >>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Bill Burke >>>>>> Red Hat >>>>> >>>> >>>> >>>> -- >>>> Bill Burke >>>> Red Hat >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> >> -- >> Bill Burke >> Red Hat >> > _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev