Re: [hibernate-dev] replace Pax Exam with Docker

I guess the way I'm looking at this is Docker will be primarily used by Jenkins, and myself or anyone working directly on hibernate-osgi itself. Otherwise, it'll be disabled by default and hidden behind a profile. We'll make sure that most contributors running the entire Hibernate test suite won't be affected... On 1/12/18 1:13 PM, andrea boriero wrote: > I already have Docker running on my machine, so it seems not a big > issue for me,but not sure about the impact for others. > > Anyway It's worth giving a try. > > On 12 January 2018 at 17:54, Sanne Grinovero <sanne(a)hibernate.org > <mailto:sanne@hibernate.org>> wrote: > > On 12 January 2018 at 17:32, Brett Meyer <brett(a)hibernate.org > <mailto:brett@hibernate.org>> wrote: > > If I don't have time to contribute to Pax Exam, I certainly > don't have > > time to start a new project haha... > > > > And realistically, that "something new" would likely involve > containers > > anyway. > > > > At this point, mostly a question of 1) status quo, 2) Docker (or any > > other container-based solution), or 3) try screwing around with > Pax Exam > > in "server-only" mode (but I don't have high hopes there). > > Sure. Docker is now available on the CI slaves too, so that's not > a problem. > > The only annoyance is that the whole ORM team - and anyone > contributing - would need to have Docker as well, but that doesn't > seem too bad to me... and was likely bound to happen for other tools > :) > > Steve, Chris and Andrea? Ok with that? Maybe you have Docker > running already? > > > > > > > On 1/12/18 12:27 PM, Sanne Grinovero wrote: > >> Ok, looks like you really should start something new :) > >> > >> Hopefully many of those other annoyed Karaf developers will follow. > >> > >> On 12 January 2018 at 13:59, Brett Meyer <brett(a)hibernate.org > <mailto:brett@hibernate.org>> wrote: > >>> Plus, for me, it's more a question of time. I only have a bit > available > >>> for open source work these days, and I'd rather spend that > knocking out > >>> some of the hibernate-osgi tasks we've had on our plate for a > while. I > >>> unfortunately don't have anything left to contribute to Pax > Exam itself, > >>> assuming that would even fix the problem. > >>> > >>> Even worse, we're barely using the integration tests for > anything more > >>> than a simple smoke test at this point, since it seems like > every time > >>> we touch it something new goes wrong. Looking for a more > *consistent* > >>> solution -- need more confidence in the backbone. > >>> > >>> > >>> On 1/12/18 8:56 AM, Brett Meyer wrote: > >>>> Sorry Gunnar/Sanne, should have clarified this first: > >>>> > >>>> We actually used Arquillian before Pax Exam, and the > experience was > >>>> far worse (somewhat of a long story)... > >>>> > >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, so I > >>>> can't imagine using Karaf without the helpers being a walk in > the park > >>>> > >>>> That's not actually the case. The way Pax Exam currently > runs our > >>>> tests is fundamentally part of the problem. The test code is > >>>> dynamically wrapped in an actual bundle, using something like > >>>> tiny-bundles, and executed *within* the container itself. Pax > >>>> overrides runs with additional probes, overrides logging > >>>> infrastructure, etc. Those nuances can often be the source > of many of > >>>> the bugs (there are a ton of classloader implications, etc. > -- IIRC, > >>>> this was one area where Arquillian was much, much worse). > There are > >>>> some benefits to that setup, but for Hibernate it mainly gets > in the way. > >>>> > >>>> It *does* have a "server mode" where tests run outside of the > >>>> container, but I vaguely remember going down that path early > on and > >>>> hitting a roadblock. For the life of me, I can't remember the > >>>> specifics. But my pushback here is that ultimately Docker > might be > >>>> more preferable, giving us more of a real world scenario to > do true > >>>> e2e tests without something else in the middle. > >>>> > >>>>> so I can't imagine using Karaf without the helpers being a > walk in > >>>> the park; e.g. having to deal with HTTP operations comes with > its own > >>>> baggage {dependencies, complexity, speed, .. } and generally more > >>>> stuff to maintain. > >>>> > >>>> I guess I respectfully disagree with that, but purely due to > Karaf > >>>> features. Our features.xml does most of the heavy lifting for us > >>>> w/r/t getting Hibernate provisioned. The same would be true > with the > >>>> test harness bundle/feature. REST is simple and > out-of-the-box thanks > >>>> to Karaf + CXF or Camel. For other possible routes (Karaf > commands), > >>>> we already have code available in our demo/quickstart projects. > >>>> > >>>>> Also: considered contributing to Pax? > >>>> Yes, of course. But the fact that numerous Karaf *committers* > >>>> themselves have a long history of built-up frustration on it > doesn't > >>>> leave me optimistic. A couple of them had tried to pitch in > at one > >>>> point and weren't able to get anywhere. > >>>> > >>>>> but it seems their developers really expect their users to > be deeply > >>>> familiar with it all > >>>> > >>>> Absolutely! But again, our struggles also come down to the > >>>> fundamental way Pax Exam works... > >>>> > >>>> > >>>> On 1/12/18 6:27 AM, Sanne Grinovero wrote: > >>>>> +1 to explore alternatives to Pax Exam, but I'd be wary of > maintining > >>>>> our own test infrastructure. > >>>>> > >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, > so I can't > >>>>> imagine using Karaf without the helpers being a walk in the > park; e.g. > >>>>> having to deal with HTTP operations comes with its own baggage > >>>>> {dependencies, complexity, speed, .. } and generally more > stuff to > >>>>> maintain. > >>>>> > >>>>> So.. +1 to try out Arquillian or anything else. Or maybe you > could > >>>>> start your own tool, but I'd prefer to see it in a separate > repository > >>>>> :) e.g. a nice Gradle plugin so maybe you get more people > helping? > >>>>> > >>>>> Also: considered contributing to Pax? My personal experience > with it > >>>>> has always been a pain but if I had to try identify the > reason, it was > >>>>> mostly caused by me being unfamiliar with Karaf and not > having good > >>>>> clues to track down the real failure; maybe some minor error > reporting > >>>>> improvements could make a big difference to its usability? Just > >>>>> saying, I don't feel like Pax is bad, but it seems their > developers > >>>>> really expect their users to be deeply familiar with it all > - feels > >>>>> like the typical case in which they could use some feedback > and a > >>>>> hand. > >>>>> > >>>>> Thanks, > >>>>> Sanne > >>>>> > >>>>> On 12 January 2018 at 08:22, Gunnar > Morling&lt;gunnar(a)hibernate.org <mailto:gunnar@hibernate.org>> wrote: > >>>>>> Hi Brett, > >>>>>> > >>>>>> We also had our fair share of frustration with Pax Exam in > HV, and I was > >>>>>> (more than once) at the point of dropping it. > >>>>>> > >>>>>> Docker could work, but as you say it's a bit of a heavy > dependency, if not > >>>>>> required anyways. Not sure whether I'd like to add this as > a prerequisite > >>>>>> for the HV build to be executed. And tests in separate > profiles tend to be > >>>>>> "forgotten" in my experience. > >>>>>> > >>>>>> One other approach could be to use Arquillian's OSGi > support (see > >>>>>> https://github.com/arquillian/arquillian-container-osgi > <https://github.com/arquillian/arquillian-container-osgi>), did > you consider > >>>>>> to use that one as an alternative? > >>>>>> > >>>>>> Cheers, > >>>>>> > >>>>>> --Gunnar > >>>>>> > >>>>>> > >>>>>> 2018-01-12 3:34 GMT+01:00 Brett Meyer&lt;brett(a)hibernate.org > <mailto:brett@hibernate.org>>: > >>>>>> > >>>>>>> <tired-rant> > >>>>>>> > >>>>>>> I'm fed up with Pax Exam and would love to replace it as the > >>>>>>> hibernate-osgi integration test harness. Most of the > Karaf committers > >>>>>>> I've been working with hate it more than I do. Every > single time we > >>>>>>> upgrade the Karaf version, something less-than-minor in > hibernate-osgi, > >>>>>>> upgrade/change dependencies, or attempt to upgrade Pax > Exam itself, > >>>>>>> there's some new obfuscated failure. And no matter how > much I pray, it > >>>>>>> refuses to let us get to the container logs to figure out what > >>>>>>> happened. Tis a house of cards. > >>>>>>> > >>>>>>> </tired-rant> > >>>>>>> > >>>>>>> One alternative that recently came up elsewhere: use > Docker to bootstrap > >>>>>>> the container, hit it with our features.xml, install a > test bundle that > >>>>>>> exposes functionality externally (over HTTP, Karaf > commands, etc), then > >>>>>>> hit the endpoints and run assertions. > >>>>>>> > >>>>>>> Pros: true "integration test", plain vanilla Karaf, direct > access to all > >>>>>>> logs, easier to eventually support and test other containers. > >>>>>>> > >>>>>>> Cons: Need Docker installed for local test runs, probably > safer to > >>>>>>> isolate the integration test behind a disabled-by-default > Maven profile. > >>>>>>> > >>>>>>> Any gut reactions? > >>>>>>> > >>>>>>> OSGi is fun and I'm not at all bitter, > >>>>>>> > >>>>>>> -Brett- > >>>>>>> > >>>>>>> ;) > >>>>>>> > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> hibernate-dev mailing list > >>>>>>> hibernate-dev(a)lists.jboss.org > <mailto:hibernate-dev@lists.jboss.org> > >>>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > >>>>>> _______________________________________________ > >>>>>> hibernate-dev mailing list > >>>>>> hibernate-dev(a)lists.jboss.org > <mailto:hibernate-dev@lists.jboss.org> > >>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > >>>>> _______________________________________________ > >>>>> hibernate-dev mailing list > >>>>> hibernate-dev(a)lists.jboss.org > <mailto:hibernate-dev@lists.jboss.org> > >>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > >>> _______________________________________________ > >>> hibernate-dev mailing list > >>> hibernate-dev(a)lists.jboss.org > <mailto:hibernate-dev@lists.jboss.org> > >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > > > > > > _______________________________________________ > > hibernate-dev mailing list > > hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists. jboss.org> > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > _______________________________________________ > hibernate-dev mailing list > hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/hibernate-dev > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > > _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev