[hibernate-dev] replace Pax Exam with Docker
Steve Ebersole
steve at hibernate.org
Fri Jan 12 16:04:14 EST 2018
I believe most of this is already handled for various build tools (like the
gradle docker plugin) and simplified by docker image repos (bintray, e.g.).
But this is a good point. I'd be interested to see if you've done this
before Brett..
On Fri, Jan 12, 2018 at 2:56 PM Gunnar Morling <gunnar at hibernate.org> wrote:
> Brett,
>
> What's still unclear to me is when going the Docker route, won't you still
> need some code which deploys your tests to Karaf, runs them there and
> fetches the test results, so e.g. your Gradle build will fail if there are
> test failures? Would you envision to write these bits yourself? And
> wouldn't this amount to re-implementing PaxExam yourself? Seems I'm still
> missing a piece of the story :)
>
> Cheers,
>
> --Gunnar
>
>
> 2018-01-12 19:16 GMT+01:00 Brett Meyer <brett at hibernate.org>:
>
> > I guess the way I'm looking at this is Docker will be primarily used by
> > Jenkins, and myself or anyone working directly on hibernate-osgi
> > itself. Otherwise, it'll be disabled by default and hidden behind a
> > profile. We'll make sure that most contributors running the entire
> > Hibernate test suite won't be affected...
> >
> >
> > On 1/12/18 1:13 PM, andrea boriero wrote:
> > > I already have Docker running on my machine, so it seems not a big
> > > issue for me,but not sure about the impact for others.
> > >
> > > Anyway It's worth giving a try.
> > >
> > > On 12 January 2018 at 17:54, Sanne Grinovero <sanne at hibernate.org
> > > <mailto:sanne at hibernate.org>> wrote:
> > >
> > > On 12 January 2018 at 17:32, Brett Meyer <brett at hibernate.org
> > > <mailto:brett at hibernate.org>> wrote:
> > > > If I don't have time to contribute to Pax Exam, I certainly
> > > don't have
> > > > time to start a new project haha...
> > > >
> > > > And realistically, that "something new" would likely involve
> > > containers
> > > > anyway.
> > > >
> > > > At this point, mostly a question of 1) status quo, 2) Docker (or
> > any
> > > > other container-based solution), or 3) try screwing around with
> > > Pax Exam
> > > > in "server-only" mode (but I don't have high hopes there).
> > >
> > > Sure. Docker is now available on the CI slaves too, so that's not
> > > a problem.
> > >
> > > The only annoyance is that the whole ORM team - and anyone
> > > contributing - would need to have Docker as well, but that doesn't
> > > seem too bad to me... and was likely bound to happen for other
> tools
> > > :)
> > >
> > > Steve, Chris and Andrea? Ok with that? Maybe you have Docker
> > > running already?
> > >
> > > >
> > > >
> > > > On 1/12/18 12:27 PM, Sanne Grinovero wrote:
> > > >> Ok, looks like you really should start something new :)
> > > >>
> > > >> Hopefully many of those other annoyed Karaf developers will
> > follow.
> > > >>
> > > >> On 12 January 2018 at 13:59, Brett Meyer <brett at hibernate.org
> > > <mailto:brett at hibernate.org>> wrote:
> > > >>> Plus, for me, it's more a question of time. I only have a bit
> > > available
> > > >>> for open source work these days, and I'd rather spend that
> > > knocking out
> > > >>> some of the hibernate-osgi tasks we've had on our plate for a
> > > while. I
> > > >>> unfortunately don't have anything left to contribute to Pax
> > > Exam itself,
> > > >>> assuming that would even fix the problem.
> > > >>>
> > > >>> Even worse, we're barely using the integration tests for
> > > anything more
> > > >>> than a simple smoke test at this point, since it seems like
> > > every time
> > > >>> we touch it something new goes wrong. Looking for a more
> > > *consistent*
> > > >>> solution -- need more confidence in the backbone.
> > > >>>
> > > >>>
> > > >>> On 1/12/18 8:56 AM, Brett Meyer wrote:
> > > >>>> Sorry Gunnar/Sanne, should have clarified this first:
> > > >>>>
> > > >>>> We actually used Arquillian before Pax Exam, and the
> > > experience was
> > > >>>> far worse (somewhat of a long story)...
> > > >>>>
> > > >>>>> Pax Exam was just "helping" to deploy/run things in Karaf,
> so I
> > > >>>> can't imagine using Karaf without the helpers being a walk in
> > > the park
> > > >>>>
> > > >>>> That's not actually the case. The way Pax Exam currently
> > > runs our
> > > >>>> tests is fundamentally part of the problem. The test code is
> > > >>>> dynamically wrapped in an actual bundle, using something like
> > > >>>> tiny-bundles, and executed *within* the container itself. Pax
> > > >>>> overrides runs with additional probes, overrides logging
> > > >>>> infrastructure, etc. Those nuances can often be the source
> > > of many of
> > > >>>> the bugs (there are a ton of classloader implications, etc.
> > > -- IIRC,
> > > >>>> this was one area where Arquillian was much, much worse).
> > > There are
> > > >>>> some benefits to that setup, but for Hibernate it mainly gets
> > > in the way.
> > > >>>>
> > > >>>> It *does* have a "server mode" where tests run outside of the
> > > >>>> container, but I vaguely remember going down that path early
> > > on and
> > > >>>> hitting a roadblock. For the life of me, I can't remember the
> > > >>>> specifics. But my pushback here is that ultimately Docker
> > > might be
> > > >>>> more preferable, giving us more of a real world scenario to
> > > do true
> > > >>>> e2e tests without something else in the middle.
> > > >>>>
> > > >>>>> so I can't imagine using Karaf without the helpers being a
> > > walk in
> > > >>>> the park; e.g. having to deal with HTTP operations comes with
> > > its own
> > > >>>> baggage {dependencies, complexity, speed, .. } and generally
> > more
> > > >>>> stuff to maintain.
> > > >>>>
> > > >>>> I guess I respectfully disagree with that, but purely due to
> > > Karaf
> > > >>>> features. Our features.xml does most of the heavy lifting for
> > us
> > > >>>> w/r/t getting Hibernate provisioned. The same would be true
> > > with the
> > > >>>> test harness bundle/feature. REST is simple and
> > > out-of-the-box thanks
> > > >>>> to Karaf + CXF or Camel. For other possible routes (Karaf
> > > commands),
> > > >>>> we already have code available in our demo/quickstart
> projects.
> > > >>>>
> > > >>>>> Also: considered contributing to Pax?
> > > >>>> Yes, of course. But the fact that numerous Karaf *committers*
> > > >>>> themselves have a long history of built-up frustration on it
> > > doesn't
> > > >>>> leave me optimistic. A couple of them had tried to pitch in
> > > at one
> > > >>>> point and weren't able to get anywhere.
> > > >>>>
> > > >>>>> but it seems their developers really expect their users to
> > > be deeply
> > > >>>> familiar with it all
> > > >>>>
> > > >>>> Absolutely! But again, our struggles also come down to the
> > > >>>> fundamental way Pax Exam works...
> > > >>>>
> > > >>>>
> > > >>>> On 1/12/18 6:27 AM, Sanne Grinovero wrote:
> > > >>>>> +1 to explore alternatives to Pax Exam, but I'd be wary of
> > > maintining
> > > >>>>> our own test infrastructure.
> > > >>>>>
> > > >>>>> Pax Exam was just "helping" to deploy/run things in Karaf,
> > > so I can't
> > > >>>>> imagine using Karaf without the helpers being a walk in the
> > > park; e.g.
> > > >>>>> having to deal with HTTP operations comes with its own
> baggage
> > > >>>>> {dependencies, complexity, speed, .. } and generally more
> > > stuff to
> > > >>>>> maintain.
> > > >>>>>
> > > >>>>> So.. +1 to try out Arquillian or anything else. Or maybe you
> > > could
> > > >>>>> start your own tool, but I'd prefer to see it in a separate
> > > repository
> > > >>>>> :) e.g. a nice Gradle plugin so maybe you get more people
> > > helping?
> > > >>>>>
> > > >>>>> Also: considered contributing to Pax? My personal experience
> > > with it
> > > >>>>> has always been a pain but if I had to try identify the
> > > reason, it was
> > > >>>>> mostly caused by me being unfamiliar with Karaf and not
> > > having good
> > > >>>>> clues to track down the real failure; maybe some minor error
> > > reporting
> > > >>>>> improvements could make a big difference to its usability?
> Just
> > > >>>>> saying, I don't feel like Pax is bad, but it seems their
> > > developers
> > > >>>>> really expect their users to be deeply familiar with it all
> > > - feels
> > > >>>>> like the typical case in which they could use some feedback
> > > and a
> > > >>>>> hand.
> > > >>>>>
> > > >>>>> Thanks,
> > > >>>>> Sanne
> > > >>>>>
> > > >>>>> On 12 January 2018 at 08:22, Gunnar
> > > Morling<gunnar at hibernate.org <mailto:gunnar at hibernate.org>> wrote:
> > > >>>>>> Hi Brett,
> > > >>>>>>
> > > >>>>>> We also had our fair share of frustration with Pax Exam in
> > > HV, and I was
> > > >>>>>> (more than once) at the point of dropping it.
> > > >>>>>>
> > > >>>>>> Docker could work, but as you say it's a bit of a heavy
> > > dependency, if not
> > > >>>>>> required anyways. Not sure whether I'd like to add this as
> > > a prerequisite
> > > >>>>>> for the HV build to be executed. And tests in separate
> > > profiles tend to be
> > > >>>>>> "forgotten" in my experience.
> > > >>>>>>
> > > >>>>>> One other approach could be to use Arquillian's OSGi
> > > support (see
> > > >>>>>> https://github.com/arquillian/arquillian-container-osgi
> > > <https://github.com/arquillian/arquillian-container-osgi>), did
> > > you consider
> > > >>>>>> to use that one as an alternative?
> > > >>>>>>
> > > >>>>>> Cheers,
> > > >>>>>>
> > > >>>>>> --Gunnar
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> 2018-01-12 3:34 GMT+01:00 Brett Meyer<brett at hibernate.org
> > > <mailto:brett at hibernate.org>>:
> > > >>>>>>
> > > >>>>>>> <tired-rant>
> > > >>>>>>>
> > > >>>>>>> I'm fed up with Pax Exam and would love to replace it as
> the
> > > >>>>>>> hibernate-osgi integration test harness. Most of the
> > > Karaf committers
> > > >>>>>>> I've been working with hate it more than I do. Every
> > > single time we
> > > >>>>>>> upgrade the Karaf version, something less-than-minor in
> > > hibernate-osgi,
> > > >>>>>>> upgrade/change dependencies, or attempt to upgrade Pax
> > > Exam itself,
> > > >>>>>>> there's some new obfuscated failure. And no matter how
> > > much I pray, it
> > > >>>>>>> refuses to let us get to the container logs to figure out
> > what
> > > >>>>>>> happened. Tis a house of cards.
> > > >>>>>>>
> > > >>>>>>> </tired-rant>
> > > >>>>>>>
> > > >>>>>>> One alternative that recently came up elsewhere: use
> > > Docker to bootstrap
> > > >>>>>>> the container, hit it with our features.xml, install a
> > > test bundle that
> > > >>>>>>> exposes functionality externally (over HTTP, Karaf
> > > commands, etc), then
> > > >>>>>>> hit the endpoints and run assertions.
> > > >>>>>>>
> > > >>>>>>> Pros: true "integration test", plain vanilla Karaf, direct
> > > access to all
> > > >>>>>>> logs, easier to eventually support and test other
> containers.
> > > >>>>>>>
> > > >>>>>>> Cons: Need Docker installed for local test runs, probably
> > > safer to
> > > >>>>>>> isolate the integration test behind a disabled-by-default
> > > Maven profile.
> > > >>>>>>>
> > > >>>>>>> Any gut reactions?
> > > >>>>>>>
> > > >>>>>>> OSGi is fun and I'm not at all bitter,
> > > >>>>>>>
> > > >>>>>>> -Brett-
> > > >>>>>>>
> > > >>>>>>> ;)
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> _______________________________________________
> > > >>>>>>> hibernate-dev mailing list
> > > >>>>>>> hibernate-dev at lists.jboss.org
> > > <mailto:hibernate-dev at lists.jboss.org>
> > > >>>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> > > <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
> > > >>>>>> _______________________________________________
> > > >>>>>> hibernate-dev mailing list
> > > >>>>>> hibernate-dev at lists.jboss.org
> > > <mailto:hibernate-dev at lists.jboss.org>
> > > >>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> > > <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
> > > >>>>> _______________________________________________
> > > >>>>> hibernate-dev mailing list
> > > >>>>> hibernate-dev at lists.jboss.org
> > > <mailto:hibernate-dev at lists.jboss.org>
> > > >>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> > > <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
> > > >>> _______________________________________________
> > > >>> hibernate-dev mailing list
> > > >>> hibernate-dev at lists.jboss.org
> > > <mailto:hibernate-dev at lists.jboss.org>
> > > >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> > > <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
> > > >
> > > >
> > > > _______________________________________________
> > > > hibernate-dev mailing list
> > > > hibernate-dev at lists.jboss.org <mailto:hibernate-dev at lists.
> > jboss.org>
> > > > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> > > <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
> > > _______________________________________________
> > > hibernate-dev mailing list
> > > hibernate-dev at lists.jboss.org <mailto:
> hibernate-dev at lists.jboss.org>
> > > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> > > <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
> > >
> > >
> >
> > _______________________________________________
> > hibernate-dev mailing list
> > hibernate-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
More information about the hibernate-dev
mailing list