[hibernate-dev] replace Pax Exam with Docker

Brett Meyer brett at hibernate.org
Fri Jan 12 13:16:49 EST 2018


I guess the way I'm looking at this is Docker will be primarily used by 
Jenkins, and myself or anyone working directly on hibernate-osgi 
itself.  Otherwise, it'll be disabled by default and hidden behind a 
profile.  We'll make sure that most contributors running the entire 
Hibernate test suite won't be affected...


On 1/12/18 1:13 PM, andrea boriero wrote:
> I already have Docker running on my machine, so it seems not a big 
> issue for me,but not sure about the impact for others.
>
> Anyway It's worth giving a try.
>
> On 12 January 2018 at 17:54, Sanne Grinovero <sanne at hibernate.org 
> <mailto:sanne at hibernate.org>> wrote:
>
>     On 12 January 2018 at 17:32, Brett Meyer <brett at hibernate.org
>     <mailto:brett at hibernate.org>> wrote:
>     > If I don't have time to contribute to Pax Exam, I certainly
>     don't have
>     > time to start a new project haha...
>     >
>     > And realistically, that "something new" would likely involve
>     containers
>     > anyway.
>     >
>     > At this point, mostly a question of 1) status quo, 2) Docker (or any
>     > other container-based solution), or 3) try screwing around with
>     Pax Exam
>     > in "server-only" mode (but I don't have high hopes there).
>
>     Sure. Docker is now available on the CI slaves too, so that's not
>     a problem.
>
>     The only annoyance is that the whole ORM team - and anyone
>     contributing - would need to have Docker as well, but that doesn't
>     seem too bad to me... and was likely bound to happen for other tools
>     :)
>
>     Steve, Chris and Andrea? Ok with that? Maybe you have Docker
>     running already?
>
>     >
>     >
>     > On 1/12/18 12:27 PM, Sanne Grinovero wrote:
>     >> Ok, looks like you really should start something new :)
>     >>
>     >> Hopefully many of those other annoyed Karaf developers will follow.
>     >>
>     >> On 12 January 2018 at 13:59, Brett Meyer <brett at hibernate.org
>     <mailto:brett at hibernate.org>> wrote:
>     >>> Plus, for me, it's more a question of time.  I only have a bit
>     available
>     >>> for open source work these days, and I'd rather spend that
>     knocking out
>     >>> some of the hibernate-osgi tasks we've had on our plate for a
>     while.  I
>     >>> unfortunately don't have anything left to contribute to Pax
>     Exam itself,
>     >>> assuming that would even fix the problem.
>     >>>
>     >>> Even worse, we're barely using the integration tests for
>     anything more
>     >>> than a simple smoke test at this point, since it seems like
>     every time
>     >>> we touch it something new goes wrong. Looking for a more
>     *consistent*
>     >>> solution -- need more confidence in the backbone.
>     >>>
>     >>>
>     >>> On 1/12/18 8:56 AM, Brett Meyer wrote:
>     >>>> Sorry Gunnar/Sanne, should have clarified this first:
>     >>>>
>     >>>> We actually used Arquillian before Pax Exam, and the
>     experience was
>     >>>> far worse (somewhat of a long story)...
>     >>>>
>     >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, so I
>     >>>> can't imagine using Karaf without the helpers being a walk in
>     the park
>     >>>>
>     >>>> That's not actually the case.  The way Pax Exam currently
>     runs our
>     >>>> tests is fundamentally part of the problem.  The test code is
>     >>>> dynamically wrapped in an actual bundle, using something like
>     >>>> tiny-bundles, and executed *within* the container itself. Pax
>     >>>> overrides runs with additional probes, overrides logging
>     >>>> infrastructure, etc.  Those nuances can often be the source
>     of many of
>     >>>> the bugs (there are a ton of classloader implications, etc.
>     -- IIRC,
>     >>>> this was one area where Arquillian was much, much worse). 
>     There are
>     >>>> some benefits to that setup, but for Hibernate it mainly gets
>     in the way.
>     >>>>
>     >>>> It *does* have a "server mode" where tests run outside of the
>     >>>> container, but I vaguely remember going down that path early
>     on and
>     >>>> hitting a roadblock.  For the life of me, I can't remember the
>     >>>> specifics.  But my pushback here is that ultimately Docker
>     might be
>     >>>> more preferable, giving us more of a real world scenario to
>     do true
>     >>>> e2e tests without something else in the middle.
>     >>>>
>     >>>>> so I can't imagine using Karaf without the helpers being a
>     walk in
>     >>>> the park; e.g. having to deal with HTTP operations comes with
>     its own
>     >>>> baggage {dependencies, complexity, speed, .. } and generally more
>     >>>> stuff to maintain.
>     >>>>
>     >>>> I guess I respectfully disagree with that, but purely due to
>     Karaf
>     >>>> features.  Our features.xml does most of the heavy lifting for us
>     >>>> w/r/t getting Hibernate provisioned. The same would be true
>     with the
>     >>>> test harness bundle/feature.  REST is simple and
>     out-of-the-box thanks
>     >>>> to Karaf + CXF or Camel.  For other possible routes (Karaf
>     commands),
>     >>>> we already have code available in our demo/quickstart projects.
>     >>>>
>     >>>>> Also: considered contributing to Pax?
>     >>>> Yes, of course.  But the fact that numerous Karaf *committers*
>     >>>> themselves have a long history of built-up frustration on it
>     doesn't
>     >>>> leave me optimistic.  A couple of them had tried to pitch in
>     at one
>     >>>> point and weren't able to get anywhere.
>     >>>>
>     >>>>> but it seems their developers really expect their users to
>     be deeply
>     >>>> familiar with it all
>     >>>>
>     >>>> Absolutely!  But again, our struggles also come down to the
>     >>>> fundamental way Pax Exam works...
>     >>>>
>     >>>>
>     >>>> On 1/12/18 6:27 AM, Sanne Grinovero wrote:
>     >>>>> +1 to explore alternatives to Pax Exam, but I'd be wary of
>     maintining
>     >>>>> our own test infrastructure.
>     >>>>>
>     >>>>> Pax Exam was just "helping" to deploy/run things in Karaf,
>     so I can't
>     >>>>> imagine using Karaf without the helpers being a walk in the
>     park; e.g.
>     >>>>> having to deal with HTTP operations comes with its own baggage
>     >>>>> {dependencies, complexity, speed, .. } and generally more
>     stuff to
>     >>>>> maintain.
>     >>>>>
>     >>>>> So.. +1 to try out Arquillian or anything else. Or maybe you
>     could
>     >>>>> start your own tool, but I'd prefer to see it in a separate
>     repository
>     >>>>> :) e.g. a nice Gradle plugin so maybe you get more people
>     helping?
>     >>>>>
>     >>>>> Also: considered contributing to Pax? My personal experience
>     with it
>     >>>>> has always been a pain but if I had to try identify the
>     reason, it was
>     >>>>> mostly caused by me being unfamiliar with Karaf and not
>     having good
>     >>>>> clues to track down the real failure; maybe some minor error
>     reporting
>     >>>>> improvements could make a big difference to its usability? Just
>     >>>>> saying, I don't feel like Pax is bad, but it seems their
>     developers
>     >>>>> really expect their users to be deeply familiar with it all
>     - feels
>     >>>>> like the typical case in which they could use some feedback
>     and a
>     >>>>> hand.
>     >>>>>
>     >>>>> Thanks,
>     >>>>> Sanne
>     >>>>>
>     >>>>> On 12 January 2018 at 08:22, Gunnar
>     Morling<gunnar at hibernate.org <mailto:gunnar at hibernate.org>> wrote:
>     >>>>>> Hi Brett,
>     >>>>>>
>     >>>>>> We also had our fair share of frustration with Pax Exam in
>     HV, and I was
>     >>>>>> (more than once) at the point of dropping it.
>     >>>>>>
>     >>>>>> Docker could work, but as you say it's a bit of a heavy
>     dependency, if not
>     >>>>>> required anyways. Not sure whether I'd like to add this as
>     a prerequisite
>     >>>>>> for the HV build to be executed. And tests in separate
>     profiles tend to be
>     >>>>>> "forgotten" in my experience.
>     >>>>>>
>     >>>>>> One other approach could be to use Arquillian's OSGi
>     support (see
>     >>>>>> https://github.com/arquillian/arquillian-container-osgi
>     <https://github.com/arquillian/arquillian-container-osgi>), did
>     you consider
>     >>>>>> to use that one as an alternative?
>     >>>>>>
>     >>>>>> Cheers,
>     >>>>>>
>     >>>>>> --Gunnar
>     >>>>>>
>     >>>>>>
>     >>>>>> 2018-01-12 3:34 GMT+01:00 Brett Meyer<brett at hibernate.org
>     <mailto:brett at hibernate.org>>:
>     >>>>>>
>     >>>>>>> <tired-rant>
>     >>>>>>>
>     >>>>>>> I'm fed up with Pax Exam and would love to replace it as the
>     >>>>>>> hibernate-osgi integration test harness.  Most of the
>     Karaf committers
>     >>>>>>> I've been working with hate it more than I do.  Every
>     single time we
>     >>>>>>> upgrade the Karaf version, something less-than-minor in
>     hibernate-osgi,
>     >>>>>>> upgrade/change dependencies, or attempt to upgrade Pax
>     Exam itself,
>     >>>>>>> there's some new obfuscated failure.  And no matter how
>     much I pray, it
>     >>>>>>> refuses to let us get to the container logs to figure out what
>     >>>>>>> happened.  Tis a house of cards.
>     >>>>>>>
>     >>>>>>> </tired-rant>
>     >>>>>>>
>     >>>>>>> One alternative that recently came up elsewhere: use
>     Docker to bootstrap
>     >>>>>>> the container, hit it with our features.xml, install a
>     test bundle that
>     >>>>>>> exposes functionality externally (over HTTP, Karaf
>     commands, etc), then
>     >>>>>>> hit the endpoints and run assertions.
>     >>>>>>>
>     >>>>>>> Pros: true "integration test", plain vanilla Karaf, direct
>     access to all
>     >>>>>>> logs, easier to eventually support and test other containers.
>     >>>>>>>
>     >>>>>>> Cons: Need Docker installed for local test runs, probably
>     safer to
>     >>>>>>> isolate the integration test behind a disabled-by-default
>     Maven profile.
>     >>>>>>>
>     >>>>>>> Any gut reactions?
>     >>>>>>>
>     >>>>>>> OSGi is fun and I'm not at all bitter,
>     >>>>>>>
>     >>>>>>> -Brett-
>     >>>>>>>
>     >>>>>>> ;)
>     >>>>>>>
>     >>>>>>>
>     >>>>>>> _______________________________________________
>     >>>>>>> hibernate-dev mailing list
>     >>>>>>> hibernate-dev at lists.jboss.org
>     <mailto:hibernate-dev at lists.jboss.org>
>     >>>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>     <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
>     >>>>>> _______________________________________________
>     >>>>>> hibernate-dev mailing list
>     >>>>>> hibernate-dev at lists.jboss.org
>     <mailto:hibernate-dev at lists.jboss.org>
>     >>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>     <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
>     >>>>> _______________________________________________
>     >>>>> hibernate-dev mailing list
>     >>>>> hibernate-dev at lists.jboss.org
>     <mailto:hibernate-dev at lists.jboss.org>
>     >>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>     <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
>     >>> _______________________________________________
>     >>> hibernate-dev mailing list
>     >>> hibernate-dev at lists.jboss.org
>     <mailto:hibernate-dev at lists.jboss.org>
>     >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>     <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
>     >
>     >
>     > _______________________________________________
>     > hibernate-dev mailing list
>     > hibernate-dev at lists.jboss.org <mailto:hibernate-dev at lists.jboss.org>
>     > https://lists.jboss.org/mailman/listinfo/hibernate-dev
>     <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
>     _______________________________________________
>     hibernate-dev mailing list
>     hibernate-dev at lists.jboss.org <mailto:hibernate-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/hibernate-dev
>     <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
>
>



More information about the hibernate-dev mailing list