[hibernate-dev] replace Pax Exam with Docker
Brett Meyer
brett at hibernate.org
Fri Jan 12 13:16:49 EST 2018
I guess the way I'm looking at this is Docker will be primarily used by
Jenkins, and myself or anyone working directly on hibernate-osgi
itself. Otherwise, it'll be disabled by default and hidden behind a
profile. We'll make sure that most contributors running the entire
Hibernate test suite won't be affected...
On 1/12/18 1:13 PM, andrea boriero wrote:
> I already have Docker running on my machine, so it seems not a big
> issue for me,but not sure about the impact for others.
>
> Anyway It's worth giving a try.
>
> On 12 January 2018 at 17:54, Sanne Grinovero <sanne at hibernate.org
> <mailto:sanne at hibernate.org>> wrote:
>
> On 12 January 2018 at 17:32, Brett Meyer <brett at hibernate.org
> <mailto:brett at hibernate.org>> wrote:
> > If I don't have time to contribute to Pax Exam, I certainly
> don't have
> > time to start a new project haha...
> >
> > And realistically, that "something new" would likely involve
> containers
> > anyway.
> >
> > At this point, mostly a question of 1) status quo, 2) Docker (or any
> > other container-based solution), or 3) try screwing around with
> Pax Exam
> > in "server-only" mode (but I don't have high hopes there).
>
> Sure. Docker is now available on the CI slaves too, so that's not
> a problem.
>
> The only annoyance is that the whole ORM team - and anyone
> contributing - would need to have Docker as well, but that doesn't
> seem too bad to me... and was likely bound to happen for other tools
> :)
>
> Steve, Chris and Andrea? Ok with that? Maybe you have Docker
> running already?
>
> >
> >
> > On 1/12/18 12:27 PM, Sanne Grinovero wrote:
> >> Ok, looks like you really should start something new :)
> >>
> >> Hopefully many of those other annoyed Karaf developers will follow.
> >>
> >> On 12 January 2018 at 13:59, Brett Meyer <brett at hibernate.org
> <mailto:brett at hibernate.org>> wrote:
> >>> Plus, for me, it's more a question of time. I only have a bit
> available
> >>> for open source work these days, and I'd rather spend that
> knocking out
> >>> some of the hibernate-osgi tasks we've had on our plate for a
> while. I
> >>> unfortunately don't have anything left to contribute to Pax
> Exam itself,
> >>> assuming that would even fix the problem.
> >>>
> >>> Even worse, we're barely using the integration tests for
> anything more
> >>> than a simple smoke test at this point, since it seems like
> every time
> >>> we touch it something new goes wrong. Looking for a more
> *consistent*
> >>> solution -- need more confidence in the backbone.
> >>>
> >>>
> >>> On 1/12/18 8:56 AM, Brett Meyer wrote:
> >>>> Sorry Gunnar/Sanne, should have clarified this first:
> >>>>
> >>>> We actually used Arquillian before Pax Exam, and the
> experience was
> >>>> far worse (somewhat of a long story)...
> >>>>
> >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, so I
> >>>> can't imagine using Karaf without the helpers being a walk in
> the park
> >>>>
> >>>> That's not actually the case. The way Pax Exam currently
> runs our
> >>>> tests is fundamentally part of the problem. The test code is
> >>>> dynamically wrapped in an actual bundle, using something like
> >>>> tiny-bundles, and executed *within* the container itself. Pax
> >>>> overrides runs with additional probes, overrides logging
> >>>> infrastructure, etc. Those nuances can often be the source
> of many of
> >>>> the bugs (there are a ton of classloader implications, etc.
> -- IIRC,
> >>>> this was one area where Arquillian was much, much worse).
> There are
> >>>> some benefits to that setup, but for Hibernate it mainly gets
> in the way.
> >>>>
> >>>> It *does* have a "server mode" where tests run outside of the
> >>>> container, but I vaguely remember going down that path early
> on and
> >>>> hitting a roadblock. For the life of me, I can't remember the
> >>>> specifics. But my pushback here is that ultimately Docker
> might be
> >>>> more preferable, giving us more of a real world scenario to
> do true
> >>>> e2e tests without something else in the middle.
> >>>>
> >>>>> so I can't imagine using Karaf without the helpers being a
> walk in
> >>>> the park; e.g. having to deal with HTTP operations comes with
> its own
> >>>> baggage {dependencies, complexity, speed, .. } and generally more
> >>>> stuff to maintain.
> >>>>
> >>>> I guess I respectfully disagree with that, but purely due to
> Karaf
> >>>> features. Our features.xml does most of the heavy lifting for us
> >>>> w/r/t getting Hibernate provisioned. The same would be true
> with the
> >>>> test harness bundle/feature. REST is simple and
> out-of-the-box thanks
> >>>> to Karaf + CXF or Camel. For other possible routes (Karaf
> commands),
> >>>> we already have code available in our demo/quickstart projects.
> >>>>
> >>>>> Also: considered contributing to Pax?
> >>>> Yes, of course. But the fact that numerous Karaf *committers*
> >>>> themselves have a long history of built-up frustration on it
> doesn't
> >>>> leave me optimistic. A couple of them had tried to pitch in
> at one
> >>>> point and weren't able to get anywhere.
> >>>>
> >>>>> but it seems their developers really expect their users to
> be deeply
> >>>> familiar with it all
> >>>>
> >>>> Absolutely! But again, our struggles also come down to the
> >>>> fundamental way Pax Exam works...
> >>>>
> >>>>
> >>>> On 1/12/18 6:27 AM, Sanne Grinovero wrote:
> >>>>> +1 to explore alternatives to Pax Exam, but I'd be wary of
> maintining
> >>>>> our own test infrastructure.
> >>>>>
> >>>>> Pax Exam was just "helping" to deploy/run things in Karaf,
> so I can't
> >>>>> imagine using Karaf without the helpers being a walk in the
> park; e.g.
> >>>>> having to deal with HTTP operations comes with its own baggage
> >>>>> {dependencies, complexity, speed, .. } and generally more
> stuff to
> >>>>> maintain.
> >>>>>
> >>>>> So.. +1 to try out Arquillian or anything else. Or maybe you
> could
> >>>>> start your own tool, but I'd prefer to see it in a separate
> repository
> >>>>> :) e.g. a nice Gradle plugin so maybe you get more people
> helping?
> >>>>>
> >>>>> Also: considered contributing to Pax? My personal experience
> with it
> >>>>> has always been a pain but if I had to try identify the
> reason, it was
> >>>>> mostly caused by me being unfamiliar with Karaf and not
> having good
> >>>>> clues to track down the real failure; maybe some minor error
> reporting
> >>>>> improvements could make a big difference to its usability? Just
> >>>>> saying, I don't feel like Pax is bad, but it seems their
> developers
> >>>>> really expect their users to be deeply familiar with it all
> - feels
> >>>>> like the typical case in which they could use some feedback
> and a
> >>>>> hand.
> >>>>>
> >>>>> Thanks,
> >>>>> Sanne
> >>>>>
> >>>>> On 12 January 2018 at 08:22, Gunnar
> Morling<gunnar at hibernate.org <mailto:gunnar at hibernate.org>> wrote:
> >>>>>> Hi Brett,
> >>>>>>
> >>>>>> We also had our fair share of frustration with Pax Exam in
> HV, and I was
> >>>>>> (more than once) at the point of dropping it.
> >>>>>>
> >>>>>> Docker could work, but as you say it's a bit of a heavy
> dependency, if not
> >>>>>> required anyways. Not sure whether I'd like to add this as
> a prerequisite
> >>>>>> for the HV build to be executed. And tests in separate
> profiles tend to be
> >>>>>> "forgotten" in my experience.
> >>>>>>
> >>>>>> One other approach could be to use Arquillian's OSGi
> support (see
> >>>>>> https://github.com/arquillian/arquillian-container-osgi
> <https://github.com/arquillian/arquillian-container-osgi>), did
> you consider
> >>>>>> to use that one as an alternative?
> >>>>>>
> >>>>>> Cheers,
> >>>>>>
> >>>>>> --Gunnar
> >>>>>>
> >>>>>>
> >>>>>> 2018-01-12 3:34 GMT+01:00 Brett Meyer<brett at hibernate.org
> <mailto:brett at hibernate.org>>:
> >>>>>>
> >>>>>>> <tired-rant>
> >>>>>>>
> >>>>>>> I'm fed up with Pax Exam and would love to replace it as the
> >>>>>>> hibernate-osgi integration test harness. Most of the
> Karaf committers
> >>>>>>> I've been working with hate it more than I do. Every
> single time we
> >>>>>>> upgrade the Karaf version, something less-than-minor in
> hibernate-osgi,
> >>>>>>> upgrade/change dependencies, or attempt to upgrade Pax
> Exam itself,
> >>>>>>> there's some new obfuscated failure. And no matter how
> much I pray, it
> >>>>>>> refuses to let us get to the container logs to figure out what
> >>>>>>> happened. Tis a house of cards.
> >>>>>>>
> >>>>>>> </tired-rant>
> >>>>>>>
> >>>>>>> One alternative that recently came up elsewhere: use
> Docker to bootstrap
> >>>>>>> the container, hit it with our features.xml, install a
> test bundle that
> >>>>>>> exposes functionality externally (over HTTP, Karaf
> commands, etc), then
> >>>>>>> hit the endpoints and run assertions.
> >>>>>>>
> >>>>>>> Pros: true "integration test", plain vanilla Karaf, direct
> access to all
> >>>>>>> logs, easier to eventually support and test other containers.
> >>>>>>>
> >>>>>>> Cons: Need Docker installed for local test runs, probably
> safer to
> >>>>>>> isolate the integration test behind a disabled-by-default
> Maven profile.
> >>>>>>>
> >>>>>>> Any gut reactions?
> >>>>>>>
> >>>>>>> OSGi is fun and I'm not at all bitter,
> >>>>>>>
> >>>>>>> -Brett-
> >>>>>>>
> >>>>>>> ;)
> >>>>>>>
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> hibernate-dev mailing list
> >>>>>>> hibernate-dev at lists.jboss.org
> <mailto:hibernate-dev at lists.jboss.org>
> >>>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
> >>>>>> _______________________________________________
> >>>>>> hibernate-dev mailing list
> >>>>>> hibernate-dev at lists.jboss.org
> <mailto:hibernate-dev at lists.jboss.org>
> >>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
> >>>>> _______________________________________________
> >>>>> hibernate-dev mailing list
> >>>>> hibernate-dev at lists.jboss.org
> <mailto:hibernate-dev at lists.jboss.org>
> >>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
> >>> _______________________________________________
> >>> hibernate-dev mailing list
> >>> hibernate-dev at lists.jboss.org
> <mailto:hibernate-dev at lists.jboss.org>
> >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
> >
> >
> > _______________________________________________
> > hibernate-dev mailing list
> > hibernate-dev at lists.jboss.org <mailto:hibernate-dev at lists.jboss.org>
> > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev at lists.jboss.org <mailto:hibernate-dev at lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
>
>
More information about the hibernate-dev
mailing list