[hibernate-dev] replace Pax Exam with Docker

Fri Jan 12 12:54:11 EST 2018

On 12 January 2018 at 17:32, Brett Meyer <brett at hibernate.org> wrote:
> If I don't have time to contribute to Pax Exam, I certainly don't have
> time to start a new project haha...
>
> And realistically, that "something new" would likely involve containers
> anyway.
>
> At this point, mostly a question of 1) status quo, 2) Docker (or any
> other container-based solution), or 3) try screwing around with Pax Exam
> in "server-only" mode (but I don't have high hopes there).

Sure. Docker is now available on the CI slaves too, so that's not a problem.

The only annoyance is that the whole ORM team - and anyone
contributing - would need to have Docker as well, but that doesn't
seem too bad to me... and was likely bound to happen for other tools
:)

Steve, Chris and Andrea? Ok with that? Maybe you have Docker running already?

>
>
> On 1/12/18 12:27 PM, Sanne Grinovero wrote:
>> Ok, looks like you really should start something new :)
>>
>> Hopefully many of those other annoyed Karaf developers will follow.
>>
>> On 12 January 2018 at 13:59, Brett Meyer <brett at hibernate.org> wrote:
>>> Plus, for me, it's more a question of time.  I only have a bit available
>>> for open source work these days, and I'd rather spend that knocking out
>>> some of the hibernate-osgi tasks we've had on our plate for a while.  I
>>> unfortunately don't have anything left to contribute to Pax Exam itself,
>>> assuming that would even fix the problem.
>>>
>>> Even worse, we're barely using the integration tests for anything more
>>> than a simple smoke test at this point, since it seems like every time
>>> we touch it something new goes wrong.  Looking for a more *consistent*
>>> solution -- need more confidence in the backbone.
>>>
>>>
>>> On 1/12/18 8:56 AM, Brett Meyer wrote:
>>>> Sorry Gunnar/Sanne, should have clarified this first:
>>>>
>>>> We actually used Arquillian before Pax Exam, and the experience was
>>>> far worse (somewhat of a long story)...
>>>>
>>>>> Pax Exam was just "helping" to deploy/run things in Karaf, so I
>>>> can't imagine using Karaf without the helpers being a walk in the park
>>>>
>>>> That's not actually the case.  The way Pax Exam currently runs our
>>>> tests is fundamentally part of the problem.  The test code is
>>>> dynamically wrapped in an actual bundle, using something like
>>>> tiny-bundles, and executed *within* the container itself. Pax
>>>> overrides runs with additional probes, overrides logging
>>>> infrastructure, etc.  Those nuances can often be the source of many of
>>>> the bugs (there are a ton of classloader implications, etc. -- IIRC,
>>>> this was one area where Arquillian was much, much worse).  There are
>>>> some benefits to that setup, but for Hibernate it mainly gets in the way.
>>>>
>>>> It *does* have a "server mode" where tests run outside of the
>>>> container, but I vaguely remember going down that path early on and
>>>> hitting a roadblock.  For the life of me, I can't remember the
>>>> specifics.  But my pushback here is that ultimately Docker might be
>>>> more preferable, giving us more of a real world scenario to do true
>>>> e2e tests without something else in the middle.
>>>>
>>>>> so I can't imagine using Karaf without the helpers being a walk in
>>>> the park; e.g. having to deal with HTTP operations comes with its own
>>>> baggage {dependencies, complexity, speed, .. } and generally more
>>>> stuff to maintain.
>>>>
>>>> I guess I respectfully disagree with that, but purely due to Karaf
>>>> features.  Our features.xml does most of the heavy lifting for us
>>>> w/r/t getting Hibernate provisioned.  The same would be true with the
>>>> test harness bundle/feature.  REST is simple and out-of-the-box thanks
>>>> to Karaf + CXF or Camel.  For other possible routes (Karaf commands),
>>>> we already have code available in our demo/quickstart projects.
>>>>
>>>>> Also: considered contributing to Pax?
>>>> Yes, of course.  But the fact that numerous Karaf *committers*
>>>> themselves have a long history of built-up frustration on it doesn't
>>>> leave me optimistic.  A couple of them had tried to pitch in at one
>>>> point and weren't able to get anywhere.
>>>>
>>>>> but it seems their developers really expect their users to be deeply
>>>> familiar with it all
>>>>
>>>> Absolutely!  But again, our struggles also come down to the
>>>> fundamental way Pax Exam works...
>>>>
>>>>
>>>> On 1/12/18 6:27 AM, Sanne Grinovero wrote:
>>>>> +1 to explore alternatives to Pax Exam, but I'd be wary of maintining
>>>>> our own test infrastructure.
>>>>>
>>>>> Pax Exam was just "helping" to deploy/run things in Karaf, so I can't
>>>>> imagine using Karaf without the helpers being a walk in the park; e.g.
>>>>> having to deal with HTTP operations comes with its own baggage
>>>>> {dependencies, complexity, speed, .. } and generally more stuff to
>>>>> maintain.
>>>>>
>>>>> So.. +1 to try out Arquillian or anything else. Or maybe you could
>>>>> start your own tool, but I'd prefer to see it in a separate repository
>>>>> :) e.g. a nice Gradle plugin so maybe you get more people helping?
>>>>>
>>>>> Also: considered contributing to Pax? My personal experience with it
>>>>> has always been a pain but if I had to try identify the reason, it was
>>>>> mostly caused by me being unfamiliar with Karaf and not having good
>>>>> clues to track down the real failure; maybe some minor error reporting
>>>>> improvements could make a big difference to its usability? Just
>>>>> saying, I don't feel like Pax is bad, but it seems their developers
>>>>> really expect their users to be deeply familiar with it all - feels
>>>>> like the typical case in which they could use some feedback and a
>>>>> hand.
>>>>>
>>>>> Thanks,
>>>>> Sanne
>>>>>
>>>>> On 12 January 2018 at 08:22, Gunnar Morling<gunnar at hibernate.org>  wrote:
>>>>>> Hi Brett,
>>>>>>
>>>>>> We also had our fair share of frustration with Pax Exam in HV, and I was
>>>>>> (more than once) at the point of dropping it.
>>>>>>
>>>>>> Docker could work, but as you say it's a bit of a heavy dependency, if not
>>>>>> required anyways. Not sure whether I'd like to add this as a prerequisite
>>>>>> for the HV build to be executed. And tests in separate profiles tend to be
>>>>>> "forgotten" in my experience.
>>>>>>
>>>>>> One other approach could be to use Arquillian's OSGi support (see
>>>>>> https://github.com/arquillian/arquillian-container-osgi), did you consider
>>>>>> to use that one as an alternative?
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> --Gunnar
>>>>>>
>>>>>>
>>>>>> 2018-01-12 3:34 GMT+01:00 Brett Meyer<brett at hibernate.org>:
>>>>>>
>>>>>>> <tired-rant>
>>>>>>>
>>>>>>> I'm fed up with Pax Exam and would love to replace it as the
>>>>>>> hibernate-osgi integration test harness.  Most of the Karaf committers
>>>>>>> I've been working with hate it more than I do.  Every single time we
>>>>>>> upgrade the Karaf version, something less-than-minor in hibernate-osgi,
>>>>>>> upgrade/change dependencies, or attempt to upgrade Pax Exam itself,
>>>>>>> there's some new obfuscated failure.  And no matter how much I pray, it
>>>>>>> refuses to let us get to the container logs to figure out what
>>>>>>> happened.  Tis a house of cards.
>>>>>>>
>>>>>>> </tired-rant>
>>>>>>>
>>>>>>> One alternative that recently came up elsewhere: use Docker to bootstrap
>>>>>>> the container, hit it with our features.xml, install a test bundle that
>>>>>>> exposes functionality externally (over HTTP, Karaf commands, etc), then
>>>>>>> hit the endpoints and run assertions.
>>>>>>>
>>>>>>> Pros: true "integration test", plain vanilla Karaf, direct access to all
>>>>>>> logs, easier to eventually support and test other containers.
>>>>>>>
>>>>>>> Cons: Need Docker installed for local test runs, probably safer to
>>>>>>> isolate the integration test behind a disabled-by-default Maven profile.
>>>>>>>
>>>>>>> Any gut reactions?
>>>>>>>
>>>>>>> OSGi is fun and I'm not at all bitter,
>>>>>>>
>>>>>>> -Brett-
>>>>>>>
>>>>>>> ;)
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> hibernate-dev mailing list
>>>>>>> hibernate-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>>>>> _______________________________________________
>>>>>> hibernate-dev mailing list
>>>>>> hibernate-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>>>> _______________________________________________
>>>>> hibernate-dev mailing list
>>>>> hibernate-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>> _______________________________________________
>>> hibernate-dev mailing list
>>> hibernate-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
>
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev