Re: [hibernate-dev] replace Pax Exam with Docker

Plus, for me, it's more a question of time. I only have a bit available for open source work these days, and I'd rather spend that knocking out some of the hibernate-osgi tasks we've had on our plate for a while. I unfortunately don't have anything left to contribute to Pax Exam itself, assuming that would even fix the problem. Even worse, we're barely using the integration tests for anything more than a simple smoke test at this point, since it seems like every time we touch it something new goes wrong. Looking for a more *consistent* solution -- need more confidence in the backbone. On 1/12/18 8:56 AM, Brett Meyer wrote: > > Sorry Gunnar/Sanne, should have clarified this first: > > We actually used Arquillian before Pax Exam, and the experience was > far worse (somewhat of a long story)... > > > Pax Exam was just "helping" to deploy/run things in Karaf, so I > can't imagine using Karaf without the helpers being a walk in the park > > That's not actually the case. The way Pax Exam currently runs our > tests is fundamentally part of the problem. The test code is > dynamically wrapped in an actual bundle, using something like > tiny-bundles, and executed *within* the container itself. Pax > overrides runs with additional probes, overrides logging > infrastructure, etc. Those nuances can often be the source of many of > the bugs (there are a ton of classloader implications, etc. -- IIRC, > this was one area where Arquillian was much, much worse). There are > some benefits to that setup, but for Hibernate it mainly gets in the way. > > It *does* have a "server mode" where tests run outside of the > container, but I vaguely remember going down that path early on and > hitting a roadblock. For the life of me, I can't remember the > specifics. But my pushback here is that ultimately Docker might be > more preferable, giving us more of a real world scenario to do true > e2e tests without something else in the middle. > > > so I can't imagine using Karaf without the helpers being a walk in > the park; e.g. having to deal with HTTP operations comes with its own > baggage {dependencies, complexity, speed, .. } and generally more > stuff to maintain. > > I guess I respectfully disagree with that, but purely due to Karaf > features. Our features.xml does most of the heavy lifting for us > w/r/t getting Hibernate provisioned. The same would be true with the > test harness bundle/feature. REST is simple and out-of-the-box thanks > to Karaf + CXF or Camel. For other possible routes (Karaf commands), > we already have code available in our demo/quickstart projects. > > > Also: considered contributing to Pax? > > Yes, of course. But the fact that numerous Karaf *committers* > themselves have a long history of built-up frustration on it doesn't > leave me optimistic. A couple of them had tried to pitch in at one > point and weren't able to get anywhere. > > > but it seems their developers really expect their users to be deeply > familiar with it all > > Absolutely! But again, our struggles also come down to the > fundamental way Pax Exam works... > > > On 1/12/18 6:27 AM, Sanne Grinovero wrote: >> +1 to explore alternatives to Pax Exam, but I'd be wary of maintining >> our own test infrastructure. >> >> Pax Exam was just "helping" to deploy/run things in Karaf, so I can't >> imagine using Karaf without the helpers being a walk in the park; e.g. >> having to deal with HTTP operations comes with its own baggage >> {dependencies, complexity, speed, .. } and generally more stuff to >> maintain. >> >> So.. +1 to try out Arquillian or anything else. Or maybe you could >> start your own tool, but I'd prefer to see it in a separate repository >> :) e.g. a nice Gradle plugin so maybe you get more people helping? >> >> Also: considered contributing to Pax? My personal experience with it >> has always been a pain but if I had to try identify the reason, it was >> mostly caused by me being unfamiliar with Karaf and not having good >> clues to track down the real failure; maybe some minor error reporting >> improvements could make a big difference to its usability? Just >> saying, I don't feel like Pax is bad, but it seems their developers >> really expect their users to be deeply familiar with it all - feels >> like the typical case in which they could use some feedback and a >> hand. >> >> Thanks, >> Sanne >> >> On 12 January 2018 at 08:22, Gunnar Morling&lt;gunnar(a)hibernate.org&gt; wrote: >>> Hi Brett, >>> >>> We also had our fair share of frustration with Pax Exam in HV, and I was >>> (more than once) at the point of dropping it. >>> >>> Docker could work, but as you say it's a bit of a heavy dependency, if not >>> required anyways. Not sure whether I'd like to add this as a prerequisite >>> for the HV build to be executed. And tests in separate profiles tend to be >>> "forgotten" in my experience. >>> >>> One other approach could be to use Arquillian's OSGi support (see >>> https://github.com/arquillian/arquillian-container-osgi), did you consider >>> to use that one as an alternative? >>> >>> Cheers, >>> >>> --Gunnar >>> >>> >>> 2018-01-12 3:34 GMT+01:00 Brett Meyer&lt;brett(a)hibernate.org&gt;: >>> >>>> <tired-rant> >>>> >>>> I'm fed up with Pax Exam and would love to replace it as the >>>> hibernate-osgi integration test harness. Most of the Karaf committers >>>> I've been working with hate it more than I do. Every single time we >>>> upgrade the Karaf version, something less-than-minor in hibernate-osgi, >>>> upgrade/change dependencies, or attempt to upgrade Pax Exam itself, >>>> there's some new obfuscated failure. And no matter how much I pray, it >>>> refuses to let us get to the container logs to figure out what >>>> happened. Tis a house of cards. >>>> >>>> </tired-rant> >>>> >>>> One alternative that recently came up elsewhere: use Docker to bootstrap >>>> the container, hit it with our features.xml, install a test bundle that >>>> exposes functionality externally (over HTTP, Karaf commands, etc), then >>>> hit the endpoints and run assertions. >>>> >>>> Pros: true "integration test", plain vanilla Karaf, direct access to all >>>> logs, easier to eventually support and test other containers. >>>> >>>> Cons: Need Docker installed for local test runs, probably safer to >>>> isolate the integration test behind a disabled-by-default Maven profile. >>>> >>>> Any gut reactions? >>>> >>>> OSGi is fun and I'm not at all bitter, >>>> >>>> -Brett- >>>> >>>> ;) >>>> >>>> >>>> _______________________________________________ >>>> hibernate-dev mailing list >>>> hibernate-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> _______________________________________________ >>> hibernate-dev mailing list >>> hibernate-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> _______________________________________________ >> hibernate-dev mailing list >> hibernate-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev > _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev