Hi Brett, We also had our fair share of frustration with Pax Exam in HV, and I was (more than once) at the point of dropping it. Docker could work, but as you say it's a bit of a heavy dependency, if not required anyways. Not sure whether I'd like to add this as a prerequisite for the HV build to be executed. And tests in separate profiles tend to be "forgotten" in my experience. One other approach could be to use Arquillian's OSGi support (see https://github.com/arquillian/arquillian-container-osgi), did you consider to use that one as an alternative? Cheers, --Gunnar 2018-01-12 3:34 GMT+01:00 Brett Meyer <brett(a)hibernate.org&gt;: > <tired-rant> > > I'm fed up with Pax Exam and would love to replace it as the > hibernate-osgi integration test harness. Most of the Karaf committers > I've been working with hate it more than I do. Every single time we > upgrade the Karaf version, something less-than-minor in hibernate-osgi, > upgrade/change dependencies, or attempt to upgrade Pax Exam itself, > there's some new obfuscated failure. And no matter how much I pray, it > refuses to let us get to the container logs to figure out what > happened. Tis a house of cards. > > </tired-rant> > > One alternative that recently came up elsewhere: use Docker to bootstrap > the container, hit it with our features.xml, install a test bundle that > exposes functionality externally (over HTTP, Karaf commands, etc), then > hit the endpoints and run assertions. > > Pros: true "integration test", plain vanilla Karaf, direct access to all > logs, easier to eventually support and test other containers. > > Cons: Need Docker installed for local test runs, probably safer to > isolate the integration test behind a disabled-by-default Maven profile. > > Any gut reactions? > > OSGi is fun and I'm not at all bitter, > > -Brett- > > ;) > > > _______________________________________________ > hibernate-dev mailing list > hibernate-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

Sorry Gunnar/Sanne, should have clarified this first: We actually used Arquillian before Pax Exam, and the experience was far worse (somewhat of a long story)... > Pax Exam was just "helping" to deploy/run things in Karaf, so I can't imagine using Karaf without the helpers being a walk in the park That's not actually the case.  The way Pax Exam currently runs our tests is fundamentally part of the problem.  The test code is dynamically wrapped in an actual bundle, using something like tiny-bundles, and executed *within* the container itself. Pax overrides runs with additional probes, overrides logging infrastructure, etc.  Those nuances can often be the source of many of the bugs (there are a ton of classloader implications, etc. -- IIRC, this was one area where Arquillian was much, much worse).  There are some benefits to that setup, but for Hibernate it mainly gets in the way. It *does* have a "server mode" where tests run outside of the container, but I vaguely remember going down that path early on and hitting a roadblock.  For the life of me, I can't remember the specifics.  But my pushback here is that ultimately Docker might be more preferable, giving us more of a real world scenario to do true e2e tests without something else in the middle. > so I can't imagine using Karaf without the helpers being a walk in the park; e.g. having to deal with HTTP operations comes with its own baggage {dependencies, complexity, speed, .. } and generally more stuff to maintain. I guess I respectfully disagree with that, but purely due to Karaf features.  Our features.xml does most of the heavy lifting for us w/r/t getting Hibernate provisioned.  The same would be true with the test harness bundle/feature.  REST is simple and out-of-the-box thanks to Karaf + CXF or Camel.  For other possible routes (Karaf commands), we already have code available in our demo/quickstart projects. > Also: considered contributing to Pax? Yes, of course.  But the fact that numerous Karaf *committers* themselves have a long history of built-up frustration on it doesn't leave me optimistic.  A couple of them had tried to pitch in at one point and weren't able to get anywhere. > but it seems their developers really expect their users to be deeply familiar with it all Absolutely!  But again, our struggles also come down to the fundamental way Pax Exam works... On 1/12/18 6:27 AM, Sanne Grinovero wrote: > +1 to explore alternatives to Pax Exam, but I'd be wary of maintining > our own test infrastructure. > > Pax Exam was just "helping" to deploy/run things in Karaf, so I can't > imagine using Karaf without the helpers being a walk in the park; e.g. > having to deal with HTTP operations comes with its own baggage > {dependencies, complexity, speed, .. } and generally more stuff to > maintain. > > So.. +1 to try out Arquillian or anything else. Or maybe you could > start your own tool, but I'd prefer to see it in a separate repository > :) e.g. a nice Gradle plugin so maybe you get more people helping? > > Also: considered contributing to Pax? My personal experience with it > has always been a pain but if I had to try identify the reason, it was > mostly caused by me being unfamiliar with Karaf and not having good > clues to track down the real failure; maybe some minor error reporting > improvements could make a big difference to its usability? Just > saying, I don't feel like Pax is bad, but it seems their developers > really expect their users to be deeply familiar with it all - feels > like the typical case in which they could use some feedback and a > hand. > > Thanks, > Sanne > > On 12 January 2018 at 08:22, Gunnar Morling&lt;gunnar(a)hibernate.org&gt; wrote: >> Hi Brett, >> >> We also had our fair share of frustration with Pax Exam in HV, and I was >> (more than once) at the point of dropping it. >> >> Docker could work, but as you say it's a bit of a heavy dependency, if not >> required anyways. Not sure whether I'd like to add this as a prerequisite >> for the HV build to be executed. And tests in separate profiles tend to be >> "forgotten" in my experience. >> >> One other approach could be to use Arquillian's OSGi support (see >> https://github.com/arquillian/arquillian-container-osgi), did you consider >> to use that one as an alternative? >> >> Cheers, >> >> --Gunnar >> >> >> 2018-01-12 3:34 GMT+01:00 Brett Meyer&lt;brett(a)hibernate.org&gt;: >> >>> <tired-rant> >>> >>> I'm fed up with Pax Exam and would love to replace it as the >>> hibernate-osgi integration test harness. Most of the Karaf committers >>> I've been working with hate it more than I do. Every single time we >>> upgrade the Karaf version, something less-than-minor in hibernate-osgi, >>> upgrade/change dependencies, or attempt to upgrade Pax Exam itself, >>> there's some new obfuscated failure. And no matter how much I pray, it >>> refuses to let us get to the container logs to figure out what >>> happened. Tis a house of cards. >>> >>> </tired-rant> >>> >>> One alternative that recently came up elsewhere: use Docker to bootstrap >>> the container, hit it with our features.xml, install a test bundle that >>> exposes functionality externally (over HTTP, Karaf commands, etc), then >>> hit the endpoints and run assertions. >>> >>> Pros: true "integration test", plain vanilla Karaf, direct access to all >>> logs, easier to eventually support and test other containers. >>> >>> Cons: Need Docker installed for local test runs, probably safer to >>> isolate the integration test behind a disabled-by-default Maven profile. >>> >>> Any gut reactions? >>> >>> OSGi is fun and I'm not at all bitter, >>> >>> -Brett- >>> >>> ;) >>> >>> >>> _______________________________________________ >>> hibernate-dev mailing list >>> hibernate-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> _______________________________________________ >> hibernate-dev mailing list >> hibernate-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev > _______________________________________________ > hibernate-dev mailing list > hibernate-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev

Ok, looks like you really should start something new :) Hopefully many of those other annoyed Karaf developers will follow. On 12 January 2018 at 13:59, Brett Meyer <brett(a)hibernate.org&gt; wrote: > Plus, for me, it's more a question of time. I only have a bit available > for open source work these days, and I'd rather spend that knocking out > some of the hibernate-osgi tasks we've had on our plate for a while. I > unfortunately don't have anything left to contribute to Pax Exam itself, > assuming that would even fix the problem. > > Even worse, we're barely using the integration tests for anything more > than a simple smoke test at this point, since it seems like every time > we touch it something new goes wrong. Looking for a more *consistent* > solution -- need more confidence in the backbone. > > > On 1/12/18 8:56 AM, Brett Meyer wrote: >> Sorry Gunnar/Sanne, should have clarified this first: >> >> We actually used Arquillian before Pax Exam, and the experience was >> far worse (somewhat of a long story)... >> >>> Pax Exam was just "helping" to deploy/run things in Karaf, so I >> can't imagine using Karaf without the helpers being a walk in the park >> >> That's not actually the case. The way Pax Exam currently runs our >> tests is fundamentally part of the problem. The test code is >> dynamically wrapped in an actual bundle, using something like >> tiny-bundles, and executed *within* the container itself. Pax >> overrides runs with additional probes, overrides logging >> infrastructure, etc. Those nuances can often be the source of many of >> the bugs (there are a ton of classloader implications, etc. -- IIRC, >> this was one area where Arquillian was much, much worse). There are >> some benefits to that setup, but for Hibernate it mainly gets in the way. >> >> It *does* have a "server mode" where tests run outside of the >> container, but I vaguely remember going down that path early on and >> hitting a roadblock. For the life of me, I can't remember the >> specifics. But my pushback here is that ultimately Docker might be >> more preferable, giving us more of a real world scenario to do true >> e2e tests without something else in the middle. >> >>> so I can't imagine using Karaf without the helpers being a walk in >> the park; e.g. having to deal with HTTP operations comes with its own >> baggage {dependencies, complexity, speed, .. } and generally more >> stuff to maintain. >> >> I guess I respectfully disagree with that, but purely due to Karaf >> features. Our features.xml does most of the heavy lifting for us >> w/r/t getting Hibernate provisioned. The same would be true with the >> test harness bundle/feature. REST is simple and out-of-the-box thanks >> to Karaf + CXF or Camel. For other possible routes (Karaf commands), >> we already have code available in our demo/quickstart projects. >> >>> Also: considered contributing to Pax? >> Yes, of course. But the fact that numerous Karaf *committers* >> themselves have a long history of built-up frustration on it doesn't >> leave me optimistic. A couple of them had tried to pitch in at one >> point and weren't able to get anywhere. >> >>> but it seems their developers really expect their users to be deeply >> familiar with it all >> >> Absolutely! But again, our struggles also come down to the >> fundamental way Pax Exam works... >> >> >> On 1/12/18 6:27 AM, Sanne Grinovero wrote: >>> +1 to explore alternatives to Pax Exam, but I'd be wary of maintining >>> our own test infrastructure. >>> >>> Pax Exam was just "helping" to deploy/run things in Karaf, so I can't >>> imagine using Karaf without the helpers being a walk in the park; e.g. >>> having to deal with HTTP operations comes with its own baggage >>> {dependencies, complexity, speed, .. } and generally more stuff to >>> maintain. >>> >>> So.. +1 to try out Arquillian or anything else. Or maybe you could >>> start your own tool, but I'd prefer to see it in a separate repository >>> :) e.g. a nice Gradle plugin so maybe you get more people helping? >>> >>> Also: considered contributing to Pax? My personal experience with it >>> has always been a pain but if I had to try identify the reason, it was >>> mostly caused by me being unfamiliar with Karaf and not having good >>> clues to track down the real failure; maybe some minor error reporting >>> improvements could make a big difference to its usability? Just >>> saying, I don't feel like Pax is bad, but it seems their developers >>> really expect their users to be deeply familiar with it all - feels >>> like the typical case in which they could use some feedback and a >>> hand. >>> >>> Thanks, >>> Sanne >>> >>> On 12 January 2018 at 08:22, Gunnar Morling&lt;gunnar(a)hibernate.org&gt; wrote: >>>> Hi Brett, >>>> >>>> We also had our fair share of frustration with Pax Exam in HV, and I was >>>> (more than once) at the point of dropping it. >>>> >>>> Docker could work, but as you say it's a bit of a heavy dependency, if not >>>> required anyways. Not sure whether I'd like to add this as a prerequisite >>>> for the HV build to be executed. And tests in separate profiles tend to be >>>> "forgotten" in my experience. >>>> >>>> One other approach could be to use Arquillian's OSGi support (see >>>> https://github.com/arquillian/arquillian-container-osgi), did you consider >>>> to use that one as an alternative? >>>> >>>> Cheers, >>>> >>>> --Gunnar >>>> >>>> >>>> 2018-01-12 3:34 GMT+01:00 Brett Meyer&lt;brett(a)hibernate.org&gt;: >>>> >>>>> <tired-rant> >>>>> >>>>> I'm fed up with Pax Exam and would love to replace it as the >>>>> hibernate-osgi integration test harness. Most of the Karaf committers >>>>> I've been working with hate it more than I do. Every single time we >>>>> upgrade the Karaf version, something less-than-minor in hibernate-osgi, >>>>> upgrade/change dependencies, or attempt to upgrade Pax Exam itself, >>>>> there's some new obfuscated failure. And no matter how much I pray, it >>>>> refuses to let us get to the container logs to figure out what >>>>> happened. Tis a house of cards. >>>>> >>>>> </tired-rant> >>>>> >>>>> One alternative that recently came up elsewhere: use Docker to bootstrap >>>>> the container, hit it with our features.xml, install a test bundle that >>>>> exposes functionality externally (over HTTP, Karaf commands, etc), then >>>>> hit the endpoints and run assertions. >>>>> >>>>> Pros: true "integration test", plain vanilla Karaf, direct access to all >>>>> logs, easier to eventually support and test other containers. >>>>> >>>>> Cons: Need Docker installed for local test runs, probably safer to >>>>> isolate the integration test behind a disabled-by-default Maven profile. >>>>> >>>>> Any gut reactions? >>>>> >>>>> OSGi is fun and I'm not at all bitter, >>>>> >>>>> -Brett- >>>>> >>>>> ;) >>>>> >>>>> >>>>> _______________________________________________ >>>>> hibernate-dev mailing list >>>>> hibernate-dev(a)lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>>> _______________________________________________ >>>> hibernate-dev mailing list >>>> hibernate-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> _______________________________________________ >>> hibernate-dev mailing list >>> hibernate-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > _______________________________________________ > hibernate-dev mailing list > hibernate-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev

On 12 January 2018 at 17:32, Brett Meyer <brett(a)hibernate.org&gt; wrote: > If I don't have time to contribute to Pax Exam, I certainly don't have > time to start a new project haha... > > And realistically, that "something new" would likely involve containers > anyway. > > At this point, mostly a question of 1) status quo, 2) Docker (or any > other container-based solution), or 3) try screwing around with Pax Exam > in "server-only" mode (but I don't have high hopes there). Sure. Docker is now available on the CI slaves too, so that's not a problem. The only annoyance is that the whole ORM team - and anyone contributing - would need to have Docker as well, but that doesn't seem too bad to me... and was likely bound to happen for other tools :) Steve, Chris and Andrea? Ok with that? Maybe you have Docker running already? > > > On 1/12/18 12:27 PM, Sanne Grinovero wrote: >> Ok, looks like you really should start something new :) >> >> Hopefully many of those other annoyed Karaf developers will follow. >> >> On 12 January 2018 at 13:59, Brett Meyer <brett(a)hibernate.org&gt; wrote: >>> Plus, for me, it's more a question of time. I only have a bit available >>> for open source work these days, and I'd rather spend that knocking out >>> some of the hibernate-osgi tasks we've had on our plate for a while. I >>> unfortunately don't have anything left to contribute to Pax Exam itself, >>> assuming that would even fix the problem. >>> >>> Even worse, we're barely using the integration tests for anything more >>> than a simple smoke test at this point, since it seems like every time >>> we touch it something new goes wrong. Looking for a more *consistent* >>> solution -- need more confidence in the backbone. >>> >>> >>> On 1/12/18 8:56 AM, Brett Meyer wrote: >>>> Sorry Gunnar/Sanne, should have clarified this first: >>>> >>>> We actually used Arquillian before Pax Exam, and the experience was >>>> far worse (somewhat of a long story)... >>>> >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, so I >>>> can't imagine using Karaf without the helpers being a walk in the park >>>> >>>> That's not actually the case. The way Pax Exam currently runs our >>>> tests is fundamentally part of the problem. The test code is >>>> dynamically wrapped in an actual bundle, using something like >>>> tiny-bundles, and executed *within* the container itself. Pax >>>> overrides runs with additional probes, overrides logging >>>> infrastructure, etc. Those nuances can often be the source of many of >>>> the bugs (there are a ton of classloader implications, etc. -- IIRC, >>>> this was one area where Arquillian was much, much worse). There are >>>> some benefits to that setup, but for Hibernate it mainly gets in the way. >>>> >>>> It *does* have a "server mode" where tests run outside of the >>>> container, but I vaguely remember going down that path early on and >>>> hitting a roadblock. For the life of me, I can't remember the >>>> specifics. But my pushback here is that ultimately Docker might be >>>> more preferable, giving us more of a real world scenario to do true >>>> e2e tests without something else in the middle. >>>> >>>>> so I can't imagine using Karaf without the helpers being a walk in >>>> the park; e.g. having to deal with HTTP operations comes with its own >>>> baggage {dependencies, complexity, speed, .. } and generally more >>>> stuff to maintain. >>>> >>>> I guess I respectfully disagree with that, but purely due to Karaf >>>> features. Our features.xml does most of the heavy lifting for us >>>> w/r/t getting Hibernate provisioned. The same would be true with the >>>> test harness bundle/feature. REST is simple and out-of-the-box thanks >>>> to Karaf + CXF or Camel. For other possible routes (Karaf commands), >>>> we already have code available in our demo/quickstart projects. >>>> >>>>> Also: considered contributing to Pax? >>>> Yes, of course. But the fact that numerous Karaf *committers* >>>> themselves have a long history of built-up frustration on it doesn't >>>> leave me optimistic. A couple of them had tried to pitch in at one >>>> point and weren't able to get anywhere. >>>> >>>>> but it seems their developers really expect their users to be deeply >>>> familiar with it all >>>> >>>> Absolutely! But again, our struggles also come down to the >>>> fundamental way Pax Exam works... >>>> >>>> >>>> On 1/12/18 6:27 AM, Sanne Grinovero wrote: >>>>> +1 to explore alternatives to Pax Exam, but I'd be wary of maintining >>>>> our own test infrastructure. >>>>> >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, so I can't >>>>> imagine using Karaf without the helpers being a walk in the park; e.g. >>>>> having to deal with HTTP operations comes with its own baggage >>>>> {dependencies, complexity, speed, .. } and generally more stuff to >>>>> maintain. >>>>> >>>>> So.. +1 to try out Arquillian or anything else. Or maybe you could >>>>> start your own tool, but I'd prefer to see it in a separate repository >>>>> :) e.g. a nice Gradle plugin so maybe you get more people helping? >>>>> >>>>> Also: considered contributing to Pax? My personal experience with it >>>>> has always been a pain but if I had to try identify the reason, it was >>>>> mostly caused by me being unfamiliar with Karaf and not having good >>>>> clues to track down the real failure; maybe some minor error reporting >>>>> improvements could make a big difference to its usability? Just >>>>> saying, I don't feel like Pax is bad, but it seems their developers >>>>> really expect their users to be deeply familiar with it all - feels >>>>> like the typical case in which they could use some feedback and a >>>>> hand. >>>>> >>>>> Thanks, >>>>> Sanne >>>>> >>>>> On 12 January 2018 at 08:22, Gunnar Morling&lt;gunnar(a)hibernate.org&gt; wrote: >>>>>> Hi Brett, >>>>>> >>>>>> We also had our fair share of frustration with Pax Exam in HV, and I was >>>>>> (more than once) at the point of dropping it. >>>>>> >>>>>> Docker could work, but as you say it's a bit of a heavy dependency, if not >>>>>> required anyways. Not sure whether I'd like to add this as a prerequisite >>>>>> for the HV build to be executed. And tests in separate profiles tend to be >>>>>> "forgotten" in my experience. >>>>>> >>>>>> One other approach could be to use Arquillian's OSGi support (see >>>>>> https://github.com/arquillian/arquillian-container-osgi), did you consider >>>>>> to use that one as an alternative? >>>>>> >>>>>> Cheers, >>>>>> >>>>>> --Gunnar >>>>>> >>>>>> >>>>>> 2018-01-12 3:34 GMT+01:00 Brett Meyer&lt;brett(a)hibernate.org&gt;: >>>>>> >>>>>>> <tired-rant> >>>>>>> >>>>>>> I'm fed up with Pax Exam and would love to replace it as the >>>>>>> hibernate-osgi integration test harness. Most of the Karaf committers >>>>>>> I've been working with hate it more than I do. Every single time we >>>>>>> upgrade the Karaf version, something less-than-minor in hibernate-osgi, >>>>>>> upgrade/change dependencies, or attempt to upgrade Pax Exam itself, >>>>>>> there's some new obfuscated failure. And no matter how much I pray, it >>>>>>> refuses to let us get to the container logs to figure out what >>>>>>> happened. Tis a house of cards. >>>>>>> >>>>>>> </tired-rant> >>>>>>> >>>>>>> One alternative that recently came up elsewhere: use Docker to bootstrap >>>>>>> the container, hit it with our features.xml, install a test bundle that >>>>>>> exposes functionality externally (over HTTP, Karaf commands, etc), then >>>>>>> hit the endpoints and run assertions. >>>>>>> >>>>>>> Pros: true "integration test", plain vanilla Karaf, direct access to all >>>>>>> logs, easier to eventually support and test other containers. >>>>>>> >>>>>>> Cons: Need Docker installed for local test runs, probably safer to >>>>>>> isolate the integration test behind a disabled-by-default Maven profile. >>>>>>> >>>>>>> Any gut reactions? >>>>>>> >>>>>>> OSGi is fun and I'm not at all bitter, >>>>>>> >>>>>>> -Brett- >>>>>>> >>>>>>> ;) >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> hibernate-dev mailing list >>>>>>> hibernate-dev(a)lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>>>>> _______________________________________________ >>>>>> hibernate-dev mailing list >>>>>> hibernate-dev(a)lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>>>> _______________________________________________ >>>>> hibernate-dev mailing list >>>>> hibernate-dev(a)lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> _______________________________________________ >>> hibernate-dev mailing list >>> hibernate-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > _______________________________________________ > hibernate-dev mailing list > hibernate-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

On 12 January 2018 at 17:32, Brett Meyer <brett(a)hibernate.org&gt; wrote: > If I don't have time to contribute to Pax Exam, I certainly don't have > time to start a new project haha... > > And realistically, that "something new" would likely involve containers > anyway. > > At this point, mostly a question of 1) status quo, 2) Docker (or any > other container-based solution), or 3) try screwing around with Pax Exam > in "server-only" mode (but I don't have high hopes there). Sure. Docker is now available on the CI slaves too, so that's not a problem. The only annoyance is that the whole ORM team - and anyone contributing - would need to have Docker as well, but that doesn't seem too bad to me... and was likely bound to happen for other tools :) Steve, Chris and Andrea? Ok with that? Maybe you have Docker running already? > > > On 1/12/18 12:27 PM, Sanne Grinovero wrote: >> Ok, looks like you really should start something new :) >> >> Hopefully many of those other annoyed Karaf developers will follow. >> >> On 12 January 2018 at 13:59, Brett Meyer <brett(a)hibernate.org&gt; wrote: >>> Plus, for me, it's more a question of time. I only have a bit available >>> for open source work these days, and I'd rather spend that knocking out >>> some of the hibernate-osgi tasks we've had on our plate for a while. I >>> unfortunately don't have anything left to contribute to Pax Exam itself, >>> assuming that would even fix the problem. >>> >>> Even worse, we're barely using the integration tests for anything more >>> than a simple smoke test at this point, since it seems like every time >>> we touch it something new goes wrong. Looking for a more *consistent* >>> solution -- need more confidence in the backbone. >>> >>> >>> On 1/12/18 8:56 AM, Brett Meyer wrote: >>>> Sorry Gunnar/Sanne, should have clarified this first: >>>> >>>> We actually used Arquillian before Pax Exam, and the experience was >>>> far worse (somewhat of a long story)... >>>> >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, so I >>>> can't imagine using Karaf without the helpers being a walk in the park >>>> >>>> That's not actually the case. The way Pax Exam currently runs our >>>> tests is fundamentally part of the problem. The test code is >>>> dynamically wrapped in an actual bundle, using something like >>>> tiny-bundles, and executed *within* the container itself. Pax >>>> overrides runs with additional probes, overrides logging >>>> infrastructure, etc. Those nuances can often be the source of many of >>>> the bugs (there are a ton of classloader implications, etc. -- IIRC, >>>> this was one area where Arquillian was much, much worse). There are >>>> some benefits to that setup, but for Hibernate it mainly gets in the way. >>>> >>>> It *does* have a "server mode" where tests run outside of the >>>> container, but I vaguely remember going down that path early on and >>>> hitting a roadblock. For the life of me, I can't remember the >>>> specifics. But my pushback here is that ultimately Docker might be >>>> more preferable, giving us more of a real world scenario to do true >>>> e2e tests without something else in the middle. >>>> >>>>> so I can't imagine using Karaf without the helpers being a walk in >>>> the park; e.g. having to deal with HTTP operations comes with its own >>>> baggage {dependencies, complexity, speed, .. } and generally more >>>> stuff to maintain. >>>> >>>> I guess I respectfully disagree with that, but purely due to Karaf >>>> features. Our features.xml does most of the heavy lifting for us >>>> w/r/t getting Hibernate provisioned. The same would be true with the >>>> test harness bundle/feature. REST is simple and out-of-the-box thanks >>>> to Karaf + CXF or Camel. For other possible routes (Karaf commands), >>>> we already have code available in our demo/quickstart projects. >>>> >>>>> Also: considered contributing to Pax? >>>> Yes, of course. But the fact that numerous Karaf *committers* >>>> themselves have a long history of built-up frustration on it doesn't >>>> leave me optimistic. A couple of them had tried to pitch in at one >>>> point and weren't able to get anywhere. >>>> >>>>> but it seems their developers really expect their users to be deeply >>>> familiar with it all >>>> >>>> Absolutely! But again, our struggles also come down to the >>>> fundamental way Pax Exam works... >>>> >>>> >>>> On 1/12/18 6:27 AM, Sanne Grinovero wrote: >>>>> +1 to explore alternatives to Pax Exam, but I'd be wary of maintining >>>>> our own test infrastructure. >>>>> >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, so I can't >>>>> imagine using Karaf without the helpers being a walk in the park; e.g. >>>>> having to deal with HTTP operations comes with its own baggage >>>>> {dependencies, complexity, speed, .. } and generally more stuff to >>>>> maintain. >>>>> >>>>> So.. +1 to try out Arquillian or anything else. Or maybe you could >>>>> start your own tool, but I'd prefer to see it in a separate repository >>>>> :) e.g. a nice Gradle plugin so maybe you get more people helping? >>>>> >>>>> Also: considered contributing to Pax? My personal experience with it >>>>> has always been a pain but if I had to try identify the reason, it was >>>>> mostly caused by me being unfamiliar with Karaf and not having good >>>>> clues to track down the real failure; maybe some minor error reporting >>>>> improvements could make a big difference to its usability? Just >>>>> saying, I don't feel like Pax is bad, but it seems their developers >>>>> really expect their users to be deeply familiar with it all - feels >>>>> like the typical case in which they could use some feedback and a >>>>> hand. >>>>> >>>>> Thanks, >>>>> Sanne >>>>> >>>>> On 12 January 2018 at 08:22, Gunnar Morling&lt;gunnar(a)hibernate.org&gt; wrote: >>>>>> Hi Brett, >>>>>> >>>>>> We also had our fair share of frustration with Pax Exam in HV, and I was >>>>>> (more than once) at the point of dropping it. >>>>>> >>>>>> Docker could work, but as you say it's a bit of a heavy dependency, if not >>>>>> required anyways. Not sure whether I'd like to add this as a prerequisite >>>>>> for the HV build to be executed. And tests in separate profiles tend to be >>>>>> "forgotten" in my experience. >>>>>> >>>>>> One other approach could be to use Arquillian's OSGi support (see >>>>>> https://github.com/arquillian/arquillian-container-osgi), did you consider >>>>>> to use that one as an alternative? >>>>>> >>>>>> Cheers, >>>>>> >>>>>> --Gunnar >>>>>> >>>>>> >>>>>> 2018-01-12 3:34 GMT+01:00 Brett Meyer&lt;brett(a)hibernate.org&gt;: >>>>>> >>>>>>> <tired-rant> >>>>>>> >>>>>>> I'm fed up with Pax Exam and would love to replace it as the >>>>>>> hibernate-osgi integration test harness. Most of the Karaf committers >>>>>>> I've been working with hate it more than I do. Every single time we >>>>>>> upgrade the Karaf version, something less-than-minor in hibernate-osgi, >>>>>>> upgrade/change dependencies, or attempt to upgrade Pax Exam itself, >>>>>>> there's some new obfuscated failure. And no matter how much I pray, it >>>>>>> refuses to let us get to the container logs to figure out what >>>>>>> happened. Tis a house of cards. >>>>>>> >>>>>>> </tired-rant> >>>>>>> >>>>>>> One alternative that recently came up elsewhere: use Docker to bootstrap >>>>>>> the container, hit it with our features.xml, install a test bundle that >>>>>>> exposes functionality externally (over HTTP, Karaf commands, etc), then >>>>>>> hit the endpoints and run assertions. >>>>>>> >>>>>>> Pros: true "integration test", plain vanilla Karaf, direct access to all >>>>>>> logs, easier to eventually support and test other containers. >>>>>>> >>>>>>> Cons: Need Docker installed for local test runs, probably safer to >>>>>>> isolate the integration test behind a disabled-by-default Maven profile. >>>>>>> >>>>>>> Any gut reactions? >>>>>>> >>>>>>> OSGi is fun and I'm not at all bitter, >>>>>>> >>>>>>> -Brett- >>>>>>> >>>>>>> ;) >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> hibernate-dev mailing list >>>>>>> hibernate-dev(a)lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>>>>> _______________________________________________ >>>>>> hibernate-dev mailing list >>>>>> hibernate-dev(a)lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>>>> _______________________________________________ >>>>> hibernate-dev mailing list >>>>> hibernate-dev(a)lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> _______________________________________________ >>> hibernate-dev mailing list >>> hibernate-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > _______________________________________________ > hibernate-dev mailing list > hibernate-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

I guess the way I'm looking at this is Docker will be primarily used by Jenkins, and myself or anyone working directly on hibernate-osgi itself. Otherwise, it'll be disabled by default and hidden behind a profile. We'll make sure that most contributors running the entire Hibernate test suite won't be affected... On 1/12/18 1:13 PM, andrea boriero wrote: > I already have Docker running on my machine, so it seems not a big > issue for me,but not sure about the impact for others. > > Anyway It's worth giving a try. > > On 12 January 2018 at 17:54, Sanne Grinovero <sanne(a)hibernate.org > <mailto:sanne@hibernate.org>> wrote: > > On 12 January 2018 at 17:32, Brett Meyer <brett(a)hibernate.org > <mailto:brett@hibernate.org>> wrote: > > If I don't have time to contribute to Pax Exam, I certainly > don't have > > time to start a new project haha... > > > > And realistically, that "something new" would likely involve > containers > > anyway. > > > > At this point, mostly a question of 1) status quo, 2) Docker (or any > > other container-based solution), or 3) try screwing around with > Pax Exam > > in "server-only" mode (but I don't have high hopes there). > > Sure. Docker is now available on the CI slaves too, so that's not > a problem. > > The only annoyance is that the whole ORM team - and anyone > contributing - would need to have Docker as well, but that doesn't > seem too bad to me... and was likely bound to happen for other tools > :) > > Steve, Chris and Andrea? Ok with that? Maybe you have Docker > running already? > > > > > > > On 1/12/18 12:27 PM, Sanne Grinovero wrote: > >> Ok, looks like you really should start something new :) > >> > >> Hopefully many of those other annoyed Karaf developers will follow. > >> > >> On 12 January 2018 at 13:59, Brett Meyer <brett(a)hibernate.org > <mailto:brett@hibernate.org>> wrote: > >>> Plus, for me, it's more a question of time. I only have a bit > available > >>> for open source work these days, and I'd rather spend that > knocking out > >>> some of the hibernate-osgi tasks we've had on our plate for a > while. I > >>> unfortunately don't have anything left to contribute to Pax > Exam itself, > >>> assuming that would even fix the problem. > >>> > >>> Even worse, we're barely using the integration tests for > anything more > >>> than a simple smoke test at this point, since it seems like > every time > >>> we touch it something new goes wrong. Looking for a more > *consistent* > >>> solution -- need more confidence in the backbone. > >>> > >>> > >>> On 1/12/18 8:56 AM, Brett Meyer wrote: > >>>> Sorry Gunnar/Sanne, should have clarified this first: > >>>> > >>>> We actually used Arquillian before Pax Exam, and the > experience was > >>>> far worse (somewhat of a long story)... > >>>> > >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, so I > >>>> can't imagine using Karaf without the helpers being a walk in > the park > >>>> > >>>> That's not actually the case. The way Pax Exam currently > runs our > >>>> tests is fundamentally part of the problem. The test code is > >>>> dynamically wrapped in an actual bundle, using something like > >>>> tiny-bundles, and executed *within* the container itself. Pax > >>>> overrides runs with additional probes, overrides logging > >>>> infrastructure, etc. Those nuances can often be the source > of many of > >>>> the bugs (there are a ton of classloader implications, etc. > -- IIRC, > >>>> this was one area where Arquillian was much, much worse). > There are > >>>> some benefits to that setup, but for Hibernate it mainly gets > in the way. > >>>> > >>>> It *does* have a "server mode" where tests run outside of the > >>>> container, but I vaguely remember going down that path early > on and > >>>> hitting a roadblock. For the life of me, I can't remember the > >>>> specifics. But my pushback here is that ultimately Docker > might be > >>>> more preferable, giving us more of a real world scenario to > do true > >>>> e2e tests without something else in the middle. > >>>> > >>>>> so I can't imagine using Karaf without the helpers being a > walk in > >>>> the park; e.g. having to deal with HTTP operations comes with > its own > >>>> baggage {dependencies, complexity, speed, .. } and generally more > >>>> stuff to maintain. > >>>> > >>>> I guess I respectfully disagree with that, but purely due to > Karaf > >>>> features. Our features.xml does most of the heavy lifting for us > >>>> w/r/t getting Hibernate provisioned. The same would be true > with the > >>>> test harness bundle/feature. REST is simple and > out-of-the-box thanks > >>>> to Karaf + CXF or Camel. For other possible routes (Karaf > commands), > >>>> we already have code available in our demo/quickstart projects. > >>>> > >>>>> Also: considered contributing to Pax? > >>>> Yes, of course. But the fact that numerous Karaf *committers* > >>>> themselves have a long history of built-up frustration on it > doesn't > >>>> leave me optimistic. A couple of them had tried to pitch in > at one > >>>> point and weren't able to get anywhere. > >>>> > >>>>> but it seems their developers really expect their users to > be deeply > >>>> familiar with it all > >>>> > >>>> Absolutely! But again, our struggles also come down to the > >>>> fundamental way Pax Exam works... > >>>> > >>>> > >>>> On 1/12/18 6:27 AM, Sanne Grinovero wrote: > >>>>> +1 to explore alternatives to Pax Exam, but I'd be wary of > maintining > >>>>> our own test infrastructure. > >>>>> > >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, > so I can't > >>>>> imagine using Karaf without the helpers being a walk in the > park; e.g. > >>>>> having to deal with HTTP operations comes with its own baggage > >>>>> {dependencies, complexity, speed, .. } and generally more > stuff to > >>>>> maintain. > >>>>> > >>>>> So.. +1 to try out Arquillian or anything else. Or maybe you > could > >>>>> start your own tool, but I'd prefer to see it in a separate > repository > >>>>> :) e.g. a nice Gradle plugin so maybe you get more people > helping? > >>>>> > >>>>> Also: considered contributing to Pax? My personal experience > with it > >>>>> has always been a pain but if I had to try identify the > reason, it was > >>>>> mostly caused by me being unfamiliar with Karaf and not > having good > >>>>> clues to track down the real failure; maybe some minor error > reporting > >>>>> improvements could make a big difference to its usability? Just > >>>>> saying, I don't feel like Pax is bad, but it seems their > developers > >>>>> really expect their users to be deeply familiar with it all > - feels > >>>>> like the typical case in which they could use some feedback > and a > >>>>> hand. > >>>>> > >>>>> Thanks, > >>>>> Sanne > >>>>> > >>>>> On 12 January 2018 at 08:22, Gunnar > Morling&lt;gunnar(a)hibernate.org <mailto:gunnar@hibernate.org>> wrote: > >>>>>> Hi Brett, > >>>>>> > >>>>>> We also had our fair share of frustration with Pax Exam in > HV, and I was > >>>>>> (more than once) at the point of dropping it. > >>>>>> > >>>>>> Docker could work, but as you say it's a bit of a heavy > dependency, if not > >>>>>> required anyways. Not sure whether I'd like to add this as > a prerequisite > >>>>>> for the HV build to be executed. And tests in separate > profiles tend to be > >>>>>> "forgotten" in my experience. > >>>>>> > >>>>>> One other approach could be to use Arquillian's OSGi > support (see > >>>>>> https://github.com/arquillian/arquillian-container-osgi > <https://github.com/arquillian/arquillian-container-osgi>), did > you consider > >>>>>> to use that one as an alternative? > >>>>>> > >>>>>> Cheers, > >>>>>> > >>>>>> --Gunnar > >>>>>> > >>>>>> > >>>>>> 2018-01-12 3:34 GMT+01:00 Brett Meyer&lt;brett(a)hibernate.org > <mailto:brett@hibernate.org>>: > >>>>>> > >>>>>>> <tired-rant> > >>>>>>> > >>>>>>> I'm fed up with Pax Exam and would love to replace it as the > >>>>>>> hibernate-osgi integration test harness. Most of the > Karaf committers > >>>>>>> I've been working with hate it more than I do. Every > single time we > >>>>>>> upgrade the Karaf version, something less-than-minor in > hibernate-osgi, > >>>>>>> upgrade/change dependencies, or attempt to upgrade Pax > Exam itself, > >>>>>>> there's some new obfuscated failure. And no matter how > much I pray, it > >>>>>>> refuses to let us get to the container logs to figure out what > >>>>>>> happened. Tis a house of cards. > >>>>>>> > >>>>>>> </tired-rant> > >>>>>>> > >>>>>>> One alternative that recently came up elsewhere: use > Docker to bootstrap > >>>>>>> the container, hit it with our features.xml, install a > test bundle that > >>>>>>> exposes functionality externally (over HTTP, Karaf > commands, etc), then > >>>>>>> hit the endpoints and run assertions. > >>>>>>> > >>>>>>> Pros: true "integration test", plain vanilla Karaf, direct > access to all > >>>>>>> logs, easier to eventually support and test other containers. > >>>>>>> > >>>>>>> Cons: Need Docker installed for local test runs, probably > safer to > >>>>>>> isolate the integration test behind a disabled-by-default > Maven profile. > >>>>>>> > >>>>>>> Any gut reactions? > >>>>>>> > >>>>>>> OSGi is fun and I'm not at all bitter, > >>>>>>> > >>>>>>> -Brett- > >>>>>>> > >>>>>>> ;) > >>>>>>> > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> hibernate-dev mailing list > >>>>>>> hibernate-dev(a)lists.jboss.org > <mailto:hibernate-dev@lists.jboss.org> > >>>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > >>>>>> _______________________________________________ > >>>>>> hibernate-dev mailing list > >>>>>> hibernate-dev(a)lists.jboss.org > <mailto:hibernate-dev@lists.jboss.org> > >>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > >>>>> _______________________________________________ > >>>>> hibernate-dev mailing list > >>>>> hibernate-dev(a)lists.jboss.org > <mailto:hibernate-dev@lists.jboss.org> > >>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > >>> _______________________________________________ > >>> hibernate-dev mailing list > >>> hibernate-dev(a)lists.jboss.org > <mailto:hibernate-dev@lists.jboss.org> > >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > > > > > > _______________________________________________ > > hibernate-dev mailing list > > hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists. jboss.org> > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > _______________________________________________ > hibernate-dev mailing list > hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/hibernate-dev > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > > _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

I believe most of this is already handled for various build tools

I believe most of this is already handled for various build tools (like the gradle docker plugin) and simplified by docker image repos (bintray, e.g.). But this is a good point.  I'd be interested to see if you've done this before Brett.. On Fri, Jan 12, 2018 at 2:56 PM Gunnar Morling <gunnar(a)hibernate.org <mailto:gunnar@hibernate.org>> wrote: Brett, What's still unclear to me is when going the Docker route, won't you still need some code which deploys your tests to Karaf, runs them there and fetches the test results, so e.g. your Gradle build will fail if there are test failures? Would you envision to write these bits yourself? And wouldn't this amount to re-implementing PaxExam yourself? Seems I'm still missing a piece of the story :) Cheers, --Gunnar 2018-01-12 19:16 GMT+01:00 Brett Meyer <brett(a)hibernate.org <mailto:brett@hibernate.org>>: > I guess the way I'm looking at this is Docker will be primarily used by > Jenkins, and myself or anyone working directly on hibernate-osgi > itself.  Otherwise, it'll be disabled by default and hidden behind a > profile.  We'll make sure that most contributors running the entire > Hibernate test suite won't be affected... > > > On 1/12/18 1:13 PM, andrea boriero wrote: > > I already have Docker running on my machine, so it seems not a big > > issue for me,but not sure about the impact for others. > > > > Anyway It's worth giving a try. > > > > On 12 January 2018 at 17:54, Sanne Grinovero <sanne(a)hibernate.org <mailto:sanne@hibernate.org> > > <mailto:sanne@hibernate.org <mailto:sanne@hibernate.org>>> wrote: > > > >     On 12 January 2018 at 17:32, Brett Meyer <brett(a)hibernate.org <mailto:brett@hibernate.org> > >     <mailto:brett@hibernate.org <mailto:brett@hibernate.org>>> wrote: > >     > If I don't have time to contribute to Pax Exam, I certainly > >     don't have > >     > time to start a new project haha... > >     > > >     > And realistically, that "something new" would likely involve > >     containers > >     > anyway. > >     > > >     > At this point, mostly a question of 1) status quo, 2) Docker (or > any > >     > other container-based solution), or 3) try screwing around with > >     Pax Exam > >     > in "server-only" mode (but I don't have high hopes there). > > > >     Sure. Docker is now available on the CI slaves too, so that's not > >     a problem. > > > >     The only annoyance is that the whole ORM team - and anyone > >     contributing - would need to have Docker as well, but that doesn't > >     seem too bad to me... and was likely bound to happen for other tools > >     :) > > > >     Steve, Chris and Andrea? Ok with that? Maybe you have Docker > >     running already? > > > >     > > >     > > >     > On 1/12/18 12:27 PM, Sanne Grinovero wrote: > >     >> Ok, looks like you really should start something new :) > >     >> > >     >> Hopefully many of those other annoyed Karaf developers will > follow. > >     >> > >     >> On 12 January 2018 at 13:59, Brett Meyer <brett(a)hibernate.org <mailto:brett@hibernate.org> > >     <mailto:brett@hibernate.org <mailto:brett@hibernate.org>>> wrote: > >     >>> Plus, for me, it's more a question of time.  I only have a bit > >     available > >     >>> for open source work these days, and I'd rather spend that > >     knocking out > >     >>> some of the hibernate-osgi tasks we've had on our plate for a > >     while.  I > >     >>> unfortunately don't have anything left to contribute to Pax > >     Exam itself, > >     >>> assuming that would even fix the problem. > >     >>> > >     >>> Even worse, we're barely using the integration tests for > >     anything more > >     >>> than a simple smoke test at this point, since it seems like > >     every time > >     >>> we touch it something new goes wrong. Looking for a more > >     *consistent* > >     >>> solution -- need more confidence in the backbone. > >     >>> > >     >>> > >     >>> On 1/12/18 8:56 AM, Brett Meyer wrote: > >     >>>> Sorry Gunnar/Sanne, should have clarified this first: > >     >>>> > >     >>>> We actually used Arquillian before Pax Exam, and the > >     experience was > >     >>>> far worse (somewhat of a long story)... > >     >>>> > >     >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, so I > >     >>>> can't imagine using Karaf without the helpers being a walk in > >     the park > >     >>>> > >     >>>> That's not actually the case. The way Pax Exam currently > >     runs our > >     >>>> tests is fundamentally part of the problem.  The test code is > >     >>>> dynamically wrapped in an actual bundle, using something like > >     >>>> tiny-bundles, and executed *within* the container itself. Pax > >     >>>> overrides runs with additional probes, overrides logging > >     >>>> infrastructure, etc.  Those nuances can often be the source > >     of many of > >     >>>> the bugs (there are a ton of classloader implications, etc. > >     -- IIRC, > >     >>>> this was one area where Arquillian was much, much worse). > >     There are > >     >>>> some benefits to that setup, but for Hibernate it mainly gets > >     in the way. > >     >>>> > >     >>>> It *does* have a "server mode" where tests run outside of the > >     >>>> container, but I vaguely remember going down that path early > >     on and > >     >>>> hitting a roadblock.  For the life of me, I can't remember the > >     >>>> specifics.  But my pushback here is that ultimately Docker > >     might be > >     >>>> more preferable, giving us more of a real world scenario to > >     do true > >     >>>> e2e tests without something else in the middle. > >     >>>> > >     >>>>> so I can't imagine using Karaf without the helpers being a > >     walk in > >     >>>> the park; e.g. having to deal with HTTP operations comes with > >     its own > >     >>>> baggage {dependencies, complexity, speed, .. } and generally > more > >     >>>> stuff to maintain. > >     >>>> > >     >>>> I guess I respectfully disagree with that, but purely due to > >     Karaf > >     >>>> features.  Our features.xml does most of the heavy lifting for > us > >     >>>> w/r/t getting Hibernate provisioned. The same would be true > >     with the > >     >>>> test harness bundle/feature. REST is simple and > >     out-of-the-box thanks > >     >>>> to Karaf + CXF or Camel.  For other possible routes (Karaf > >     commands), > >     >>>> we already have code available in our demo/quickstart projects. > >     >>>> > >     >>>>> Also: considered contributing to Pax? > >     >>>> Yes, of course.  But the fact that numerous Karaf *committers* > >     >>>> themselves have a long history of built-up frustration on it > >     doesn't > >     >>>> leave me optimistic.  A couple of them had tried to pitch in > >     at one > >     >>>> point and weren't able to get anywhere. > >     >>>> > >     >>>>> but it seems their developers really expect their users to > >     be deeply > >     >>>> familiar with it all > >     >>>> > >     >>>> Absolutely!  But again, our struggles also come down to the > >     >>>> fundamental way Pax Exam works... > >     >>>> > >     >>>> > >     >>>> On 1/12/18 6:27 AM, Sanne Grinovero wrote: > >     >>>>> +1 to explore alternatives to Pax Exam, but I'd be wary of > >     maintining > >     >>>>> our own test infrastructure. > >     >>>>> > >     >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, > >     so I can't > >     >>>>> imagine using Karaf without the helpers being a walk in the > >     park; e.g. > >     >>>>> having to deal with HTTP operations comes with its own baggage > >     >>>>> {dependencies, complexity, speed, .. } and generally more > >     stuff to > >     >>>>> maintain. > >     >>>>> > >     >>>>> So.. +1 to try out Arquillian or anything else. Or maybe you > >     could > >     >>>>> start your own tool, but I'd prefer to see it in a separate > >     repository > >     >>>>> :) e.g. a nice Gradle plugin so maybe you get more people > >     helping? > >     >>>>> > >     >>>>> Also: considered contributing to Pax? My personal experience > >     with it > >     >>>>> has always been a pain but if I had to try identify the > >     reason, it was > >     >>>>> mostly caused by me being unfamiliar with Karaf and not > >     having good > >     >>>>> clues to track down the real failure; maybe some minor error > >     reporting > >     >>>>> improvements could make a big difference to its usability? Just > >     >>>>> saying, I don't feel like Pax is bad, but it seems their > >     developers > >     >>>>> really expect their users to be deeply familiar with it all > >     - feels > >     >>>>> like the typical case in which they could use some feedback > >     and a > >     >>>>> hand. > >     >>>>> > >     >>>>> Thanks, > >     >>>>> Sanne > >     >>>>> > >     >>>>> On 12 January 2018 at 08:22, Gunnar > >     Morling&lt;gunnar(a)hibernate.org <mailto:gunnar@hibernate.org> <mailto:gunnar@hibernate.org <mailto:gunnar@hibernate.org>>> wrote: > >     >>>>>> Hi Brett, > >     >>>>>> > >     >>>>>> We also had our fair share of frustration with Pax Exam in > >     HV, and I was > >     >>>>>> (more than once) at the point of dropping it. > >     >>>>>> > >     >>>>>> Docker could work, but as you say it's a bit of a heavy > >     dependency, if not > >     >>>>>> required anyways. Not sure whether I'd like to add this as > >     a prerequisite > >     >>>>>> for the HV build to be executed. And tests in separate > >     profiles tend to be > >     >>>>>> "forgotten" in my experience. > >     >>>>>> > >     >>>>>> One other approach could be to use Arquillian's OSGi > >     support (see > >     >>>>>> https://github.com/arquillian/arquillian-container-osgi > >     <https://github.com/arquillian/arquillian-container-osgi>), did > >     you consider > >     >>>>>> to use that one as an alternative? > >     >>>>>> > >     >>>>>> Cheers, > >     >>>>>> > >     >>>>>> --Gunnar > >     >>>>>> > >     >>>>>> > >     >>>>>> 2018-01-12 3:34 GMT+01:00 Brett Meyer&lt;brett(a)hibernate.org <mailto:brett@hibernate.org> > >     <mailto:brett@hibernate.org <mailto:brett@hibernate.org>>>: > >     >>>>>> > >     >>>>>>> <tired-rant> > >     >>>>>>> > >     >>>>>>> I'm fed up with Pax Exam and would love to replace it as the > >     >>>>>>> hibernate-osgi integration test harness.  Most of the > >     Karaf committers > >     >>>>>>> I've been working with hate it more than I do.  Every > >     single time we > >     >>>>>>> upgrade the Karaf version, something less-than-minor in > >     hibernate-osgi, > >     >>>>>>> upgrade/change dependencies, or attempt to upgrade Pax > >     Exam itself, > >     >>>>>>> there's some new obfuscated failure.  And no matter how > >     much I pray, it > >     >>>>>>> refuses to let us get to the container logs to figure out > what > >     >>>>>>> happened.  Tis a house of cards. > >     >>>>>>> > >     >>>>>>> </tired-rant> > >     >>>>>>> > >     >>>>>>> One alternative that recently came up elsewhere: use > >     Docker to bootstrap > >     >>>>>>> the container, hit it with our features.xml, install a > >     test bundle that > >     >>>>>>> exposes functionality externally (over HTTP, Karaf > >     commands, etc), then > >     >>>>>>> hit the endpoints and run assertions. > >     >>>>>>> > >     >>>>>>> Pros: true "integration test", plain vanilla Karaf, direct > >     access to all > >     >>>>>>> logs, easier to eventually support and test other containers. > >     >>>>>>> > >     >>>>>>> Cons: Need Docker installed for local test runs, probably > >     safer to > >     >>>>>>> isolate the integration test behind a disabled-by-default > >     Maven profile. > >     >>>>>>> > >     >>>>>>> Any gut reactions? > >     >>>>>>> > >     >>>>>>> OSGi is fun and I'm not at all bitter, > >     >>>>>>> > >     >>>>>>> -Brett- > >     >>>>>>> > >     >>>>>>> ;) > >     >>>>>>> > >     >>>>>>> > >     >>>>>>> _______________________________________________ > >     >>>>>>> hibernate-dev mailing list > >     >>>>>>> hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> > >     <mailto:hibernate-dev@lists.jboss.org <mailto:hibernate-dev@lists.jboss.org>> > >     >>>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > >     <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > >     >>>>>> _______________________________________________ > >     >>>>>> hibernate-dev mailing list > >     >>>>>> hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> > >     <mailto:hibernate-dev@lists.jboss.org <mailto:hibernate-dev@lists.jboss.org>> > >     >>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > >     <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > >     >>>>> _______________________________________________ > >     >>>>> hibernate-dev mailing list > >     >>>>> hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> > >     <mailto:hibernate-dev@lists.jboss.org <mailto:hibernate-dev@lists.jboss.org>> > >     >>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > >     <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > >     >>> _______________________________________________ > >     >>> hibernate-dev mailing list > >     >>> hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> > >     <mailto:hibernate-dev@lists.jboss.org <mailto:hibernate-dev@lists.jboss.org>> > >     >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > >     <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > >     > > >     > > >     > _______________________________________________ > >     > hibernate-dev mailing list > >     > hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> <mailto:hibernate-dev@lists <mailto:hibernate-dev@lists>. > jboss.org <http://jboss.org>> > >     > https://lists.jboss.org/mailman/listinfo/hibernate-dev > >     <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > >     _______________________________________________ > >     hibernate-dev mailing list > > hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> <mailto:hibernate-dev@lists.jboss.org <mailto:hibernate-dev@lists.jboss.org>> > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > >     <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > > > > > > _______________________________________________ > hibernate-dev mailing list > hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/hibernate-dev > _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/hibernate-dev

On 01/12/2018 12:54 PM, Sanne Grinovero wrote: > On 12 January 2018 at 17:32, Brett Meyer <brett(a)hibernate.org&gt; wrote: >> If I don't have time to contribute to Pax Exam, I certainly don't have >> time to start a new project haha... >> >> And realistically, that "something new" would likely involve containers >> anyway. >> >> At this point, mostly a question of 1) status quo, 2) Docker (or any >> other container-based solution), or 3) try screwing around with Pax Exam >> in "server-only" mode (but I don't have high hopes there). > Sure. Docker is now available on the CI slaves too, so that's not a problem. > > The only annoyance is that the whole ORM team - and anyone > contributing - would need to have Docker as well, but that doesn't > seem too bad to me... and was likely bound to happen for other tools > :) > > Steve, Chris and Andrea? Ok with that? Maybe you have Docker running already? I am fine with it; I have Docker installed already.

I'm ok with it.  But we use Gradle, so it'll be hard to use a Maven profile ;) On Thu, Jan 18, 2018, 4:14 PM Brett Meyer <brett(a)hibernate.org <mailto:brett@hibernate.org>> wrote: Thanks Chris!  Steve, not sure if you officially voted -- any concerns? Andrea?  Anyone else? Again, this would all be disabled by default, behind a Maven profile... On 1/16/18 3:15 PM, Chris Cranford wrote: > On 01/12/2018 12:54 PM, Sanne Grinovero wrote: >> On 12 January 2018 at 17:32, Brett Meyer <brett(a)hibernate.org <mailto:brett@hibernate.org>> wrote: >>> If I don't have time to contribute to Pax Exam, I certainly don't have >>> time to start a new project haha... >>> >>> And realistically, that "something new" would likely involve containers >>> anyway. >>> >>> At this point, mostly a question of 1) status quo, 2) Docker (or any >>> other container-based solution), or 3) try screwing around with Pax Exam >>> in "server-only" mode (but I don't have high hopes there). >> Sure. Docker is now available on the CI slaves too, so that's not a problem. >> >> The only annoyance is that the whole ORM team - and anyone >> contributing - would need to have Docker as well, but that doesn't >> seem too bad to me... and was likely bound to happen for other tools >> :) >> >> Steve, Chris and Andrea? Ok with that? Maybe you have Docker running already? > I am fine with it; I have Docker installed already. > _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/hibernate-dev