Hi,

Did I misunderstood something? Are there counterpoints to use fragments and maven-surefire-plugin? Are there other examples which are matching more closely to my usecase in jboss tools codebase?

About using fragments, the only drawback I'm aware of is that one cannot add a dependency to a fragment from their plugin. So it's not possible for other tests to reuse some logic that is inside a fragment. In the spirit of "treating tests as 1st class citizen", I believe it's generally better to allow composition of test bundles and to make them regular bundles. If reusable, the logic can be moved to a dedicated utility test plugin.

On the other hand, some integration tests should be able to detect that.

> The QE people could tell you their opinion on what's more important > between fast tests, and longer tests that are closer from production. > I guess it all depends, as always: we could imagine the unit test > running in plain Java to verify only their logic, but it shouldn't > become a replacement to some integration test to verify that the > logic also works in the right context. Not planned to have them as a replacement. I already created the structure to write also higher integration tests.

When running locally, I may want to run only unit tests first. Without launching the OSGi platform it will be faster. It will also allow to use some tools such as Infinitest to have continuous feedback while developing.

On 01/27/2016 10:28 AM, Aurelien Pupier wrote: > Hi, Hi Aurelien! > Did I misunderstood something? > Are there counterpoints to use fragments and maven-surefire-plugin? > Are there other examples which are matching more closely to my > usecase > in jboss tools codebase? About using fragments, the only drawback I'm aware of is that one cannot add a dependency to a fragment from their plugin. So it's not possible for other tests to reuse some logic that is inside a fragment. In the spirit of "treating tests as 1st class citizen", I believe it's generally better to allow composition of test bundles and to make them regular bundles. About maven-surefire (plain Java) vs tycho-surefire (OSGi/Eclipse), so far, we do not test the code of JBoss Tools outside of the "Eclipse" context. We can debate about the meaning of "unit tests", but what you see can be seen as "unit tests running in integration context". Our experience with some other parts of JBoss Tools have been that the difference of behavior between inside or outside of Eclipse can be important. So our policy has been so far to test everything with tycho-surefire-plugin to make sure what we're testing isn't too far from what will actually happen in production. The QE people could tell you their opinion on what's more important between fast tests, and longer tests that are closer from production. I guess it all depends, as always: we could imagine the unit test running in plain Java to verify only their logic, but it shouldn't become a replacement to some integration test to verify that the logic also works in the right context. I would say that as long as Eclipse is the *only* deployment context for a piece of code, then starting unit tests inside Eclipse is fine. Also, for fast unit tests, then they are neglictable compared to starting up the OSGi Platform. So if we want to start an OSGi platform anyway for integration tests, it seems simpler to run those tests inside that Platform. HTH -- Mickael Istria Eclipse developer at JBoss, by Red Hat <http://www.jboss.org/tools> My blog <http://mickaelistria.wordpress.com> - My Tweets <http://twitter.com/mickaelistria> _______________________________________________ jbosstools-dev mailing list jbosstools-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jbosstools-dev

On 01/27/2016 11:16 AM, Aurelien Pupier wrote: > About using fragments, the only drawback I'm aware of is that one > cannot add a dependency to a fragment from their plugin. So it's not > possible for other tests to reuse some logic that is inside a > fragment. In the spirit of "treating tests as 1st class citizen", I > believe it's generally better to allow composition of test bundles > and to make them regular bundles. > > If reusable, the logic can be moved to a dedicated utility test plugin. Yes, but it requires an effort in moving the code to the right plugin or turning fragment into a bundle everytime one identifies something to reuse; whereas using bundles only doesn't require any effort from the "producer" side.

> On the other hand, some integration tests should be able to detect that. That would require writing some additional integration tests, whereas the current approach to run "unit" tests inside a workbench provide that verification without additional thing to write. >> The QE people could tell you their opinion on what's more important >> between fast tests, and longer tests that are closer from production. >> I guess it all depends, as always: we could imagine the unit test >> running in plain Java to verify only their logic, but it shouldn't >> become a replacement to some integration test to verify that the >> logic also works in the right context. > Not planned to have them as a replacement. I already created the > structure to write also higher integration tests. If, during automated tests/build, you run both unit tests and then start a workbench to run integration tests, then you pay the price of the workbench startup anyway. So why not running those unit tests in the workbench?

> When running locally, I may want to run only unit tests first. > Without launching the OSGi platform it will be faster. It will also > allow to use some tools such as Infinitest to have continuous > feedback while developing. I don't know much of Infinitest, but I believe it doesn't rely on how Maven runs tests. Eclipse already provide the ability to run a test class in a bundle in plain Java without starting the workbench; it's "Run As > JUnit Test" instead of "Run As > JUnit Plugin Test". I guess Infinitest can rely on that, can't it?

Just to be clear, I'm not saying it's wrong to change that; I'm more wonderint how much profitable is it, what changing test structure would provide better than the current one does. The Infinitest story seems the only one "worth it" IMO, and it doesn't seem to be correlated to tycho-surefire vs maven-surefire.

-- Mickael Istria Eclipse developer at JBoss, by Red Hat <http://www.jboss.org/tools> My blog <http://mickaelistria.wordpress.com> - My Tweets <http://twitter.com/mickaelistria> _______________________________________________ jbosstools-dev mailing list jbosstools-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jbosstools-dev

On 02/01/2016 07:25 PM, Aurelien Pupier wrote: Le 1/27/2016 11:53 AM, Mickael Istria a écrit : Yes, but it requires an effort in moving the code to the right plugin or turning fragment into a bundle everytime one identifies something to reuse; whereas using bundles only doesn't require any effort from the "producer" side. Considering this code as first citizen, it seems to me an acceptable effort ;-) Do you put your actual "first citizen" business code into fragments too? When running locally, I may want to run only unit tests first. Without launching the OSGi platform it will be faster. It will also allow to use some tools such as Infinitest to have continuous feedback while developing. I don't know much of Infinitest, but I believe it doesn't rely on how Maven runs tests. Eclipse already provide the ability to run a test class in a bundle in plain Java without starting the workbench; it's "Run As > JUnit Test" instead of "Run As > JUnit Plugin Test". I guess Infinitest can rely on that, can't it? Yes Infinitest can rely on it but it means that you are able to launch as "JUnit test" and so not starting the workbench. Yes, and if you're able to do it from the Eclipse IDE for your test class, isn't it enough? So in your IDE, you can run "unit tests" without the workbench/Platform actually started, just the API available; and with Maven and tycho-surefire-plugin, you get a Platform/workbench started, like it will happen in real-life, to find real issues that wouldn't happen in plain Java environment. Just to be clear, I'm not saying it's wrong to change that; I'm more wonderint how much profitable is it, what changing test structure would provide better than the current one does. The Infinitest story seems the only one "worth it" IMO, and it doesn't seem to be correlated to tycho-surefire vs maven-surefire. InfiniTest or only launching test of a single plugin very fast (less than a second versus tens seconds) it is the difference between keeping concentrated on the task and have our brain switching to another idea. It seems to me that only tests launched with surefire will be able and ensured - to run with Junit test. I believe this sentence illustrates our divergence: you mainly think about tests as a tool for the developer, and they have to be fast for the developer to use them. That's right. I see them as an armor for the application, I want them to find as many bugs possible in the actual environment where the code will actually run, and if it has a cost for the developer, so be it, as long as tests are still run and checked by continuous integration. That's right too. Overall, both are right and useful. Depending on the test, you might want one or the other. What's possible and that seems like the best thing to me is to have the unit-tests in a bundle that run fast with maven-surefire-plugin, and have integration tests in another bundle also running the unit-test suite (+ some other tests) with tycho-surefire-plugin. So you get the best of both: unit tests are run fast and keep running fast because they are automatically tested using plain JUnit and maven-surefire-plugin; and you also have them running as part of the integration tests to also detect bugs that depend on the Eclipse runtime mechanism. WDYT? -- Mickael Istria Eclipse developer at JBoss, by Red Hat <http://www.jboss.org/tools> My blog <http://mickaelistria.wordpress.com> - My Tweets <http://twitter.com/mickaelistria> _______________________________________________ jbosstools-dev mailing list jbosstools-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jbosstools-dev