[forge-dev] Again on testing Forge commands

[George Gastaldi]

I strongly agree and support that we should improve this area.

Here is the test documentation about the 
@AddonDependencies/AddonDependencyEntry stuff: 
http://forge.jboss.org/document/test-your-addon

I need to test this, but I think we could improve this by making the 
Maven Settings object offline (to avoid remote dependency lookups) .
We already have a JIRA about this annoying error while running tests: 
https://issues.jboss.org/browse/FORGE-2125

I think for the first option, we could assume that the test dependencies 
are the addons already declared in the project's pom.xml (or we could 
have some method in the ForgeArchive to assume that)

Best Regards,

George Gastaldi

On 12/22/2014 08:12 PM, Ivan St. Ivanov wrote:
> Hi everybody,
>
> I resumed my Security addon development and reached my "favorite" 
> point: writing and executing UI command tests. I have attached here 
> the output of the test harness as well as the sample test that I wrote.
>
> Here are some observations:
>   - It took one minute for Forge to run a simple UI test. And this is 
> on Linux. From my experience, if I run the same test on Windows, it 
> would take at least twice more
>
>   - Even though Lincoln explained it to me at least twice, setting up 
> @Deployment @AddonDependencies and AddonDependencyEntry's is still 
> black magic to me. I usually copy those hoping that I didn't miss 
> anything, but the result of this test proves that I missed something
>
>   - For the most part the test was starting furnace, checking the 
> missing dependencies, installing them one by one, but in the mean time 
> it installed their transitive dependencies and for each of these 
> operations, Forge was again shutting down and starting up furnace and 
> weld. And then again calculating missing dependencies. Most of these 
> operations take usually less than a second, but still there are so 
> many of them that at the end it piles up to a whole minute
>
>   - To be fair, some big chunks of this minute was taken by, what it 
> seems to me, resolution of transitive dependencies:
> Dec 22, 2014 11:15:49 PM 
> org.jboss.forge.furnace.impl.addons.AddonRunnable run
> INFO: >> Started container 
> [org.jboss.forge.addon:ui-test-harness,2.13.1-SNAPSHOT] - 133ms
> Dec 22, 2014 11:15:58 PM 
> org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy
> INFO: Deploying addon org.jboss.forge.addon:parser-xml,2.13.1-SNAPSHOT
> ....
> Dec 22, 2014 11:16:12 PM 
> org.jboss.forge.furnace.impl.addons.AddonRunnable run
> INFO: >> Started container 
> [org.jboss.forge.addon:javaee,2.13.1-SNAPSHOT] - 1802ms
> Dec 22, 2014 11:16:24 PM 
> org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy
> INFO: Deploying addon org.jboss.forge.addon:maven,2.13.1-SNAPSHOT
>
>   - The test failed with the following exception:
> java.lang.IllegalStateException: Test runner could not locate test 
> class 
> [org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest] in 
> any deployed Addon.
> at 
> org.jboss.forge.arquillian.ForgeTestMethodExecutor.invoke(ForgeTestMethodExecutor.java:234)
> at 
> org.jboss.arquillian.container.test.impl.execution.RemoteTestExecuter.execute(RemoteTestExecuter.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at 
> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
> at 
> org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
> ...
>
>     However, the real reason was hidden in the massive console output 
> a bit above it:
>
> Dec 22, 2014 11:16:25 PM 
> org.jboss.weld.bootstrap.MissingDependenciesRegistry 
> handleResourceLoadingException
> INFO: WELD-000119: Not generating any bean definitions from 
> org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest 
> because of underlying class loading error: Type 
> org.jboss.forge.addon.javaee.ProjectHelper from [Module 
> "_DEFAULT_:2fba4fbf-9342-4566-9879-eebe1b753d2d_3ccd4af3-6ec9-4385-9aab-1693a53753fa" 
> from AddonModuleLoader] not found.  If this is unexpected, enable 
> DEBUG logging to see the full error.
>
> Enough with the observations. What can we do about it? Well, I see the 
> following areas of improvement:
>
>   - Fight the black magic. It shouldn't be so hard to setup a test. 
> What I usually need is a UI test harness, project utilities, sometimes 
> a parser and the addon that I am testing
>   - Fight the slow startup time. So, we are using Arquillian. Imagine 
> how would you feel if Arquillian was setting up from scratch Wildfly 
> or (oh my!) WebLogic every time you run a Java EE test? Instead, it 
> just relies on the fact that the target runtime is there
>
> So, can't we just create a composite test addon or something like 
> that? That we use as kind of arquillian container and we just update 
> the needed addons there. Instead of setting up everything from 
> scratch. And in the @Deployment method we simply list the addons (or 
> even at smaller granularity: files) that are changed and we want to be 
> redeployed on top.
>
> This doesn't look too far away form the Arquillian model that we are 
> all used to. And I believe that will be much faster to start 
> (especially in the so called 'remote' arquillian mode).
>
> What do you think?
>
> Cheers,
> Ivan
>
>
>
> _______________________________________________
> forge-dev mailing list
> forge-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/forge-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/forge-dev/attachments/20141222/2d0884de/attachment.html