9:09 a.m.
Hey Ivan,
Great observations, definitely things that we want to improve:
Fight the black magic. It shouldn't be so hard to setup a test. What I
usually need is a UI test harness, project utilities, sometimes a
parser
and the addon that I am testing
Yes, this is an issue. We have a number of ideas for how to improve this,
such as introducing a new @DeployAddon() or @AddonDeployment() annotation,
which is really what our current @AddonDependency() annotation is doing -
deploying an addon. The @AddonDependency() annotation would then deploy AND
establish an addon dependency on the specified coordinate, eliminating the
need to specify .withAddonDependency() in the @Deployment method itself.
Essentially our current situation is one in which we must specify addon
deployment and dependency metadata separately, which leads to quite a bit
of duplicate information (boilerplate.) This should be easy to resolve.
Unfortunately we have a problem with backwards compatability if we change
the existing annotations, so we may need to deprecate them and introduce
entirely new ones in a new package.
I would actually like to solve this problem by refactoring the "Forge Test
Harness" in Furnace and calling it the "Furnace Test Harness" which is
really what it is.
Fight the slow startup time. So, we are using Arquillian. Imagine how would
you feel if Arquillian was setting up from scratch Wildfly or (oh
my!)
WebLogic every time you run a Java EE test? Instead, it just relies on the
fact that the target runtime is there
So one thing to note, is that the slowness here is not the fact that we are
setting up Furnace/Forge from scratch. Just like Wildfly, the runtime
environment is already there, but we still have to deploy our application
(the addons.) There is a great deal of inefficiency, as George says, in the
maven deployment of the addons, and this is (in my opinion) where our
greatest opportunity for improvement lies.
Very open to ideas here. It's been a while since i attached a profiler, but
if nothing has changed, most of the time is spent in maven itself -
resolving dependencies.
On Mon, Dec 22, 2014 at 7:49 PM, George Gastaldi <ggastald(a)redhat.com>
wrote:
I strongly agree and support that we should improve this area.
Here is the test documentation about the
@AddonDependencies/AddonDependencyEntry stuff:
http://forge.jboss.org/document/test-your-addon
I need to test this, but I think we could improve this by making the Maven
Settings object offline (to avoid remote dependency lookups) .
We already have a JIRA about this annoying error while running tests:
https://issues.jboss.org/browse/FORGE-2125
I think for the first option, we could assume that the test dependencies
are the addons already declared in the project's pom.xml (or we could have
some method in the ForgeArchive to assume that)
Best Regards,
George Gastaldi
On 12/22/2014 08:12 PM, Ivan St. Ivanov wrote:
Hi everybody,
I resumed my Security addon development and reached my "favorite" point:
writing and executing UI command tests. I have attached here the output of
the test harness as well as the sample test that I wrote.
Here are some observations:
- It took one minute for Forge to run a simple UI test. And this is on
Linux. From my experience, if I run the same test on Windows, it would take
at least twice more
- Even though Lincoln explained it to me at least twice, setting up
@Deployment @AddonDependencies and AddonDependencyEntry's is still black
magic to me. I usually copy those hoping that I didn't miss anything, but
the result of this test proves that I missed something
- For the most part the test was starting furnace, checking the
missing dependencies, installing them one by one, but in the mean time it
installed their transitive dependencies and for each of these operations,
Forge was again shutting down and starting up furnace and weld. And then
again calculating missing dependencies. Most of these operations take
usually less than a second, but still there are so many of them that at the
end it piles up to a whole minute
- To be fair, some big chunks of this minute was taken by, what it
seems to me, resolution of transitive dependencies:
Dec 22, 2014 11:15:49 PM org.jboss.forge.furnace.impl.addons.AddonRunnable
run
INFO: >> Started container
[org.jboss.forge.addon:ui-test-harness,2.13.1-SNAPSHOT] - 133ms
Dec 22, 2014 11:15:58 PM
org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy
INFO: Deploying addon org.jboss.forge.addon:parser-xml,2.13.1-SNAPSHOT
....
Dec 22, 2014 11:16:12 PM
org.jboss.forge.furnace.impl.addons.AddonRunnable run
INFO: >> Started container [org.jboss.forge.addon:javaee,2.13.1-SNAPSHOT]
- 1802ms
Dec 22, 2014 11:16:24 PM
org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy
INFO: Deploying addon org.jboss.forge.addon:maven,2.13.1-SNAPSHOT
- The test failed with the following exception:
java.lang.IllegalStateException: Test runner could not locate test class
[org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest] in any
deployed Addon.
at
org.jboss.forge.arquillian.ForgeTestMethodExecutor.invoke(ForgeTestMethodExecutor.java:234)
at
org.jboss.arquillian.container.test.impl.execution.RemoteTestExecuter.execute(RemoteTestExecuter.java:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at
org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
at
org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
...
However, the real reason was hidden in the massive console output a
bit above it:
Dec 22, 2014 11:16:25 PM
org.jboss.weld.bootstrap.MissingDependenciesRegistry
handleResourceLoadingException
INFO: WELD-000119: Not generating any bean definitions from
org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest because
of underlying class loading error: Type
org.jboss.forge.addon.javaee.ProjectHelper from [Module
"_DEFAULT_:2fba4fbf-9342-4566-9879-eebe1b753d2d_3ccd4af3-6ec9-4385-9aab-1693a53753fa"
from AddonModuleLoader] not found. If this is unexpected, enable DEBUG
logging to see the full error.
Enough with the observations. What can we do about it? Well, I see the
following areas of improvement:
- Fight the black magic. It shouldn't be so hard to setup a test. What
I usually need is a UI test harness, project utilities, sometimes a parser
and the addon that I am testing
- Fight the slow startup time. So, we are using Arquillian. Imagine how
would you feel if Arquillian was setting up from scratch Wildfly or (oh
my!) WebLogic every time you run a Java EE test? Instead, it just relies on
the fact that the target runtime is there
So, can't we just create a composite test addon or something like that?
That we use as kind of arquillian container and we just update the needed
addons there. Instead of setting up everything from scratch. And in the
@Deployment method we simply list the addons (or even at smaller
granularity: files) that are changed and we want to be redeployed on top.
This doesn't look too far away form the Arquillian model that we are all
used to. And I believe that will be much faster to start (especially in the
so called 'remote' arquillian mode).
What do you think?
Cheers,
Ivan
_______________________________________________
forge-dev mailing
listforge-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/forge-dev
_______________________________________________
forge-dev mailing list
forge-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/forge-dev
--
Lincoln Baxter, III
http://ocpsoft.org
"Simpler is better."