<div dir="ltr">Hi everybody,<div><br></div><div>Thank you for the answers, guys! :)</div><div><br></div><div>Lincoln, you don't need a profiler here. You and George both are right that the reason for the slowness is resolving of Maven dependencies. So most probably, if we have Forge already setup and you just deploy your addon on top, maybe this could be fixed. I don't imagine too many addons at the moment that depend on anything besides what's in the core.</div><div><br></div><div><span style="font-size:12.7272720336914px">This would need to be provided as an extension to the test-harness, however, because we can't assume that people are using Furnace to test Forge.</span><br></div><div><br></div><div>Again, in 99% of the cases people will be using Furnace to test Forge. So it should be the default and whoever is not using that should do some configurations.</div><div><br></div><div>Again, what I really want is simplicity like the one I get with Arquillian tests: just tell Forge which is the addon under test (again, in 99% it is the addon of the test case) and whether I am relying on anything different than Furnace. So Forge should just take my addon, deploy it in an already prepared repository and run the test.</div><div><br></div><div>I know that it might sound very simplistic, but this is the role of people that do not work on the core: create impossible looking requirements for you, lol :)</div><div><br></div><div>Cheers,</div><div>Ivan</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Dec 27, 2014 at 5:11 PM, Lincoln Baxter, III <span dir="ltr"><<a href="mailto:lincolnbaxter@gmail.com" target="_blank">lincolnbaxter@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">One possible solution for #2 is to provide a new "all-of-forge" dependency that is unzipped from the forge-distribution-offline artifact based on the version specified. This would need to be provided as an extension to the test-harness, however, because we can't assume that people are using Furnace to test Forge. (The Forge test harness could include this extension on top of the Furnace test harness.)<div><br></div><div>Aren't all these F-names easy to keep track of? ;)</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Dec 27, 2014 at 10:09 AM, Lincoln Baxter, III <span dir="ltr"><<a href="mailto:lincolnbaxter@gmail.com" target="_blank">lincolnbaxter@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hey Ivan,<div><br></div><div>Great observations, definitely things that we want to improve:</div><div><br></div><div><span><blockquote style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex" class="gmail_quote">Fight the black magic. It shouldn't be so hard to setup a test. What I usually need is a UI test harness, project utilities, sometimes a parser and the addon that I am testing</blockquote><div style="font-size:13px"><br></div></span><div style="font-size:13px">Yes, this is an issue. We have a number of ideas for how to improve this, such as introducing a new @DeployAddon() or @AddonDeployment() annotation, which is really what our current @AddonDependency() annotation is doing - deploying an addon. The @AddonDependency() annotation would then deploy AND establish an addon dependency on the specified coordinate, eliminating the need to specify .withAddonDependency() in the @Deployment method itself. Essentially our current situation is one in which we must specify addon deployment and dependency metadata separately, which leads to quite a bit of duplicate information (boilerplate.) This should be easy to resolve. Unfortunately we have a problem with backwards compatability if we change the existing annotations, so we may need to deprecate them and introduce entirely new ones in a new package. </div><div style="font-size:13px"><br></div><div style="font-size:13px">I would actually like to solve this problem by refactoring the "Forge Test Harness" in Furnace and calling it the "Furnace Test Harness" which is really what it is.</div><span><div style="font-size:13px"><br></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex" class="gmail_quote">Fight the slow startup time. So, we are using Arquillian. Imagine how would you feel if Arquillian was setting up from scratch Wildfly or (oh my!) WebLogic every time you run a Java EE test? Instead, it just relies on the fact that the target runtime is there</blockquote><div><br></div></span><div>So one thing to note, is that the slowness here is not the fact that we are setting up Furnace/Forge from scratch. Just like Wildfly, the runtime environment is already there, but we still have to deploy our application (the addons.) There is a great deal of inefficiency, as George says, in the maven deployment of the addons, and this is (in my opinion) where our greatest opportunity for improvement lies.</div></div><div><br></div><div>Very open to ideas here. It's been a while since i attached a profiler, but if nothing has changed, most of the time is spent in maven itself - resolving dependencies.</div></div><div class="gmail_extra"><div><div><br><div class="gmail_quote">On Mon, Dec 22, 2014 at 7:49 PM, George Gastaldi <span dir="ltr"><<a href="mailto:ggastald@redhat.com" target="_blank">ggastald@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
I strongly agree and support that we should improve this area. <br>
<br>
Here is the test documentation about the
@AddonDependencies/AddonDependencyEntry stuff:
<a href="http://forge.jboss.org/document/test-your-addon" target="_blank">http://forge.jboss.org/document/test-your-addon</a> <br>
<br>
I need to test this, but I think we could improve this by making the
Maven Settings object offline (to avoid remote dependency lookups) .
<br>
We already have a JIRA about this annoying error while running
tests: <a href="https://issues.jboss.org/browse/FORGE-2125" target="_blank">https://issues.jboss.org/browse/FORGE-2125</a><br>
<br>
I think for the first option, we could assume that the test
dependencies are the addons already declared in the project's
pom.xml (or we could have some method in the ForgeArchive to assume
that)<br>
<br>
Best Regards,<br>
<br>
George Gastaldi<div><div><br>
<br>
<div>On 12/22/2014 08:12 PM, Ivan St. Ivanov
wrote:<br>
</div>
</div></div><blockquote type="cite"><div><div>
<div dir="ltr">Hi everybody,
<div><br>
</div>
<div>I resumed my Security addon development and reached my
"favorite" point: writing and executing UI command tests. I
have attached here the output of the test harness as well as
the sample test that I wrote.</div>
<div><br>
</div>
<div>Here are some observations:</div>
<div> - It took one minute for Forge to run a simple UI test.
And this is on Linux. From my experience, if I run the same
test on Windows, it would take at least twice more</div>
<div><br>
</div>
<div> - Even though Lincoln explained it to me at least twice,
setting up @Deployment @AddonDependencies and
AddonDependencyEntry's is still black magic to me. I usually
copy those hoping that I didn't miss anything, but the result
of this test proves that I missed something</div>
<div><br>
</div>
<div> - For the most part the test was starting furnace,
checking the missing dependencies, installing them one by one,
but in the mean time it installed their transitive
dependencies and for each of these operations, Forge was again
shutting down and starting up furnace and weld. And then again
calculating missing dependencies. Most of these operations
take usually less than a second, but still there are so many
of them that at the end it piles up to a whole minute</div>
<div><br>
</div>
<div> - To be fair, some big chunks of this minute was taken
by, what it seems to me, resolution of transitive
dependencies:</div>
<div>Dec 22, 2014 11:15:49 PM
org.jboss.forge.furnace.impl.addons.AddonRunnable run</div>
<div>INFO: >> Started container
[org.jboss.forge.addon:ui-test-harness,2.13.1-SNAPSHOT] -
133ms</div>
<div>Dec 22, 2014 11:15:58 PM
org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl
deploy</div>
<div>INFO: Deploying addon
org.jboss.forge.addon:parser-xml,2.13.1-SNAPSHOT</div>
<div>....</div>
<div>
<div>Dec 22, 2014 11:16:12 PM
org.jboss.forge.furnace.impl.addons.AddonRunnable run</div>
<div>INFO: >> Started container
[org.jboss.forge.addon:javaee,2.13.1-SNAPSHOT] - 1802ms</div>
<div>Dec 22, 2014 11:16:24 PM
org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl
deploy</div>
<div>INFO: Deploying addon
org.jboss.forge.addon:maven,2.13.1-SNAPSHOT</div>
</div>
<div><br>
</div>
<div> - The test failed with the following exception:</div>
<div>
<div>java.lang.IllegalStateException: Test runner could not
locate test class
[org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest]
in any deployed Addon.</div>
<div><span style="white-space:pre-wrap"> </span>at
org.jboss.forge.arquillian.ForgeTestMethodExecutor.invoke(ForgeTestMethodExecutor.java:234)</div>
<div><span style="white-space:pre-wrap"> </span>at
org.jboss.arquillian.container.test.impl.execution.RemoteTestExecuter.execute(RemoteTestExecuter.java:109)</div>
<div><span style="white-space:pre-wrap"> </span>at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)</div>
<div><span style="white-space:pre-wrap"> </span>at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)</div>
<div><span style="white-space:pre-wrap"> </span>at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)</div>
<div><span style="white-space:pre-wrap"> </span>at
org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)</div>
<div><span style="white-space:pre-wrap"> </span>at
org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)</div>
</div>
<div>...</div>
<div><br>
</div>
<div> However, the real reason was hidden in the massive
console output a bit above it:</div>
<div>
<div><br>
</div>
<div>Dec 22, 2014 11:16:25 PM
org.jboss.weld.bootstrap.MissingDependenciesRegistry
handleResourceLoadingException</div>
<div>INFO: WELD-000119: Not generating any bean definitions
from
org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest
because of underlying class loading error: Type
org.jboss.forge.addon.javaee.ProjectHelper from [Module
"_DEFAULT_:2fba4fbf-9342-4566-9879-eebe1b753d2d_3ccd4af3-6ec9-4385-9aab-1693a53753fa"
from AddonModuleLoader] not found. If this is unexpected,
enable DEBUG logging to see the full error.</div>
</div>
<div><br>
</div>
<div>Enough with the observations. What can we do about it?
Well, I see the following areas of improvement:</div>
<div><br>
</div>
<div> - Fight the black magic. It shouldn't be so hard to setup
a test. What I usually need is a UI test harness, project
utilities, sometimes a parser and the addon that I am testing</div>
<div> - Fight the slow startup time. So, we are using
Arquillian. Imagine how would you feel if Arquillian was
setting up from scratch Wildfly or (oh my!) WebLogic every
time you run a Java EE test? Instead, it just relies on the
fact that the target runtime is there</div>
<div><br>
</div>
<div>So, can't we just create a composite test addon or
something like that? That we use as kind of arquillian
container and we just update the needed addons there. Instead
of setting up everything from scratch. And in the @Deployment
method we simply list the addons (or even at smaller
granularity: files) that are changed and we want to be
redeployed on top.</div>
<div><br>
</div>
<div>This doesn't look too far away form the Arquillian model
that we are all used to. And I believe that will be much
faster to start (especially in the so called 'remote'
arquillian mode).</div>
<div><br>
</div>
<div>What do you think?</div>
<div><br>
</div>
<div>Cheers,</div>
<div>Ivan</div>
<div><br>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</div></div><pre>_______________________________________________
forge-dev mailing list
<a href="mailto:forge-dev@lists.jboss.org" target="_blank">forge-dev@lists.jboss.org</a>
<a href="https://lists.jboss.org/mailman/listinfo/forge-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/forge-dev</a></pre>
</blockquote>
<br>
</div>
<br>_______________________________________________<br>
forge-dev mailing list<br>
<a href="mailto:forge-dev@lists.jboss.org" target="_blank">forge-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/forge-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/forge-dev</a><br></blockquote></div><br><br clear="all"><div><br></div></div></div><span><font color="#888888">-- <br><div>Lincoln Baxter, III<br><a href="http://ocpsoft.org" target="_blank">http://ocpsoft.org</a><br>"Simpler is better."</div>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div>Lincoln Baxter, III<br><a href="http://ocpsoft.org" target="_blank">http://ocpsoft.org</a><br>"Simpler is better."</div>
</div>
</div></div><br>_______________________________________________<br>
forge-dev mailing list<br>
<a href="mailto:forge-dev@lists.jboss.org">forge-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/forge-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/forge-dev</a><br></blockquote></div><br></div>