[forge-dev] Again on testing Forge commands

Sat Dec 27 10:11:04 EST 2014

One possible solution for #2 is to provide a new "all-of-forge" dependency
that is unzipped from the forge-distribution-offline artifact based on the
version specified. This would need to be provided as an extension to the
test-harness, however, because we can't assume that people are using
Furnace to test Forge. (The Forge test harness could include this extension
on top of the Furnace test harness.)

Aren't all these F-names easy to keep track of? ;)

On Sat, Dec 27, 2014 at 10:09 AM, Lincoln Baxter, III <
lincolnbaxter at gmail.com> wrote:

> Hey Ivan,
>
> Great observations, definitely things that we want to improve:
>
> Fight the black magic. It shouldn't be so hard to setup a test. What I
>> usually need is a UI test harness, project utilities, sometimes a parser
>> and the addon that I am testing
>
>
> Yes, this is an issue. We have a number of ideas for how to improve this,
> such as introducing a new @DeployAddon() or @AddonDeployment() annotation,
> which is really what our current @AddonDependency() annotation is doing -
> deploying an addon. The @AddonDependency() annotation would then deploy AND
> establish an addon dependency on the specified coordinate, eliminating the
> need to specify .withAddonDependency() in the @Deployment method itself.
> Essentially our current situation is one in which we must specify addon
> deployment and dependency metadata separately, which leads to quite a bit
> of duplicate information (boilerplate.) This should be easy to resolve.
> Unfortunately we have a problem with backwards compatability if we change
> the existing annotations, so we may need to deprecate them and introduce
> entirely new ones in a new package.
>
> I would actually like to solve this problem by refactoring the "Forge Test
> Harness" in Furnace and calling it the "Furnace Test Harness" which is
> really what it is.
>
> Fight the slow startup time. So, we are using Arquillian. Imagine how
>> would you feel if Arquillian was setting up from scratch Wildfly or (oh
>> my!) WebLogic every time you run a Java EE test? Instead, it just relies on
>> the fact that the target runtime is there
>
>
> So one thing to note, is that the slowness here is not the fact that we
> are setting up Furnace/Forge from scratch. Just like Wildfly, the runtime
> environment is already there, but we still have to deploy our application
> (the addons.) There is a great deal of inefficiency, as George says, in the
> maven deployment of the addons, and this is (in my opinion) where our
> greatest opportunity for improvement lies.
>
> Very open to ideas here. It's been a while since i attached a profiler,
> but if nothing has changed, most of the time is spent in maven itself -
> resolving dependencies.
>
> On Mon, Dec 22, 2014 at 7:49 PM, George Gastaldi <ggastald at redhat.com>
> wrote:
>
>>  I strongly agree and support that we should improve this area.
>>
>> Here is the test documentation about the
>> @AddonDependencies/AddonDependencyEntry stuff:
>> http://forge.jboss.org/document/test-your-addon
>>
>> I need to test this, but I think we could improve this by making the
>> Maven Settings object offline (to avoid remote dependency lookups) .
>> We already have a JIRA about this annoying error while running tests:
>> https://issues.jboss.org/browse/FORGE-2125
>>
>> I think for the first option, we could assume that the test dependencies
>> are the addons already declared in the project's pom.xml (or we could have
>> some method in the ForgeArchive to assume that)
>>
>> Best Regards,
>>
>> George Gastaldi
>>
>>
>> On 12/22/2014 08:12 PM, Ivan St. Ivanov wrote:
>>
>> Hi everybody,
>>
>>  I resumed my Security addon development and reached my "favorite"
>> point: writing and executing UI command tests. I have attached here the
>> output of the test harness as well as the sample test that I wrote.
>>
>>  Here are some observations:
>>   - It took one minute for Forge to run a simple UI test. And this is on
>> Linux. From my experience, if I run the same test on Windows, it would take
>> at least twice more
>>
>>    - Even though Lincoln explained it to me at least twice, setting up
>> @Deployment @AddonDependencies and AddonDependencyEntry's is still black
>> magic to me. I usually copy those hoping that I didn't miss anything, but
>> the result of this test proves that I missed something
>>
>>    - For the most part the test was starting furnace, checking the
>> missing dependencies, installing them one by one, but in the mean time it
>> installed their transitive dependencies and for each of these operations,
>> Forge was again shutting down and starting up furnace and weld. And then
>> again calculating missing dependencies. Most of these operations take
>> usually less than a second, but still there are so many of them that at the
>> end it piles up to a whole minute
>>
>>    - To be fair, some big chunks of this minute was taken by, what it
>> seems to me, resolution of transitive dependencies:
>> Dec 22, 2014 11:15:49 PM
>> org.jboss.forge.furnace.impl.addons.AddonRunnable run
>> INFO: >> Started container
>> [org.jboss.forge.addon:ui-test-harness,2.13.1-SNAPSHOT] - 133ms
>> Dec 22, 2014 11:15:58 PM
>> org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy
>> INFO: Deploying addon org.jboss.forge.addon:parser-xml,2.13.1-SNAPSHOT
>> ....
>>  Dec 22, 2014 11:16:12 PM
>> org.jboss.forge.furnace.impl.addons.AddonRunnable run
>> INFO: >> Started container [org.jboss.forge.addon:javaee,2.13.1-SNAPSHOT]
>> - 1802ms
>> Dec 22, 2014 11:16:24 PM
>> org.jboss.forge.furnace.manager.impl.request.DeployRequestImpl deploy
>> INFO: Deploying addon org.jboss.forge.addon:maven,2.13.1-SNAPSHOT
>>
>>    - The test failed with the following exception:
>>  java.lang.IllegalStateException: Test runner could not locate test
>> class [org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest]
>> in any deployed Addon.
>>  at
>> org.jboss.forge.arquillian.ForgeTestMethodExecutor.invoke(ForgeTestMethodExecutor.java:234)
>>  at
>> org.jboss.arquillian.container.test.impl.execution.RemoteTestExecuter.execute(RemoteTestExecuter.java:109)
>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>  at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>  at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>  at
>> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
>>  at
>> org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
>>  ...
>>
>>      However, the real reason was hidden in the massive console output a
>> bit above it:
>>
>>  Dec 22, 2014 11:16:25 PM
>> org.jboss.weld.bootstrap.MissingDependenciesRegistry
>> handleResourceLoadingException
>> INFO: WELD-000119: Not generating any bean definitions from
>> org.jboss.forge.addon.javaee.security.ui.SecuritySetupCommandTest because
>> of underlying class loading error: Type
>> org.jboss.forge.addon.javaee.ProjectHelper from [Module
>> "_DEFAULT_:2fba4fbf-9342-4566-9879-eebe1b753d2d_3ccd4af3-6ec9-4385-9aab-1693a53753fa"
>> from AddonModuleLoader] not found.  If this is unexpected, enable DEBUG
>> logging to see the full error.
>>
>>  Enough with the observations. What can we do about it? Well, I see the
>> following areas of improvement:
>>
>>    - Fight the black magic. It shouldn't be so hard to setup a test.
>> What I usually need is a UI test harness, project utilities, sometimes a
>> parser and the addon that I am testing
>>   - Fight the slow startup time. So, we are using Arquillian. Imagine how
>> would you feel if Arquillian was setting up from scratch Wildfly or (oh
>> my!) WebLogic every time you run a Java EE test? Instead, it just relies on
>> the fact that the target runtime is there
>>
>>  So, can't we just create a composite test addon or something like that?
>> That we use as kind of arquillian container and we just update the needed
>> addons there. Instead of setting up everything from scratch. And in the
>> @Deployment method we simply list the addons (or even at smaller
>> granularity: files) that are changed and we want to be redeployed on top.
>>
>>  This doesn't look too far away form the Arquillian model that we are
>> all used to. And I believe that will be much faster to start (especially in
>> the so called 'remote' arquillian mode).
>>
>>  What do you think?
>>
>>  Cheers,
>> Ivan
>>
>>
>>
>> _______________________________________________
>> forge-dev mailing listforge-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/forge-dev
>>
>>
>>
>> _______________________________________________
>> forge-dev mailing list
>> forge-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/forge-dev
>>
>
>
>
> --
> Lincoln Baxter, III
> http://ocpsoft.org
> "Simpler is better."
>

-- 
Lincoln Baxter, III
http://ocpsoft.org
"Simpler is better."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/forge-dev/attachments/20141227/39945c85/attachment.html 