[forge-dev] Testing Forge Addons

[Bartosz Majsak]

 Hi Forgers ;)

First and foremost many thanks for this great tool :)

We are working on Arquillian Addon
<https://github.com/forge/addon-arquillian> bringing more capabilities as
our universe is growing. One thing which has been quite challenging is
testing. There are several ways of doing it as one can figure from
different addons available out in the wild.

I would like to get better understanding of what would be the preferred way
of testing an addon.

I personally liketestShellapproach as it gives you end-to-end flow and
ability to verify the behaviour of what user would do with the tool. This
has obvious performance drawbacks, but I believe we could make it more
efficient. I have developed small set of testing assertions
<https://github.com/forge/addon-arquillian/tree/e61fad56cab4a1684eb283345d93adeeb0789741/src/test/java/test/integration/support/assertions>
based on assertjfor verifying the state of a project (e.g. classes or
artifacts present). It is still work in progress and I’m eagerly looking
for feedback, but eventually I would like to bring it upstream to test
harness if that makes sense. You can see how it looks like for example in
the test verifying junit test scaffolding
<https://github.com/forge/addon-arquillian/blob/e61fad56cab4a1684eb283345d93adeeb0789741/src/test/java/test/integration/JUnitTestGenerationIntegrationTest.java>
.

Another problematic aspect is Forge addon deployment - the documentation
<https://forge.jboss.org/document/test-your-addon> seems not to reflect the
current state and I failed badly trying to create my own at Deployment with
assertj bundled. What I ended up with is an Arquillian extension
<https://github.com/forge/addon-arquillian/tree/e61fad56cab4a1684eb283345d93adeeb0789741/src/test/java/test/integration/extension>
which lets you add dependencies (resolved by Shrinkwrap under the hood) as
well as packages by simply using annotations - you can see both in action
in the junit test mentioned above. Is it the right way? We can also think
about bringing it upstream. Can we improve the docs to show what is
required to create proper @Deployment for the addon?

As Arquillian Addon is around for quite some time and several folks were
involved in it, there is also another way of testing forge commands by
using CommandController - can someone point me to the doc or briefly
explain what is the difference? Here’s one test
<https://github.com/forge/addon-arquillian/blob/e61fad56cab4a1684eb283345d93adeeb0789741/src/test/java/test/integration/AddArquillianExtensionTest.java#L47>
from our addon.

Cheers,
Bartosz.

​