[jbosstools-dev] Time for a new SOA test repo?

[Nick Boldt]

Capturing the requirements and TODOs from this thread, I've opened 3 jiras:

1. produce a JBTIS unit tests update site
    * https://issues.jboss.org/browse/JBTIS-28

(this is currently being done in 
http://download.jboss.org/jbosstools/updates/nightly/soatests/4.0.juno/)

2. migrate soa/brms tooling tests from jbosstools-integration-tests to 
new repo jbosstools-integration-stack-itests
    * https://issues.jboss.org/browse/JBTIS-27

3. rename jbosstools-integration-tests to jbosstools-itests

    * https://issues.jboss.org/browse/JBIDE-13796

Once all the refactoring is done, you won't need to worry about 
branching to support the divergent streams, since all the Juno-based 
JBTIS tests will be in one repo, and the Kepler-based JBT tests will be 
in another.

This will also resolve the Zest issue, as JBTIS itests (Juno level) can 
depend on the JBTIS TP, like the rest of the stack does.

In JBT (Kepler level), once we move to Kepler M6 next week, I'll add 
Zest in there too (SpringIDE 3.2 requires it).

If any questions remain, feel free to ask them here or in the above JIRAs.

Cheers,

N

On 03/14/2013 10:07 AM, Martin Malina wrote:
>
> On 14. 3. 2013, at 14:02, Nick Boldt <nboldt at redhat.com
> <mailto:nboldt at redhat.com>> wrote:
>
>> So after much discussion, I'm hearing that:
>>
>> 1) individual projects' non-integration tests (that is, relatively
>> fast UNIT tests) will run as part of those projects' builds, and their
>> features & plugins will be published to the staging composite site,
>> then aggregated into two aggregate sites, as we did in JBT 4.0.0 and
>> before:
>>
>> * http://download.jboss.org/jbosstools/updates/nightly/coretests/
>> * http://download.jboss.org/jbosstools/updates/nightly/soatests/
>
> Right, so we will have:
> http://download.jboss.org/jbosstools/updates/nightly/coretests/4.0.juno/
> http://download.jboss.org/jbosstools/updates/nightly/coretests/4.1.kepler/
> http://download.jboss.org/jbosstools/updates/nightly/soatests/4.0.juno/
> http://download.jboss.org/jbosstools/updates/nightly/soatests/4.1.kepler/
>
>> 2) jbosstools-integration-tests repo will be split into:
>>
>> * jbosstools-itests and
>> * jbosstools-integration-stack-itests
>>
>> which will publish 2 SEPARATE update sites. Name choice is based on
>> Maven "itests" convention:
>>
>> http://docs.codehaus.org/display/MAVENUSER/Maven+and+Integration+Testing
>
> As discussed in https://issues.jboss.org/browse/JBIDE-12974 before, I
> still don't understand why we need a new site for the itests - before
> moving to git, we had all the unit tests and integration tests in the
> coretests repo and there wasn't a problem with it AFAIK. But if you
> think it's better to have unit tests in coretests and itests in an
> additional update site, then why not (but it's 4 test update sites vs. 2).
> What I'm saying is that I would be perfectly fine with just having
> coretests and soatests as before, just with the extra plugins from
> integration-tests and integration-stack-tests respectively.
>
>> 3) jbosstools-itests site will build using the JBT target platform
>> (Kepler). We should also ensure we are building a Juno version so that
>> this site can exist and the jbosstools-integration-stack-itests build
>> can depend on it. This is where UI testing framework code such as
>> org.jboss.tools.ui.bot.ext should go.
>
> I think everything should be done per each main version, no?
> Right we have just one for master:
> http://download.jboss.org/jbosstools/builds/staging/jbosstools-integration-tests.aggregate_master/
> But it should be more like:
> http://download.jboss.org/jbosstools/updates/nightly/itests/4.0.juno/
> http://download.jboss.org/jbosstools/updates/nightly/itests/4.1.kepler/
> http://download.jboss.org/jbosstools/updates/nightly/integration-stack-itests/4.0.juno/
> http://download.jboss.org/jbosstools/updates/nightly/integration-stack-itests/4.1.kepler/
> Or course the last one would be kind of dead for now - nobody works on
> JBTIS for kepler yet.
>
>> 4) jbosstools-integration-stack-itests site will build using the JBTIS
>> target platform (Juno) and will depend on the jbosstools-itests update
>> site from 4.0.x, as well as the upstream coretests and/or soatests
>> sites, if necessary.
>
> Agreed.
>
>>
>> ---
>>
>> As to the Zest question, it will only be added to the JBT 4.1 target
>> platform for *Kepler*, since it is only that version of Central which
>> is planned to be updated w/ SpringIDE 3.2. Should we decide we want to
>> include SpringIDE 3.2 in Central for JBT 4.0.1 / JBDS 6.0.1 as well,
>> we would need to add Zest to the Juno target platform too.
>
> Yesterday, I had to remove support for zest from bot ext in 4.0.x [1]
> because we updated our parent pom reference to 4.0.1 and that uses the
> new Juno TP with Zest removed. So right now we don't need it. But you
> can see a possible problem here - integration-stack-itests will still
> use bot ext from integration-tests. And they might want to test zest and
> actually might want bot ext to have that support. So if that happens it
> may be easier to just put zest back to the JBT TP.
>
> [1]
> https://github.com/jbosstools/jbosstools-integration-tests/commit/2634752f82017c92e75c71fcf030f4cdd9613f32
>
>>
>> If that is something that PM and QE would like to happen, please
>> provide input in https://issues.jboss.org/browse/JBDS-2395 so I can
>> set that up ASAP.
>
> I guess we first need to find an agreement on this - but we're getting
> there :)
>
> Thanks,
> Martin
>
>>
>> Thanks!
>>
>> Nick
>>
>> On 03/14/2013 08:29 AM, Max Rydahl Andersen wrote:
>>>>>> The problem is (was) that:
>>>>>> * The ESB tests require the ESB UI which requires Zest - tests
>>>>>> build fails on master for 7.0
>>>>>> * Zest is not in the JBT test package, but it is in the JBTIS test
>>>>>> package
>>>>>
>>>>> we probably need Zest for other reasons anyway so that probably
>>>>> makes sense to add to core TP anyway.
>>>>
>>>> ok, so one problem will be solved. but spliting the repos make sense
>>>> anyway imo.
>>>
>>> Yes, totally agree - remember I was the one objecting when you put
>>> all tests into the integration test repo :)
>>>
>>>>>> * Adding JBTIS TP repo to our root pom in integration-tests would
>>>>>> affect all our tests:
>>>>>
>>>>>> -> The tests would run slower
>>>>>
>>>>> Hopefully not...did you see a big effect ?
>>>>
>>>> It's more of a theoretical possibility - I think it was you who
>>>> raised this concern in the past - that each new repo that is
>>>> searched in a build introduces more complexity for resolving of the
>>>> dependencies
>>>
>>> Okey, yes - I misunderstood the statement as if the tests would run
>>> slower which would be wrong, but the *builds* would for sure get an
>>> impact - question is just how much.
>>>
>>> But even ignoring that the core tests should not depend on any JBTIS
>>> target platform since it might not even exist in a compatible form
>>> before later in development.
>>>
>>>>>> -> The JBTIS test package might affect our tests - we might miss
>>>>>> issues of plugins not being in the JBT test package
>>>>>
>>>>> not following what you mean here ? if JBTIS test package does
>>>>> affect your test we want to know about it ;)
>>>>
>>>> The core integration tests should be able to resolve all
>>>> dependencies from the JBT core TP and update site - we're trying to
>>>> emulate the usual use case when somebody installs JBT core - they
>>>> won't have JBTIS update site available.
>>>> So if we add JBTIS update site for all tests we might miss some
>>>> problem - a missing dependency on the core update site. Sure, this
>>>> would probably be picked up by a failing build of a component, but
>>>> even so, I think it's cleaner if we just use the intended update
>>>> sites (core TP and core update site for core tests).
>>>
>>> Yes, completely agree that should be fixed.
>>>>
>>>>> That said you should be using JBTIS since that is where the esb
>>>>> plugins are. Can't just use jbosstools core as base for this.
>>>>
>>>> Sure, that's why it would be good for JBTIS integration tests to be
>>>> separate - they need to use JBTIS TP and update site.
>>>>
>>>>>> So - what are the solutions?
>>>>>
>>>>> Split out the github repository and move the integration-platform
>>>>> tests to a separate repo.
>>>>
>>>> +1
>>>
>>>>>> Second, long-term
>>>>>> * We had talked about this in the past - creating a new SOA test repo
>>>>>
>>>>> integration-test repo - its no longer just SOA.
>>>>
>>>> We tend to say "soa integration tests" because integration stack
>>>> integration tests is even more confusing. Do you have a better
>>>> suggestion here? :)
>>>
>>> integration-stack-tests ?
>>>
>>>>>> -> This will mean that existing 4.0.x work would move to the new repo
>>>>>
>>>>> you'll need to migrate the history very carefully yes.
>>>>
>>>> Yeah, another exercise for filter-branch/fast-filter I guess.
>>>
>>> Yeah - if you guys can create a script for it i'm more than willing
>>> to help review it.
>>>
>>> /max
>>> _______________________________________________
>>> jbosstools-dev mailing list
>>> jbosstools-dev at lists.jboss.org <mailto:jbosstools-dev at lists.jboss.org>
>>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>>>
>>
>> --
>> Nick Boldt :: JBoss by Red Hat
>> Productization Lead :: JBoss Tools & Dev Studio
>> http://nick.divbyzero.com
>
> --
> Martin Malina
> JBoss QA Engineer
> Red Hat Czech s.r.o.
> Purkynova 99
> 612 45 Brno, Czech Republic
>
> Tel.: +420 532 294 265
>
>
>
>

-- 
Nick Boldt :: JBoss by Red Hat
Productization Lead :: JBoss Tools & Dev Studio
http://nick.divbyzero.com