[jboss-as7-dev] Revisited: Integration TestSuite Organization and Maintenance

[Andrew Lee Rubinger]

On 08/16/2011 12:40 PM, Jason T. Greene wrote:
> On 8/16/11 11:37 AM, Jason T. Greene wrote:
>> On 8/16/11 11:29 AM, Andrew Lee Rubinger wrote:
>>> On 08/16/2011 09:33 AM, Jason T. Greene wrote:
>>>> On 8/16/11 2:51 AM, Andrew Lee Rubinger wrote:
>>>>> In short:
>>>>>
>>>>> https://issues.jboss.org/browse/AS7-1489
>>>>> https://issues.jboss.org/browse/AS7-1479
>>>>> https://issues.jboss.org/browse/AS7-1478
>>>>> https://issues.jboss.org/browse/AS7-1493
>>>>>
>>>>> ...are the issues I uncovered after moving to a restricted dependency
>>>>> chain. With the status quo in place, issues like these go unnoticed.
>>>>
>>>> Yeah but no one made any attempt to verify this pom was correct.
>>>
>>> Thanks for making my point for me. :) It's about maintenance. No one is
>>> going to be verifying that these POMs are complete.
>>> Even if we make some extra suite, who is going to think to go in there
>>> and add
>>> @SecurityDomain to it? It was the testsuite which exposed that this was
>>> needed, not a manual review.
>>
>> No actually I didn't :) You do admit that the indirect process proposal
>> has holes (e.g. it does not catch a leaked API)? Right?
>
> Basically my point is that the only way you get this pom correct is if
> you have a human involved in creating the requirements/verfication list.
> Tools can help with that, but hacking up the AS testsuite to verify it
> indirectly is no substitute.

Let me take a different tack.

I proposed we organize the current testsuite by dependency.  If we don't 
want to do that, what would we like to do?

* Status Quo?
* Some other organizational criteria?

I've heard a lot of criticisms with the proposal, but not too much in 
the way of alternate solutions.  So the concerns I'm putting forth here 
are not too important to others, or we're happy with the way things 
currently are?

S,
ALR

-- 
Andrew Lee Rubinger
Senior Software Engineer
JBoss by Red Hat
Twitter: @ALRubinger
http://about.me/alrubinger