[wildfly-dev] Call For Help (Testsuite)

[Aleksandar Kostadinov]

My experience shows that slow machines tend to reveal a lot of problems. 
That might be more of test suite deficiencies than project issues but 
they still do a lot of harm when found too late after being introduced.

I'm all for running the test suite at least daily on a range of machine 
types but I think a slow and a fast run will mostly help. A large 
instance will be an improvement in any case though.

Jason Greene wrote, On 06/14/2013 07:46 PM (EEST):
> Well the problem is that you really need more than one core to catch timing & memory visibility issues.
>
> On Jun 14, 2013, at 11:40 AM, Aleksandar Kostadinov <akostadi at redhat.com> wrote:
>
>> My hope was that test suite would run reliably on a smaller instance - at most medium which is the largest 32bit instance.
>>
>> The test suite should as well be run on modern multi-cpu/core machines but being run on slower ones will avoid issues later when certifying legacy hardware and cloud environments.
>>
>> Jason Greene wrote, On 06/14/2013 07:01 PM (EEST):
>>> Ah yes duh. We have to do some Linux hacks to get IPv6 multicast to work on the loopback with 2 addresses, but that should work on EC2 as well.
>>>
>>> We'd have to get a large instance because we need more than 1 core to adequately test. So I'll have to ask around about budget.
>>>
>>> On Jun 14, 2013, at 10:47 AM, Aleksandar Kostadinov <akostadi at redhat.com> wrote:
>>>
>>>> Multicast is not a problem. It works fine on the local machine as long as you don't expect another machine to see the messages.
>>>>
>>>> Jason Greene wrote, On 06/14/2013 06:35 PM (EEST):
>>>>> Unfortunately EC2 won't run the test suite since some of the tests rely on multicast. Lightning is virtualize but its at capacity. We may be getting some more hardware though which would help.
>>>>>
>>>>> On Jun 14, 2013, at 10:32 AM, Aleksandar Kostadinov <akostadi at redhat.com> wrote:
>>>>>
>>>>>> Not exactly an answer but can we have a persistent amazon EC2 small instance to run the testsuite? This way such intermittent issues would be caught early after being introduced.
>>>>>>
>>>>>> Jason Greene wrote, On 06/12/2013 10:01 PM (EEST):
>>>>>>> We still have a number intermittent test failures that have been around for over a year now. I'm asking for everyone's help in doing what we can to make them stable. If you submit a PR, and you see what looks like an intermittent failure, can you do some investigation and report your findings even if it is not your area? It would be awesome if you can report what you find to the mailing list, and rope in help.
>>>>>>>
>>>>>>> Nearly all of these seem to only occur when virtualization is involved, so if need be we can work out a plan to create either a special run to capture diagnostic info, or I can give access to a dedicated slave.
>>>>>>>
>>>>>>> If anyone has any further ideas on how to tackle these issues I am all ears.
>>>>>>>
>>>>>>> --
>>>>>>> Jason T. Greene
>>>>>>> WildFly Lead / JBoss EAP Platform Architect
>>>>>>> JBoss, a division of Red Hat
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> wildfly-dev mailing list
>>>>>>> wildfly-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>>>>
>>>>>
>>>>> --
>>>>> Jason T. Greene
>>>>> WildFly Lead / JBoss EAP Platform Architect
>>>>> JBoss, a division of Red Hat
>>>>>
>>>
>>> --
>>> Jason T. Greene
>>> WildFly Lead / JBoss EAP Platform Architect
>>> JBoss, a division of Red Hat
>>>
>
> --
> Jason T. Greene
> WildFly Lead / JBoss EAP Platform Architect
> JBoss, a division of Red Hat
>