2011/6/10 Manik Surtani <msurtani(a)redhat.com&gt;: > Some tests - like ExpiryTest - rely on certain timings for the test to run, > and due to thread scheduling on our parallel test suite, tend to > occasionally fail on certain environments such as CloudBees: > https://infinispan.ci.cloudbees.com/job/Infinispan-master-JDK6-tcp/org.in... > In this example, entries are placed in the data container with a 30 second > lifespan, tested for existence, wait 30s, and test for non-existence. The > failure here is that the first test for existence fails since the thread is > de-scheduled for a period of time between storing the entry and the first > test. Ah, I was looking at the same test and came with this solution, I'm not sure if we where thinking the same: https://github.com/infinispan/infinispan/pull/375 > Upping the lifespan just moves the problem - and makes the test suite run > slower (got to wait for that lifespan before testing again). > How about we group such tests into a new group, "timeSensitiveTests", and > *don't* run these on CI environments (but *do* run them on local > environments where response times are more reasonable/predictable)? Can't we make it unlikely to happen, so to not test exactly at the millisecond in which the state is changed (without being able to check if it was just before or just after the event) ? > Thoughts? > Cheers > Manik > -- > Manik Surtani > manik(a)jboss.org > twitter.com/maniksurtani > Lead, Infinispan > http://www.infinispan.org > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

This is a very tricky one indeed. These might fail as well e.g. if we increase the number of concurrent threads running the test suite. I remember when switching from 1 thread to 10 there were many of these failing. I think your suggestion makes sense. On 10 Jun 2011, at 13:02, Manik Surtani wrote: > Some tests - like ExpiryTest - rely on certain timings for the test to run, and due to thread scheduling on our parallel test suite, tend to occasionally fail on certain environments such as CloudBees: > > https://infinispan.ci.cloudbees.com/job/Infinispan-master-JDK6-tcp/org.in... > > In this example, entries are placed in the data container with a 30 second lifespan, tested for existence, wait 30s, and test for non-existence. The failure here is that the first test for existence fails since the thread is de-scheduled for a period of time between storing the entry and the first test. > > Upping the lifespan just moves the problem - and makes the test suite run slower (got to wait for that lifespan before testing again). > > How about we group such tests into a new group, "timeSensitiveTests", and *don't* run these on CI environments (but *do* run them on local environments where response times are more reasonable/predictable)? > > Thoughts? > > Cheers > Manik > -- > Manik Surtani > manik(a)jboss.org > twitter.com/maniksurtani > > Lead, Infinispan > http://www.infinispan.org > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev