[infinispan-dev] Concerns about the testsuite state

[Sanne Grinovero]

Hi all,
I was mentioning on IRC today that I've seen many failures in the past
hours, as I was trying to verify a simple PR.

Tristan suggested to share some failures, so here are the outcomes of
my first attempts to build Infinispan, each time restarting after a
module would fail:

Failed tests:
  ClusterListenerDistTest>AbstractClusterListenerNonTxTest.testPrimaryOwnerGoesDownAfterSendingEvent:81
expected [ClusterListenerDistTest-NodeC-54048] but found
[ClusterListenerDistTest-NodeA-38805]
  ClusterListenerDistTest.testPrimaryOwnerGoesDownBeforeSendingEvent:78
expected [ClusterListenerDistTest-NodeDR-1944] but found
[ClusterListenerDistTest-NodeDP-23754]

Failed tests:
  LocalModeNoPassivationTest>LocalModePassivationTest.testValuesWithEvictedEntries:219
Value: 295 was not found!
  InfinispanNodeFailureTest.killedNodeDoesNotBreakReplaceCommand:135
expected:<false> but was:<true>

Failed tests:
  DistL1WriteSkewTest>AbstractClusteredWriteSkewTest.testConditionalPutFailWriteSkewWithPassivation:184->AbstractClusteredWriteSkewTest.doTestWriteSkewWithPassivation:316
The key was not evicted after 10 inserts
  RecoveryEnabledWriteSkewTest>AbstractClusteredWriteSkewTest.testRemoveFailWriteSkewWithPassivationOnNonOwner:160->AbstractClusteredWriteSkewTest.doTestWriteSkewWithPassivation:316
The key was not evicted after 10 inserts

Failed tests:
  NonTxBackupOwnerBecomingPrimaryOwnerTest.testPrimaryOwnerChangingDuringPutOverwrite:75->doTest:165
» Runtime
  ClusteredTxConditionalCommandTest>ClusteredConditionalCommandTest.testPutIfAbsentOnNonOwnerShared:246->ClusteredConditionalCommandTest.doTest:121->assertLoadAfterOperation:46
primary owner load expected:<1> but was:<0>
  ReplCommandForwardingTest.testForwardToJoinerAsyncPrepare:119->testForwardToJoinerAsyncTx:161
» IllegalState

Failed tests:
org.infinispan.functional.FunctionalCachestoreTest.testWriteLoad[passivation=false](org.infinispan.functional.FunctionalCachestoreTest)
  Run 1: PASS
  Run 2: PASS
  Run 3: PASS
  Run 4: PASS
  Run 5: PASS
  Run 6: PASS
  Run 7: PASS
  Run 8: PASS
  Run 9: FunctionalCachestoreTest.testWriteLoad:58->lambda$testWriteLoad$3:58
FunctionalCachestoreTest[passivation=false]-NodeB-46507 expected
[false] but found [true]
  Run 10: PASS
  Run 11: PASS
  Run 12: PASS
  Run 13: PASS
  Run 14: PASS
  Run 15: PASS
  Run 16: PASS

org.infinispan.functional.FunctionalCachestoreTest.testWriteLoad[passivation=true](org.infinispan.functional.FunctionalCachestoreTest)
  Run 1: PASS
  Run 2: PASS
  Run 3: PASS
  Run 4: PASS
  Run 5: PASS
  Run 6: PASS
  Run 7: PASS
  Run 8: PASS
  Run 9: PASS
  Run 10: PASS
  Run 11: PASS
  Run 12: PASS
  Run 13: PASS
  Run 14: FunctionalCachestoreTest.testWriteLoad:58->lambda$testWriteLoad$3:58
FunctionalCachestoreTest[passivation=true]-NodeB-39167 expected
[false] but found [true]
  Run 15: PASS
  Run 16: PASS

  DistTotalOrderL1WriteSkewTest>AbstractClusteredWriteSkewTest.testConditionalPutFailWriteSkewWithPassivation:184->AbstractClusteredWriteSkewTest.doTestWriteSkewWithPassivation:316
The key was not evicted after 10 inserts
  DistTotalOrderWriteSkewTest>AbstractClusteredWriteSkewTest.testConditionalRemoveFailWriteSkewWithPassivation:200->AbstractClusteredWriteSkewTest.doTestWriteSkewWithPassivation:316
The key was not evicted after 10 inserts

Tests run: 8427, Failures: 4, Errors: 0, Skipped: 0

Failed tests:
  SecureServerFailureRetryTest>HitsAwareCacheManagersTest.createBeforeMethod:114->MultipleCacheManagersTest.createBeforeMethod:119->MultipleCacheManagersTest.callCreateCacheManagers:109->AbstractRetryTest.createCacheManagers:63->createStartHotRodServer:27
» IllegalState

Failed tests:
  DistWriteSkewTest>AbstractClusteredWriteSkewTest.testConditionalPutFailWriteSkewWithPassivationOnNonOwner:192->AbstractClusteredWriteSkewTest.doTestWriteSkewWithPassivation:316
The key was not evicted after 10 inserts
  DistWriteSkewTest>AbstractClusteredWriteSkewTest.testConditionalReplaceWriteSkewWithPassivationOnNonOwner:220->AbstractClusteredWriteSkewTest.doTestWriteSkewWithPassivation:316
The key was not evicted after 10 inserts

Tests run: 8457, Failures: 2, Errors: 0, Skipped: 0

AFAIR in one case it was able to go beyond infinispan-core, in all
others these are failures in core.

Thanks,
Sanne