[infinispan-issues] [JBoss JIRA] (ISPN-10241) ListenerExceptionTest random failures in async test methods

ListenerExceptionTest random failures in async test methods ----------------------------------------------------------- Key: ISPN-10241 URL: https://issues.jboss.org/browse/ISPN-10241 Project: Infinispan Issue Type: Bug Components: Test Suite - Core Affects Versions: 10.0.0.Beta3, 9.4.14.Final Reporter: Dan Berindei Assignee: Dan Berindei Priority: Major Labels: testsuite_stability Fix For: 10.0.0.Beta4, 9.4.15.Final {{ListenerExceptionTest.doCallsWithExcepListAsync}}, used by all the async test methods in the class, adds an {{ErrorInducingListenerAsync}} listener to the cache but never removes it. This means instead of having a single listener notification on the async notification executor for each of the 4 methods, there are 55 notifications in total. Tests do not wait for the listeners to finish, so they could in theory all be running or queued at the same time. In tests the async notification executor is smaller (max threads 6, max queue size 20), so it can reject some tasks: {noformat} 10:43:52,171 WARN (testng-ListenerExceptionTest:[]) [CacheNotifierImpl] ISPN000331: Unable to invoke method public void org.infinispan.notifications.cachelistener.ListenerExceptionTest$ErrorInducingListenerAsync.entryCreated(org.infinispan.notifications.cachelistener.event.CacheEntryEvent) throws java.lang.Exception on Object instance org.infinispan.notifications.cachelistener.ListenerExceptionTest$ErrorInducingListenerAsync@3fbdc638 org.infinispan.remoting.transport.jgroups.SuspectException: Simulated ASYNC suspicion when isPre=true and in ON_CREATE at org.infinispan.notifications.cachelistener.ListenerExceptionTest$ErrorInducingListenerAsync.throwSuspectException(ListenerExceptionTest.java:204) ~[test-classes/:?] at org.infinispan.notifications.cachelistener.ListenerExceptionTest$ErrorInducingListenerAsync.injectFailure(ListenerExceptionTest.java:197) ~[test-classes/:?] at org.infinispan.notifications.cachelistener.ListenerExceptionTest$ErrorInducingListenerAsync.entryCreated(ListenerExceptionTest.java:183) ~[test-classes/:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.infinispan.notifications.impl.AbstractListenerImpl$ListenerInvocationImpl.lambda$invoke$1(AbstractListenerImpl.java:416) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExecutor.java:2027) [?:?] at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:825) [?:?] at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1355) [?:?] at org.infinispan.executors.LazyInitializingExecutorService.execute(LazyInitializingExecutorService.java:119) [classes/:?] at org.infinispan.notifications.impl.AbstractListenerImpl$ListenerInvocationImpl.invoke(AbstractListenerImpl.java:448) [classes/:?] at org.infinispan.notifications.cachelistener.CacheNotifierImpl$BaseCacheEntryListenerInvocation.doRealInvocation(CacheNotifierImpl.java:1853) [classes/:?] at org.infinispan.notifications.cachelistener.CacheNotifierImpl$BaseCacheEntryListenerInvocation.doRealInvocation(CacheNotifierImpl.java:1849) [classes/:?] at org.infinispan.notifications.cachelistener.CacheNotifierImpl$BaseCacheEntryListenerInvocation.invokeNoChecks(CacheNotifierImpl.java:1844) [classes/:?] at org.infinispan.notifications.cachelistener.CacheNotifierImpl$BaseCacheEntryListenerInvocation.invoke(CacheNotifierImpl.java:1817) [classes/:?] at org.infinispan.notifications.cachelistener.CacheNotifierImpl.doNotifyCreated(CacheNotifierImpl.java:410) [classes/:?] at org.infinispan.notifications.cachelistener.CacheNotifierImpl.notifyCacheEntryCreated(CacheNotifierImpl.java:392) [classes/:?] at org.infinispan.interceptors.impl.CallInterceptor.performPut(CallInterceptor.java:213) [classes/:?] at org.infinispan.interceptors.impl.CallInterceptor.visitPutKeyValueCommand(CallInterceptor.java:200) [classes/:?] at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:61) [classes/:?] at org.infinispan.interceptors.impl.CallInterceptor.visitCommand(CallInterceptor.java:167) [classes/:?] ... at org.infinispan.notifications.cachelistener.ListenerExceptionTest.testPreOpExceptionListenerOnPutAsync(ListenerExceptionTest.java:72) [test-classes/:?] {noformat} The test expects the listener to run on an async notification thread, not on the main thread, and it fails: {noformat} 10:43:52,175 ERROR (testng-ListenerExceptionTest:[]) [TestSuiteProgress] Test failed: org.infinispan.notifications.cachelistener.ListenerExceptionTest.testPreOpExceptionListenerOnPutAsync java.lang.AssertionError: null at org.infinispan.notifications.cachelistener.ListenerExceptionTest.doCallsWithExcepListAsync(ListenerExceptionTest.java:121) ~[test-classes/:?] at org.infinispan.notifications.cachelistener.ListenerExceptionTest.testPreOpExceptionListenerOnPutAsync(ListenerExceptionTest.java:73) ~[test-classes/:?] {noformat}