[JBoss JIRA] (ISPN-8431) ScatteredSplitAndMergeTest random failures
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-8431?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-8431:
------------------------------------
It seems much easier to reproduce if I added a random delay before updating the topology in {{StateConsumerImpl}}. To be fair I was testing ISPN-7682 my changes, replacing all uses of {{StateConsumerImpl}}'s topology with the {{DistributionManager}} topology, so I was only delaying the {{DistributionManagerImpl.setCacheTopology()}} call.
> ScatteredSplitAndMergeTest random failures
> ------------------------------------------
>
> Key: ISPN-8431
> URL: https://issues.jboss.org/browse/ISPN-8431
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 9.2.0.Alpha1
> Environment: Jenkins
> Reporter: Tristan Tarrant
> Assignee: Radim Vansa
> Labels: testsuite_stability
> Attachments: ScatteredSplitAndMergeTest_20180129.log.gz, ScatteredSplitAndMergeTest_ISPN-7919_RpcManager_ResponseCollector_20171212.log.gz
>
>
> http://ci.infinispan.org/job/Infinispan/job/master/214/testReport/junit/o...
> java.lang.AssertionError: expected [null] but found [v0]
> at org.infinispan.partitionhandling.ScatteredSplitAndMergeTest.testSplitAndMerge(ScatteredSplitAndMergeTest.java:80)
> at org.infinispan.partitionhandling.ScatteredSplitAndMergeTest.testSplitAndMerge5(ScatteredSplitAndMergeTest.java:51)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> ... Removed 20 stack frames
>
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 10 months
[JBoss JIRA] (ISPN-9139) CacheWriterInterceptor invokes batch persistence operations with potentially empty entry sets
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-9139?page=com.atlassian.jira.plugin.... ]
Ryan Emerson updated ISPN-9139:
-------------------------------
Sprint: Sprint 9.3.0.CR1
> CacheWriterInterceptor invokes batch persistence operations with potentially empty entry sets
> ---------------------------------------------------------------------------------------------
>
> Key: ISPN-9139
> URL: https://issues.jboss.org/browse/ISPN-9139
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 9.2.2.Final
> Reporter: Moritz Becker
> Assignee: Ryan Emerson
>
> In {{org.infinispan.interceptors.impl.CacheWriterInterceptor#store}}, you have the following:
> {code:java}
> persistenceManager.writeBatchToAllNonTxStores(sharedMods.getMarshalledEntries(), BOTH, 0);
> persistenceManager.writeBatchToAllNonTxStores(nonSharedMods.getMarshalledEntries(), PRIVATE, 0);
> persistenceManager.deleteBatchFromAllNonTxStores(sharedMods.getKeysToRemove(), BOTH, 0);
> persistenceManager.deleteBatchFromAllNonTxStores(nonSharedMods.getKeysToRemove(), PRIVATE, 0);
> {code}
> The batch deletion causes an error with JPAStore in case of an empty key set because the an SQL query like {{where id in ()}}, i.e. the comparison set for the in clause is empty. At least with postgres, this causes the following exception:
> {code}
> ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (EE-ManagedExecutorService-default-Thread-2) ISPN000136: Error executing command CommitCommand, writing keys []: org.infinispan.persistence.jpa.JpaStoreException: Exception caught in deleteBatch()
> at org.infinispan.persistence.jpa.JpaStore.deleteBatch(JpaStore.java:350)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.lambda$deleteBatchFromAllNonTxStores$14(PersistenceManagerImpl.java:562)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.deleteBatchFromAllNonTxStores(PersistenceManagerImpl.java:562)
> at org.infinispan.interceptors.impl.CacheWriterInterceptor.store(CacheWriterInterceptor.java:406)
> at org.infinispan.interceptors.impl.CacheWriterInterceptor.commitCommand(CacheWriterInterceptor.java:123)
> at org.infinispan.interceptors.impl.CacheWriterInterceptor.visitCommitCommand(CacheWriterInterceptor.java:102)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:54)
> at org.infinispan.interceptors.impl.TransactionalStoreInterceptor.visitCommitCommand(TransactionalStoreInterceptor.java:53)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:54)
> at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54)
> at org.infinispan.interceptors.DDAsyncInterceptor.visitCommitCommand(DDAsyncInterceptor.java:142)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndFinally(BaseAsyncInterceptor.java:150)
> at org.infinispan.interceptors.impl.EntryWrappingInterceptor.visitCommitCommand(EntryWrappingInterceptor.java:183)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:98)
> at org.infinispan.interceptors.impl.NotificationInterceptor.visitCommitCommand(NotificationInterceptor.java:46)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndFinally(BaseAsyncInterceptor.java:150)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.visitCommitCommand(AbstractTxLockingInterceptor.java:51)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:98)
> at org.infinispan.interceptors.impl.TxInterceptor.finishCommit(TxInterceptor.java:189)
> at org.infinispan.interceptors.impl.TxInterceptor.visitCommitCommand(TxInterceptor.java:183)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:54)
> at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54)
> at org.infinispan.interceptors.DDAsyncInterceptor.visitCommitCommand(DDAsyncInterceptor.java:142)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:123)
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:90)
> at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137)
> at org.infinispan.transaction.impl.TransactionCoordinator.commitInternal(TransactionCoordinator.java:219)
> at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:161)
> at org.infinispan.transaction.impl.TransactionTable.afterCompletion(TransactionTable.java:851)
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:33)
> at org.wildfly.transaction.client.AbstractTransaction.performConsumer(AbstractTransaction.java:196)
> at org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization.afterCompletion(AbstractTransaction.java:279)
> at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:542)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:101)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1289)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> at org.wildfly.transaction.client.LocalTransaction.commitAndDissociate(LocalTransaction.java:73)
> at org.wildfly.transaction.client.ContextTransactionManager.commit(ContextTransactionManager.java:71)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.commitIfNeeded(PersistenceManagerImpl.java:874)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.preloadKey(PersistenceManagerImpl.java:830)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.lambda$preload$2(PersistenceManagerImpl.java:279)
> at org.infinispan.persistence.jpa.JpaStore$LoadingProcessTask.call(JpaStore.java:915)
> at org.infinispan.persistence.jpa.JpaStore$LoadingProcessTask.call(JpaStore.java:843)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:20)
> at java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:181)
> at org.infinispan.executors.ExecutorAllCompletionService.submit(ExecutorAllCompletionService.java:31)
> at org.infinispan.persistence.jpa.JpaStore.process(JpaStore.java:644)
> at org.infinispan.persistence.jpa.JpaStore.process(JpaStore.java:583)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.preload(PersistenceManagerImpl.java:271)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:229)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:1020)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:421)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:644)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:589)
> at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:475)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:461)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:447)
> at com.ordami.geo.service.impl.service.MatrixUpdateServiceImpl.updateMatrixEntry(MatrixUpdateServiceImpl.java:157)
> at com.ordami.geo.service.impl.service.MatrixUpdateServiceImpl$Proxy$_$$_WeldSubclass.updateMatrixEntry$$super(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:49)
> at org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:77)
> at com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:120)
> at com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorRequired.doIntercept(TransactionalInterceptorRequired.java:53)
> at com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorBase.intercept(TransactionalInterceptorBase.java:79)
> at com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorRequired.intercept(TransactionalInterceptorRequired.java:47)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73)
> at org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84)
> at org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72)
> at org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56)
> at org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79)
> at org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68)
> at com.ordami.geo.service.impl.service.MatrixUpdateServiceImpl$Proxy$_$$_WeldSubclass.updateMatrixEntry(Unknown Source)
> at com.ordami.geo.service.impl.service.MatrixUpdateServiceImpl$Proxy$_$$_WeldClientProxy.updateMatrixEntry(Unknown Source)
> at com.ordami.geo.service.impl.service.MatrixUpdateServiceImpl$MatrixEntryUpdateTask.run(MatrixUpdateServiceImpl.java:207)
> at org.jboss.as.ee.concurrent.ControlPointUtils$ControlledRunnable.run(ControlPointUtils.java:105)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at org.glassfish.enterprise.concurrent.internal.ManagedFutureTask.run(ManagedFutureTask.java:141)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
> at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)
> Caused by: javax.persistence.PersistenceException: org.hibernate.exception.SQLGrammarException: could not execute statement
> at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:149)
> at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:157)
> at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1514)
> at org.infinispan.persistence.jpa.JpaStore.deleteBatch(JpaStore.java:334)
> ... 133 more
> Caused by: org.hibernate.exception.SQLGrammarException: could not execute statement
> at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:106)
> at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
> at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111)
> at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97)
> at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:178)
> at org.hibernate.hql.internal.ast.exec.BasicExecutor.doExecute(BasicExecutor.java:100)
> at org.hibernate.hql.internal.ast.exec.BasicExecutor.execute(BasicExecutor.java:59)
> at org.hibernate.hql.internal.ast.exec.DeleteExecutor.execute(DeleteExecutor.java:110)
> at org.hibernate.hql.internal.ast.QueryTranslatorImpl.executeUpdate(QueryTranslatorImpl.java:450)
> at org.hibernate.engine.query.spi.HQLQueryPlan.performExecuteUpdate(HQLQueryPlan.java:374)
> at org.hibernate.internal.SessionImpl.executeUpdate(SessionImpl.java:1514)
> at org.hibernate.query.internal.AbstractProducedQuery.doExecuteUpdate(AbstractProducedQuery.java:1526)
> at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1504)
> ... 134 more
> Caused by: org.postgresql.util.PSQLException: ERROR: syntax error at or near ")"
> Position: 105
> at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2477)
> at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2190)
> at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:300)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:428)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354)
> at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:169)
> at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:136)
> at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeUpdate(WrappedPreparedStatement.java:537)
> at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:175)
> ... 142 more
> {code}
> Since you are doing expensive locking operations in all batch operations, I propose to change is code part to:
> {code:java}
> if (!sharedMods.getMarshalledEntries().isEmpty()) {
> persistenceManager.writeBatchToAllNonTxStores(sharedMods.getMarshalledEntries(), BOTH, 0);
> }
> if (!nonSharedMods.getMarshalledEntries().isEmpty()) {
> persistenceManager.writeBatchToAllNonTxStores(nonSharedMods.getMarshalledEntries(), PRIVATE, 0);
> }
> if (!sharedMods.getKeysToRemove().isEmpty()) {
> persistenceManager.deleteBatchFromAllNonTxStores(sharedMods.getKeysToRemove(), BOTH, 0);
> }
> if (!nonSharedMods.getKeysToRemove().isEmpty()) {
> persistenceManager.deleteBatchFromAllNonTxStores(nonSharedMods.getKeysToRemove(), PRIVATE, 0);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 10 months
[JBoss JIRA] (ISPN-9139) CacheWriterInterceptor invokes batch persistence operations with potentially empty entry sets
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-9139?page=com.atlassian.jira.plugin.... ]
Ryan Emerson updated ISPN-9139:
-------------------------------
Component/s: Loaders and Stores
> CacheWriterInterceptor invokes batch persistence operations with potentially empty entry sets
> ---------------------------------------------------------------------------------------------
>
> Key: ISPN-9139
> URL: https://issues.jboss.org/browse/ISPN-9139
> Project: Infinispan
> Issue Type: Bug
> Components: Loaders and Stores
> Affects Versions: 9.2.2.Final
> Reporter: Moritz Becker
> Assignee: Ryan Emerson
>
> In {{org.infinispan.interceptors.impl.CacheWriterInterceptor#store}}, you have the following:
> {code:java}
> persistenceManager.writeBatchToAllNonTxStores(sharedMods.getMarshalledEntries(), BOTH, 0);
> persistenceManager.writeBatchToAllNonTxStores(nonSharedMods.getMarshalledEntries(), PRIVATE, 0);
> persistenceManager.deleteBatchFromAllNonTxStores(sharedMods.getKeysToRemove(), BOTH, 0);
> persistenceManager.deleteBatchFromAllNonTxStores(nonSharedMods.getKeysToRemove(), PRIVATE, 0);
> {code}
> The batch deletion causes an error with JPAStore in case of an empty key set because the an SQL query like {{where id in ()}}, i.e. the comparison set for the in clause is empty. At least with postgres, this causes the following exception:
> {code}
> ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (EE-ManagedExecutorService-default-Thread-2) ISPN000136: Error executing command CommitCommand, writing keys []: org.infinispan.persistence.jpa.JpaStoreException: Exception caught in deleteBatch()
> at org.infinispan.persistence.jpa.JpaStore.deleteBatch(JpaStore.java:350)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.lambda$deleteBatchFromAllNonTxStores$14(PersistenceManagerImpl.java:562)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.deleteBatchFromAllNonTxStores(PersistenceManagerImpl.java:562)
> at org.infinispan.interceptors.impl.CacheWriterInterceptor.store(CacheWriterInterceptor.java:406)
> at org.infinispan.interceptors.impl.CacheWriterInterceptor.commitCommand(CacheWriterInterceptor.java:123)
> at org.infinispan.interceptors.impl.CacheWriterInterceptor.visitCommitCommand(CacheWriterInterceptor.java:102)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:54)
> at org.infinispan.interceptors.impl.TransactionalStoreInterceptor.visitCommitCommand(TransactionalStoreInterceptor.java:53)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:54)
> at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54)
> at org.infinispan.interceptors.DDAsyncInterceptor.visitCommitCommand(DDAsyncInterceptor.java:142)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndFinally(BaseAsyncInterceptor.java:150)
> at org.infinispan.interceptors.impl.EntryWrappingInterceptor.visitCommitCommand(EntryWrappingInterceptor.java:183)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:98)
> at org.infinispan.interceptors.impl.NotificationInterceptor.visitCommitCommand(NotificationInterceptor.java:46)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndFinally(BaseAsyncInterceptor.java:150)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.visitCommitCommand(AbstractTxLockingInterceptor.java:51)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:98)
> at org.infinispan.interceptors.impl.TxInterceptor.finishCommit(TxInterceptor.java:189)
> at org.infinispan.interceptors.impl.TxInterceptor.visitCommitCommand(TxInterceptor.java:183)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:54)
> at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54)
> at org.infinispan.interceptors.DDAsyncInterceptor.visitCommitCommand(DDAsyncInterceptor.java:142)
> at org.infinispan.commands.tx.CommitCommand.acceptVisitor(CommitCommand.java:38)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:123)
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:90)
> at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137)
> at org.infinispan.transaction.impl.TransactionCoordinator.commitInternal(TransactionCoordinator.java:219)
> at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:161)
> at org.infinispan.transaction.impl.TransactionTable.afterCompletion(TransactionTable.java:851)
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:33)
> at org.wildfly.transaction.client.AbstractTransaction.performConsumer(AbstractTransaction.java:196)
> at org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization.afterCompletion(AbstractTransaction.java:279)
> at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:542)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:101)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1289)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
> at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
> at org.wildfly.transaction.client.LocalTransaction.commitAndDissociate(LocalTransaction.java:73)
> at org.wildfly.transaction.client.ContextTransactionManager.commit(ContextTransactionManager.java:71)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.commitIfNeeded(PersistenceManagerImpl.java:874)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.preloadKey(PersistenceManagerImpl.java:830)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.lambda$preload$2(PersistenceManagerImpl.java:279)
> at org.infinispan.persistence.jpa.JpaStore$LoadingProcessTask.call(JpaStore.java:915)
> at org.infinispan.persistence.jpa.JpaStore$LoadingProcessTask.call(JpaStore.java:843)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:20)
> at java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:181)
> at org.infinispan.executors.ExecutorAllCompletionService.submit(ExecutorAllCompletionService.java:31)
> at org.infinispan.persistence.jpa.JpaStore.process(JpaStore.java:644)
> at org.infinispan.persistence.jpa.JpaStore.process(JpaStore.java:583)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.preload(PersistenceManagerImpl.java:271)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
> at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
> at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
> at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
> at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
> at org.infinispan.security.Security.doPrivileged(Security.java:76)
> at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
> at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:229)
> at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:1020)
> at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:421)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:644)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:589)
> at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:475)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:461)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:447)
> at com.ordami.geo.service.impl.service.MatrixUpdateServiceImpl.updateMatrixEntry(MatrixUpdateServiceImpl.java:157)
> at com.ordami.geo.service.impl.service.MatrixUpdateServiceImpl$Proxy$_$$_WeldSubclass.updateMatrixEntry$$super(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:49)
> at org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:77)
> at com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:120)
> at com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorRequired.doIntercept(TransactionalInterceptorRequired.java:53)
> at com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorBase.intercept(TransactionalInterceptorBase.java:79)
> at com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorRequired.intercept(TransactionalInterceptorRequired.java:47)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73)
> at org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84)
> at org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72)
> at org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56)
> at org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79)
> at org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68)
> at com.ordami.geo.service.impl.service.MatrixUpdateServiceImpl$Proxy$_$$_WeldSubclass.updateMatrixEntry(Unknown Source)
> at com.ordami.geo.service.impl.service.MatrixUpdateServiceImpl$Proxy$_$$_WeldClientProxy.updateMatrixEntry(Unknown Source)
> at com.ordami.geo.service.impl.service.MatrixUpdateServiceImpl$MatrixEntryUpdateTask.run(MatrixUpdateServiceImpl.java:207)
> at org.jboss.as.ee.concurrent.ControlPointUtils$ControlledRunnable.run(ControlPointUtils.java:105)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at org.glassfish.enterprise.concurrent.internal.ManagedFutureTask.run(ManagedFutureTask.java:141)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
> at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)
> Caused by: javax.persistence.PersistenceException: org.hibernate.exception.SQLGrammarException: could not execute statement
> at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:149)
> at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:157)
> at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1514)
> at org.infinispan.persistence.jpa.JpaStore.deleteBatch(JpaStore.java:334)
> ... 133 more
> Caused by: org.hibernate.exception.SQLGrammarException: could not execute statement
> at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:106)
> at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
> at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111)
> at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97)
> at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:178)
> at org.hibernate.hql.internal.ast.exec.BasicExecutor.doExecute(BasicExecutor.java:100)
> at org.hibernate.hql.internal.ast.exec.BasicExecutor.execute(BasicExecutor.java:59)
> at org.hibernate.hql.internal.ast.exec.DeleteExecutor.execute(DeleteExecutor.java:110)
> at org.hibernate.hql.internal.ast.QueryTranslatorImpl.executeUpdate(QueryTranslatorImpl.java:450)
> at org.hibernate.engine.query.spi.HQLQueryPlan.performExecuteUpdate(HQLQueryPlan.java:374)
> at org.hibernate.internal.SessionImpl.executeUpdate(SessionImpl.java:1514)
> at org.hibernate.query.internal.AbstractProducedQuery.doExecuteUpdate(AbstractProducedQuery.java:1526)
> at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1504)
> ... 134 more
> Caused by: org.postgresql.util.PSQLException: ERROR: syntax error at or near ")"
> Position: 105
> at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2477)
> at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2190)
> at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:300)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:428)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354)
> at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:169)
> at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:136)
> at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeUpdate(WrappedPreparedStatement.java:537)
> at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:175)
> ... 142 more
> {code}
> Since you are doing expensive locking operations in all batch operations, I propose to change is code part to:
> {code:java}
> if (!sharedMods.getMarshalledEntries().isEmpty()) {
> persistenceManager.writeBatchToAllNonTxStores(sharedMods.getMarshalledEntries(), BOTH, 0);
> }
> if (!nonSharedMods.getMarshalledEntries().isEmpty()) {
> persistenceManager.writeBatchToAllNonTxStores(nonSharedMods.getMarshalledEntries(), PRIVATE, 0);
> }
> if (!sharedMods.getKeysToRemove().isEmpty()) {
> persistenceManager.deleteBatchFromAllNonTxStores(sharedMods.getKeysToRemove(), BOTH, 0);
> }
> if (!nonSharedMods.getKeysToRemove().isEmpty()) {
> persistenceManager.deleteBatchFromAllNonTxStores(nonSharedMods.getKeysToRemove(), PRIVATE, 0);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 10 months
[JBoss JIRA] (ISPN-9094) ArrayIndexOutOfBoundsException on server using scattered cache
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-9094?page=com.atlassian.jira.plugin.... ]
Ryan Emerson updated ISPN-9094:
-------------------------------
Fix Version/s: 9.2.4.Final
> ArrayIndexOutOfBoundsException on server using scattered cache
> ---------------------------------------------------------------
>
> Key: ISPN-9094
> URL: https://issues.jboss.org/browse/ISPN-9094
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.2.1.Final
> Reporter: Paul Ferraro
> Assignee: Radim Vansa
> Fix For: 9.3.0.Beta1, 9.2.4.Final
>
>
> We hit ArrayIndexOutOfBoundsException when running tests for RFE EAP7-867.
> EAP distribution was built from {{https://github.com/pferraro/wildfly/tree/scattered}} .
> Test description: Positive stress test (no failover), 4-node EAP cluster, clients: starting with 400 clients in the beginning, raising the number of clients to 6000 in the end of the test.
> Error occured on server dev215 around 7th iteration (can be seen in the performance report, link below):
> {code}
> [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) Exception in thread "transport-thread--p15-t25" java.lang.ArrayIndexOutOfBoundsException: 129
> [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) at org.infinispan.scattered.impl.ScatteredVersionManagerImpl.lambda$tryRegularInvalidations$4(ScatteredVersionManagerImpl.java:413)
> [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) at org.wildfly.clustering.service.concurrent.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:47)
> [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) at java.lang.Thread.run(Thread.java:748)
> {code}
> Clients were getting "SocketTimeoutException: Read timed out" exceptions even before the ArrayIndexOutOfBoundsException ocurred, but also after.
> Performance report (accessible only when connected to VPN):
> http://download.eng.brq.redhat.com/scratch/mvinkler/reports/2018-04-19_15...
> One can observe that dev215 CPU usage and network usage dropped after 7th iteration.
> dev215 server log link:
> https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-se...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 10 months