[JBoss JIRA] (ISPN-3455) Cache replication not warranted under load
by Lukasz Szelag (JIRA)
Lukasz Szelag created ISPN-3455:
-----------------------------------
Summary: Cache replication not warranted under load
Key: ISPN-3455
URL: https://issues.jboss.org/browse/ISPN-3455
Project: Infinispan
Issue Type: Feature Request
Affects Versions: 5.3.0.Final
Environment: JSE 1.6.0_45; Windows 7
Reporter: Lukasz Szelag
Assignee: Mircea Markus
Problem:
When running a replicated cache and repeatedly calling a cacheable method (using Spring cache abstraction), Infinispan enters an infinite replication loop. This can be confirmed by observing replication counts growing over time, where there are no cache misses.
Expected behavior:
Caches shouldn't be replicated when there is a cache hit.
Test case:
- 3 cluster members; asynchronous replication with a replication queue
- a cacheable method is executed repeatedly using 2 different keys
Notes:
- for some reason, this issue only occurs when using Enum arguments for a cache key; I was not able to replicate this when using int or String types (see com.designamus.infinispan.Main.works())
- the behavior is not deterministic (random), which points to a race condition
- the problem does not seem to be related to the Spring's default cache key generator; I was able to reproduce the same behavior with a custom cache key generator, which was thread-safe
- the cacheable method is executed only twice (once both keys are stored in the cache); subsequent invocations retrieve stored values from the cache; this can be confirmed by inspecting the log file
- the cache doesn't expire and entries are not evicted
- the memory usage grows over time, eventually causing OOM on a heavily loaded system
- since the issue is random in nature it may take a 3-4 attempts to reproduce it; I was successful in reproducing this behavior numerous times
Steps to reproduce:
1. Build a test project (mvn clean compile)
2. Execute /run.sh (this will spawn 3 JVMs)
3. Start JConsole to monitor 3 cluster members (jconsole localhost:17001 localhost:17002 localhost:17003)
4. Monitor "replicationCount" attribute under RpcManager for cache "MyCache" for all JVMs (see /replication-counts.png)
5. Observe that replication counts grow over time
6. Observe that all caches are of size 2 and there are no cache misses (see /cache-statistics.png)
If the issue cannot be reproduced (replication counts stay at the same level):
5. Terminate all 3 JVM processes (as a convenience you can execute /stop.sh)
6. Repeat steps 2 through 5 above
When testing the above scenario using a distributed mode, I observed some other anomalies (i.e. the cacheable method was executed multiple times, as if the value was not there). While this may be related, it deserves a separate JIRA.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (ISPN-3200) Allow KeyFilters to be applied to listeners
by Manik Surtani (JIRA)
[ https://issues.jboss.org/browse/ISPN-3200?page=com.atlassian.jira.plugin.... ]
Manik Surtani updated ISPN-3200:
--------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/2037
> Allow KeyFilters to be applied to listeners
> -------------------------------------------
>
> Key: ISPN-3200
> URL: https://issues.jboss.org/browse/ISPN-3200
> Project: Infinispan
> Issue Type: Feature Request
> Components: Listeners
> Affects Versions: 5.3.0.Final
> Reporter: Manik Surtani
> Assignee: Manik Surtani
> Labels: leads
> Fix For: 6.0.0.Beta1, 6.0.0.Final
>
>
> When registering a listener, users should be able to provide a {{KeyFilter}}, a simple interface that determines whether a listener is invoked or not based on whether the affected key(s) matches the filter.
> The proposed API may overload the {{addListener(Object listener)}} method on {{Cache}}, adding:
> {code}
> void addListener(Object listener, KeyFilter filter);
> {code}
> where {{KeyFilter}} may be an interface that looks like:
> {code}
> public interface KeyFilter {
> boolean match(Object key);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (ISPN-2815) Put to Transactional cache with enabled indexing to InfinispanDirectory doesn't work properly
by Anna Manukyan (JIRA)
[ https://issues.jboss.org/browse/ISPN-2815?page=com.atlassian.jira.plugin.... ]
Anna Manukyan updated ISPN-2815:
--------------------------------
Description:
In case when using Index enabled cache (no-matter in which mode), with enabled InfinispanDirectory and enabled transactions, the cache put throws exception, although then the data appears in the cache.
But then, when query is performed on that cache, it doesn't return any result.
The cache config is:
{code}
ConfigurationBuilder c = getDefaultStandaloneCacheConfig(true);
c.indexing()
.enable()
.indexLocalOnly(true)
.addProperty("hibernate.search.default.indexmanager", "org.infinispan.query.indexmanager.InfinispanIndexManager")
.addProperty("hibernate.search.default.directory_provider", "infinispan")
.addProperty("hibernate.search.default.exclusive_index_use", "false")
.addProperty("lucene_version", "LUCENE_36");
TestCacheManagerFactory.createCacheManager(c);
{code}
While performing put to the created cache, the following exception appears in the logs, although the program flow continues properly:
{code}
2013-02-12 13:03:00,733 ERROR [InvocationContextInterceptor] (Hibernate Search: Index updates queue processor for index indexA-1) ISPN000136: Execution error
org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [10 seconds] on key [write.lock|M|indexA] for requestor [GlobalTransaction:<null>:5:local]! Lock held by [GlobalTransaction:<null>:3:local]
at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:213)
at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:186)
at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:186)
at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:123)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.lockAndRecord(OptimisticLockingInterceptor.java:254)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitSingleKeyCommand(OptimisticLockingInterceptor.java:249)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitPutKeyValueCommand(OptimisticLockingInterceptor.java:242)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.acquireLocksVisitingCommands(OptimisticLockingInterceptor.java:335)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:123)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:128)
at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:115)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:141)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2013-02-12 13:03:00,738 ERROR [TransactionCoordinator] (Hibernate Search: Index updates queue processor for index indexA-1) Error while processing prepare
org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [10 seconds] on key [write.lock|M|indexA] for requestor [GlobalTransaction:<null>:5:local]! Lock held by [GlobalTransaction:<null>:3:local]
at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:213)
at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:186)
at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:186)
at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:123)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.lockAndRecord(OptimisticLockingInterceptor.java:254)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitSingleKeyCommand(OptimisticLockingInterceptor.java:249)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitPutKeyValueCommand(OptimisticLockingInterceptor.java:242)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.acquireLocksVisitingCommands(OptimisticLockingInterceptor.java:335)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:123)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:128)
at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:115)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:141)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2013-02-12 13:03:00,742 WARN [arjuna] (Hibernate Search: Index updates queue processor for index indexA-1) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a22814f:cd67:511a2f6a:9, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, topologyId=-1, isFromStateTransfer=false} org.infinispan.transaction.synchronization.SyncLocalTransaction@5} org.infinispan.transaction.synchronization.SynchronizationAdapter@24 >
org.infinispan.CacheException: Could not prepare.
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: javax.transaction.xa.XAException
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
... 25 more
2013-02-12 13:03:00,744 WARN [CacheImpl] (Hibernate Search: Index updates queue processor for index indexA-1) ISPN000160: Could not complete injected transaction.
javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.infinispan.CacheException: Could not prepare.
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
... 20 more
Caused by: javax.transaction.xa.XAException
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
... 25 more
2013-02-12 13:03:00,745 ERROR [LuceneBackendQueueTask] (Hibernate Search: Index updates queue processor for index indexA-1) HSEARCH000073: Error in backend
org.infinispan.CacheException: Could not commit implicit transaction
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1175)
at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
... 18 more
Caused by: org.infinispan.CacheException: Could not prepare.
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
... 20 more
Caused by: javax.transaction.xa.XAException
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
... 25 more
2013-02-12 13:03:00,748 ERROR [LogErrorHandler] (Hibernate Search: Index updates queue processor for index indexA-1) HSEARCH000058: Exception occurred org.infinispan.CacheException: Could not commit implicit transaction
Primary Failure:
Entity org.infinispan.query.api.TestEntity Id L:10 Work Type org.hibernate.search.backend.UpdateLuceneWork
org.infinispan.CacheException: Could not commit implicit transaction
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1175)
at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
... 18 more
Caused by: org.infinispan.CacheException: Could not prepare.
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
... 20 more
Caused by: javax.transaction.xa.XAException
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
... 25 more
{code}
After the put, the entries are in the cache, but the querying doesn't return results.
You can find attached the failing test.
IMPORTANT UPDATE: After looking deeper into the failure and into the failed cases, I've found out that this issue appears only if the ISPN Directory related caches are TRANSACTIONAL. If only the data storage cache is transactional, then no issue appears. E.g. if there is no specific caches are defined for ISPN directory (for index storing), then the caches are created from the default configuration (LuceneIndexesMetadata, LuceneIndexesData, LuceneIndexesLocking), and if the default configuration is set to be TRANSACTIONAL (which is the case described in the attachments), then this issue will appear.
was:
In case when using Index enabled cache (no-matter in which mode), with enabled InfinispanDirectory and enabled transactions, the cache put throws exception, although then the data appears in the cache.
But then, when query is performed on that cache, it doesn't return any result.
The cache config is:
{code}
ConfigurationBuilder c = getDefaultStandaloneCacheConfig(true);
c.indexing()
.enable()
.indexLocalOnly(true)
.addProperty("hibernate.search.default.indexmanager", "org.infinispan.query.indexmanager.InfinispanIndexManager")
.addProperty("hibernate.search.default.directory_provider", "infinispan")
.addProperty("hibernate.search.default.exclusive_index_use", "false")
.addProperty("lucene_version", "LUCENE_36");
TestCacheManagerFactory.createCacheManager(c);
{code}
While performing put to the created cache, the following exception appears in the logs, although the program flow continues properly:
{code}
2013-02-12 13:03:00,733 ERROR [InvocationContextInterceptor] (Hibernate Search: Index updates queue processor for index indexA-1) ISPN000136: Execution error
org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [10 seconds] on key [write.lock|M|indexA] for requestor [GlobalTransaction:<null>:5:local]! Lock held by [GlobalTransaction:<null>:3:local]
at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:213)
at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:186)
at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:186)
at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:123)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.lockAndRecord(OptimisticLockingInterceptor.java:254)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitSingleKeyCommand(OptimisticLockingInterceptor.java:249)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitPutKeyValueCommand(OptimisticLockingInterceptor.java:242)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.acquireLocksVisitingCommands(OptimisticLockingInterceptor.java:335)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:123)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:128)
at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:115)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:141)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2013-02-12 13:03:00,738 ERROR [TransactionCoordinator] (Hibernate Search: Index updates queue processor for index indexA-1) Error while processing prepare
org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [10 seconds] on key [write.lock|M|indexA] for requestor [GlobalTransaction:<null>:5:local]! Lock held by [GlobalTransaction:<null>:3:local]
at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:213)
at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:186)
at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:186)
at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:123)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.lockAndRecord(OptimisticLockingInterceptor.java:254)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitSingleKeyCommand(OptimisticLockingInterceptor.java:249)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitPutKeyValueCommand(OptimisticLockingInterceptor.java:242)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.acquireLocksVisitingCommands(OptimisticLockingInterceptor.java:335)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:123)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:128)
at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:115)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:141)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2013-02-12 13:03:00,742 WARN [arjuna] (Hibernate Search: Index updates queue processor for index indexA-1) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a22814f:cd67:511a2f6a:9, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, topologyId=-1, isFromStateTransfer=false} org.infinispan.transaction.synchronization.SyncLocalTransaction@5} org.infinispan.transaction.synchronization.SynchronizationAdapter@24 >
org.infinispan.CacheException: Could not prepare.
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: javax.transaction.xa.XAException
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
... 25 more
2013-02-12 13:03:00,744 WARN [CacheImpl] (Hibernate Search: Index updates queue processor for index indexA-1) ISPN000160: Could not complete injected transaction.
javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.infinispan.CacheException: Could not prepare.
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
... 20 more
Caused by: javax.transaction.xa.XAException
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
... 25 more
2013-02-12 13:03:00,745 ERROR [LuceneBackendQueueTask] (Hibernate Search: Index updates queue processor for index indexA-1) HSEARCH000073: Error in backend
org.infinispan.CacheException: Could not commit implicit transaction
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1175)
at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
... 18 more
Caused by: org.infinispan.CacheException: Could not prepare.
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
... 20 more
Caused by: javax.transaction.xa.XAException
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
... 25 more
2013-02-12 13:03:00,748 ERROR [LogErrorHandler] (Hibernate Search: Index updates queue processor for index indexA-1) HSEARCH000058: Exception occurred org.infinispan.CacheException: Could not commit implicit transaction
Primary Failure:
Entity org.infinispan.query.api.TestEntity Id L:10 Work Type org.hibernate.search.backend.UpdateLuceneWork
org.infinispan.CacheException: Could not commit implicit transaction
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1175)
at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
... 18 more
Caused by: org.infinispan.CacheException: Could not prepare.
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
... 20 more
Caused by: javax.transaction.xa.XAException
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
... 25 more
{code}
After the put, the entries are in the cache, but the querying doesn't return results.
You can find attached the failing test.
> Put to Transactional cache with enabled indexing to InfinispanDirectory doesn't work properly
> ---------------------------------------------------------------------------------------------
>
> Key: ISPN-2815
> URL: https://issues.jboss.org/browse/ISPN-2815
> Project: Infinispan
> Issue Type: Bug
> Components: Querying
> Affects Versions: 5.2.0.Final
> Reporter: Anna Manukyan
> Assignee: Sanne Grinovero
> Labels: stable_embedded_query
> Fix For: 6.0.0.Final
>
> Attachments: InfinispanDirectoryNonIndexedValuesTest.java, NullCollectionElementsClusteredTest.java
>
>
> In case when using Index enabled cache (no-matter in which mode), with enabled InfinispanDirectory and enabled transactions, the cache put throws exception, although then the data appears in the cache.
> But then, when query is performed on that cache, it doesn't return any result.
> The cache config is:
> {code}
> ConfigurationBuilder c = getDefaultStandaloneCacheConfig(true);
> c.indexing()
> .enable()
> .indexLocalOnly(true)
> .addProperty("hibernate.search.default.indexmanager", "org.infinispan.query.indexmanager.InfinispanIndexManager")
> .addProperty("hibernate.search.default.directory_provider", "infinispan")
> .addProperty("hibernate.search.default.exclusive_index_use", "false")
> .addProperty("lucene_version", "LUCENE_36");
> TestCacheManagerFactory.createCacheManager(c);
> {code}
> While performing put to the created cache, the following exception appears in the logs, although the program flow continues properly:
> {code}
> 2013-02-12 13:03:00,733 ERROR [InvocationContextInterceptor] (Hibernate Search: Index updates queue processor for index indexA-1) ISPN000136: Execution error
> org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [10 seconds] on key [write.lock|M|indexA] for requestor [GlobalTransaction:<null>:5:local]! Lock held by [GlobalTransaction:<null>:3:local]
> at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:213)
> at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:186)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:186)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:123)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.lockAndRecord(OptimisticLockingInterceptor.java:254)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitSingleKeyCommand(OptimisticLockingInterceptor.java:249)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitPutKeyValueCommand(OptimisticLockingInterceptor.java:242)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.acquireLocksVisitingCommands(OptimisticLockingInterceptor.java:335)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:123)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:128)
> at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:115)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
> at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
> at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:141)
> at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
> at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
> at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
> at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
> at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
> at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
> at org.apache.lucene.store.Lock.obtain(Lock.java:72)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
> at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
> at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
> at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
> at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
> at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
> at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 2013-02-12 13:03:00,738 ERROR [TransactionCoordinator] (Hibernate Search: Index updates queue processor for index indexA-1) Error while processing prepare
> org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [10 seconds] on key [write.lock|M|indexA] for requestor [GlobalTransaction:<null>:5:local]! Lock held by [GlobalTransaction:<null>:3:local]
> at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:213)
> at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:186)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:186)
> at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:123)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.lockAndRecord(OptimisticLockingInterceptor.java:254)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitSingleKeyCommand(OptimisticLockingInterceptor.java:249)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor$LockAcquisitionVisitor.visitPutKeyValueCommand(OptimisticLockingInterceptor.java:242)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.acquireLocksVisitingCommands(OptimisticLockingInterceptor.java:335)
> at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:123)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:128)
> at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:115)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
> at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126)
> at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
> at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:141)
> at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
> at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
> at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
> at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
> at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
> at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
> at org.apache.lucene.store.Lock.obtain(Lock.java:72)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
> at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
> at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
> at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
> at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
> at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
> at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 2013-02-12 13:03:00,742 WARN [arjuna] (Hibernate Search: Index updates queue processor for index indexA-1) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a22814f:cd67:511a2f6a:9, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, topologyId=-1, isFromStateTransfer=false} org.infinispan.transaction.synchronization.SyncLocalTransaction@5} org.infinispan.transaction.synchronization.SynchronizationAdapter@24 >
> org.infinispan.CacheException: Could not prepare.
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
> at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
> at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
> at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
> at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
> at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
> at org.apache.lucene.store.Lock.obtain(Lock.java:72)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
> at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
> at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
> at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
> at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
> at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
> at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: javax.transaction.xa.XAException
> at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
> at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
> ... 25 more
> 2013-02-12 13:03:00,744 WARN [CacheImpl] (Hibernate Search: Index updates queue processor for index indexA-1) ISPN000160: Could not complete injected transaction.
> javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
> at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
> at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
> at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
> at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
> at org.apache.lucene.store.Lock.obtain(Lock.java:72)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
> at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
> at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
> at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
> at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
> at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
> at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: org.infinispan.CacheException: Could not prepare.
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
> at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
> ... 20 more
> Caused by: javax.transaction.xa.XAException
> at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
> at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
> ... 25 more
> 2013-02-12 13:03:00,745 ERROR [LuceneBackendQueueTask] (Hibernate Search: Index updates queue processor for index indexA-1) HSEARCH000073: Error in backend
> org.infinispan.CacheException: Could not commit implicit transaction
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1175)
> at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
> at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
> at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
> at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
> at org.apache.lucene.store.Lock.obtain(Lock.java:72)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
> at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
> at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
> at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
> at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
> at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
> at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
> ... 18 more
> Caused by: org.infinispan.CacheException: Could not prepare.
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
> at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
> ... 20 more
> Caused by: javax.transaction.xa.XAException
> at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
> at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
> ... 25 more
> 2013-02-12 13:03:00,748 ERROR [LogErrorHandler] (Hibernate Search: Index updates queue processor for index indexA-1) HSEARCH000058: Exception occurred org.infinispan.CacheException: Could not commit implicit transaction
> Primary Failure:
> Entity org.infinispan.query.api.TestEntity Id L:10 Work Type org.hibernate.search.backend.UpdateLuceneWork
> org.infinispan.CacheException: Could not commit implicit transaction
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1175)
> at org.infinispan.CacheImpl.putIfAbsentInternal(CacheImpl.java:779)
> at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:771)
> at org.infinispan.DecoratedCache.putIfAbsent(DecoratedCache.java:355)
> at org.infinispan.lucene.locking.BaseLuceneLock.obtain(BaseLuceneLock.java:63)
> at org.apache.lucene.store.Lock.obtain(Lock.java:72)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
> at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:148)
> at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:115)
> at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)
> at org.hibernate.search.backend.impl.lucene.SharedIndexWorkspaceImpl.getIndexWriter(SharedIndexWorkspaceImpl.java:78)
> at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:99)
> at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1172)
> ... 18 more
> Caused by: org.infinispan.CacheException: Could not prepare.
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70)
> at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
> at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
> at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164)
> at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
> ... 20 more
> Caused by: javax.transaction.xa.XAException
> at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161)
> at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123)
> at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68)
> ... 25 more
> {code}
> After the put, the entries are in the cache, but the querying doesn't return results.
> You can find attached the failing test.
> IMPORTANT UPDATE: After looking deeper into the failure and into the failed cases, I've found out that this issue appears only if the ISPN Directory related caches are TRANSACTIONAL. If only the data storage cache is transactional, then no issue appears. E.g. if there is no specific caches are defined for ISPN directory (for index storing), then the caches are created from the default configuration (LuceneIndexesMetadata, LuceneIndexesData, LuceneIndexesLocking), and if the default configuration is set to be TRANSACTIONAL (which is the case described in the attachments), then this issue will appear.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (ISPN-3454) Hot Rod client doesn't retry operation on RemoteException
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-3454?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-3454:
-----------------------------------------------
Michal Linhard <mlinhard(a)redhat.com> made a comment on [bug 1002957|https://bugzilla.redhat.com/show_bug.cgi?id=1002957]
Found in JDG 6.2.0.DR3
see
https://issues.jboss.org/browse/ISPN-3454
> Hot Rod client doesn't retry operation on RemoteException
> ---------------------------------------------------------
>
> Key: ISPN-3454
> URL: https://issues.jboss.org/browse/ISPN-3454
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 6.0.0.Alpha3
> Reporter: Michal Linhard
> Assignee: Mircea Markus
>
> This is a client-side problem.
> In a resilience test with 4 nodes where 1 is killed, I'm getting a lot of these:
> {code}
> 08:30:55,198 ERROR [org.jboss.smartfrog.jdg.loaddriver.DriverThread] (DriverThread-369) Error doing: PUT key399869 to node node04, took 493 ms
> org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for message id[821188] returned server error (status=0x85): org.infinispan.remoting.RemoteException: ISPN000217: Received exception from node01/default, see cause for remote stack trace
> at org.infinispan.client.hotrod.impl.protocol.Codec10.checkForErrorsInResponseStatus(Codec10.java:143)
> at org.infinispan.client.hotrod.impl.protocol.Codec10.readHeader(Codec10.java:99)
> at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:56)
> at org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:50)
> at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:30)
> at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:19)
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:46)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:209)
> at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:79)
> at org.jboss.qa.jdg.adapter.Infinispan60Adapter$HotRodRemoteCacheAdapter.put(Infinispan60Adapter.java:269)
> at org.jboss.smartfrog.jdg.loaddriver.DriverThreadImpl.makeRequest(DriverThreadImpl.java:265)
> at org.jboss.smartfrog.jdg.loaddriver.DriverThreadImpl.run(DriverThreadImpl.java:378)
> {code}
> Isn't this a recoverable problem that shouldn't be left to user to handle ?
> source:
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JDG/view/RESILIENCE...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (ISPN-3454) Hot Rod client doesn't retry operation on RemoteException
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-3454?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration updated ISPN-3454:
------------------------------------------
Bugzilla Update: Perform
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1002957
> Hot Rod client doesn't retry operation on RemoteException
> ---------------------------------------------------------
>
> Key: ISPN-3454
> URL: https://issues.jboss.org/browse/ISPN-3454
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 6.0.0.Alpha3
> Reporter: Michal Linhard
> Assignee: Mircea Markus
>
> This is a client-side problem.
> In a resilience test with 4 nodes where 1 is killed, I'm getting a lot of these:
> {code}
> 08:30:55,198 ERROR [org.jboss.smartfrog.jdg.loaddriver.DriverThread] (DriverThread-369) Error doing: PUT key399869 to node node04, took 493 ms
> org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for message id[821188] returned server error (status=0x85): org.infinispan.remoting.RemoteException: ISPN000217: Received exception from node01/default, see cause for remote stack trace
> at org.infinispan.client.hotrod.impl.protocol.Codec10.checkForErrorsInResponseStatus(Codec10.java:143)
> at org.infinispan.client.hotrod.impl.protocol.Codec10.readHeader(Codec10.java:99)
> at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:56)
> at org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:50)
> at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:30)
> at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:19)
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:46)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:209)
> at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:79)
> at org.jboss.qa.jdg.adapter.Infinispan60Adapter$HotRodRemoteCacheAdapter.put(Infinispan60Adapter.java:269)
> at org.jboss.smartfrog.jdg.loaddriver.DriverThreadImpl.makeRequest(DriverThreadImpl.java:265)
> at org.jboss.smartfrog.jdg.loaddriver.DriverThreadImpl.run(DriverThreadImpl.java:378)
> {code}
> Isn't this a recoverable problem that shouldn't be left to user to handle ?
> source:
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JDG/view/RESILIENCE...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (ISPN-3454) Hot Rod client doesn't retry operation on RemoteException
by Michal Linhard (JIRA)
Michal Linhard created ISPN-3454:
------------------------------------
Summary: Hot Rod client doesn't retry operation on RemoteException
Key: ISPN-3454
URL: https://issues.jboss.org/browse/ISPN-3454
Project: Infinispan
Issue Type: Bug
Affects Versions: 6.0.0.Alpha3
Reporter: Michal Linhard
Assignee: Mircea Markus
This is a client-side problem.
In a resilience test with 4 nodes where 1 is killed, I'm getting a lot of these:
{code}
08:30:55,198 ERROR [org.jboss.smartfrog.jdg.loaddriver.DriverThread] (DriverThread-369) Error doing: PUT key399869 to node node04, took 493 ms
org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for message id[821188] returned server error (status=0x85): org.infinispan.remoting.RemoteException: ISPN000217: Received exception from node01/default, see cause for remote stack trace
at org.infinispan.client.hotrod.impl.protocol.Codec10.checkForErrorsInResponseStatus(Codec10.java:143)
at org.infinispan.client.hotrod.impl.protocol.Codec10.readHeader(Codec10.java:99)
at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:56)
at org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:50)
at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:30)
at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:19)
at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:46)
at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:209)
at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:79)
at org.jboss.qa.jdg.adapter.Infinispan60Adapter$HotRodRemoteCacheAdapter.put(Infinispan60Adapter.java:269)
at org.jboss.smartfrog.jdg.loaddriver.DriverThreadImpl.makeRequest(DriverThreadImpl.java:265)
at org.jboss.smartfrog.jdg.loaddriver.DriverThreadImpl.run(DriverThreadImpl.java:378)
{code}
Isn't this a recoverable problem that shouldn't be left to user to handle ?
source:
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JDG/view/RESILIENCE...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months