[JBoss JIRA] (ISPN-4321) leveldb.config.ConfigurationTest.testXmlConfig60 fails randomly on RHEL6
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-4321?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-4321:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.0.0.Beta1
9.0.0.Final
Resolution: Done
> leveldb.config.ConfigurationTest.testXmlConfig60 fails randomly on RHEL6
> ------------------------------------------------------------------------
>
> Key: ISPN-4321
> URL: https://issues.jboss.org/browse/ISPN-4321
> Project: Infinispan
> Issue Type: Bug
> Components: Loaders and Stores
> Affects Versions: 7.0.0.Alpha2, 7.0.0.Alpha3, 7.0.0.Alpha4
> Environment: RHEL6 with {OpenJDK6, IBM7}
> Reporter: Vitalii Chepeliuk
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 9.0.0.Beta1, 9.0.0.Final
>
>
> {noformat}
> org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object of type PersistenceManagerImpl
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
> at org.infinispan.CacheImpl.start(CacheImpl.java:684)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:574)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:529)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:409)
> at org.infinispan.persistence.leveldb.config.ConfigurationTest.testXmlConfig60(ConfigurationTest.java:107)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
> at java.lang.reflect.Method.invoke(Method.java:619)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
> at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
> at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
> at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
> at org.testng.TestRunner.privateRun(TestRunner.java:767)
> at org.testng.TestRunner.run(TestRunner.java:617)
> at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
> at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
> at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
> at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
> at java.util.concurrent.FutureTask.run(FutureTask.java:273)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1170)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:640)
> at java.lang.Thread.run(Thread.java:853)
> Caused by: org.infinispan.commons.CacheException: Unable to start cache loaders
> at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:155)
> at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
> at java.lang.reflect.Method.invoke(Method.java:619)
> at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
> ... 30 more
> Caused by: org.infinispan.commons.CacheConfigurationException: Unable to open database
> at org.infinispan.persistence.leveldb.LevelDBStore.start(LevelDBStore.java:103)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:122)
> ... 34 more
> Caused by: org.fusesource.leveldbjni.internal.NativeDB$DBException: Corruption: 1 missing files; e.g.: /tmp/leveldb/52/datatestCache/000005.sst
> at org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:200)
> at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:218)
> at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:168)
> at org.infinispan.persistence.leveldb.LevelDBStore.openDatabase(LevelDBStore.java:148)
> at org.infinispan.persistence.leveldb.LevelDBStore.start(LevelDBStore.java:100)
> ... 35 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
6 years, 5 months
[JBoss JIRA] (ISPN-6857) OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-6857?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant reopened ISPN-6857:
-----------------------------------
Still needs fixing on 8.1.x
> OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
> -----------------------------------------------------------------------------------------------------------
>
> Key: ISPN-6857
> URL: https://issues.jboss.org/browse/ISPN-6857
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 8.1.0.Final, 9.0.0.Alpha4, 8.2.4.Final
> Reporter: Marek Posolda
> Assignee: Dan Berindei
> Fix For: 9.0.0.Final, 8.2.5.Final
>
> Attachments: OutdatedTopologyExceptionReproducerTest.java
>
>
> I have the following setup:
> - 2 nodes in cluster with mode INVALIDATION_SYNC. No-transaction cache.
> - Node1 is started
> - Called "cache.remove" on some key on node1. At the same time, node2 is starting, which is causing topology change.
> - The "cache.remove" call on node1 is throwing OutdatedTopologyException.
> I found the cause is that StateTransferInterceptor is not added in InterceptorChain during INVALIDATION mode. It's just available during REPLICATION or DISTRIBUTED modes - https://github.com/infinispan/infinispan/blob/master/core/src/main/java/o...
> Indeed when I manually added StateTransferInterceptor to my invalidation cache:
> {code}
> invalidationConfigBuilder.customInterceptors()
> .addInterceptor()
> .before(NonTransactionalLockingInterceptor.class)
> .interceptorClass(StateTransferInterceptor.class);
> {code}
>
> I can see that issue is gone as OutdatedTopologyException is catched and command is retried with new topology.
> I am attaching the Java unit test for reproducing issue. On my laptop when I run it, I can almost always simulate the issue.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
6 years, 5 months
[JBoss JIRA] (ISPN-7075) [8.1.x] : OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-7075?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant closed ISPN-7075.
---------------------------------
Resolution: Duplicate Issue
Let's reopen the original issue instead.
> [8.1.x] : OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-7075
> URL: https://issues.jboss.org/browse/ISPN-7075
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 8.1.0.Final
> Reporter: Panagiotis Sotiropoulos
> Assignee: Panagiotis Sotiropoulos
> Fix For: 9.0.0.Final, 8.2.5.Final
>
> Attachments: OutdatedTopologyExceptionReproducerTest.java
>
>
> I have the following setup:
> - 2 nodes in cluster with mode INVALIDATION_SYNC. No-transaction cache.
> - Node1 is started
> - Called "cache.remove" on some key on node1. At the same time, node2 is starting, which is causing topology change.
> - The "cache.remove" call on node1 is throwing OutdatedTopologyException.
> I found the cause is that StateTransferInterceptor is not added in InterceptorChain during INVALIDATION mode. It's just available during REPLICATION or DISTRIBUTED modes - https://github.com/infinispan/infinispan/blob/master/core/src/main/java/o...
> Indeed when I manually added StateTransferInterceptor to my invalidation cache:
> {code}
> invalidationConfigBuilder.customInterceptors()
> .addInterceptor()
> .before(NonTransactionalLockingInterceptor.class)
> .interceptorClass(StateTransferInterceptor.class);
> {code}
>
> I can see that issue is gone as OutdatedTopologyException is catched and command is retried with new topology.
> I am attaching the Java unit test for reproducing issue. On my laptop when I run it, I can almost always simulate the issue.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
6 years, 5 months
[JBoss JIRA] (ISPN-6857) OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-6857?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-6857:
----------------------------------
Affects Version/s: 8.2.4.Final
9.0.0.Alpha4
> OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
> -----------------------------------------------------------------------------------------------------------
>
> Key: ISPN-6857
> URL: https://issues.jboss.org/browse/ISPN-6857
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 8.1.0.Final, 9.0.0.Alpha4, 8.2.4.Final
> Reporter: Marek Posolda
> Assignee: Dan Berindei
> Fix For: 9.0.0.Final, 8.2.5.Final
>
> Attachments: OutdatedTopologyExceptionReproducerTest.java
>
>
> I have the following setup:
> - 2 nodes in cluster with mode INVALIDATION_SYNC. No-transaction cache.
> - Node1 is started
> - Called "cache.remove" on some key on node1. At the same time, node2 is starting, which is causing topology change.
> - The "cache.remove" call on node1 is throwing OutdatedTopologyException.
> I found the cause is that StateTransferInterceptor is not added in InterceptorChain during INVALIDATION mode. It's just available during REPLICATION or DISTRIBUTED modes - https://github.com/infinispan/infinispan/blob/master/core/src/main/java/o...
> Indeed when I manually added StateTransferInterceptor to my invalidation cache:
> {code}
> invalidationConfigBuilder.customInterceptors()
> .addInterceptor()
> .before(NonTransactionalLockingInterceptor.class)
> .interceptorClass(StateTransferInterceptor.class);
> {code}
>
> I can see that issue is gone as OutdatedTopologyException is catched and command is retried with new topology.
> I am attaching the Java unit test for reproducing issue. On my laptop when I run it, I can almost always simulate the issue.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
6 years, 5 months
[JBoss JIRA] (ISPN-7075) [8.1.x] : OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-7075?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant commented on ISPN-7075:
---------------------------------------
Don't clone issues just because they happen on different branches please.
> [8.1.x] : OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-7075
> URL: https://issues.jboss.org/browse/ISPN-7075
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 8.1.0.Final
> Reporter: Panagiotis Sotiropoulos
> Assignee: Panagiotis Sotiropoulos
> Fix For: 9.0.0.Final, 8.2.5.Final
>
> Attachments: OutdatedTopologyExceptionReproducerTest.java
>
>
> I have the following setup:
> - 2 nodes in cluster with mode INVALIDATION_SYNC. No-transaction cache.
> - Node1 is started
> - Called "cache.remove" on some key on node1. At the same time, node2 is starting, which is causing topology change.
> - The "cache.remove" call on node1 is throwing OutdatedTopologyException.
> I found the cause is that StateTransferInterceptor is not added in InterceptorChain during INVALIDATION mode. It's just available during REPLICATION or DISTRIBUTED modes - https://github.com/infinispan/infinispan/blob/master/core/src/main/java/o...
> Indeed when I manually added StateTransferInterceptor to my invalidation cache:
> {code}
> invalidationConfigBuilder.customInterceptors()
> .addInterceptor()
> .before(NonTransactionalLockingInterceptor.class)
> .interceptorClass(StateTransferInterceptor.class);
> {code}
>
> I can see that issue is gone as OutdatedTopologyException is catched and command is retried with new topology.
> I am attaching the Java unit test for reproducing issue. On my laptop when I run it, I can almost always simulate the issue.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
6 years, 5 months
[JBoss JIRA] (ISPN-7067) Cache.evict() sometimes performs a DataContainer.remove()
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-7067?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-7067:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.0.0.Final
Resolution: Done
> Cache.evict() sometimes performs a DataContainer.remove()
> ---------------------------------------------------------
>
> Key: ISPN-7067
> URL: https://issues.jboss.org/browse/ISPN-7067
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.0.0.Alpha4, 8.2.4.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.0.0.Beta1, 9.0.0.Final
>
>
> {{Cache.evict()}} generally uses {{DataContainer.evict()}} to move an entry from the data container to the store.
> However, when {{EntryWrappingInterceptor}} doesn't find the entry in the data container, {{EvictCommand.perform()}} doesn't set the {{EVICTED}} flag on the context entry, and then {{ReadCommittedEntry.commit()}} calls {{DataContainer.remove()}} instead of {{DataContainer.evict()}}.
> If another command activated the entry between the entry wrapping and the commit, this will remove the entry altogether instead of moving it to the store.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
6 years, 5 months
[JBoss JIRA] (ISPN-6981) AffinityIndexManager fails to index documents in async mode
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-6981?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-6981:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.0.0.Beta1
9.0.0.Final
Resolution: Done
> AffinityIndexManager fails to index documents in async mode
> -----------------------------------------------------------
>
> Key: ISPN-6981
> URL: https://issues.jboss.org/browse/ISPN-6981
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Affects Versions: 9.0.0.Alpha4
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Fix For: 9.0.0.Beta1, 9.0.0.Final
>
>
> During topology changes in async mode ("<index>.worker.execution" set to "async"), the {{AfinityIndexManager}} sometimes fails to index entries. Some of errors thrown:
> {noformat}
> 19:06:04,638 ERROR (Hibernate Search: Index updates queue processor for index entity.5-1) [LogErrorHandler] HSEARCH000058: Exception occurred
> org.apache.lucene.store.LockObtainFailedException: lock instance already assigned
> {noformat}
> {noformat}
> 18:53:59,553 ERROR (Hibernate Search: Index updates queue processor for index entity.2-1) [LuceneBackendQueueTask] HSEARCH000073: Error in
> backend
> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
> at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
> at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
> {noformat}
> {noformat}
> 18:55:31,328 ERROR (Hibernate Search: Commit Scheduler for index entity.143-1) [AffinityErrorHandler] HSEARCH000117: IOException
> on the IndexWriter
> java.io.IOException: This lock is no longer being held
> at org.infinispan.lucene.impl.BaseLuceneLock.ensureValid(BaseLuceneLock.java:89)
> at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:43)
> at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
> at org.apache.lucene.codecs.lucene50.Lucene50PostingsWriter.<init>(Lucene50PostingsWriter.java:105)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
6 years, 5 months