[JBoss JIRA] (ISPN-3767) MassIndexer breaks search feature with one node cluster
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-3767?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero updated ISPN-3767:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> MassIndexer breaks search feature with one node cluster
> -------------------------------------------------------
>
> Key: ISPN-3767
> URL: https://issues.jboss.org/browse/ISPN-3767
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 5.2.4.Final
> Reporter: Romain Pelisse
> Assignee: Adrian Nistor
> Fix For: 6.1.0.Final
>
>
> Hi,
> Trying to cope with the extreme slowliness of put() operation with indexing [1], I've tried to use the MassIndexer, to create the index AFTER adding all the data in the grid. This actually works pretty well, but, when running in a "single node" grid, it "breaks" the search, which always returns 0 result to any kind of query.
> I've modified one of the test suite of InfiniSpan to reproduce the issue:
> https://github.com/rpelisse/infinispan/tree/mass-indexer-breaks-search-wi...
> Once this branch is checked out, just run :
> $ cd ./query
> $ mvn clean -Dtest=org.infinispan.query.distributed.DistributedMassIndexingTest test
> Note: MassIndexer being implemented using the Map/Reduce algorithm, it requires to run within a cluster (ie several instances of ISPN).
> [1] http://stackoverflow.com/questions/10090361/infinispan-very-slow-for-load...
> If run within a single node cluster, the MassIndexer
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3712) HotRod Rolling Upgrades from Delirium 5.2.4 to latest server needs dropped org.infinispan.util.ByteArrayKey
by Vojtech Juranek (JIRA)
[ https://issues.jboss.org/browse/ISPN-3712?page=com.atlassian.jira.plugin.... ]
Vojtech Juranek commented on ISPN-3712:
---------------------------------------
PRs [1,2] seems to fix the issue
[1] https://github.com/infinispan/infinispan-server/pull/202
[2] https://github.com/infinispan/infinispan/pull/2260
> HotRod Rolling Upgrades from Delirium 5.2.4 to latest server needs dropped org.infinispan.util.ByteArrayKey
> -----------------------------------------------------------------------------------------------------------
>
> Key: ISPN-3712
> URL: https://issues.jboss.org/browse/ISPN-3712
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 6.0.0.CR1
> Environment: Processing HotRod rolling upgrades from source Infinispan 'Delirium' 5.2.4.Final-redhat-1 server to the latest Infinispan 'Infinium' 6.0.0-SNAPSHOT. Using old standalone.xml for source node and examples/standalone-hotrod-rolling-upgrade.xml for new target server.
> Reporter: Tomas Sykora
> Assignee: Tristan Tarrant
> Labels: 620
> Fix For: 6.0.0.Final
>
>
> When it is running from 6.0.0-SNAPSHOT to 6.0.0-SNAPSHOT it is ok. It looks like some back-porting compatibility issue.
> See stacktrace:
> javax.management.MBeanException
> at org.infinispan.jmx.ResourceDMBean.invoke(ResourceDMBean.java:273)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:791)
> at org.jboss.as.jmx.PluggableMBeanServerImpl$TcclMBeanServer.invoke(PluggableMBeanServerImpl.java:527)
> at org.jboss.as.jmx.PluggableMBeanServerImpl.invoke(PluggableMBeanServerImpl.java:263)
> at org.jboss.remotingjmx.protocol.v2.ServerProxy$InvokeHandler.handle(ServerProxy.java:915)
> at org.jboss.remotingjmx.protocol.v2.ServerCommon$MessageReciever$1.run(ServerCommon.java:152)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.infinispan.jmx.ResourceDMBean.invoke(ResourceDMBean.java:271)
> ... 9 more
> Caused by: org.infinispan.commons.CacheException: java.lang.ClassNotFoundException: org.infinispan.util.ByteArrayKey from [Module "org.infinispan:main" from local module loader @4099a992 (finder: local module finder @284bd160 (roots: /home/tsykora/programs/eclipseWorkspace/Infinispan-server-tsykora/infinispan-server/testsuite/target/server/node1/modules,/home/tsykora/programs/eclipseWorkspace/Infinispan-server-tsykora/infinispan-server/testsuite/target/server/node1/modules/system/layers/base))]
> at org.infinispan.persistence.remote.upgrade.HotRodTargetMigrator.synchronizeData(HotRodTargetMigrator.java:63)
> at org.infinispan.upgrade.RollingUpgradeManager.synchronizeData(RollingUpgradeManager.java:59)
> ... 14 more
> Caused by: java.lang.ClassNotFoundException: org.infinispan.util.ByteArrayKey from [Module "org.infinispan:main" from local module loader @4099a992 (finder: local module finder @284bd160 (roots: /home/tsykora/programs/eclipseWorkspace/Infinispan-server-tsykora/infinispan-server/testsuite/target/server/node1/modules,/home/tsykora/programs/eclipseWorkspace/Infinispan-server-tsykora/infinispan-server/testsuite/target/server/node1/modules/system/layers/base))]
> at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190)
> at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468)
> at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456)
> at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)
> at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:135)
> at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:116)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:947)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1259)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:213)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadCollectionObject(RiverUnmarshaller.java:184)
> at org.jboss.marshalling.river.RiverUnmarshaller.readCollectionData(RiverUnmarshaller.java:777)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:656)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:213)
> at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
> at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:140)
> at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromByteBuffer(AbstractJBossMarshaller.java:118)
> at org.infinispan.commons.marshall.AbstractMarshaller.objectFromByteBuffer(AbstractMarshaller.java:82)
> at org.infinispan.persistence.remote.upgrade.HotRodTargetMigrator.synchronizeData(HotRodTargetMigrator.java:61)
> ... 15 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3794) Add tests to confirm that L1LastChanceInterceptor doesn't cause issues with ST
by William Burns (JIRA)
William Burns created ISPN-3794:
-----------------------------------
Summary: Add tests to confirm that L1LastChanceInterceptor doesn't cause issues with ST
Key: ISPN-3794
URL: https://issues.jboss.org/browse/ISPN-3794
Project: Infinispan
Issue Type: Bug
Components: State transfer
Reporter: William Burns
Assignee: William Burns
L1LastChanceInterceptor currently fires it's invalidations asynchronously. This could cause an issue in that if there was a topology change in between there is nothing to guarantee a retry. This would be to add some tests to verify this is not an issue.
This should be okay since EntryWrappingInterceptor only adds the updatedKeys if there was a change (shouldn't happen since InvalidateL1Command doesn't update if it is an owner) and ISPN-3780 fixes the issue with no value and a L1 invalidation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3780) CLONE - InvalidateL1Command during ST should not cancel writing the entry by ST in nontx
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-3780?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-3780:
-------------------------------------
Trying to write a test for this wasn't quite what I thought at first. I found this issue is only reproducible when an owner sends a L1 invalidation but the node doesn't actually have the L1 value present (as you mentioned). In this case the EntryWrappingInterceptor is guaranteed to add to the updatedKeys as long as a ST is in progress.
> CLONE - InvalidateL1Command during ST should not cancel writing the entry by ST in nontx
> ----------------------------------------------------------------------------------------
>
> Key: ISPN-3780
> URL: https://issues.jboss.org/browse/ISPN-3780
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 5.3.0.Final
> Reporter: Radim Vansa
> Assignee: William Burns
> Priority: Critical
> Labels: 620
> Fix For: 6.1.0.Final
>
>
> When a node which is about to receive the entries for some segment receives InvalidateL1Command, this puts the key into StateConsumer.updatedKeys. After the entries for ST are received, the entry which was invalidated is not updated -> this result in losing the entry.
> Btw., in EntryWrappingInterceptor.visitInvalidateL1Command the trace logs all looked up entries for each entry - this causes some performance problems when tracing is on and there are many invalidated entries. Please, do this only once.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3743) Silence "IllegalStateException: Default cache is in 'STOPPING' state" exceptions
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-3743?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration updated ISPN-3743:
------------------------------------------
Bugzilla Update: Perform
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1039090
> Silence "IllegalStateException: Default cache is in 'STOPPING' state" exceptions
> --------------------------------------------------------------------------------
>
> Key: ISPN-3743
> URL: https://issues.jboss.org/browse/ISPN-3743
> Project: Infinispan
> Issue Type: Feature Request
> Components: RPC
> Affects Versions: 6.0.0.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 7.0.0.Final
>
>
> When a cache in STOPPING state receives a command, the exception is propagated all the way to the caller:
> {noformat}
> 09:33:02,005 ERROR (testng-NonTxPutIfAbsentDuringLeaveStressTest:) [UnitTestTestNGListener] Test testNodeLeavingDuringPutIfAbsent(org.infinispan.distribution.rehash.NonTxPutIfAbsentDuringLeaveStressTest) failed.
> java.util.concurrent.ExecutionException: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from NodeC-25987, see cause for remote stack trace
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> Caused by: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from NodeC-25987, see cause for remote stack trace
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:41)
> Caused by: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from NodeD-57301, see cause for remote stack trace
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:41)
> Caused by: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:93)
> {noformat}
> This causes random failures in NonTxPutIfAbsentDuringLeaveStressTest.testNodeLeavingDuringPutIfAbsent.
> The originator should either ignore the IllegalStateException or wait for a new cache topology and retry the operation, just like it should for SuspectExceptions (ISPN-2577).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3743) Silence "IllegalStateException: Default cache is in 'STOPPING' state" exceptions
by Michal Linhard (JIRA)
[ https://issues.jboss.org/browse/ISPN-3743?page=com.atlassian.jira.plugin.... ]
Michal Linhard commented on ISPN-3743:
--------------------------------------
This also bubbled up to hot rod client in resilience tests for JDG 6.2.0.ER5 with graceful shutdown of one node of 4.
https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JDG/view/RESILIENCE...
IMO the client should handle such scenario with retry. Should I open a new JIRA for this or can it be a part of this one ?
> Silence "IllegalStateException: Default cache is in 'STOPPING' state" exceptions
> --------------------------------------------------------------------------------
>
> Key: ISPN-3743
> URL: https://issues.jboss.org/browse/ISPN-3743
> Project: Infinispan
> Issue Type: Feature Request
> Components: RPC
> Affects Versions: 6.0.0.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 7.0.0.Final
>
>
> When a cache in STOPPING state receives a command, the exception is propagated all the way to the caller:
> {noformat}
> 09:33:02,005 ERROR (testng-NonTxPutIfAbsentDuringLeaveStressTest:) [UnitTestTestNGListener] Test testNodeLeavingDuringPutIfAbsent(org.infinispan.distribution.rehash.NonTxPutIfAbsentDuringLeaveStressTest) failed.
> java.util.concurrent.ExecutionException: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from NodeC-25987, see cause for remote stack trace
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> Caused by: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from NodeC-25987, see cause for remote stack trace
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:41)
> Caused by: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from NodeD-57301, see cause for remote stack trace
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:41)
> Caused by: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:93)
> {noformat}
> This causes random failures in NonTxPutIfAbsentDuringLeaveStressTest.testNodeLeavingDuringPutIfAbsent.
> The originator should either ignore the IllegalStateException or wait for a new cache topology and retry the operation, just like it should for SuspectExceptions (ISPN-2577).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3792) Optional Cache.putForExternalRead expiration arguments
by Vladimir Dzhuvinov (JIRA)
[ https://issues.jboss.org/browse/ISPN-3792?page=com.atlassian.jira.plugin.... ]
Vladimir Dzhuvinov commented on ISPN-3792:
------------------------------------------
No, we don't use the Hibernate. We chose to try PFER as the docs claimed it was the least expensive cache put op and that was what we were looking to improve. Unfortunately some of our tests that involve updates failed when we switched from putAsync to PFER. We couldn't determine what the exact cause of that was (wrong use of PFER I suppose).
Thanks for the Flag API pointer, that seems to be a working alternative to the PFER arguments. I wasn't aware of this API.
> Optional Cache.putForExternalRead expiration arguments
> ------------------------------------------------------
>
> Key: ISPN-3792
> URL: https://issues.jboss.org/browse/ISPN-3792
> Project: Infinispan
> Issue Type: Feature Request
> Reporter: Vladimir Dzhuvinov
> Assignee: Mircea Markus
> Priority: Optional
>
> BasicCache has optional lifespan and idle time arguments for its put methods, I was hoping to have that for putForExternalRead as well.
> Cheers,
> Vladimir
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3793) L1 tracks requestors for nodes that don't retrieve a value
by William Burns (JIRA)
William Burns created ISPN-3793:
-----------------------------------
Summary: L1 tracks requestors for nodes that don't retrieve a value
Key: ISPN-3793
URL: https://issues.jboss.org/browse/ISPN-3793
Project: Infinispan
Issue Type: Bug
Components: Distributed Cache
Affects Versions: 6.0.0.Final
Reporter: William Burns
Assignee: Mircea Markus
Currently we always register any non local requestor in the L1 requestors map. This can be a performance issue with values that aren't present in the cache, since a new value put in for that value will require invalidating those nodes in addition to storing the key in memory while waiting for that.
The easiest fix would have been to check the command invocation to see if it returns a value, however ISPN-3737 moved the registering to before the command is ran to prevent a different issue.
Also thinking about it we could remove the requestor after the command returns, however this also has a race condition if you had data committed and the node retrieved the value concurrently.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3793) L1 tracks requestors for nodes that don't retrieve a value
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-3793?page=com.atlassian.jira.plugin.... ]
William Burns reassigned ISPN-3793:
-----------------------------------
Assignee: William Burns (was: Mircea Markus)
> L1 tracks requestors for nodes that don't retrieve a value
> ----------------------------------------------------------
>
> Key: ISPN-3793
> URL: https://issues.jboss.org/browse/ISPN-3793
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 6.0.0.Final
> Reporter: William Burns
> Assignee: William Burns
>
> Currently we always register any non local requestor in the L1 requestors map. This can be a performance issue with values that aren't present in the cache, since a new value put in for that value will require invalidating those nodes in addition to storing the key in memory while waiting for that.
> The easiest fix would have been to check the command invocation to see if it returns a value, however ISPN-3737 moved the registering to before the command is ran to prevent a different issue.
> Also thinking about it we could remove the requestor after the command returns, however this also has a race condition if you had data committed and the node retrieved the value concurrently.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months