[JBoss JIRA] (ISPN-3335) JMX statistics for Queries partially doesn't work
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-3335?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-3335:
-----------------------------------------------
Vojtech Juranek <vjuranek(a)redhat.com> changed the Status of [bug 1024914|https://bugzilla.redhat.com/show_bug.cgi?id=1024914] from ON_QA to VERIFIED
> JMX statistics for Queries partially doesn't work
> -------------------------------------------------
>
> Key: ISPN-3335
> URL: https://issues.jboss.org/browse/ISPN-3335
> Project: Infinispan
> Issue Type: Bug
> Components: Querying
> Affects Versions: 6.0.0.Alpha1
> Reporter: Anna Manukyan
> Assignee: Adrian Nistor
> Priority: Critical
> Labels: 620
> Fix For: 6.1.0.Final
>
> Attachments: QueryMBeanTest.java
>
>
> I was playing around with Query JMX statistics, and found out that there are several attributes like SearchQueryTotalTime are always 0 (this attr. represents the duration of query in nano-seconds), even though I'm running the infinispan query.
> The only attribute which is updated and returns proper value is StatisticsEnabled. Also the following operations work as expected:
> getNumberOfIndexedEntities(String entity)
> clear()
> I've tried also to retrieve the statistics using the following method:
> {code}
> Search.getSearchManager(cacheManager.getCache(CACHE_NAME)).getSearchFactory().getStatistics().getSearchQueryTotalTime()
> {code}
> and it returns the same results as if getting via JMX.
> You can find the whole running test attached.
> Best regards,
> Anna.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3792) Optional Cache.putForExternalRead expiration arguments
by Vladimir Dzhuvinov (JIRA)
Vladimir Dzhuvinov created ISPN-3792:
----------------------------------------
Summary: Optional Cache.putForExternalRead expiration arguments
Key: ISPN-3792
URL: https://issues.jboss.org/browse/ISPN-3792
Project: Infinispan
Issue Type: Feature Request
Reporter: Vladimir Dzhuvinov
Assignee: Mircea Markus
Priority: Optional
BasicCache has optional lifespan and idle time arguments for its put methods, I was hoping to have that for putForExternalRead as well.
Cheers,
Vladimir
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3738) Entry version gets lost during topology change -> NPE
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-3738?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-3738:
------------------------------
Fix Version/s: 6.1.0.Final
> Entry version gets lost during topology change -> NPE
> -----------------------------------------------------
>
> Key: ISPN-3738
> URL: https://issues.jboss.org/browse/ISPN-3738
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 6.0.0.Final
> Reporter: Radim Vansa
> Assignee: Pedro Ruivo
> Priority: Critical
> Labels: 620
> Fix For: 6.1.0.Final
>
>
> Replicated TX cache with WSC, A, B are in cluster, C is joining
> 0. The current CH already contains A and B as owners, C is joining (is not primary owner of anything yet). B is primary owner of K=V.
> 1. A sends PrepareCommand to B and C with put(K, V) (V is null on all nodes)
> 2. C receives PrepareCommand and responds with no versions (it is not primary owner)
> 3. topology changes on B - primary ownership of K is transfered to C
> 4. B receives PrepareCommand, responds without K's version (it is not primary)
> 5. B forwards the Prepare to C as it sees that the command has lower topology ID
> 6. C responds to B's prepare with version of K
> 7. K version is *not* added to B's response, B responds to A
> 8. A finds out that topology has changed, forwards prepare to C
> 9. C responds to C's prepare with version of K
> 10. A receives C's response, but the versions are not added to transaction
> 11. A sends out CommitCommand missing version of K
> 12. all nodes record K=V without version as usual ImmortalCacheEntry
> 13. the next time we try to increase version of K=V, we fail with NPE in SimpleClusteredVersionGenerator (actually when it tries to throw IllegalArgumentException because the null version is unexpected version class)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3738) Entry version gets lost during topology change -> NPE
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-3738?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-3738:
------------------------------
Status: Pull Request Sent (was: Coding In Progress)
Git Pull Request: https://github.com/infinispan/infinispan/pull/2261
> Entry version gets lost during topology change -> NPE
> -----------------------------------------------------
>
> Key: ISPN-3738
> URL: https://issues.jboss.org/browse/ISPN-3738
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 6.0.0.Final
> Reporter: Radim Vansa
> Assignee: Pedro Ruivo
> Priority: Critical
> Labels: 620
>
> Replicated TX cache with WSC, A, B are in cluster, C is joining
> 0. The current CH already contains A and B as owners, C is joining (is not primary owner of anything yet). B is primary owner of K=V.
> 1. A sends PrepareCommand to B and C with put(K, V) (V is null on all nodes)
> 2. C receives PrepareCommand and responds with no versions (it is not primary owner)
> 3. topology changes on B - primary ownership of K is transfered to C
> 4. B receives PrepareCommand, responds without K's version (it is not primary)
> 5. B forwards the Prepare to C as it sees that the command has lower topology ID
> 6. C responds to B's prepare with version of K
> 7. K version is *not* added to B's response, B responds to A
> 8. A finds out that topology has changed, forwards prepare to C
> 9. C responds to C's prepare with version of K
> 10. A receives C's response, but the versions are not added to transaction
> 11. A sends out CommitCommand missing version of K
> 12. all nodes record K=V without version as usual ImmortalCacheEntry
> 13. the next time we try to increase version of K=V, we fail with NPE in SimpleClusteredVersionGenerator (actually when it tries to throw IllegalArgumentException because the null version is unexpected version class)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3473) CDI Cache interceptor tests are failing while running in Tomcat container
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-3473?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-3473:
-----------------------------------------------
Vojtech Juranek <vjuranek(a)redhat.com> changed the Status of [bug 1024412|https://bugzilla.redhat.com/show_bug.cgi?id=1024412] from NEW to VERIFIED
> CDI Cache interceptor tests are failing while running in Tomcat container
> -------------------------------------------------------------------------
>
> Key: ISPN-3473
> URL: https://issues.jboss.org/browse/ISPN-3473
> Project: Infinispan
> Issue Type: Bug
> Components: CDI integration
> Affects Versions: 6.0.0.Alpha3
> Reporter: Anna Manukyan
> Assignee: Galder Zamarreño
> Labels: 620
>
> There is a problem with Infinispan CDI interceptors under the Tomcat container.
> While running the infinispan/cdi-extension module's tests under the tomcat container, the following 2 related issues appear:
> 1) When the beans.xml contains the definition for the interceptors, as given below:
> {code}
> <beans xmlns="http://java.sun.com/xml/ns/javaee"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/beans_1_0.xsd">
> <interceptors>
> <class>org.infinispan.jcache.annotation.CacheResultInterceptor</class>
> <class>org.infinispan.jcache.annotation.CachePutInterceptor</class>
> <class>org.infinispan.jcache.annotation.CacheRemoveEntryInterceptor</class>
> <class>org.infinispan.jcache.annotation.CacheRemoveAllInterceptor</class>
> </interceptors>
> <alternatives>
> <class>org.infinispan.cdi.InjectedCacheResolver</class>
> </alternatives>
> </beans>
> {code}
> Then while the deployment on the container, the following exception is thrown:
> {code}
> org.jboss.weld.exceptions.DeploymentException: WELD-001416 Enabled interceptor class [<class>org.infinispan.jcache.annotation.CacheResultInterceptor</class> in jar:file:/home/anna.manukyan/.jenkins/jobs/invm-tomcat/workspace/tomcat-server/jboss-ews-2.0/tomcat7/webapps/test/WEB-INF/lib/infinispan-jcache.jar!/META-INF/beans.xml@7, <class>org.infinispan.jcache.annotation.CachePutInterceptor</class> in jar:file:/home/anna.manukyan/.jenkins/jobs/invm-tomcat/workspace/tomcat-server/jboss-ews-2.0/tomcat7/webapps/test/WEB-INF/lib/infinispan-jcache.jar!/META-INF/beans.xml@8, <class>org.infinispan.jcache.annotation.CacheRemoveEntryInterceptor</class> in jar:file:/home/anna.manukyan/.jenkins/jobs/invm-tomcat/workspace/tomcat-server/jboss-ews-2.0/tomcat7/webapps/test/WEB-INF/lib/infinispan-jcache.jar!/META-INF/beans.xml@9, <class>org.infinispan.jcache.annotation.CacheRemoveAllInterceptor</class> in jar:file:/home/anna.manukyan/.jenkins/jobs/invm-tomcat/workspace/tomcat-server/jboss-ews-2.0/tomcat7/webapps/test/WEB-INF/lib/infinispan-jcache.jar!/META-INF/beans.xml@10, <class>org.infinispan.jcache.annotation.CacheResultInterceptor</class> in jndi:/localhost/test/WEB-INF/beans.xml@29, <class>org.infinispan.jcache.annotation.CachePutInterceptor</class> in jndi:/localhost/test/WEB-INF/beans.xml@30, <class>org.infinispan.jcache.annotation.CacheRemoveEntryInterceptor</class> in jndi:/localhost/test/WEB-INF/beans.xml@31, <class>org.infinispan.jcache.annotation.CacheRemoveAllInterceptor</class> in jndi:/localhost/test/WEB-INF/beans.xml@32] specified twice
> at org.jboss.weld.manager.Enabled.createMetadataMap(Enabled.java:123)
> at org.jboss.weld.manager.Enabled.<init>(Enabled.java:96)
> at org.jboss.weld.manager.Enabled.of(Enabled.java:79)
> at org.jboss.weld.bootstrap.BeanDeployment.<init>(BeanDeployment.java:114)
> at org.jboss.weld.bootstrap.WeldBootstrap$DeploymentVisitor.visit(WeldBootstrap.java:183)
> at org.jboss.weld.bootstrap.WeldBootstrap$DeploymentVisitor.visit(WeldBootstrap.java:152)
> at org.jboss.weld.bootstrap.WeldBootstrap.startContainer(WeldBootstrap.java:283)
> at org.jboss.weld.environment.servlet.Listener.contextInitialized(Listener.java:151)
> at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4791)
> at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5285)
> at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
> at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
> at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
> at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:618)
> at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:963)
> at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:537)
> at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1468)
> at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:301)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:791)
> at org.apache.catalina.manager.ManagerServlet.check(ManagerServlet.java:1444)
> at org.apache.catalina.manager.ManagerServlet.deploy(ManagerServlet.java:677)
> at org.apache.catalina.manager.ManagerServlet.doPut(ManagerServlet.java:435)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:581)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
> at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1002)
> at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585)
> at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> {code}
> If there is no need declare the interceptors in the beans.xml (as far as I know it is necessary), then please let me know and ignore this case.
> 2) When the interceptors declaration is commented, then the interceptors doesn't work properly. So all tests located under the package org.infinispan.cdi.test.interceptor.* are failing with assertion errors.
> Please note that this behaviour is noticed for infinispan 6.0.0.Alpha3. It is not tried for the earlier versions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3791) Silence "Received invalid rebalance confirmation from NodeX" exceptions
by Dan Berindei (JIRA)
Dan Berindei created ISPN-3791:
----------------------------------
Summary: Silence "Received invalid rebalance confirmation from NodeX" exceptions
Key: ISPN-3791
URL: https://issues.jboss.org/browse/ISPN-3791
Project: Infinispan
Issue Type: Bug
Components: State transfer
Affects Versions: 6.0.0.Final
Reporter: Dan Berindei
Assignee: Dan Berindei
Priority: Minor
When the coordinator shuts down, it tries to shut down each of its caches first. This triggers a rebalance for the rest of the members, but the rebalance usually finishes only after the coordinator's channel also shuts down.
The nodes who finish their state transfer will then send a REBALANCE_CONFIRM command to the new coordinator, but the new coordinator doesn't know about that rebalance (it will start the rebalance process from scratch). This results in exceptions like this in the new coordinator's log:
{noformat}
12:36:04,977 WARN [org.infinispan.topology.CacheTopologyControlCommand] (remote-thread-2,ISPN-Node-1) ISPN000071: Caught exception when handling command CacheTopologyControlCommand{cache=MyCoolCache, type=REBALANCE_CONFIRM, sender=ISPN-Node-3-54019, joinInfo=null, topologyId=8, currentCH=null, pendingCH=null, throwable=null, viewId=4}: org.infinispan.commons.CacheException: Received invalid rebalance confirmation from ISPN-Node-3-54019 for cache MyCoolCache, we don't have a rebalance in progress
at org.infinispan.topology.ClusterTopologyManagerImpl.handleRebalanceCompleted(ClusterTopologyManagerImpl.java:190) [infinispan-core-6.0.0.Final.jar:6.0.0.Final]
at org.infinispan.topology.CacheTopologyControlCommand.doPerform(CacheTopologyControlCommand.java:147) [infinispan-core-6.0.0.Final.jar:6.0.0.Final]
at org.infinispan.topology.CacheTopologyControlCommand.perform(CacheTopologyControlCommand.java:124) [infinispan-core-6.0.0.Final.jar:6.0.0.Final]
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher$4.run(CommandAwareRpcDispatcher.java:270) [infinispan-core-6.0.0.Final.jar:6.0.0.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
{noformat}
A simple way to avoid these warnings would be to keep track of the coordinator that initiated a particular rebalance on each node, and only send the confirmation message to that coordinator. The same warnings seem to appear on the old coordinator, when it receives a confirmation after its ClusterTopologyManager started shutting down, so we may need another check there.
A more ambitious approach would be to keep the old rebalance when the new coordinator takes over, and have another round in the cluster state recovery asking if any members have already sent REBALANCE_CONFIRMATION commands (after the new coordinator is ready to process those commands). This should eliminate the duplicate state transfer that happens now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3780) CLONE - InvalidateL1Command during ST should not cancel writing the entry by ST in nontx
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-3780?page=com.atlassian.jira.plugin.... ]
William Burns edited comment on ISPN-3780 at 12/5/13 4:06 PM:
--------------------------------------------------------------
{quote}The topology change is started on A only after the invalidation has been sent to C.{quote}
Yeah it appears that the new topology was only installed on C before the invalidation arrived and A doesn't get the new topology until after the put command is completed which is why it wasn't retried.
Now normally I would think the topology check in the entryCommit in EntryWrappingInterceptor would then throw an error since A topolyg is 1 behind, however the topologyId is always -1 for InvalidateL1Commands. It is never set in StateTransferInterceptor and the setting of the topology doesn't work in RpcMangerImpl because the command is wrapped in a CacheRpcCommand which doesn't implement TopologyAffectedCommand. I think the easiest fix is to move the if check higher up in RpcManagerImpl, but I will get a test going first to confirm this. Also I will try to fix the issue I found with the LastChanceInterceptor which this may also fix.
was (Author: william.burns):
{quote}The topology change is started on A only after the invalidation has been sent to C.{quote}
Yeah it appears that the new topology was only installed on C before the invalidation arrived and A doesn't get the new topology until after the put command is completed.
Now normally I would think the topology check in the entryCommit in EntryWrappingInterceptor would then throw an error. However the topologyId is always a -1 for InvalidateL1Commands. It is never set in StateTransferInterceptor and the setting of the topology doesn't work in RpcMangerImpl because the command is wrapped in a CacheRpcCommand which doesn't implement TopologyAffectedCommand. I think the easiest fix is to move the if check higher up in RpcManagerImpl, but I will get a test going first to confirm this. Also I will try to fix the issue I found with the LastChanceInterceptor which this may also fix.
> CLONE - InvalidateL1Command during ST should not cancel writing the entry by ST in nontx
> ----------------------------------------------------------------------------------------
>
> Key: ISPN-3780
> URL: https://issues.jboss.org/browse/ISPN-3780
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 5.3.0.Final
> Reporter: Radim Vansa
> Assignee: William Burns
> Priority: Critical
> Labels: 620
> Fix For: 6.1.0.Final
>
>
> When a node which is about to receive the entries for some segment receives InvalidateL1Command, this puts the key into StateConsumer.updatedKeys. After the entries for ST are received, the entry which was invalidated is not updated -> this result in losing the entry.
> Btw., in EntryWrappingInterceptor.visitInvalidateL1Command the trace logs all looked up entries for each entry - this causes some performance problems when tracing is on and there are many invalidated entries. Please, do this only once.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3780) CLONE - InvalidateL1Command during ST should not cancel writing the entry by ST in nontx
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-3780?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-3780:
-------------------------------------
{quote}The topology change is started on A only after the invalidation has been sent to C.{quote}
Yeah it appears that the new topology was only installed on C before the invalidation arrived and A doesn't get the new topology until after the put command is completed.
Now normally I would think the topology check in the entryCommit in EntryWrappingInterceptor would then throw an error. However the topologyId is always a -1 for InvalidateL1Commands. It is never set in StateTransferInterceptor and the setting of the topology doesn't work in RpcMangerImpl because the command is wrapped in a CacheRpcCommand which doesn't implement TopologyAffectedCommand. I think the easiest fix is to move the if check higher up in RpcManagerImpl, but I will get a test going first to confirm this. Also I will try to fix the issue I found with the LastChanceInterceptor which this may also fix.
> CLONE - InvalidateL1Command during ST should not cancel writing the entry by ST in nontx
> ----------------------------------------------------------------------------------------
>
> Key: ISPN-3780
> URL: https://issues.jboss.org/browse/ISPN-3780
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 5.3.0.Final
> Reporter: Radim Vansa
> Assignee: William Burns
> Priority: Critical
> Labels: 620
> Fix For: 6.1.0.Final
>
>
> When a node which is about to receive the entries for some segment receives InvalidateL1Command, this puts the key into StateConsumer.updatedKeys. After the entries for ST are received, the entry which was invalidated is not updated -> this result in losing the entry.
> Btw., in EntryWrappingInterceptor.visitInvalidateL1Command the trace logs all looked up entries for each entry - this causes some performance problems when tracing is on and there are many invalidated entries. Please, do this only once.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months
[JBoss JIRA] (ISPN-3375) Map/Reduce jobs with small value sizes fail because of timeouts putting intermediate keys into the cache
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-3375?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-3375:
-----------------------------------------------
Alan Field <afield(a)redhat.com> changed the Status of [bug 1013001|https://bugzilla.redhat.com/show_bug.cgi?id=1013001] from ON_QA to ASSIGNED
> Map/Reduce jobs with small value sizes fail because of timeouts putting intermediate keys into the cache
> --------------------------------------------------------------------------------------------------------
>
> Key: ISPN-3375
> URL: https://issues.jboss.org/browse/ISPN-3375
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 5.3.0.Final
> Reporter: Alan Field
> Assignee: Dan Berindei
> Priority: Critical
> Labels: 620
> Fix For: 6.0.0.Final
>
> Attachments: jgroups-udp-mr.xml, stdout.zip
>
>
> The following exception occurs on the primary node running a Map/Reduce job with 1KB values:
> {noformat}
> 12:57:32,755 ERROR [org.radargun.stages.MapReduceStage] (pool-1-thread-1) executeMapReduceTask() returned an exception
> org.infinispan.CacheException: java.util.concurrent.ExecutionException: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from edg-perf03-15313, see cause for remote stack trace
> at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:369)
> at org.radargun.cachewrappers.InfinispanMapReduceWrapper.executeMapReduceTask(InfinispanMapReduceWrapper.java:113)
> at org.radargun.stages.MapReduceStage.executeMapReduceTask(MapReduceStage.java:207)
> at org.radargun.stages.MapReduceStage.executeOnSlave(MapReduceStage.java:149)
> at org.radargun.Slave$2.run(Slave.java:103)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: java.util.concurrent.ExecutionException: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from edg-perf03-15313, see cause for remote stack trace
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at org.infinispan.distexec.mapreduce.MapReduceTask$TaskPart.get(MapReduceTask.java:867)
> at org.infinispan.distexec.mapreduce.MapReduceTask.executeMapPhase(MapReduceTask.java:461)
> at org.infinispan.distexec.mapreduce.MapReduceTask.execute(MapReduceTask.java:363)
> ... 10 more
> Caused by: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from edg-perf03-15313, see cause for remote stack trace
> at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:70)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:384)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:189)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:531)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:303)
> at org.infinispan.remoting.rpc.RpcManagerImpl$2.call(RpcManagerImpl.java:340)
> ... 5 more
> Caused by: org.infinispan.CacheException: org.infinispan.CacheException: Could not move intermediate keys/values for M/R task edc7a51c-d25a-4650-b39a-8d8a87d7cd61
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:114)
> at org.infinispan.commands.read.MapCombineCommand.perform(MapCombineCommand.java:93)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:122)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.access$000(InboundInvocationHandlerImpl.java:68)
> at org.infinispan.remoting.InboundInvocationHandlerImpl$2.run(InboundInvocationHandlerImpl.java:194)
> ... 3 more
> Caused by: org.infinispan.CacheException: Could not move intermediate keys/values for M/R task edc7a51c-d25a-4650-b39a-8d8a87d7cd61
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.combine(MapReduceManagerImpl.java:331)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:112)
> ... 7 more
> Caused by: org.infinispan.util.concurrent.TimeoutException: Node edg-perf02-56077 timed out
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:196)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:531)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:303)
> at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.handleNonTxWriteCommand(BaseDistributionInterceptor.java:151)
> at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.visitPutKeyValueCommand(NonTxDistributionInterceptor.java:91)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:290)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:157)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:70)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:194)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:136)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:160)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1337)
> at org.infinispan.CacheImpl.putInternal(CacheImpl.java:898)
> at org.infinispan.CacheImpl.put(CacheImpl.java:890)
> at org.infinispan.CacheImpl.put(CacheImpl.java:1390)
> at org.infinispan.CacheImpl.put(CacheImpl.java:229)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.combine(MapReduceManagerImpl.java:326)
> ... 8 more
> Caused by: org.jgroups.TimeoutException: timeout sending message to edg-perf02-56077
> at org.jgroups.blocks.MessageDispatcher.sendMessage(MessageDispatcher.java:419)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:375)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:189)
> ... 48 more
> {noformat}
> And this exception is occurring on the other node in the cluster:
> {noformat}
> 12:57:32,484 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (remote-thread-0) ISPN000136: Execution error
> org.infinispan.util.concurrent.TimeoutException: Node edg-perf02-56077 timed out
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:196)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:531)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:303)
> at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.handleNonTxWriteCommand(BaseDistributionInterceptor.java:151)
> at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.visitPutKeyValueCommand(NonTxDistributionInterceptor.java:91)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:290)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:157)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:70)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:194)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:136)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:160)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1337)
> at org.infinispan.CacheImpl.putInternal(CacheImpl.java:898)
> at org.infinispan.CacheImpl.put(CacheImpl.java:890)
> at org.infinispan.CacheImpl.put(CacheImpl.java:1390)
> at org.infinispan.CacheImpl.put(CacheImpl.java:229)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.combine(MapReduceManagerImpl.java:326)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:112)
> at org.infinispan.commands.read.MapCombineCommand.perform(MapCombineCommand.java:93)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:122)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.access$000(InboundInvocationHandlerImpl.java:68)
> at org.infinispan.remoting.InboundInvocationHandlerImpl$2.run(InboundInvocationHandlerImpl.java:194)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: org.jgroups.TimeoutException: timeout sending message to edg-perf02-56077
> at org.jgroups.blocks.MessageDispatcher.sendMessage(MessageDispatcher.java:419)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:375)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:189)
> ... 48 more
> 12:57:32,494 ERROR [org.infinispan.remoting.InboundInvocationHandlerImpl] (remote-thread-0) Exception executing command
> org.infinispan.CacheException: org.infinispan.CacheException: Could not move intermediate keys/values for M/R task edc7a51c-d25a-4650-b39a-8d8a87d7cd61
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:114)
> at org.infinispan.commands.read.MapCombineCommand.perform(MapCombineCommand.java:93)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.handleInternal(InboundInvocationHandlerImpl.java:122)
> at org.infinispan.remoting.InboundInvocationHandlerImpl.access$000(InboundInvocationHandlerImpl.java:68)
> at org.infinispan.remoting.InboundInvocationHandlerImpl$2.run(InboundInvocationHandlerImpl.java:194)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: org.infinispan.CacheException: Could not move intermediate keys/values for M/R task edc7a51c-d25a-4650-b39a-8d8a87d7cd61
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.combine(MapReduceManagerImpl.java:331)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.mapAndCombineForDistributedReduction(MapReduceManagerImpl.java:112)
> ... 7 more
> Caused by: org.infinispan.util.concurrent.TimeoutException: Node edg-perf02-56077 timed out
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:196)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:531)
> at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:303)
> at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.handleNonTxWriteCommand(BaseDistributionInterceptor.java:151)
> at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.visitPutKeyValueCommand(NonTxDistributionInterceptor.java:91)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:290)
> at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:157)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:70)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:194)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:136)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:160)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:54)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:83)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
> at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1337)
> at org.infinispan.CacheImpl.putInternal(CacheImpl.java:898)
> at org.infinispan.CacheImpl.put(CacheImpl.java:890)
> at org.infinispan.CacheImpl.put(CacheImpl.java:1390)
> at org.infinispan.CacheImpl.put(CacheImpl.java:229)
> at org.infinispan.distexec.mapreduce.MapReduceManagerImpl.combine(MapReduceManagerImpl.java:326)
> ... 8 more
> Caused by: org.jgroups.TimeoutException: timeout sending message to edg-perf02-56077
> at org.jgroups.blocks.MessageDispatcher.sendMessage(MessageDispatcher.java:419)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:375)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:189)
> ... 48 more
> {noformat}
> ISPN-2836 added a timeout for the entire MapReduceTask, but this timeout is happening when the intermediate keys are put into the cache by the map phase. This error does not occur when the value size is 8KB or larger.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 6 months