[JBoss JIRA] (ISPN-6087) Hadoop MapReduce task returns wrong results after one of the servers from the cluster is stopped
by Anna Manukyan (JIRA)
Anna Manukyan created ISPN-6087:
-----------------------------------
Summary: Hadoop MapReduce task returns wrong results after one of the servers from the cluster is stopped
Key: ISPN-6087
URL: https://issues.jboss.org/browse/ISPN-6087
Project: Infinispan
Issue Type: Bug
Components: Hadoop Integration
Reporter: Anna Manukyan
Attachments: FailoverTest.log
The test is performed on 4 node cluster. The data read is performed from Infinispan cache, the write is performed to HDFS.
There are 200000 entries in the cache. As soon as the Hadoop Job is submitted for performing map/reduce task, one of the servers from the cluster is stopped.
The MapReduce task performs calculation of word count.
As a result, the Hadoop Map/Reduce result map contains wrong results, that is the number of the certain words found is greater than it is expected.
The failure doesn't always happen. It depends on which stage was the mapreduce task when the server stopped.
You can find the test execution log attached.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 11 months
[JBoss JIRA] (ISPN-6086) Hadoop MapReduce task fails after the prefered ispn server stopped
by Anna Manukyan (JIRA)
[ https://issues.jboss.org/browse/ISPN-6086?page=com.atlassian.jira.plugin.... ]
Anna Manukyan reassigned ISPN-6086:
-----------------------------------
Assignee: Gustavo Fernandes
> Hadoop MapReduce task fails after the prefered ispn server stopped
> -------------------------------------------------------------------
>
> Key: ISPN-6086
> URL: https://issues.jboss.org/browse/ISPN-6086
> Project: Infinispan
> Issue Type: Bug
> Components: Hadoop Integration
> Reporter: Anna Manukyan
> Assignee: Gustavo Fernandes
> Attachments: FailoverTest_when_prefered_stopped.log
>
>
> I have added tests to the infinispan-hadoop testsuite testing the load balancing of mapreduce task in case if the input server which is specified as a parameter to Hadoop configuration is stopped.
> The test is performed on 4 node cluster. The data read is performed from Infinispan cache, the write is performed to HDFS.
> There are 200000 entries in the cache. As soon as the Hadoop Job is submitted for performing map/reduce task, the preferred server is stopped.
> As a result, the Hadoop Map/Reduce result map is empty.
> You can find the test execution log attached.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 11 months
[JBoss JIRA] (ISPN-6086) Hadoop MapReduce task fails after the prefered ispn server stopped
by Anna Manukyan (JIRA)
Anna Manukyan created ISPN-6086:
-----------------------------------
Summary: Hadoop MapReduce task fails after the prefered ispn server stopped
Key: ISPN-6086
URL: https://issues.jboss.org/browse/ISPN-6086
Project: Infinispan
Issue Type: Bug
Components: Hadoop Integration
Reporter: Anna Manukyan
Attachments: FailoverTest_when_prefered_stopped.log
I have added tests to the infinispan-hadoop testsuite testing the load balancing of mapreduce task in case if the input server which is specified as a parameter to Hadoop configuration is stopped.
The test is performed on 4 node cluster. The data read is performed from Infinispan cache, the write is performed to HDFS.
There are 200000 entries in the cache. As soon as the Hadoop Job is submitted for performing map/reduce task, the preferred server is stopped.
As a result, the Hadoop Map/Reduce result map is empty.
You can find the test execution log attached.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 11 months
[JBoss JIRA] (ISPN-6085) Avoid using sun.misc.VM
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-6085?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero updated ISPN-6085:
----------------------------------
Status: Open (was: New)
> Avoid using sun.misc.VM
> -----------------------
>
> Key: ISPN-6085
> URL: https://issues.jboss.org/browse/ISPN-6085
> Project: Infinispan
> Issue Type: Task
> Components: Core
> Affects Versions: 8.1.0.Final
> Reporter: Sanne Grinovero
> Assignee: William Burns
>
> The function {{org.infinispan.container.entries.PrimitiveEntrySizeCalculator.IntegerCacheConstraints.calculateMaxIntCache()}} is using {{sun.misc.VM}}.
> This class is being moved and "idden" in Java 9:
> - http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/037853....
> Could we try avoiding the need for this, or explore alternatives?
> One could use reflection to see if the class is available without loading it explicitly, but that still needs some sensible fallback for the case in which the class isn't available.
> Even if this looks like a small detail, this completely blocks the possibility to test Infinispan on Java 9.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 11 months
[JBoss JIRA] (ISPN-6085) Avoid using sun.misc.VM
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-6085?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero reassigned ISPN-6085:
-------------------------------------
Assignee: Sanne Grinovero (was: William Burns)
> Avoid using sun.misc.VM
> -----------------------
>
> Key: ISPN-6085
> URL: https://issues.jboss.org/browse/ISPN-6085
> Project: Infinispan
> Issue Type: Task
> Components: Core
> Affects Versions: 8.1.0.Final
> Reporter: Sanne Grinovero
> Assignee: Sanne Grinovero
>
> The function {{org.infinispan.container.entries.PrimitiveEntrySizeCalculator.IntegerCacheConstraints.calculateMaxIntCache()}} is using {{sun.misc.VM}}.
> This class is being moved and "idden" in Java 9:
> - http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/037853....
> Could we try avoiding the need for this, or explore alternatives?
> One could use reflection to see if the class is available without loading it explicitly, but that still needs some sensible fallback for the case in which the class isn't available.
> Even if this looks like a small detail, this completely blocks the possibility to test Infinispan on Java 9.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 11 months
[JBoss JIRA] (ISPN-6022) Unable to query cache when data is preloaded via AdvancedCacheLoader#process
by Dan Siviter (JIRA)
[ https://issues.jboss.org/browse/ISPN-6022?page=com.atlassian.jira.plugin.... ]
Dan Siviter commented on ISPN-6022:
-----------------------------------
Any update?
> Unable to query cache when data is preloaded via AdvancedCacheLoader#process
> ----------------------------------------------------------------------------
>
> Key: ISPN-6022
> URL: https://issues.jboss.org/browse/ISPN-6022
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying, Loaders and Stores
> Affects Versions: 8.1.0.Final
> Reporter: Dan Siviter
>
> When preloading from a {{AdvancedCacheLoader}} the index doesn't get updated. Therefore it is only possible to query items that have been {{#put(...)}} into the cache. I am able to get preloaded items from the cache using their key which leads me to think the index is never built on pre-load.
> I've seen no implicit rebuilding of caches in any of the existing {{AdvancedCacheLoader#process(...)}} which leads me to think this will not work with any of them.
> I've verified this reindexing using {{searchManager.getMassIndexer().start()}} and the query will then return results.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 11 months