[JBoss JIRA] (ISPN-999) Support eventual consistency
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-999?page=com.atlassian.jira.plugin.s... ]
Dan Berindei commented on ISPN-999:
-----------------------------------
I bumped the target version again, to 8.0, since we haven't made any progress on this yet.
> Support eventual consistency
> ----------------------------
>
> Key: ISPN-999
> URL: https://issues.jboss.org/browse/ISPN-999
> Project: Infinispan
> Issue Type: Feature Request
> Components: Core, Transactions
> Reporter: Manik Surtani
> Assignee: Manik Surtani
> Priority: Critical
> Labels: eventual_consistency
> Fix For: 8.0.0.Final
>
> Original Estimate: 1 minute
> Remaining Estimate: 1 minute
>
> Essentially, it is about supporting eventual consistency in Infinispan. Currently Infinispan is strongly consistent when using synchronous distribution mode. Each data owner receives updates synchronously so anyone anywhere on the cluster doing a GET will see the correct value. The only exception is during a rehash (when a new node joins or leaves), that consistency is eventual since the GET may reach a new joiner who may not have applied state it receives from its neighbours yet. However this is hidden from users since the GET is sent to> 1 data owner and if an UnsureResponse is received (determined by the fact that a new joiner responds and the new joiner wouldn't have finished applying state), the caller thread waits for more definite responses.
> However, there is a use case for being eventually consistent as well: the main benefits being speed and partition tolerance. E.g., if we use distribution in asynchronous mode, the writes become *much* faster. However, anyone anywhere doing a GET will have to perform the GET on all data owners, and compare the versions of the data received to determine which is the latest. And if there is a conflict, to pass back all values to the user.
> So in terms of design, what I have in mind is:
> * All cache entries are versioned using vector clocks. One vector clock per node.
> * When a node performs a GET, the GET is sent to all data owners (concurrently), and the value + version is retrieved from each.
> * If the versions are all the same (or they can be "fast forwarded"), the value is returned
> * Otherwise, all potential values and their versions are returned
> * A resolve() API should be provided where application code may provide a "hint" as to which version should be "correct" - which will cause an update.
> * In terms of implementation, this will touch the DistributionInterceptor, InternalCacheEntry and relevant factories, some config code (since this should be consistency model should be configurable), and a new public interface.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 6 months
[JBoss JIRA] (ISPN-999) Support eventual consistency
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-999?page=com.atlassian.jira.plugin.s... ]
Dan Berindei updated ISPN-999:
------------------------------
Fix Version/s: 8.0.0.Final
(was: 7.0.0.Final)
> Support eventual consistency
> ----------------------------
>
> Key: ISPN-999
> URL: https://issues.jboss.org/browse/ISPN-999
> Project: Infinispan
> Issue Type: Feature Request
> Components: Core, Transactions
> Reporter: Manik Surtani
> Assignee: Manik Surtani
> Priority: Critical
> Labels: eventual_consistency
> Fix For: 8.0.0.Final
>
> Original Estimate: 1 minute
> Remaining Estimate: 1 minute
>
> Essentially, it is about supporting eventual consistency in Infinispan. Currently Infinispan is strongly consistent when using synchronous distribution mode. Each data owner receives updates synchronously so anyone anywhere on the cluster doing a GET will see the correct value. The only exception is during a rehash (when a new node joins or leaves), that consistency is eventual since the GET may reach a new joiner who may not have applied state it receives from its neighbours yet. However this is hidden from users since the GET is sent to> 1 data owner and if an UnsureResponse is received (determined by the fact that a new joiner responds and the new joiner wouldn't have finished applying state), the caller thread waits for more definite responses.
> However, there is a use case for being eventually consistent as well: the main benefits being speed and partition tolerance. E.g., if we use distribution in asynchronous mode, the writes become *much* faster. However, anyone anywhere doing a GET will have to perform the GET on all data owners, and compare the versions of the data received to determine which is the latest. And if there is a conflict, to pass back all values to the user.
> So in terms of design, what I have in mind is:
> * All cache entries are versioned using vector clocks. One vector clock per node.
> * When a node performs a GET, the GET is sent to all data owners (concurrently), and the value + version is retrieved from each.
> * If the versions are all the same (or they can be "fast forwarded"), the value is returned
> * Otherwise, all potential values and their versions are returned
> * A resolve() API should be provided where application code may provide a "hint" as to which version should be "correct" - which will cause an update.
> * In terms of implementation, this will touch the DistributionInterceptor, InternalCacheEntry and relevant factories, some config code (since this should be consistency model should be configurable), and a new public interface.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 6 months
[JBoss JIRA] (ISPN-4735) Remove TODOs from the documentation
by Mircea Markus (JIRA)
Mircea Markus created ISPN-4735:
-----------------------------------
Summary: Remove TODOs from the documentation
Key: ISPN-4735
URL: https://issues.jboss.org/browse/ISPN-4735
Project: Infinispan
Issue Type: Task
Components: Documentation-Core, Documentation-Query, Documentation-Servers
Reporter: Mircea Markus
Assignee: Dan Berindei
Priority: Critical
Fix For: 7.0.0.CR1
ATM the documentation has 38 todos, some of them being empty chapter in User Guide - pretty annoying for a new user trying to get used to Infinispan. Let's fixe all of these before 7.0.0.CR1
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 6 months
[JBoss JIRA] (ISPN-4734) Hot Rod marshaller for custom events...etc, needs to be configurable in server
by Galder Zamarreño (JIRA)
Galder Zamarreño created ISPN-4734:
--------------------------------------
Summary: Hot Rod marshaller for custom events...etc, needs to be configurable in server
Key: ISPN-4734
URL: https://issues.jboss.org/browse/ISPN-4734
Project: Infinispan
Issue Type: Enhancement
Affects Versions: 7.0.0.Beta2
Reporter: Galder Zamarreño
Assignee: Galder Zamarreño
Fix For: 7.0.0.CR1
The marshaller used by Hot Rod to marshall custom events is configurable via HotRodServerConfigurationBuilder. This needs to be extended to server side configuration.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 6 months
[JBoss JIRA] (ISPN-4677) Map/Reduce job fails with the wrong explanation on underlying TimeoutException
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4677?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-4677:
------------------------------------
I agree with Sanne, I think it would be better to wrap exceptions thrown by mappers/combiners/reducers in an exception.
> Map/Reduce job fails with the wrong explanation on underlying TimeoutException
> ------------------------------------------------------------------------------
>
> Key: ISPN-4677
> URL: https://issues.jboss.org/browse/ISPN-4677
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 7.0.0.Beta1
> Reporter: Sanne Grinovero
> Assignee: Vladimir Blagojevic
> Fix For: 7.0.0.CR1
>
>
> I'm running a task which is throwing the following exception during the Map phase:
> {noformat}
> org.infinispan.util.concurrent.TimeoutException: Node main-NodeC-22183 timed out{noformat}
> The user facing error however is this very confusing message:
> {noformat}org.infinispan.commons.CacheException: java.util.concurrent.ExecutionException: Map phase executing at main-NodeA-39904 did not complete within 0 sec timeout{noformat}
> I have no timeout enabled.
> The problem is the instanceof operation on the cause of the error in org.infinispan.distexec.mapreduce.MapReduceTask.executeMapPhaseWithLocalReduction(Map<KOut, VOut>): the check on the type only is not good enough.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 6 months
[JBoss JIRA] (ISPN-4733) Investigate potential issues with System.nanoTime() overflow
by Dan Berindei (JIRA)
Dan Berindei created ISPN-4733:
----------------------------------
Summary: Investigate potential issues with System.nanoTime() overflow
Key: ISPN-4733
URL: https://issues.jboss.org/browse/ISPN-4733
Project: Infinispan
Issue Type: Task
Components: Core, Test Suite - Core
Affects Versions: 7.0.0.Beta2
Reporter: Dan Berindei
Assignee: William Burns
Fix For: 7.0.0.CR1
{{System.nanoTime()}} can overflow, so it's not ok to compare two {{System.nanoTime()}} return values directly. Instead, we always have to work with the difference between return values.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 6 months
[JBoss JIRA] (ISPN-4694) Apply ASYNC backend indexing options to the RemoteIndexingBackend
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-4694?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero updated ISPN-4694:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> Apply ASYNC backend indexing options to the RemoteIndexingBackend
> -----------------------------------------------------------------
>
> Key: ISPN-4694
> URL: https://issues.jboss.org/browse/ISPN-4694
> Project: Infinispan
> Issue Type: Feature Request
> Components: Embedded Querying
> Reporter: Sanne Grinovero
> Assignee: Gustavo Fernandes
> Fix For: 7.0.0.CR1, 7.0.0.Final
>
>
> The {{RemoteIndexingBackend}} in Infinispan Query generates synchronous RPCs.
> There is an option in Hibernate Search to enable the "async" worker, which decouples the indexing queue processing from the invoker by using a separate executor.
> If this option is enabled, we should transparently apply it to the {{RemoteIndexingBackend}} too by making RPCs non blocking as well.
> {code}rpcManager.getDefaultRpcOptions(sync){code}
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 6 months
[JBoss JIRA] (ISPN-999) Support eventual consistency
by Rich DiCroce (JIRA)
[ https://issues.jboss.org/browse/ISPN-999?page=com.atlassian.jira.plugin.s... ]
Rich DiCroce commented on ISPN-999:
-----------------------------------
Is this still slated for Infinispan 7? I can see it's been bumped from 5.1 to 6 to 7 so far...
> Support eventual consistency
> ----------------------------
>
> Key: ISPN-999
> URL: https://issues.jboss.org/browse/ISPN-999
> Project: Infinispan
> Issue Type: Feature Request
> Components: Core, Transactions
> Reporter: Manik Surtani
> Assignee: Manik Surtani
> Priority: Critical
> Labels: eventual_consistency
> Fix For: 7.0.0.Final
>
> Original Estimate: 1 minute
> Remaining Estimate: 1 minute
>
> Essentially, it is about supporting eventual consistency in Infinispan. Currently Infinispan is strongly consistent when using synchronous distribution mode. Each data owner receives updates synchronously so anyone anywhere on the cluster doing a GET will see the correct value. The only exception is during a rehash (when a new node joins or leaves), that consistency is eventual since the GET may reach a new joiner who may not have applied state it receives from its neighbours yet. However this is hidden from users since the GET is sent to> 1 data owner and if an UnsureResponse is received (determined by the fact that a new joiner responds and the new joiner wouldn't have finished applying state), the caller thread waits for more definite responses.
> However, there is a use case for being eventually consistent as well: the main benefits being speed and partition tolerance. E.g., if we use distribution in asynchronous mode, the writes become *much* faster. However, anyone anywhere doing a GET will have to perform the GET on all data owners, and compare the versions of the data received to determine which is the latest. And if there is a conflict, to pass back all values to the user.
> So in terms of design, what I have in mind is:
> * All cache entries are versioned using vector clocks. One vector clock per node.
> * When a node performs a GET, the GET is sent to all data owners (concurrently), and the value + version is retrieved from each.
> * If the versions are all the same (or they can be "fast forwarded"), the value is returned
> * Otherwise, all potential values and their versions are returned
> * A resolve() API should be provided where application code may provide a "hint" as to which version should be "correct" - which will cause an update.
> * In terms of implementation, this will touch the DistributionInterceptor, InternalCacheEntry and relevant factories, some config code (since this should be consistency model should be configurable), and a new public interface.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 6 months