[JBoss JIRA] (ISPN-5199) TransportStackConfigurationIT failing
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5199?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5199:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1202253|https://bugzilla.redhat.com/show_bug.cgi?id=1202253] from POST to MODIFIED
> TransportStackConfigurationIT failing
> -------------------------------------
>
> Key: ISPN-5199
> URL: https://issues.jboss.org/browse/ISPN-5199
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Server
> Affects Versions: 7.1.0.Final
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Fix For: 7.2.0.Alpha1, 7.1.1.Final
>
>
> TransportStackConfigurationIT.testTCPStackAttributes and TransportStackConfigurationIT.testUDPStackAttributes fails with:
> {code}
> org.junit.ComparisonFailure: expected:<[d]iscard> but was:<[D]iscard>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at org.infinispan.server.test.transport.TransportStackConfigurationIT.assertMBeanAttributes(TransportStackConfigurationIT.java:76)
> at org.infinispan.server.test.transport.TransportStackConfigurationIT.testUDPStackAttributes(TransportStackConfigurationIT.java:55)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years
[JBoss JIRA] (ISPN-5309) Model data consistency for get()/put() operations in Infinispan
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-5309?page=com.atlassian.jira.plugin.... ]
Radim Vansa commented on ISPN-5309:
-----------------------------------
The suggestion above looks like a good start for learning PlusCal/TLA+, but not considering membership changes in the basic design looks as we won't catch much. I think that the modeling should be based on good approximation of our view ids/topologyIds/CHs as we do use them, rather than focus on cache with multiple keys.
Off course, let's leave cachestores, passivation etc. away - we already know where these are broken. I would even suggest starting just with replicated cache (or rather single value replicated to all nodes).
> Model data consistency for get()/put() operations in Infinispan
> ---------------------------------------------------------------
>
> Key: ISPN-5309
> URL: https://issues.jboss.org/browse/ISPN-5309
> Project: Infinispan
> Issue Type: Task
> Reporter: Richard Achmatowicz
> Assignee: Richard Achmatowicz
>
> This will be the first in a series of modelling/validation exercises for some of the more critical Infinispan protocols.
> We shall use TLA+ / PlusCal / TLA+ Tools to do the following:
> - model the design of processing of get()/put() operations in an Infinispan cluster
> - model client interactions with that cluster
> - describe the data consistency requirements of get()/put() operations
> - verify that the data consistency semantics of Infinispan are preserved in the face of concurrent client interactions
> TLA+ / PLusCal can be thought of as a pseudo-code language which has a well-defined semantics and is testable using the tools in the TLA+ Toolkit.
> The benefits of such an exercise are that we end up with:
> - a specification of data consistency guarantees that Infinispan provides
> - a semantically precise pseudo-code description of the design of get()/put() processing
> - a verification that the protocol design is correct
> We start here with the simple case of modelling data consistency in the absence of failures. In later exercises, we aim to tackle rebalancing and non-blocking state transfer in the face of membership changes and partitions.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years
[JBoss JIRA] (ISPN-5298) HotRod millisecond precision for lifespan/maxidle
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-5298?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño edited comment on ISPN-5298 at 3/18/15 11:26 AM:
------------------------------------------------------------------
Ion, did you get a reply to the JSR issue? I'd wait to get an answer for that before making any immediate changes :|
If we really need to implement this, I think Dan's approach is easiest. IOW, adding flags indicating the granularity of the expiration parameter. Seconds and Milliseconds would be enough. Remember what the protocol says:
bq. Number of seconds that a entry during which the entry is allowed to life. If number of seconds is bigger than 30 days, this number of seconds is treated as UNIX time and so, represents the number of seconds since 1/1/1970. If set to 0, lifespan is unlimited.
Seconds is enough to be able to represent any other expiration that's above it.
Let's not complicate things beyond what's required :)
was (Author: galder.zamarreno):
Ion, did you get a reply to the JSR issue? I'd wait to get an answer for that before making any immediate changes :|
If we really need to implement this, I think Dan's approach is easiest. IOW, adding flags indicating the granularity of the expiration parameter. Seconds and Milliseconds would be enough. Also, remember what the protocol says:
bq. Number of seconds that a entry during which the entry is allowed to life. If number of seconds is bigger than 30 days, this number of seconds is treated as UNIX time and so, represents the number of seconds since 1/1/1970. If set to 0, lifespan is unlimited.
Seconds is enough to be able to represent any other expiration that's above it.
Let's not complicate things beyond what's required :)
> HotRod millisecond precision for lifespan/maxidle
> -------------------------------------------------
>
> Key: ISPN-5298
> URL: https://issues.jboss.org/browse/ISPN-5298
> Project: Infinispan
> Issue Type: Feature Request
> Components: Remote Protocols
> Affects Versions: 7.2.0.Alpha1
> Reporter: Ion Savin
>
> Currently lifespan/maxidle precision is SECONDS.
> JSR-107 requires MILLISECONDS precision.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years
[JBoss JIRA] (ISPN-5298) HotRod millisecond precision for lifespan/maxidle
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-5298?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño commented on ISPN-5298:
----------------------------------------
Ion, did you get a reply to the JSR issue? I'd wait to get an answer for that before making any immediate changes :|
If we really need to implement this, I think Dan's approach is easiest. IOW, adding flags indicating the granularity of the expiration parameter. Seconds and Milliseconds would be enough. Also, remember what the protocol says:
bq. Number of seconds that a entry during which the entry is allowed to life. If number of seconds is bigger than 30 days, this number of seconds is treated as UNIX time and so, represents the number of seconds since 1/1/1970. If set to 0, lifespan is unlimited.
Seconds is enough to be able to represent any other expiration that's above it.
Let's not complicate things beyond what's required :)
> HotRod millisecond precision for lifespan/maxidle
> -------------------------------------------------
>
> Key: ISPN-5298
> URL: https://issues.jboss.org/browse/ISPN-5298
> Project: Infinispan
> Issue Type: Feature Request
> Components: Remote Protocols
> Affects Versions: 7.2.0.Alpha1
> Reporter: Ion Savin
>
> Currently lifespan/maxidle precision is SECONDS.
> JSR-107 requires MILLISECONDS precision.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years
[JBoss JIRA] (ISPN-5085) Convert REST server to using RESTEasy's Netty container integration
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-5085?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-5085:
-----------------------------------
Summary: Convert REST server to using RESTEasy's Netty container integration (was: Convert REST server to using Netty instead of RESTEasy)
> Convert REST server to using RESTEasy's Netty container integration
> -------------------------------------------------------------------
>
> Key: ISPN-5085
> URL: https://issues.jboss.org/browse/ISPN-5085
> Project: Infinispan
> Issue Type: Enhancement
> Components: Remote Protocols
> Reporter: Galder Zamarreño
> Fix For: 7.2.0.Beta2, 7.2.0.Final
>
>
> REST server is currently based on RESTEasy which relies on having a web container underneath to do its job. So, we're depending on jbossweb/undertow to provide the container capabilities. Instead, we should switch to a Netty based container to better control our dependencies.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years
[JBoss JIRA] (ISPN-5085) Convert REST server to using Netty instead of RESTEasy
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-5085?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-5085:
-----------------------------------
Description: REST server is currently based on RESTEasy which relies on having a web container underneath to do its job. So, we're depending on jbossweb/undertow to provide the container capabilities. Instead, we should switch to a Netty based container to better control our dependencies. (was: REST server is currently based on RESTEasy. We should instead use Netty.)
> Convert REST server to using Netty instead of RESTEasy
> ------------------------------------------------------
>
> Key: ISPN-5085
> URL: https://issues.jboss.org/browse/ISPN-5085
> Project: Infinispan
> Issue Type: Enhancement
> Components: Remote Protocols
> Reporter: Galder Zamarreño
> Fix For: 7.2.0.Beta2, 7.2.0.Final
>
>
> REST server is currently based on RESTEasy which relies on having a web container underneath to do its job. So, we're depending on jbossweb/undertow to provide the container capabilities. Instead, we should switch to a Netty based container to better control our dependencies.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years
[JBoss JIRA] (ISPN-5309) Model data consistency for get()/put() operations in Infinispan
by Richard Achmatowicz (JIRA)
[ https://issues.jboss.org/browse/ISPN-5309?page=com.atlassian.jira.plugin.... ]
Richard Achmatowicz edited comment on ISPN-5309 at 3/18/15 11:05 AM:
---------------------------------------------------------------------
For the record:
{quote}
> Hi Dan
>
> After reviewing your document Consistency Guarantees in Infinispan, I agree the starting point would be to validate get() and put() data consistency semantics in the absence of membership changes on the simplest of caches: no data store, no transactions, no passivation nor eviction. These are all refinements of the more basic model.
>
> I have a few comments/clarifications on the following issues:
> - the expected data consistency for interleavings of gets and puts
> - use of cache topology and CH in replicated mode and distributed mode
> - some more detail on the processing of get and put operations in the context of a set of infinispan cache instances in distributed mode
> - message order assumptions
>
>
> data consistency
> ---------------
> In order to describe input sequences of get/put operations and the expected output values they return after processing by Infinispan,
> I assume the following:
>
> - operations to be considered are ordered (or not) by the happens-before relation based on causality (as opposed to wall clock time)
> - initial data consistency model for get and put operations:
> - get(): operation that happens-after a get operation get(k) -> v sees the same value v (or the value of a later write)
> - put(): operation that happens-after a write operation put(k=v) will see the updated value v (or the value of a later write)
>
> NB: this rules out any required data consistency between operations initiated at different nodes, even though one may occur after the other based on wall clock time
This sounds good.
Because a single client can connect to multiple nodes, in the future we should also model consistency for operations that happen on different but do not overlap (the first operation completely finishes before the other operation starts).
> replicated mode vs distributed mode
> -----------------------------------
> I'm a little unclear on the way in which replicated mode and distributed mode
> make use of cache topology information (which we can assume
> has been correctly distributed to all nodes):
>
> - do both of these modes make use the current cache topology (containing the CH) to lookup nodes corresponding to keys they need to access?
Yes
> - is the only difference in the way that the CH is set up (so that for cluster size N, there is one primary owner and N-1 backup owners)?
There is one more difference: replicated mode always uses broadcasts instead of unicasts. It should only be relevant when the JGroups view changes, though.
> - I assume only one CH is used in regular mode (as opposed to rehashing mode)
Yes, we only have 2 CHs during state transfer.
> distributed mode get()/put() design
> -----------------------------------
> Here is a summary of the way in which get and puts are processed:
>
> (taken almost verbatim from section 4 of Consistency Guarantees in Infinispan, reorganized a little)
>
> get():
> - when an originator performs a get(K):
> - if originator == owner, return the local value
> - if originator != owner, originator sends a remote GET_REQ command to
> all its owners in read-CH and uses the first successful non-null
> response (GET_RSP)
Correct. We do a retry when the topology id changes, but we can ignore that for now.
> put:()
> - when an originator performs a put(K,V):
> - if originator == primary owner
> - primary owner acquires the key lock, and replicates (PUT_REPL_REQ)
> the write to the backup owner(s)
> - backup owners confirm the write to the primary owner (PUT_REPL_RSP)
> - primary owner updates the value locally, unlocks the key lock, and
> replies to the originator
> - if originator != primary owner
> - originator will send a remote PUT_REQ to the primary owner of the
> modified key (looked up from which CH?)
> - primary owner acquires the key lock, and replicates (PUT_REPL_REQ)
> the write to the backup owner(s)
> - backup owners confirm the write to the primary owner (PUT_REPL_RSP)
> - primary owner updates the value locally, unlocks the key lock, and
> replies to the originator (PUT_RSP)
Correct.
> Are there any corrections to these descriptions that you would need to make?
> Do you have the actual message types for these messages passed back and
> forth?
Remote get uses ClusteredGetCommand for replication, and the response is a SuccessfulResponse with the entry value (in some cases with an InternalCacheEntry, but we can ignore that for now).
Remote put uses PutKeyValueCommand, both for the originator to talk to the primary and for the primary to talk to the backup owners. The responses are again SuccessfulResponses, containing the previous value.
> messaging assumptions
> ---------------------
> The processing of get()/put() makes use of unicast and RPC to communicate
> between nodes.
We automatically use broadcasts instead of unicasts when a message is being sent to all the nodes in the JGroups view. So we always use broadcasts in replicated mode, and mostly unicasts in distributed mode.
> What assumptions are there on message order for:
> - the unicasts from originator to primary
> - the RPCs from primary owner to backup owners
In a DIST_SYNC/REPL_SYNC cache, all messages marked as OOB, so we don't assume any ordering.
We do assume that messages are ordered in DIST_ASYNC/REPL_ASYNC mode. Let's stick with synchronous caches, though
{quote}
was (Author: rachmato):
For the record:
<quote>
> Hi Dan
>
> After reviewing your document Consistency Guarantees in Infinispan, I agree the starting point would be to validate get() and put() data consistency semantics in the absence of membership changes on the simplest of caches: no data store, no transactions, no passivation nor eviction. These are all refinements of the more basic model.
>
> I have a few comments/clarifications on the following issues:
> - the expected data consistency for interleavings of gets and puts
> - use of cache topology and CH in replicated mode and distributed mode
> - some more detail on the processing of get and put operations in the context of a set of infinispan cache instances in distributed mode
> - message order assumptions
>
>
> data consistency
> ---------------
> In order to describe input sequences of get/put operations and the expected output values they return after processing by Infinispan,
> I assume the following:
>
> - operations to be considered are ordered (or not) by the happens-before relation based on causality (as opposed to wall clock time)
> - initial data consistency model for get and put operations:
> - get(): operation that happens-after a get operation get(k) -> v sees the same value v (or the value of a later write)
> - put(): operation that happens-after a write operation put(k=v) will see the updated value v (or the value of a later write)
>
> NB: this rules out any required data consistency between operations initiated at different nodes, even though one may occur after the other based on wall clock time
This sounds good.
Because a single client can connect to multiple nodes, in the future we should also model consistency for operations that happen on different but do not overlap (the first operation completely finishes before the other operation starts).
> replicated mode vs distributed mode
> -----------------------------------
> I'm a little unclear on the way in which replicated mode and distributed mode
> make use of cache topology information (which we can assume
> has been correctly distributed to all nodes):
>
> - do both of these modes make use the current cache topology (containing the CH) to lookup nodes corresponding to keys they need to access?
Yes
> - is the only difference in the way that the CH is set up (so that for cluster size N, there is one primary owner and N-1 backup owners)?
There is one more difference: replicated mode always uses broadcasts instead of unicasts. It should only be relevant when the JGroups view changes, though.
> - I assume only one CH is used in regular mode (as opposed to rehashing mode)
Yes, we only have 2 CHs during state transfer.
> distributed mode get()/put() design
> -----------------------------------
> Here is a summary of the way in which get and puts are processed:
>
> (taken almost verbatim from section 4 of Consistency Guarantees in Infinispan, reorganized a little)
>
> get():
> - when an originator performs a get(K):
> - if originator == owner, return the local value
> - if originator != owner, originator sends a remote GET_REQ command to
> all its owners in read-CH and uses the first successful non-null
> response (GET_RSP)
Correct. We do a retry when the topology id changes, but we can ignore that for now.
> put:()
> - when an originator performs a put(K,V):
> - if originator == primary owner
> - primary owner acquires the key lock, and replicates (PUT_REPL_REQ)
> the write to the backup owner(s)
> - backup owners confirm the write to the primary owner (PUT_REPL_RSP)
> - primary owner updates the value locally, unlocks the key lock, and
> replies to the originator
> - if originator != primary owner
> - originator will send a remote PUT_REQ to the primary owner of the
> modified key (looked up from which CH?)
> - primary owner acquires the key lock, and replicates (PUT_REPL_REQ)
> the write to the backup owner(s)
> - backup owners confirm the write to the primary owner (PUT_REPL_RSP)
> - primary owner updates the value locally, unlocks the key lock, and
> replies to the originator (PUT_RSP)
Correct.
> Are there any corrections to these descriptions that you would need to make?
> Do you have the actual message types for these messages passed back and
> forth?
Remote get uses ClusteredGetCommand for replication, and the response is a SuccessfulResponse with the entry value (in some cases with an InternalCacheEntry, but we can ignore that for now).
Remote put uses PutKeyValueCommand, both for the originator to talk to the primary and for the primary to talk to the backup owners. The responses are again SuccessfulResponses, containing the previous value.
> messaging assumptions
> ---------------------
> The processing of get()/put() makes use of unicast and RPC to communicate
> between nodes.
We automatically use broadcasts instead of unicasts when a message is being sent to all the nodes in the JGroups view. So we always use broadcasts in replicated mode, and mostly unicasts in distributed mode.
> What assumptions are there on message order for:
> - the unicasts from originator to primary
> - the RPCs from primary owner to backup owners
In a DIST_SYNC/REPL_SYNC cache, all messages marked as OOB, so we don't assume any ordering.
We do assume that messages are ordered in DIST_ASYNC/REPL_ASYNC mode. Let's stick with synchronous caches, though
<quote>
> Model data consistency for get()/put() operations in Infinispan
> ---------------------------------------------------------------
>
> Key: ISPN-5309
> URL: https://issues.jboss.org/browse/ISPN-5309
> Project: Infinispan
> Issue Type: Task
> Reporter: Richard Achmatowicz
> Assignee: Richard Achmatowicz
>
> This will be the first in a series of modelling/validation exercises for some of the more critical Infinispan protocols.
> We shall use TLA+ / PlusCal / TLA+ Tools to do the following:
> - model the design of processing of get()/put() operations in an Infinispan cluster
> - model client interactions with that cluster
> - describe the data consistency requirements of get()/put() operations
> - verify that the data consistency semantics of Infinispan are preserved in the face of concurrent client interactions
> TLA+ / PLusCal can be thought of as a pseudo-code language which has a well-defined semantics and is testable using the tools in the TLA+ Toolkit.
> The benefits of such an exercise are that we end up with:
> - a specification of data consistency guarantees that Infinispan provides
> - a semantically precise pseudo-code description of the design of get()/put() processing
> - a verification that the protocol design is correct
> We start here with the simple case of modelling data consistency in the absence of failures. In later exercises, we aim to tackle rebalancing and non-blocking state transfer in the face of membership changes and partitions.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years