4:19 p.m.
Hello,
What is the recommended cluster size for Replication Mode?
Given 3 nodes, My replication configuration performs better than my distributed
configuration.
Just wanted to know, at what cluster size, distribution will perform better than
replication.
Thanks,
Faseela
3:43 p.m.
On 6 Aug 2013, at 15:19, Faseela K <faseela.k(a)ericsson.com> wrote:
What is the recommended cluster size for Replication Mode?
Given 3 nodes, My replication configuration performs better than my distributed
configuration.
Just wanted to know, at what cluster size, distribution will perform better than
replication.
There's no straight answer, it depends on the read/write ratio and the amount of data
you store.
Replication will always perform better for reads as it won't involve a remote call to
get the data.
If you're mostly doing reads and your memory allows (replication is more memory
consuming) then you should use replication.
If the amount of data increases or you're doing more writes, distribution is the way
to go.
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
9:04 a.m.
Hi,
With a 3 node cluster, even for "WRITES" my replication performance is better
than distribution.
That's why I came across this doubt.
Could some body please clarify, why the behaviour is like this?
Thanks,
Faseela
-----Original Message-----
From: infinispan-dev-bounces(a)lists.jboss.org
[mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Mircea Markus
Sent: Friday, August 09, 2013 7:14 PM
To: infinispan -Dev List
Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication Mode
On 6 Aug 2013, at 15:19, Faseela K <faseela.k(a)ericsson.com> wrote:
What is the recommended cluster size for Replication Mode?
Given 3 nodes, My replication configuration performs better than my distributed
configuration.
Just wanted to know, at what cluster size, distribution will perform better than
replication.
There's no straight answer, it depends on the read/write ratio and the amount of data
you store.
Replication will always perform better for reads as it won't involve a remote call to
get the data.
If you're mostly doing reads and your memory allows (replication is more memory
consuming) then you should use replication.
If the amount of data increases or you're doing more writes, distribution is the way
to go.
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
2:42 p.m.
Hi,
which version exactly do you use, 5.2.x, 5.3.x or 6.0.x? In 5.2 the
replication mode was implemented separately from distribution mode and
depending on your configuration (is it non-transactional synchronous?)
the message control flow could differ. Since 5.3 replication mode is
implemented in the same manner and the results should be more comparable.
I may be wrong here, but in 5.2.x concurrent writes to single key in
non-transactional mode could result in entries being out of sync on some
nodes (the writes could arrive at two nodes in different order). I think
this cannot happen in >= 5.3 anymore.
Radim
On 08/12/2013 09:04 AM, Faseela K wrote:
Hi,
With a 3 node cluster, even for "WRITES" my replication performance is
better than distribution.
That's why I came across this doubt.
Could some body please clarify, why the behaviour is like this?
Thanks,
Faseela
-----Original Message-----
From: infinispan-dev-bounces(a)lists.jboss.org
[mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Mircea Markus
Sent: Friday, August 09, 2013 7:14 PM
To: infinispan -Dev List
Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication Mode
On 6 Aug 2013, at 15:19, Faseela K <faseela.k(a)ericsson.com> wrote:
> What is the recommended cluster size for Replication Mode?
> Given 3 nodes, My replication configuration performs better than my distributed
configuration.
> Just wanted to know, at what cluster size, distribution will perform better than
replication.
There's no straight answer, it depends on the read/write ratio and the amount of data
you store.
Replication will always perform better for reads as it won't involve a remote call to
get the data.
If you're mostly doing reads and your memory allows (replication is more memory
consuming) then you should use replication.
If the amount of data increases or you're doing more writes, distribution is the way
to go.
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
6:29 a.m.
Hi,
I am using infinispan 5.2.3.
My configuration is non-transactional, synchronous.
With this configuration, is my replication supposed to perform better than
distribution, for both reads and writes?
My Cluster Size requirement is 4 nodes.
And my application involves both reads and writes heavily.
For better performance, are there any suggestions on the clustering
modes/configurations?
All my tests show, replication having better performance than distribution for reads as
well as writes, with 4 nodes.
Thanks,
Faseela
-----Original Message-----
From: infinispan-dev-bounces(a)lists.jboss.org
[mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim Vansa
Sent: Monday, August 12, 2013 6:13 PM
To: infinispan-dev(a)lists.jboss.org
Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication Mode
Hi,
which version exactly do you use, 5.2.x, 5.3.x or 6.0.x? In 5.2 the replication mode was
implemented separately from distribution mode and depending on your configuration (is it
non-transactional synchronous?) the message control flow could differ. Since 5.3
replication mode is implemented in the same manner and the results should be more
comparable.
I may be wrong here, but in 5.2.x concurrent writes to single key in non-transactional
mode could result in entries being out of sync on some nodes (the writes could arrive at
two nodes in different order). I think this cannot happen in >= 5.3 anymore.
Radim
On 08/12/2013 09:04 AM, Faseela K wrote:
Hi,
With a 3 node cluster, even for "WRITES" my replication performance is
better than distribution.
That's why I came across this doubt.
Could some body please clarify, why the behaviour is like this?
Thanks,
Faseela
-----Original Message-----
From: infinispan-dev-bounces(a)lists.jboss.org
[mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Mircea
Markus
Sent: Friday, August 09, 2013 7:14 PM
To: infinispan -Dev List
Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication
Mode
On 6 Aug 2013, at 15:19, Faseela K <faseela.k(a)ericsson.com> wrote:
> What is the recommended cluster size for Replication Mode?
> Given 3 nodes, My replication configuration performs better than my distributed
configuration.
> Just wanted to know, at what cluster size, distribution will perform better than
replication.
There's no straight answer, it depends on the read/write ratio and the amount of data
you store.
Replication will always perform better for reads as it won't involve a remote call to
get the data.
If you're mostly doing reads and your memory allows (replication is more memory
consuming) then you should use replication.
If the amount of data increases or you're doing more writes, distribution is the way
to go.
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
10:31 a.m.
Hi,
it's best to start by understanding how a read and a write operation
are different.
For a non-transactional synch read, if the entry happens to have a
copy on the local node the read is immediately satisfied with no need
for network round-trips.
That implies several things:
- read performance in the order of millions/second (assuming small entries)
- a hit-ratio of local cache of 100% for replication
- a hit-ratio of 66.6% for distribution (assuming 2 owners: each key
is stored on 2 nodes out of 3; this ratio improves by setting more
owners and gets worse when adding more nodes)
A non-transactional synch write however always requires at least one
network RPC (if not 2) to confirm safe storage on at least one
additional node (when not 2):
- you have 33% change to be the primary owner for each key: needs one
RPC to the secondary owner
- 33% to be the secondary owner: needs one RPC to the primary owner
- 33% (remaining cases): needs 2 RPCs: one to the primary and one to
the secondary
Each of these cases have slightly different expected performance
metrics, but whichever you pick, you should be in a range of about two
orders of magnitude higher latency than a read operation.
Now in your scenario, assuming your application does a balanced mix of
reads and writes, and having understood that write performance doesn't
change when switching from DIST to REPL, it's possible the read
operations are somewhat faster (33% in average), but their latency is
extremely lower than write operations latency so unless you do a lot
more reads than writes, it's possible that you don't perceive any
measurable difference.
The odds can change dramatically if you change some parameters
- if you have 100s of nodes then REPL will still have a local
hit-ratio of 100% while DIST tends to zero, making a read similar to a
write in term of performance.
- if you change the ratio of reads vs writes
- if you set number of owners to 3, and have 3 nodes, you essentially
get the same behaviour as REPL.
Also to keep in mind: DIST can store more data in memory and scales
writes linearly, while with REPL write operations require a network
RPC for each node in the grid.
So yes it's possible that in 3 nodes the difference isn't noticeable,
but picking the right cache mode depends on statistical information
about the behaviour of your application, or more simply if you have
latency requirements on reads which require REPL.
--Sanne
On 13 August 2013 05:29, Faseela K <faseela.k(a)ericsson.com> wrote:
Hi,
I am using infinispan 5.2.3.
My configuration is non-transactional, synchronous.
With this configuration, is my replication supposed to perform better than
distribution, for both reads and writes?
My Cluster Size requirement is 4 nodes.
And my application involves both reads and writes heavily.
For better performance, are there any suggestions on the clustering
modes/configurations?
All my tests show, replication having better performance than distribution for reads
as well as writes, with 4 nodes.
Thanks,
Faseela
-----Original Message-----
From: infinispan-dev-bounces(a)lists.jboss.org
[mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim Vansa
Sent: Monday, August 12, 2013 6:13 PM
To: infinispan-dev(a)lists.jboss.org
Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication Mode
Hi,
which version exactly do you use, 5.2.x, 5.3.x or 6.0.x? In 5.2 the replication mode was
implemented separately from distribution mode and depending on your configuration (is it
non-transactional synchronous?) the message control flow could differ. Since 5.3
replication mode is implemented in the same manner and the results should be more
comparable.
I may be wrong here, but in 5.2.x concurrent writes to single key in non-transactional
mode could result in entries being out of sync on some nodes (the writes could arrive at
two nodes in different order). I think this cannot happen in >= 5.3 anymore.
Radim
On 08/12/2013 09:04 AM, Faseela K wrote:
> Hi,
>
> With a 3 node cluster, even for "WRITES" my replication performance is
better than distribution.
> That's why I came across this doubt.
> Could some body please clarify, why the behaviour is like this?
>
> Thanks,
> Faseela
>
> -----Original Message-----
> From: infinispan-dev-bounces(a)lists.jboss.org
> [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Mircea
> Markus
> Sent: Friday, August 09, 2013 7:14 PM
> To: infinispan -Dev List
> Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication
> Mode
>
> On 6 Aug 2013, at 15:19, Faseela K <faseela.k(a)ericsson.com> wrote:
>
>> What is the recommended cluster size for Replication Mode?
>> Given 3 nodes, My replication configuration performs better than my
distributed configuration.
>> Just wanted to know, at what cluster size, distribution will perform better
than replication.
> There's no straight answer, it depends on the read/write ratio and the amount of
data you store.
> Replication will always perform better for reads as it won't involve a remote
call to get the data.
> If you're mostly doing reads and your memory allows (replication is more memory
consuming) then you should use replication.
> If the amount of data increases or you're doing more writes, distribution is the
way to go.
>
> Cheers,
> --
> Mircea Markus
> Infinispan lead (www.infinispan.org)
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
10:38 a.m.
Yes, with 5.2.3 it makes sense - in replication mode there were less
messages used for the write but key may have different value on
different nodes after concurrent modification. In order to evade this,
you can setup 4-node distributed cluster with numOwners=4.
All the performance optimizations depend on how consistent the storage
has to be. Do you write to the same keys in one moment from multiple
locations? Would you be OK with a bit out-of-date reads? Is it a problem
if you read different values for the key on different nodes? Is it a
problem to a write getting lost once a while?
With strong consistency requirements, you can't get top performance, but
maybe you don't really need it - or you need it just for some data in
your app which can be positioned into cache with different configuration.
There is a nice table [1] for asynchronous options guarantees.
And by the way - "my application involves both reads and writes heavily"
does not say anything - the factor is the ratio (percentage) of reads to
writes.
Radim
[1] https://docs.jboss.org/author/pages/viewpage.action?pageId=63636092
On 08/13/2013 06:29 AM, Faseela K wrote:
Hi,
I am using infinispan 5.2.3.
My configuration is non-transactional, synchronous.
With this configuration, is my replication supposed to perform better than
distribution, for both reads and writes?
My Cluster Size requirement is 4 nodes.
And my application involves both reads and writes heavily.
For better performance, are there any suggestions on the clustering
modes/configurations?
All my tests show, replication having better performance than distribution for reads
as well as writes, with 4 nodes.
Thanks,
Faseela
-----Original Message-----
From: infinispan-dev-bounces(a)lists.jboss.org
[mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim Vansa
Sent: Monday, August 12, 2013 6:13 PM
To: infinispan-dev(a)lists.jboss.org
Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication Mode
Hi,
which version exactly do you use, 5.2.x, 5.3.x or 6.0.x? In 5.2 the replication mode was
implemented separately from distribution mode and depending on your configuration (is it
non-transactional synchronous?) the message control flow could differ. Since 5.3
replication mode is implemented in the same manner and the results should be more
comparable.
I may be wrong here, but in 5.2.x concurrent writes to single key in non-transactional
mode could result in entries being out of sync on some nodes (the writes could arrive at
two nodes in different order). I think this cannot happen in >= 5.3 anymore.
Radim
On 08/12/2013 09:04 AM, Faseela K wrote:
> Hi,
>
> With a 3 node cluster, even for "WRITES" my replication performance is
better than distribution.
> That's why I came across this doubt.
> Could some body please clarify, why the behaviour is like this?
>
> Thanks,
> Faseela
>
> -----Original Message-----
> From: infinispan-dev-bounces(a)lists.jboss.org
> [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Mircea
> Markus
> Sent: Friday, August 09, 2013 7:14 PM
> To: infinispan -Dev List
> Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication
> Mode
>
> On 6 Aug 2013, at 15:19, Faseela K <faseela.k(a)ericsson.com> wrote:
>
>> What is the recommended cluster size for Replication Mode?
>> Given 3 nodes, My replication configuration performs better than my
distributed configuration.
>> Just wanted to know, at what cluster size, distribution will perform better
than replication.
> There's no straight answer, it depends on the read/write ratio and the amount of
data you store.
> Replication will always perform better for reads as it won't involve a remote
call to get the data.
> If you're mostly doing reads and your memory allows (replication is more memory
consuming) then you should use replication.
> If the amount of data increases or you're doing more writes, distribution is the
way to go.
>
> Cheers,
> --
> Mircea Markus
> Infinispan lead (www.infinispan.org)
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
11:58 a.m.
Hi Radim,
Thanks for the information.
A few more doubts :
- Is it possible to have different named caches with different clustering
configuration?
Eg : I need a cache's data to be replicated asynchronously, and one other
cache's data to be distributed synchronously.
- What will be the impact of putAsync() call on clustered performance?
- Is there any performace numbers already avaialble for various clustering
modes?
Are there any benchmarks already documented?
I will try my performance test once again with 5.3.0 version.
I will also analyse impacts of moving my clustering mode to asynchronous
replication/distribution.
Thanks,
Faseela
-----Original Message-----
From: infinispan-dev-bounces(a)lists.jboss.org
[mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim Vansa
Sent: Tuesday, August 13, 2013 2:08 PM
To: infinispan-dev(a)lists.jboss.org
Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication Mode
Yes, with 5.2.3 it makes sense - in replication mode there were less messages used for the
write but key may have different value on different nodes after concurrent modification.
In order to evade this, you can setup 4-node distributed cluster with numOwners=4.
All the performance optimizations depend on how consistent the storage has to be. Do you
write to the same keys in one moment from multiple locations? Would you be OK with a bit
out-of-date reads? Is it a problem if you read different values for the key on different
nodes? Is it a problem to a write getting lost once a while?
With strong consistency requirements, you can't get top performance, but maybe you
don't really need it - or you need it just for some data in your app which can be
positioned into cache with different configuration.
There is a nice table [1] for asynchronous options guarantees.
And by the way - "my application involves both reads and writes heavily"
does not say anything - the factor is the ratio (percentage) of reads to writes.
Radim
[1] https://docs.jboss.org/author/pages/viewpage.action?pageId=63636092
On 08/13/2013 06:29 AM, Faseela K wrote:
Hi,
I am using infinispan 5.2.3.
My configuration is non-transactional, synchronous.
With this configuration, is my replication supposed to perform better than
distribution, for both reads and writes?
My Cluster Size requirement is 4 nodes.
And my application involves both reads and writes heavily.
For better performance, are there any suggestions on the clustering
modes/configurations?
All my tests show, replication having better performance than distribution for reads
as well as writes, with 4 nodes.
Thanks,
Faseela
-----Original Message-----
From: infinispan-dev-bounces(a)lists.jboss.org
[mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim
Vansa
Sent: Monday, August 12, 2013 6:13 PM
To: infinispan-dev(a)lists.jboss.org
Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication
Mode
Hi,
which version exactly do you use, 5.2.x, 5.3.x or 6.0.x? In 5.2 the replication mode was
implemented separately from distribution mode and depending on your configuration (is it
non-transactional synchronous?) the message control flow could differ. Since 5.3
replication mode is implemented in the same manner and the results should be more
comparable.
I may be wrong here, but in 5.2.x concurrent writes to single key in non-transactional
mode could result in entries being out of sync on some nodes (the writes could arrive at
two nodes in different order). I think this cannot happen in >= 5.3 anymore.
Radim
On 08/12/2013 09:04 AM, Faseela K wrote:
> Hi,
>
> With a 3 node cluster, even for "WRITES" my replication performance is
better than distribution.
> That's why I came across this doubt.
> Could some body please clarify, why the behaviour is like this?
>
> Thanks,
> Faseela
>
> -----Original Message-----
> From: infinispan-dev-bounces(a)lists.jboss.org
> [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Mircea
> Markus
> Sent: Friday, August 09, 2013 7:14 PM
> To: infinispan -Dev List
> Subject: Re: [infinispan-dev] Recommended Cluster Size for
> Replication Mode
>
> On 6 Aug 2013, at 15:19, Faseela K <faseela.k(a)ericsson.com> wrote:
>
>> What is the recommended cluster size for Replication Mode?
>> Given 3 nodes, My replication configuration performs better than my
distributed configuration.
>> Just wanted to know, at what cluster size, distribution will perform better
than replication.
> There's no straight answer, it depends on the read/write ratio and the amount of
data you store.
> Replication will always perform better for reads as it won't involve a remote
call to get the data.
> If you're mostly doing reads and your memory allows (replication is more memory
consuming) then you should use replication.
> If the amount of data increases or you're doing more writes, distribution is the
way to go.
>
> Cheers,
> --
> Mircea Markus
> Infinispan lead (www.infinispan.org)
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
2:03 p.m.
On 08/13/2013 11:58 AM, Faseela K wrote:
Hi Radim,
Thanks for the information.
A few more doubts :
- Is it possible to have different named caches with different clustering
configuration?
Eg : I need a cache's data to be replicated asynchronously, and one
other cache's data to be distributed synchronously.
Yes, this is expected
configuration.
- What will be the impact of putAsync() call on clustered performance?
The putAsync differs from regular put call just by executing the command
in different thread (from threadpool) and returning future instead of
the previous value. Besides this the processing of the command inside
Infinispan is identical.
Btw., if you drive for highest performance, you could consider using the
IGNORE_RETURN_VALUE flag.
- Is there any performace numbers already avaialble for
various clustering modes?
Are there any benchmarks already documented?
Such results are not
disclosed. For benchmarking, you may consider using
the RadarGun tool [1] (use snapshot version, 1.0 is prehistorical release).
Radim
[1] https://github.com/radargun/radargun
I will try my performance test once again with 5.3.0 version.
I will also analyse impacts of moving my clustering mode to asynchronous
replication/distribution.
Thanks,
Faseela
-----Original Message-----
From: infinispan-dev-bounces(a)lists.jboss.org
[mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim Vansa
Sent: Tuesday, August 13, 2013 2:08 PM
To: infinispan-dev(a)lists.jboss.org
Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication Mode
Yes, with 5.2.3 it makes sense - in replication mode there were less messages used for
the write but key may have different value on different nodes after concurrent
modification. In order to evade this, you can setup 4-node distributed cluster with
numOwners=4.
All the performance optimizations depend on how consistent the storage has to be. Do you
write to the same keys in one moment from multiple locations? Would you be OK with a bit
out-of-date reads? Is it a problem if you read different values for the key on different
nodes? Is it a problem to a write getting lost once a while?
With strong consistency requirements, you can't get top performance, but maybe you
don't really need it - or you need it just for some data in your app which can be
positioned into cache with different configuration.
There is a nice table [1] for asynchronous options guarantees.
And by the way - "my application involves both reads and writes heavily"
does not say anything - the factor is the ratio (percentage) of reads to writes.
Radim
[1] https://docs.jboss.org/author/pages/viewpage.action?pageId=63636092
On 08/13/2013 06:29 AM, Faseela K wrote:
> Hi,
>
> I am using infinispan 5.2.3.
> My configuration is non-transactional, synchronous.
> With this configuration, is my replication supposed to perform better than
distribution, for both reads and writes?
> My Cluster Size requirement is 4 nodes.
> And my application involves both reads and writes heavily.
> For better performance, are there any suggestions on the clustering
modes/configurations?
> All my tests show, replication having better performance than distribution for
reads as well as writes, with 4 nodes.
>
> Thanks,
> Faseela
>
> -----Original Message-----
> From: infinispan-dev-bounces(a)lists.jboss.org
> [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim
> Vansa
> Sent: Monday, August 12, 2013 6:13 PM
> To: infinispan-dev(a)lists.jboss.org
> Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication
> Mode
>
> Hi,
>
> which version exactly do you use, 5.2.x, 5.3.x or 6.0.x? In 5.2 the replication mode
was implemented separately from distribution mode and depending on your configuration (is
it non-transactional synchronous?) the message control flow could differ. Since 5.3
replication mode is implemented in the same manner and the results should be more
comparable.
> I may be wrong here, but in 5.2.x concurrent writes to single key in
non-transactional mode could result in entries being out of sync on some nodes (the writes
could arrive at two nodes in different order). I think this cannot happen in >= 5.3
anymore.
>
> Radim
>
> On 08/12/2013 09:04 AM, Faseela K wrote:
>> Hi,
>>
>> With a 3 node cluster, even for "WRITES" my replication
performance is better than distribution.
>> That's why I came across this doubt.
>> Could some body please clarify, why the behaviour is like this?
>>
>> Thanks,
>> Faseela
>>
>> -----Original Message-----
>> From: infinispan-dev-bounces(a)lists.jboss.org
>> [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Mircea
>> Markus
>> Sent: Friday, August 09, 2013 7:14 PM
>> To: infinispan -Dev List
>> Subject: Re: [infinispan-dev] Recommended Cluster Size for
>> Replication Mode
>>
>> On 6 Aug 2013, at 15:19, Faseela K <faseela.k(a)ericsson.com> wrote:
>>
>>> What is the recommended cluster size for Replication Mode?
>>> Given 3 nodes, My replication configuration performs better than my
distributed configuration.
>>> Just wanted to know, at what cluster size, distribution will perform
better than replication.
>> There's no straight answer, it depends on the read/write ratio and the amount
of data you store.
>> Replication will always perform better for reads as it won't involve a remote
call to get the data.
>> If you're mostly doing reads and your memory allows (replication is more
memory consuming) then you should use replication.
>> If the amount of data increases or you're doing more writes, distribution is
the way to go.
>>
>> Cheers,
>> --
>> Mircea Markus
>> Infinispan lead (www.infinispan.org)
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
12:27 p.m.
Hi Radim,
I tried my performance test again with Infinispan - 5.3.0.
Now, "replication writes" are taking almost double the time as in 5.2.3.
Could you please clarify the issues with replication mode in 5.2.3?
I guess, to overcome those issues, some more messages were added to replication mode in
5.3.0, because of which writes are
taking more time in 5.3.0.
Please note that, with a 3 node cluster, now distribution and replication are taking
almost the same time for "writes".
Thanks,
Faseela
-----Original Message-----
From: infinispan-dev-bounces(a)lists.jboss.org
[mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim Vansa
Sent: Tuesday, August 13, 2013 5:34 PM
To: infinispan-dev(a)lists.jboss.org
Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication Mode
On 08/13/2013 11:58 AM, Faseela K wrote:
Hi Radim,
Thanks for the information.
A few more doubts :
- Is it possible to have different named caches with different clustering
configuration?
Eg : I need a cache's data to be replicated asynchronously, and one
other cache's data to be distributed synchronously.
Yes, this is expected
configuration.
- What will be the impact of putAsync() call on clustered performance?
The putAsync differs from regular put call just by executing the command in
different thread (from threadpool) and returning future instead of the previous value.
Besides this the processing of the command inside Infinispan is identical.
Btw., if you drive for highest performance, you could consider using the
IGNORE_RETURN_VALUE flag.
- Is there any performace numbers already avaialble for
various clustering modes?
Are there any benchmarks already documented?
Such results are not
disclosed. For benchmarking, you may consider using the RadarGun tool [1] (use snapshot
version, 1.0 is prehistorical release).
Radim
[1] https://github.com/radargun/radargun
I will try my performance test once again with 5.3.0 version.
I will also analyse impacts of moving my clustering mode to asynchronous
replication/distribution.
Thanks,
Faseela
-----Original Message-----
From: infinispan-dev-bounces(a)lists.jboss.org
[mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim
Vansa
Sent: Tuesday, August 13, 2013 2:08 PM
To: infinispan-dev(a)lists.jboss.org
Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication
Mode
Yes, with 5.2.3 it makes sense - in replication mode there were less messages used for
the write but key may have different value on different nodes after concurrent
modification. In order to evade this, you can setup 4-node distributed cluster with
numOwners=4.
All the performance optimizations depend on how consistent the storage has to be. Do you
write to the same keys in one moment from multiple locations? Would you be OK with a bit
out-of-date reads? Is it a problem if you read different values for the key on different
nodes? Is it a problem to a write getting lost once a while?
With strong consistency requirements, you can't get top performance, but maybe you
don't really need it - or you need it just for some data in your app which can be
positioned into cache with different configuration.
There is a nice table [1] for asynchronous options guarantees.
And by the way - "my application involves both reads and writes heavily"
does not say anything - the factor is the ratio (percentage) of reads to writes.
Radim
[1]
https://docs.jboss.org/author/pages/viewpage.action?pageId=63636092
On 08/13/2013 06:29 AM, Faseela K wrote:
> Hi,
>
> I am using infinispan 5.2.3.
> My configuration is non-transactional, synchronous.
> With this configuration, is my replication supposed to perform better than
distribution, for both reads and writes?
> My Cluster Size requirement is 4 nodes.
> And my application involves both reads and writes heavily.
> For better performance, are there any suggestions on the clustering
modes/configurations?
> All my tests show, replication having better performance than distribution for
reads as well as writes, with 4 nodes.
>
> Thanks,
> Faseela
>
> -----Original Message-----
> From: infinispan-dev-bounces(a)lists.jboss.org
> [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim
> Vansa
> Sent: Monday, August 12, 2013 6:13 PM
> To: infinispan-dev(a)lists.jboss.org
> Subject: Re: [infinispan-dev] Recommended Cluster Size for
> Replication Mode
>
> Hi,
>
> which version exactly do you use, 5.2.x, 5.3.x or 6.0.x? In 5.2 the replication mode
was implemented separately from distribution mode and depending on your configuration (is
it non-transactional synchronous?) the message control flow could differ. Since 5.3
replication mode is implemented in the same manner and the results should be more
comparable.
> I may be wrong here, but in 5.2.x concurrent writes to single key in
non-transactional mode could result in entries being out of sync on some nodes (the writes
could arrive at two nodes in different order). I think this cannot happen in >= 5.3
anymore.
>
> Radim
>
> On 08/12/2013 09:04 AM, Faseela K wrote:
>> Hi,
>>
>> With a 3 node cluster, even for "WRITES" my replication
performance is better than distribution.
>> That's why I came across this doubt.
>> Could some body please clarify, why the behaviour is like this?
>>
>> Thanks,
>> Faseela
>>
>> -----Original Message-----
>> From: infinispan-dev-bounces(a)lists.jboss.org
>> [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Mircea
>> Markus
>> Sent: Friday, August 09, 2013 7:14 PM
>> To: infinispan -Dev List
>> Subject: Re: [infinispan-dev] Recommended Cluster Size for
>> Replication Mode
>>
>> On 6 Aug 2013, at 15:19, Faseela K <faseela.k(a)ericsson.com> wrote:
>>
>>> What is the recommended cluster size for Replication Mode?
>>> Given 3 nodes, My replication configuration performs better than my
distributed configuration.
>>> Just wanted to know, at what cluster size, distribution will perform
better than replication.
>> There's no straight answer, it depends on the read/write ratio and the amount
of data you store.
>> Replication will always perform better for reads as it won't involve a remote
call to get the data.
>> If you're mostly doing reads and your memory allows (replication is more
memory consuming) then you should use replication.
>> If the amount of data increases or you're doing more writes, distribution is
the way to go.
>>
>> Cheers,
>> --
>> Mircea Markus
>> Infinispan lead (www.infinispan.org)
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
12:56 p.m.
On 08/16/2013 12:27 PM, Faseela K wrote:
Hi Radim,
I tried my performance test again with Infinispan - 5.3.0.
Now, "replication writes" are taking almost double the time as in 5.2.3.
You may try transactional one-phase commits for the 5.2.x behaviour
(what accounts to messages), although there may be other overhead
slowing that down.
Could you please clarify the issues with replication mode in
5.2.3?
Let's consider two concurrent writes for key K, values are V1 and V2.
There are 3 nodes A, B, C, the write is executed from A, B is the
primary owner of this key.
5.2.x:
A locally writes V1 and sends broadcast message to B and C to update K
to V1, waiting for acks
A locally writes V2 and sends broadcast message to B and C to update K
to V2, waiting for acks
B receives and applies write V1, sends ack to A
B receives and applies write V2, sends ack to A
C receives and applies write V2, sends ack to A
C receives and applies write V1, sends ack to A
A receives acks for V1, returns
A receives acks for V2, returns
result is A[K] = V2, B[K] = V2, C[K] = V1 => inconsistency
5.3.x:
A sends to B as the primary owner with write V1
A sends to B as the primary owner with write V2
B locks key K, locally writes V1 and sends message to A and C, waiting
for acks
B tries to lock key K in order to write V2, but waits for lock
A and C apply write V1 and send acks
B unlocks K and sends ack to A
A receives acks for V1, returns
B locks key K, locally writes V2 and sends message to A and C, waiting
for acks
A and C apply write V2 and send acks
B unlocks K and sends ack to A
A receives acks for V2, returns
result is A[K] = V2, B[K] = V2, C[K] = V2 => cluster is consistent
I guess, to overcome those issues, some more messages were added
to replication mode in 5.3.0, because of which writes are
taking more time in 5.3.0.
Please note that, with a 3 node cluster, now distribution and replication are taking
almost the same time for "writes".
That's because these modes do not
differ anymore in implementation.
Radim
-----Original Message-----
From: infinispan-dev-bounces(a)lists.jboss.org
[mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim Vansa
Sent: Tuesday, August 13, 2013 5:34 PM
To: infinispan-dev(a)lists.jboss.org
Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication Mode
On 08/13/2013 11:58 AM, Faseela K wrote:
> Hi Radim,
>
> Thanks for the information.
> A few more doubts :
>
> - Is it possible to have different named caches with different clustering
configuration?
> Eg : I need a cache's data to be replicated asynchronously, and one
other cache's data to be distributed synchronously.
Yes, this is expected configuration.
> - What will be the impact of putAsync() call on clustered performance?
The putAsync differs from regular put call just by executing the command in different
thread (from threadpool) and returning future instead of the previous value. Besides this
the processing of the command inside Infinispan is identical.
Btw., if you drive for highest performance, you could consider using the
IGNORE_RETURN_VALUE flag.
> - Is there any performace numbers already avaialble for various
clustering modes?
> Are there any benchmarks already documented?
Such results are not disclosed. For benchmarking, you may consider using the RadarGun
tool [1] (use snapshot version, 1.0 is prehistorical release).
Radim
[1] https://github.com/radargun/radargun
> I will try my performance test once again with 5.3.0 version.
> I will also analyse impacts of moving my clustering mode to asynchronous
replication/distribution.
>
> Thanks,
> Faseela
>
> -----Original Message-----
> From: infinispan-dev-bounces(a)lists.jboss.org
> [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim
> Vansa
> Sent: Tuesday, August 13, 2013 2:08 PM
> To: infinispan-dev(a)lists.jboss.org
> Subject: Re: [infinispan-dev] Recommended Cluster Size for Replication
> Mode
>
> Yes, with 5.2.3 it makes sense - in replication mode there were less messages used
for the write but key may have different value on different nodes after concurrent
modification. In order to evade this, you can setup 4-node distributed cluster with
numOwners=4.
>
> All the performance optimizations depend on how consistent the storage has to be. Do
you write to the same keys in one moment from multiple locations? Would you be OK with a
bit out-of-date reads? Is it a problem if you read different values for the key on
different nodes? Is it a problem to a write getting lost once a while?
> With strong consistency requirements, you can't get top performance, but maybe
you don't really need it - or you need it just for some data in your app which can be
positioned into cache with different configuration.
> There is a nice table [1] for asynchronous options guarantees.
>
> And by the way - "my application involves both reads and writes heavily"
> does not say anything - the factor is the ratio (percentage) of reads to writes.
>
> Radim
>
> [1]
> https://docs.jboss.org/author/pages/viewpage.action?pageId=63636092
>
> On 08/13/2013 06:29 AM, Faseela K wrote:
>> Hi,
>>
>> I am using infinispan 5.2.3.
>> My configuration is non-transactional, synchronous.
>> With this configuration, is my replication supposed to perform better than
distribution, for both reads and writes?
>> My Cluster Size requirement is 4 nodes.
>> And my application involves both reads and writes heavily.
>> For better performance, are there any suggestions on the clustering
modes/configurations?
>> All my tests show, replication having better performance than distribution
for reads as well as writes, with 4 nodes.
>>
>> Thanks,
>> Faseela
>>
>> -----Original Message-----
>> From: infinispan-dev-bounces(a)lists.jboss.org
>> [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Radim
>> Vansa
>> Sent: Monday, August 12, 2013 6:13 PM
>> To: infinispan-dev(a)lists.jboss.org
>> Subject: Re: [infinispan-dev] Recommended Cluster Size for
>> Replication Mode
>>
>> Hi,
>>
>> which version exactly do you use, 5.2.x, 5.3.x or 6.0.x? In 5.2 the replication
mode was implemented separately from distribution mode and depending on your configuration
(is it non-transactional synchronous?) the message control flow could differ. Since 5.3
replication mode is implemented in the same manner and the results should be more
comparable.
>> I may be wrong here, but in 5.2.x concurrent writes to single key in
non-transactional mode could result in entries being out of sync on some nodes (the writes
could arrive at two nodes in different order). I think this cannot happen in >= 5.3
anymore.
>>
>> Radim
>>
>> On 08/12/2013 09:04 AM, Faseela K wrote:
>>> Hi,
>>>
>>> With a 3 node cluster, even for "WRITES" my replication
performance is better than distribution.
>>> That's why I came across this doubt.
>>> Could some body please clarify, why the behaviour is like this?
>>>
>>> Thanks,
>>> Faseela
>>>
>>> -----Original Message-----
>>> From: infinispan-dev-bounces(a)lists.jboss.org
>>> [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf Of Mircea
>>> Markus
>>> Sent: Friday, August 09, 2013 7:14 PM
>>> To: infinispan -Dev List
>>> Subject: Re: [infinispan-dev] Recommended Cluster Size for
>>> Replication Mode
>>>
>>> On 6 Aug 2013, at 15:19, Faseela K <faseela.k(a)ericsson.com> wrote:
>>>
>>>> What is the recommended cluster size for Replication Mode?
>>>> Given 3 nodes, My replication configuration performs better than
my distributed configuration.
>>>> Just wanted to know, at what cluster size, distribution will
perform better than replication.
>>> There's no straight answer, it depends on the read/write ratio and the
amount of data you store.
>>> Replication will always perform better for reads as it won't involve a
remote call to get the data.
>>> If you're mostly doing reads and your memory allows (replication is more
memory consuming) then you should use replication.
>>> If the amount of data increases or you're doing more writes, distribution
is the way to go.
>>>
>>> Cheers,
>>> --
>>> Mircea Markus
>>> Infinispan lead (www.infinispan.org)
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
3927
days inactive
3937
days old
infinispan-dev@lists.jboss.org
10 comments
4 participants
participants (4)
-
Faseela K -
Mircea Markus -
Radim Vansa -
Sanne Grinovero