3:14 a.m.
1: Observation:
-------------
In my Infinispan perf test (IspnPerfTest), I used
cache.getAdvancedCache().withFlags(...).put(key,value) in a tight loop.
I've always thought that withFlags() was a fast operation, *but this is
not the case* !!
Once I changed this and predefined the 2 caches (sync and async) at the
start, outside the loop, things got 10x faster ! So please change this
if you made the same mistake !
2. Question:
-----------
In Infinispan 6, I defined my custom transport as follows:
<transport ... transportClass="org.perf.CustomTransport"/>
This is gone in 7. Do I now have to use programmatic configuration ? If
so, how would I do this ?
--
Bela Ban, JGroups lead (http://www.jgroups.org)
3:58 a.m.
On 7 July 2014 09:14, Bela Ban <bban(a)redhat.com> wrote:
1: Observation:
-------------
In my Infinispan perf test (IspnPerfTest), I used
cache.getAdvancedCache().withFlags(...).put(key,value) in a tight loop.
I've always thought that withFlags() was a fast operation, *but this is
not the case* !!
Once I changed this and predefined the 2 caches (sync and async) at the
start, outside the loop, things got 10x faster ! So please change this
if you made the same mistake !
Right that's the better way to use the flags; I'm pretty sure we
documented this at some point but I couldn't find it in the docs nor
javadocs now...?!! bad bad.
Where we do use flags internally (Lucene Directory), we hold on to
multiple instances of the Cache, even if the same cache content but to
use different flags. The code is quite horrible to read as it seems
like you interact with different Caches, but as you noticed it's worth
it.
10x faster? That's surprising for a benchmark which is supposed to be
network bound isn't it? If you can measure a 10X improvement, it seems
like your tests where bound by memory allocation (as that's the
resource you starve by using _withFlags_ extensively) ?
Might be worth checking with flight recorder if that's still the case,
as _withFlags_ isn't sufficient on its own to saturate your memory
bandwith, so I'd guess there are other hot consumers which might be
easy to take down.
2. Question:
-----------
In Infinispan 6, I defined my custom transport as follows:
<transport ... transportClass="org.perf.CustomTransport"/>
This is gone in 7. Do I now have to use programmatic configuration ? If
so, how would I do this ?
I don't know this one, hopefully others will.. ?
Cheers,
Sanne
--
Bela Ban, JGroups lead (http://www.jgroups.org)
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
4:37 a.m.
On 07/07/14 10:58, Sanne Grinovero wrote:
On 7 July 2014 09:14, Bela Ban <bban(a)redhat.com> wrote:
>
> 1: Observation:
> -------------
> In my Infinispan perf test (IspnPerfTest), I used
> cache.getAdvancedCache().withFlags(...).put(key,value) in a tight loop.
>
> I've always thought that withFlags() was a fast operation, *but this is
> not the case* !!
>
> Once I changed this and predefined the 2 caches (sync and async) at the
> start, outside the loop, things got 10x faster ! So please change this
> if you made the same mistake !
Right that's the better way to use the flags; I'm pretty sure we
documented this at some point but I couldn't find it in the docs nor
javadocs now...?!! bad bad.
Looking at the code, I see that withFlags() creates a new
DecoratedCache, so this can obviously be mitigated by instantiating the
caches beforehand. Haven't had time to investigate the cost of this.
Where we do use flags internally (Lucene Directory), we hold on to
multiple instances of the Cache, even if the same cache content but to
use different flags. The code is quite horrible to read as it seems
like you interact with different Caches, but as you noticed it's worth
it.
Yes
10x faster? That's surprising for a benchmark which is supposed
to be
network bound isn't it?
I ran 2 IspnPerfTest processes on my local box with numOwners=2 (no L1
cache). I guess that roughly half of the calls go to the local node, and
there the cost of withFlags() is not amortized by the network round trip.
If you can measure a 10X improvement, it seems
like your tests where bound by memory allocation (as that's the
resource you starve by using _withFlags_ extensively) ?
Might be worth checking with flight recorder if that's still the case,
as _withFlags_ isn't sufficient on its own to saturate your memory
bandwith, so I'd guess there are other hot consumers which might be
easy to take down.
>
> 2. Question:
> -----------
> In Infinispan 6, I defined my custom transport as follows:
> <transport ... transportClass="org.perf.CustomTransport"/>
>
> This is gone in 7. Do I now have to use programmatic configuration ? If
> so, how would I do this ?
I don't know this one, hopefully others will.. ?
Cheers,
Sanne
--
Bela Ban, JGroups lead (http://www.jgroups.org)
6:36 a.m.
On Jul 7, 2014, at 10:37, Bela Ban <bban(a)redhat.com> wrote:
> 10x faster? That's surprising for a benchmark which is
supposed to be
> network bound isn't it?
I ran 2 IspnPerfTest processes on my local box with numOwners=2 (no L1
cache). I guess that roughly half of the calls go to the local node, and
there the cost of withFlags() is not amortized by the network round trip.
All the reads are local as both nodes own all the data in this setup. Writes will involve
RPCs, though.
Even for local caches, the 10x performance degradation from using the flags is way too
much, there must be something else at stake. If you profile it we can take a look.
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
6:47 a.m.
Makes sense; I'll try to get a jmc dump
On 07/07/14 13:36, Mircea Markus wrote:
On Jul 7, 2014, at 10:37, Bela Ban <bban(a)redhat.com> wrote:
>> 10x faster? That's surprising for a benchmark which is supposed to be
>> network bound isn't it?
>
> I ran 2 IspnPerfTest processes on my local box with numOwners=2 (no L1
> cache). I guess that roughly half of the calls go to the local node, and
> there the cost of withFlags() is not amortized by the network round trip.
All the reads are local as both nodes own all the data in this setup. Writes will involve
RPCs, though.
Even for local caches, the 10x performance degradation from using the flags is way too
much, there must be something else at stake. If you profile it we can take a look.
Cheers,
--
Bela Ban, JGroups lead (http://www.jgroups.org)
5:04 a.m.
How ? I already have an infinispan.xml, create a CacheManager off of it
and now only want to change the transport.
I need to get another PhD to understand programmatic configuration in
Infinispan
On 07/07/14 11:51, Pedro Ruivo wrote:
On 07/07/2014 09:14 AM, Bela Ban wrote:
> This is gone in 7. Do I now have to use programmatic configuration ? If
> so, how would I do this ?
AFAIK, yes it was removed from configuration file and can only be set by
programmatic configuration.
Pedro
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Bela Ban, JGroups lead (http://www.jgroups.org)
5:52 a.m.
On 7 July 2014 11:04, Bela Ban <bban(a)redhat.com> wrote:
How ? I already have an infinispan.xml, create a CacheManager off of
it
and now only want to change the transport.
I have the same need; in Palma I asked for a CacheManager constructor
which would take
(String infinispanConfiguration, Transport customTransportInstance).
Could we have that please please?
I never opened a new specific JIRA as there is the more generally
useful ISPN-1414 already.
I need to get another PhD to understand programmatic configuration in
Infinispan
On 07/07/14 11:51, Pedro Ruivo wrote:
>
>
> On 07/07/2014 09:14 AM, Bela Ban wrote:
>> This is gone in 7. Do I now have to use programmatic configuration ? If
>> so, how would I do this ?
>
> AFAIK, yes it was removed from configuration file and can only be set by
> programmatic configuration.
>
> Pedro
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
--
Bela Ban, JGroups lead (http://www.jgroups.org)
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
6:30 a.m.
On Jul 7, 2014, at 11:52, Sanne Grinovero <sanne(a)infinispan.org> wrote:
On 7 July 2014 11:04, Bela Ban <bban(a)redhat.com> wrote:
> How ? I already have an infinispan.xml, create a CacheManager off of it
> and now only want to change the transport.
I have the same need; in Palma I asked for a CacheManager constructor
which would take
(String infinispanConfiguration, Transport customTransportInstance).
GlobalConfigurationBuilder gc = new GlobalConfigurationBuilder();
gc.transport().transport(new JGroupsTransport());
DefaultCacheManager dcm = new DefaultCacheManager(gc.build());
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
6:41 a.m.
But where do I pass in my infinispan.xml config file ? I don't want a
pure programtic configuration
On 07/07/14 13:30, Mircea Markus wrote:
On Jul 7, 2014, at 11:52, Sanne Grinovero <sanne(a)infinispan.org> wrote:
> On 7 July 2014 11:04, Bela Ban <bban(a)redhat.com> wrote:
>> How ? I already have an infinispan.xml, create a CacheManager off of it
>> and now only want to change the transport.
>
> I have the same need; in Palma I asked for a CacheManager constructor
> which would take
> (String infinispanConfiguration, Transport customTransportInstance).
GlobalConfigurationBuilder gc = new GlobalConfigurationBuilder();
gc.transport().transport(new JGroupsTransport());
DefaultCacheManager dcm = new DefaultCacheManager(gc.build());
Cheers,
--
Bela Ban, JGroups lead (http://www.jgroups.org)
7:31 a.m.
The use case is this:
#1 I want to configure all caches from a config XML file
#2 The I want to *override* one or two small aspects, such as the transport
Ideally, I could define #2 in the config file as well, but that changed
from 6 to 7.
Note that I don't want to switch to programmatic configuration entirely
just because of #2
On 07/07/14 13:30, Mircea Markus wrote:
On Jul 7, 2014, at 11:52, Sanne Grinovero <sanne(a)infinispan.org> wrote:
> On 7 July 2014 11:04, Bela Ban <bban(a)redhat.com> wrote:
>> How ? I already have an infinispan.xml, create a CacheManager off of it
>> and now only want to change the transport.
>
> I have the same need; in Palma I asked for a CacheManager constructor
> which would take
> (String infinispanConfiguration, Transport customTransportInstance).
GlobalConfigurationBuilder gc = new GlobalConfigurationBuilder();
gc.transport().transport(new JGroupsTransport());
DefaultCacheManager dcm = new DefaultCacheManager(gc.build());
Cheers,
--
Bela Ban, JGroups lead (http://www.jgroups.org)
4:06 a.m.
On 07 Jul 2014, at 12:52, Sanne Grinovero <sanne(a)infinispan.org> wrote:
On 7 July 2014 11:04, Bela Ban <bban(a)redhat.com> wrote:
> How ? I already have an infinispan.xml, create a CacheManager off of it
> and now only want to change the transport.
I have the same need; in Palma I asked for a CacheManager constructor
which would take
(String infinispanConfiguration, Transport customTransportInstance).
Could we have that please please?
I never opened a new specific JIRA as there is the more generally
useful ISPN-1414 already.
No idea where we’re at with ISPN-1414, AFAIK it has not been looked at for quite some
time.
I don’t remember the discussion in Palma on this (anyone has minutes of this discussion?),
but I suppose the current set up is not enough for your case...
Was your request related to app server integration?
Cheers,
>
> I need to get another PhD to understand programmatic configuration in
> Infinispan
>
> On 07/07/14 11:51, Pedro Ruivo wrote:
>>
>>
>> On 07/07/2014 09:14 AM, Bela Ban wrote:
>>> This is gone in 7. Do I now have to use programmatic configuration ? If
>>> so, how would I do this ?
>>
>> AFAIK, yes it was removed from configuration file and can only be set by
>> programmatic configuration.
>>
>> Pedro
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
4:51 a.m.
No more constructors.
Tristan
On 07/07/14 12:52, Sanne Grinovero wrote:
On 7 July 2014 11:04, Bela Ban <bban(a)redhat.com> wrote:
> How ? I already have an infinispan.xml, create a CacheManager off of it
> and now only want to change the transport.
I have the same need; in Palma I asked for a CacheManager constructor
which would take
(String infinispanConfiguration, Transport customTransportInstance).
Could we have that please please?
I never opened a new specific JIRA as there is the more generally
useful ISPN-1414 already.
> I need to get another PhD to understand programmatic configuration in
> Infinispan
>
> On 07/07/14 11:51, Pedro Ruivo wrote:
>>
>> On 07/07/2014 09:14 AM, Bela Ban wrote:
>>> This is gone in 7. Do I now have to use programmatic configuration ? If
>>> so, how would I do this ?
>> AFAIK, yes it was removed from configuration file and can only be set by
>> programmatic configuration.
>>
>> Pedro
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
6 a.m.
On 07/07/2014 11:04 AM, Bela Ban wrote:
How ? I already have an infinispan.xml, create a CacheManager off of
it
and now only want to change the transport.
I don't know if there is an easy way, but this may work:
ParserRegistry parser = new ParserRegistry();
ConfigurationBuilderHolder holder = parser.parse(/*filename or input
stream*/);
holder.getGlobalConfiguration().transport().transport(new
CustomTransport());
DefaultCacheManager manager = new DefaultCacheManager(holder, true);
I need to get another PhD to understand programmatic configuration in
Infinispan
lol
On 07/07/14 11:51, Pedro Ruivo wrote:
>
>
> On 07/07/2014 09:14 AM, Bela Ban wrote:
>> This is gone in 7. Do I now have to use programmatic configuration ? If
>> so, how would I do this ?
>
> AFAIK, yes it was removed from configuration file and can only be set by
> programmatic configuration.
>
> Pedro
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
6:42 a.m.
On Jul 7, 2014, at 9:14, Bela Ban <bban(a)redhat.com> wrote:
2. Question:
-----------
In Infinispan 6, I defined my custom transport as follows:
<transport ... transportClass="org.perf.CustomTransport"/>
This is gone in 7. Do I now have to use programmatic configuration ? If
so, how would I do this ?
This has disappeared as part of the configuration revamp undertaken in 7.0, not sure if
this was in purpose, Galder?
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
4 a.m.
On 07 Jul 2014, at 10:14, Bela Ban <bban(a)redhat.com> wrote:
1: Observation:
-------------
In my Infinispan perf test (IspnPerfTest), I used
cache.getAdvancedCache().withFlags(...).put(key,value) in a tight loop.
I've always thought that withFlags() was a fast operation, *but this is
not the case* !!
Once I changed this and predefined the 2 caches (sync and async) at the
start, outside the loop, things got 10x faster ! So please change this
if you made the same mistake !
2. Question:
-----------
In Infinispan 6, I defined my custom transport as follows:
<transport ... transportClass="org.perf.CustomTransport"/>
This is gone in 7. Do I now have to use programmatic configuration ? If
so, how would I do this ?
^ An oversight, it’ll be fixed before Final: https://issues.jboss.org/browse/ISPN-4510
--
Bela Ban, JGroups lead (http://www.jgroups.org)
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
2:25 a.m.
Thanks Galder, appreciated !
On 14/07/14 11:00, Galder Zamarreño wrote:
On 07 Jul 2014, at 10:14, Bela Ban <bban(a)redhat.com> wrote:
>
> 1: Observation:
> -------------
> In my Infinispan perf test (IspnPerfTest), I used
> cache.getAdvancedCache().withFlags(...).put(key,value) in a tight loop.
>
> I've always thought that withFlags() was a fast operation, *but this is
> not the case* !!
>
> Once I changed this and predefined the 2 caches (sync and async) at the
> start, outside the loop, things got 10x faster ! So please change this
> if you made the same mistake !
>
> 2. Question:
> -----------
> In Infinispan 6, I defined my custom transport as follows:
> <transport ... transportClass="org.perf.CustomTransport"/>
>
> This is gone in 7. Do I now have to use programmatic configuration ? If
> so, how would I do this ?
^ An oversight, it’ll be fixed before Final: https://issues.jboss.org/browse/ISPN-4510
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Bela Ban, JGroups lead (http://www.jgroups.org)
4277
days inactive
4291
days old
infinispan-dev@lists.jboss.org
16 comments
6 participants
participants (6)
-
Bela Ban -
Galder Zamarreño -
Mircea Markus -
Pedro Ruivo -
Sanne Grinovero -
Tristan Tarrant