[infinispan-dev] Issue with cache blocks for local read-only cache

Sanne Grinovero sanne at infinispan.org
Fri Sep 21 09:43:53 EDT 2012


On 20 September 2012 17:38, Andrig Miller <anmiller at redhat.com> wrote:
>
>
> ----- Original Message -----
>> From: "Galder Zamarreño" <galder at redhat.com>
>> To: "Andrig Miller" <anmiller at redhat.com>
>> Cc: "Steve Ebersole" <steve at hibernate.org>, "John O'Hara" <johara at redhat.com>, "Jeremy Whiting"
>> <jwhiting at redhat.com>, "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>> Sent: Thursday, September 20, 2012 6:48:59 AM
>> Subject: Re: [infinispan-dev] Issue with cache blocks for local read-only cache
>>
>>
>> On Sep 19, 2012, at 4:20 PM, Andrig Miller <anmiller at redhat.com>
>> wrote:
>>
>> > Yes, I can see how that can happen, if the data is deleted from
>> > outside the application.
>>
>> ^ The issue does not only happen if the data is deleted outside the
>> application. As indicated in
>> https://hibernate.onjira.com/browse/HHH-3817, this can happen with
>> two competing transactions.
>>
>> > If you cache something as READ_ONLY, and it gets deleted, that
>> > doesn't fit the definition of READ_ONLY though.  You are using the
>> > wrong cache concurrency strategy.
>> >
>> > Even that issue outlines the scenario where the collection is
>> > updated, which means its not a READ_ONLY.
>>
>> I think the update is irrelevant here. The issue is related to
>> putFromLoad + remove, which both AFAIK, are allowed in READ_ONLY
>> (remember that we had the discussion on whether remove should be
>> allowed in a READ_ONLY cache:
>> https://hibernate.onjira.com/browse/HHH-7350).
>>
>
> Yes, remove can be done, its just update that matters to READ_ONLY.  One thing I thought about was I thought we were using MVCC for this stuff.  Any transaction that reads from the cache, while something is being added/removed, should be reading the read consistent image, and should never wait on a lock, correct?  We see all the threads in our thread pool sitting in a blocked state based on this locking.
>
> That really shouldn't be happening with MVCC.

This would be correct, if only Infinispan would support MVCC, but
that's not the case today.

Sanne

>
> Andy
>
>> >
>> > Andy
>> >
>> > ----- Original Message -----
>> >> From: "Galder Zamarreño" <galder at redhat.com>
>> >> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
>> >> Cc: "Steve Ebersole" <steve at hibernate.org>, "John O'Hara"
>> >> <johara at redhat.com>, "Andrig Miller" <anmiller at redhat.com>,
>> >> "Jeremy Whiting" <jwhiting at redhat.com>
>> >> Sent: Wednesday, September 19, 2012 2:48:37 AM
>> >> Subject: Re: [infinispan-dev] Issue with cache blocks for local
>> >> read-only cache
>> >>
>> >> This is code written in JBoss Cache days to deal with situations
>> >> where putFromLoad might try to store stale data (if data is
>> >> deleted
>> >> in between a database read and putFromLoad being called).
>> >>
>> >> Indeed this can happen with read only data, and has nothing to do
>> >> with clustering.
>> >>
>> >> The original issue is:
>> >> https://hibernate.onjira.com/browse/HHH-3817
>> >>
>> >> I'll check how this can be improved.
>> >>
>> >> Cheers,
>> >>
>> >> On Sep 18, 2012, at 4:55 PM, Sanne Grinovero
>> >> <sanne at infinispan.org>
>> >> wrote:
>> >>
>> >>> There seems to be a single (global) reentrant lock which is
>> >>> acquired
>> >>> on any put; I don't know much about the module design, but there
>> >>> is
>> >>> a
>> >>> comment close to the lock acquisition mentioning a need to flush
>> >>> an
>> >>> operations queue to avoid a deadlock.
>> >>> Could it just make sure to reorder keys, so that deadlocks are
>> >>> avoided?
>> >>>
>> >>> On 18 September 2012 16:26, Manik Surtani <manik at jboss.org>
>> >>> wrote:
>> >>>> Agreed.
>> >>>>
>> >>>> This locking appears to be in the 2LC integration code to guard
>> >>>> a
>> >>>> local
>> >>>> collection though - there must be better, non-blocking ways to
>> >>>> guard this -
>> >>>> if it is needed at all for RO entities.
>> >>>>
>> >>>> - M
>> >>>>
>> >>>> On 18 Sep 2012, at 15:18, Andrig Miller <anmiller at redhat.com>
>> >>>> wrote:
>> >>>>
>> >>>> What I find interesting, is that in this use case, READ_ONLY
>> >>>> concurrency
>> >>>> strategy and a local cache (no invalidation and no replication),
>> >>>> there is no
>> >>>> need to do any locking.
>> >>>>
>> >>>> If we can get to a solution without any locking that would be
>> >>>> ideal.
>> >>>>
>> >>>> Andy
>> >>>>
>> >>>> ________________________________
>> >>>>
>> >>>> From: "Manik Surtani" <manik at jboss.org>
>> >>>> To: "Ståle W. Pedersen" <spederse at redhat.com>
>> >>>> Cc: "Galder Zamarreño" <galder at redhat.com>, "John O'Hara"
>> >>>> <johara at redhat.com>, "Jeremy Whiting" <jwhiting at redhat.com>,
>> >>>> "Andrig Miller"
>> >>>> <anmiller at redhat.com>, "Steve Ebersole" <steve at hibernate.org>,
>> >>>> "infinispan-dev" <infinispan-dev at lists.jboss.org>
>> >>>> Sent: Tuesday, September 18, 2012 8:13:43 AM
>> >>>> Subject: Re: Issue with cache blocks for local read-only cache
>> >>>>
>> >>>>
>> >>>> Looking at your profiler snapshot, these locks are in the
>> >>>> Hibernate 2nd
>> >>>> level cache implementation for Infinispan.  Galder, any ideas?
>> >>>>
>> >>>> - M
>> >>>>
>> >>>> On 18 Sep 2012, at 13:43, Ståle W. Pedersen
>> >>>> <spederse at redhat.com>
>> >>>> wrote:
>> >>>>
>> >>>> hi galder and manik, sorry for sending this mail to so many, but
>> >>>> we've ran
>> >>>> into a issue that prevents us from further scaling of the
>> >>>> specjenterprise2010 benchmark.
>> >>>>
>> >>>> so when doing specjenterprise2010 benchmark testing we've seen a
>> >>>> lot of
>> >>>> blocks caused by the entity/query cache. we've been testing with
>> >>>> only
>> >>>> caching a simple entity bean that's read-only and queries
>> >>>> related
>> >>>> to this
>> >>>> entity (selects).
>> >>>>
>> >>>> here is a screenshot of the hotspot
>> >>>> found:https://dl.dropbox.com/u/30971563/specjent_block.png
>> >>>>
>> >>>> here is the standalone.xml:
>> >>>> https://dl.dropbox.com/u/30971563/standalone-full.xml
>> >>>>
>> >>>> here is the orm.xml:
>> >>>> https://dl.dropbox.com/u/30971563/order_orm.xml
>> >>>>
>> >>>> what we don't understand is why there are so many puts into the
>> >>>> cache for an
>> >>>> object that is marked as read-only. when we're testing without
>> >>>> caching we do
>> >>>> not see any blocks.
>> >>>>
>> >>>> any help/ideas would be great. if anyone want a jprofiler
>> >>>> snapshot
>> >>>> of the
>> >>>> run, let me know.
>> >>>>
>> >>>> regards, ståle
>> >>>> --
>> >>>> JBoss Performance Team Lead
>> >>>> JBoss by Red Hat
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Manik Surtani
>> >>>> manik at jboss.org
>> >>>> twitter.com/maniksurtani
>> >>>>
>> >>>> Platform Architect, JBoss Data Grid
>> >>>> http://red.ht/data-grid
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Manik Surtani
>> >>>> manik at jboss.org
>> >>>> twitter.com/maniksurtani
>> >>>>
>> >>>> Platform Architect, JBoss Data Grid
>> >>>> http://red.ht/data-grid
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> infinispan-dev mailing list
>> >>>> infinispan-dev at lists.jboss.org
>> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >>>
>> >>> _______________________________________________
>> >>> infinispan-dev mailing list
>> >>> infinispan-dev at lists.jboss.org
>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >>
>> >>
>> >> --
>> >> Galder Zamarreño
>> >> galder at redhat.com
>> >> twitter.com/galderz
>> >>
>> >> Project Lead, Escalante
>> >> http://escalante.io
>> >>
>> >> Engineer, Infinispan
>> >> http://infinispan.org
>> >>
>> >>
>>
>>
>> --
>> Galder Zamarreño
>> galder at redhat.com
>> twitter.com/galderz
>>
>> Project Lead, Escalante
>> http://escalante.io
>>
>> Engineer, Infinispan
>> http://infinispan.org
>>
>>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev



More information about the infinispan-dev mailing list