[infinispan-dev] Issue with cache blocks for local read-only cache

Andrig Miller anmiller at redhat.com
Thu Sep 20 11:38:10 EDT 2012 


----- Original Message -----
> From: "Galder Zamarreño" <galder at redhat.com>
> To: "Andrig Miller" <anmiller at redhat.com>
> Cc: "Steve Ebersole" <steve at hibernate.org>, "John O'Hara" <johara at redhat.com>, "Jeremy Whiting"
> <jwhiting at redhat.com>, "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> Sent: Thursday, September 20, 2012 6:48:59 AM
> Subject: Re: [infinispan-dev] Issue with cache blocks for local read-only cache
> 
> 
> On Sep 19, 2012, at 4:20 PM, Andrig Miller <anmiller at redhat.com>
> wrote:
> 
> > Yes, I can see how that can happen, if the data is deleted from
> > outside the application.
> 
> ^ The issue does not only happen if the data is deleted outside the
> application. As indicated in
> https://hibernate.onjira.com/browse/HHH-3817, this can happen with
> two competing transactions.
> 
> > If you cache something as READ_ONLY, and it gets deleted, that
> > doesn't fit the definition of READ_ONLY though.  You are using the
> > wrong cache concurrency strategy.
> > 
> > Even that issue outlines the scenario where the collection is
> > updated, which means its not a READ_ONLY.
> 
> I think the update is irrelevant here. The issue is related to
> putFromLoad + remove, which both AFAIK, are allowed in READ_ONLY
> (remember that we had the discussion on whether remove should be
> allowed in a READ_ONLY cache:
> https://hibernate.onjira.com/browse/HHH-7350).
> 

Yes, remove can be done, its just update that matters to READ_ONLY.  One thing I thought about was I thought we were using MVCC for this stuff.  Any transaction that reads from the cache, while something is being added/removed, should be reading the read consistent image, and should never wait on a lock, correct?  We see all the threads in our thread pool sitting in a blocked state based on this locking.

That really shouldn't be happening with MVCC.

Andy

> > 
> > Andy
> > 
> > ----- Original Message -----
> >> From: "Galder Zamarreño" <galder at redhat.com>
> >> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> >> Cc: "Steve Ebersole" <steve at hibernate.org>, "John O'Hara"
> >> <johara at redhat.com>, "Andrig Miller" <anmiller at redhat.com>,
> >> "Jeremy Whiting" <jwhiting at redhat.com>
> >> Sent: Wednesday, September 19, 2012 2:48:37 AM
> >> Subject: Re: [infinispan-dev] Issue with cache blocks for local
> >> read-only cache
> >> 
> >> This is code written in JBoss Cache days to deal with situations
> >> where putFromLoad might try to store stale data (if data is
> >> deleted
> >> in between a database read and putFromLoad being called).
> >> 
> >> Indeed this can happen with read only data, and has nothing to do
> >> with clustering.
> >> 
> >> The original issue is:
> >> https://hibernate.onjira.com/browse/HHH-3817
> >> 
> >> I'll check how this can be improved.
> >> 
> >> Cheers,
> >> 
> >> On Sep 18, 2012, at 4:55 PM, Sanne Grinovero
> >> <sanne at infinispan.org>
> >> wrote:
> >> 
> >>> There seems to be a single (global) reentrant lock which is
> >>> acquired
> >>> on any put; I don't know much about the module design, but there
> >>> is
> >>> a
> >>> comment close to the lock acquisition mentioning a need to flush
> >>> an
> >>> operations queue to avoid a deadlock.
> >>> Could it just make sure to reorder keys, so that deadlocks are
> >>> avoided?
> >>> 
> >>> On 18 September 2012 16:26, Manik Surtani <manik at jboss.org>
> >>> wrote:
> >>>> Agreed.
> >>>> 
> >>>> This locking appears to be in the 2LC integration code to guard
> >>>> a
> >>>> local
> >>>> collection though - there must be better, non-blocking ways to
> >>>> guard this -
> >>>> if it is needed at all for RO entities.
> >>>> 
> >>>> - M
> >>>> 
> >>>> On 18 Sep 2012, at 15:18, Andrig Miller <anmiller at redhat.com>
> >>>> wrote:
> >>>> 
> >>>> What I find interesting, is that in this use case, READ_ONLY
> >>>> concurrency
> >>>> strategy and a local cache (no invalidation and no replication),
> >>>> there is no
> >>>> need to do any locking.
> >>>> 
> >>>> If we can get to a solution without any locking that would be
> >>>> ideal.
> >>>> 
> >>>> Andy
> >>>> 
> >>>> ________________________________
> >>>> 
> >>>> From: "Manik Surtani" <manik at jboss.org>
> >>>> To: "Ståle W. Pedersen" <spederse at redhat.com>
> >>>> Cc: "Galder Zamarreño" <galder at redhat.com>, "John O'Hara"
> >>>> <johara at redhat.com>, "Jeremy Whiting" <jwhiting at redhat.com>,
> >>>> "Andrig Miller"
> >>>> <anmiller at redhat.com>, "Steve Ebersole" <steve at hibernate.org>,
> >>>> "infinispan-dev" <infinispan-dev at lists.jboss.org>
> >>>> Sent: Tuesday, September 18, 2012 8:13:43 AM
> >>>> Subject: Re: Issue with cache blocks for local read-only cache
> >>>> 
> >>>> 
> >>>> Looking at your profiler snapshot, these locks are in the
> >>>> Hibernate 2nd
> >>>> level cache implementation for Infinispan.  Galder, any ideas?
> >>>> 
> >>>> - M
> >>>> 
> >>>> On 18 Sep 2012, at 13:43, Ståle W. Pedersen
> >>>> <spederse at redhat.com>
> >>>> wrote:
> >>>> 
> >>>> hi galder and manik, sorry for sending this mail to so many, but
> >>>> we've ran
> >>>> into a issue that prevents us from further scaling of the
> >>>> specjenterprise2010 benchmark.
> >>>> 
> >>>> so when doing specjenterprise2010 benchmark testing we've seen a
> >>>> lot of
> >>>> blocks caused by the entity/query cache. we've been testing with
> >>>> only
> >>>> caching a simple entity bean that's read-only and queries
> >>>> related
> >>>> to this
> >>>> entity (selects).
> >>>> 
> >>>> here is a screenshot of the hotspot
> >>>> found:https://dl.dropbox.com/u/30971563/specjent_block.png
> >>>> 
> >>>> here is the standalone.xml:
> >>>> https://dl.dropbox.com/u/30971563/standalone-full.xml
> >>>> 
> >>>> here is the orm.xml:
> >>>> https://dl.dropbox.com/u/30971563/order_orm.xml
> >>>> 
> >>>> what we don't understand is why there are so many puts into the
> >>>> cache for an
> >>>> object that is marked as read-only. when we're testing without
> >>>> caching we do
> >>>> not see any blocks.
> >>>> 
> >>>> any help/ideas would be great. if anyone want a jprofiler
> >>>> snapshot
> >>>> of the
> >>>> run, let me know.
> >>>> 
> >>>> regards, ståle
> >>>> --
> >>>> JBoss Performance Team Lead
> >>>> JBoss by Red Hat
> >>>> 
> >>>> 
> >>>> --
> >>>> Manik Surtani
> >>>> manik at jboss.org
> >>>> twitter.com/maniksurtani
> >>>> 
> >>>> Platform Architect, JBoss Data Grid
> >>>> http://red.ht/data-grid
> >>>> 
> >>>> 
> >>>> --
> >>>> Manik Surtani
> >>>> manik at jboss.org
> >>>> twitter.com/maniksurtani
> >>>> 
> >>>> Platform Architect, JBoss Data Grid
> >>>> http://red.ht/data-grid
> >>>> 
> >>>> 
> >>>> _______________________________________________
> >>>> infinispan-dev mailing list
> >>>> infinispan-dev at lists.jboss.org
> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>> 
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >> 
> >> 
> >> --
> >> Galder Zamarreño
> >> galder at redhat.com
> >> twitter.com/galderz
> >> 
> >> Project Lead, Escalante
> >> http://escalante.io
> >> 
> >> Engineer, Infinispan
> >> http://infinispan.org
> >> 
> >> 
> 
> 
> --
> Galder Zamarreño
> galder at redhat.com
> twitter.com/galderz
> 
> Project Lead, Escalante
> http://escalante.io
> 
> Engineer, Infinispan
> http://infinispan.org
> 
>

More information about the infinispan-dev mailing list