[infinispan-dev] lock acquisition - "finally release" it

Mon May 17 11:56:23 EDT 2010

On Mon, May 17, 2010 at 4:39 PM, Manik Surtani <manik at jboss.org> wrote:

>
> On 17 May 2010, at 12:37, Philippe Van Dyck wrote:
>
> >
> > On Mon, May 17, 2010 at 11:47 AM, Manik Surtani <manik at jboss.org> wrote:
> >
> > On 16 May 2010, at 18:35, Philippe Van Dyck wrote:
> >
> > > Hi all,
> > >
> > > in a servlet environment, you sometimes have to claim back a thread for
> your thread pool.
> > > Jetty has a nice "denial of service attack" filter and a setting
> (maxIdleTime) interrupting servlet threads if nothing comes out (nothing is
> written in the pipe).
> > >
> > > Well it does not play well with infinispan. Everybody knows that when
> you open a stream, the best way to reclaim scarce resources is to close it
> in a 'finally' statement. What about applying it to infinispan locking
> mechanism ? ;-)
> > >
> > > Calls to acquireLock should be guarded by a finally {releaseLock} (at
> least to catch an interruptedException)
> >
> > This is a good point, in the event of a failure (for whatever reason) the
> lock may still be acquired and should be cleaned up.  Care to create a JIRA
> for this?
> >
> > Done - ISPN-444
>
> Hmm, about this - I suppose it really isn't enough to wrap the tryLock call
> since a thread could be interrupted doing a multitude of other things in
> addition to just acquiring locks - e.g., waiting to write state to a cache
> store, network comms, etc.
>
> So while I have checked in a fix for ISPN-444 [1], I presume what we really
> need is a finally block around the InvocationContextInterceptor where all
> invocations come in [2], to clean up any locks acquired within the scope of
> the current call.
>
> Thoughts?
>
>
Sound like a better idea indeed!
Do you have a plan to test it ?

BTW, this it what it looks like in the logs... when jetty lose the
connection on the TCP level (connection reset by peer) it seems to interrupt
the thread... (it is happening right now and ... it is quite problematic!)

2010-05-17 14:47:08,129[Thread-13] ERROR org.eclipse.jetty.util.log  -
EXCEPTION
org.eclipse.jetty.io.RuntimeIOException: org.eclipse.jetty.io.EofException
at
org.eclipse.jetty.io.UncheckedPrintWriter.close(UncheckedPrintWriter.java:139)
at
com.netzep.common.frontend.application.DoSFilter.closeConnection(DoSFilter.java:426)
at
com.netzep.common.frontend.application.DoSFilter$3.expired(DoSFilter.java:394)
at org.eclipse.jetty.util.thread.Timeout.tick(Timeout.java:136)
at
com.netzep.common.frontend.application.DoSFilter$2.run(DoSFilter.java:242)
Caused by: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:892)
at org.eclipse.jetty.http.HttpGenerator.complete(HttpGenerator.java:791)
at
org.eclipse.jetty.server.HttpConnection.commitResponse(HttpConnection.java:701)
at
org.eclipse.jetty.server.HttpConnection$Output.close(HttpConnection.java:1077)
at org.eclipse.jetty.server.HttpWriter.close(HttpWriter.java:80)
at
org.eclipse.jetty.io.UncheckedPrintWriter.close(UncheckedPrintWriter.java:133)
... 4 more
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
...
at
org.eclipse.jetty.http.ssl.SslSelectChannelEndPoint.flush(SslSelectChannelEndPoint.java:438)
at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:831)
... 9 more
2010-05-17 14:47:08,425[Qi4j - Transaction : ] ERROR
org.infinispan.interceptors.InvocationContextInterceptor  - Execution
error:
org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock
after [60 seconds] on key [User/w916obtfCq3D0a7W2l5KrQAAABKg] for requestor
[GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:99]! Lock held by
[GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:98]
at
org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:213)
at
org.infinispan.container.EntryFactoryImpl.wrapEntryForWriting(EntryFactoryImpl.java:121)
...
at
org.qi4j.entitystore.map.MapEntityStoreMixin$1$1.visitMap(MapEntityStoreMixin.java:132)
at
org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:157)
Exception in thread "Qi4j - Transaction : "
org.qi4j.spi.entitystore.EntityStoreException: Exception during transaction
- rollback true -
at
org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:190)
Caused by: org.infinispan.util.concurrent.TimeoutException: Unable to
acquire lock after [60 seconds] on key [User/w916obtfCq3D0a7W2l5KrQAAABKg]
for requestor [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:99]! Lock
held by [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:98]
at
org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:213)
at
org.infinispan.container.EntryFactoryImpl.wrapEntryForWriting(EntryFactoryImpl.java:121)
...
at
org.qi4j.entitystore.map.MapEntityStoreMixin$1$1.visitMap(MapEntityStoreMixin.java:132)
at
org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:157)
2010-05-17 14:47:09,001[Qi4j - Transaction : ] ERROR
org.infinispan.interceptors.InvocationContextInterceptor  - Execution
error:
org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock
after [60 seconds] on key [User/w9oM7pOrl2_D3N0KMIbWQwAAAAng] for requestor
[GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:100]! Lock held by
[GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:98]
at
org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:213)
at
org.infinispan.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:119)
at
org.infinispan.interceptors.CacheLoaderInterceptor.visitPutKeyValueCommand(CacheLoaderInterceptor.java:77)
...
at
org.qi4j.entitystore.map.MapEntityStoreMixin$1$1.visitMap(MapEntityStoreMixin.java:126)
at
org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:157)
Exception in thread "Qi4j - Transaction : "
org.qi4j.spi.entitystore.EntityStoreException: Exception during transaction
- rollback true -
at
org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:190)
Caused by: org.infinispan.util.concurrent.TimeoutException: Unable to
acquire lock after [60 seconds] on key [User/w9oM7pOrl2_D3N0KMIbWQwAAAAng]
for requestor [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:100]! Lock
held by [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:98]
at
org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:213)
at
org.infinispan.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:119)
at
org.infinispan.interceptors.CacheLoaderInterceptor.visitPutKeyValueCommand(CacheLoaderInterceptor.java
...
at
org.qi4j.entitystore.map.MapEntityStoreMixin$1$1.visitMap(MapEntityStoreMixin.java:126)
at
org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:157)

cheers,

phil

> Cheers
> Manik
>
> [1] http://fisheye.jboss.org/changelog/Infinispan/trunk?cs=1794
> [2]
> http://fisheye.jboss.org/browse/Infinispan/trunk/core/src/main/java/org/infinispan/interceptors/InvocationContextInterceptor.java?r=269#l47
>
> --
> Manik Surtani
> manik at jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20100517/f2aa9d24/attachment-0001.html 