On Mon, May 17, 2010 at 4:39 PM, Manik Surtani <manik@jboss.org> wrote:

On 17 May 2010, at 12:37, Philippe Van Dyck wrote:

>
> On Mon, May 17, 2010 at 11:47 AM, Manik Surtani <manik@jboss.org> wrote:
>
> On 16 May 2010, at 18:35, Philippe Van Dyck wrote:
>
> > Hi all,
> >
> > in a servlet environment, you sometimes have to claim back a thread for your thread pool.
> > Jetty has a nice "denial of service attack" filter and a setting (maxIdleTime) interrupting servlet threads if nothing comes out (nothing is written in the pipe).
> >
> > Well it does not play well with infinispan. Everybody knows that when you open a stream, the best way to reclaim scarce resources is to close it in a 'finally' statement. What about applying it to infinispan locking mechanism ? ;-)
> >
> > Calls to acquireLock should be guarded by a finally {releaseLock} (at least to catch an interruptedException)
>
> This is a good point, in the event of a failure (for whatever reason) the lock may still be acquired and should be cleaned up.  Care to create a JIRA for this?
>
> Done - ISPN-444

Hmm, about this - I suppose it really isn't enough to wrap the tryLock call since a thread could be interrupted doing a multitude of other things in addition to just acquiring locks - e.g., waiting to write state to a cache store, network comms, etc.

So while I have checked in a fix for ISPN-444 [1], I presume what we really need is a finally block around the InvocationContextInterceptor where all invocations come in [2], to clean up any locks acquired within the scope of the current call.

Thoughts?


Sound like a better idea indeed!
Do you have a plan to test it ?

BTW, this it what it looks like in the logs... when jetty lose the connection on the TCP level (connection reset by peer) it seems to interrupt the thread... (it is happening right now and ... it is quite problematic!)

2010-05-17 14:47:08,129[Thread-13] ERROR org.eclipse.jetty.util.log  - EXCEPTION 
org.eclipse.jetty.io.RuntimeIOException: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.io.UncheckedPrintWriter.close(UncheckedPrintWriter.java:139)
at com.netzep.common.frontend.application.DoSFilter.closeConnection(DoSFilter.java:426)
at com.netzep.common.frontend.application.DoSFilter$3.expired(DoSFilter.java:394)
at org.eclipse.jetty.util.thread.Timeout.tick(Timeout.java:136)
at com.netzep.common.frontend.application.DoSFilter$2.run(DoSFilter.java:242)
Caused by: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:892)
at org.eclipse.jetty.http.HttpGenerator.complete(HttpGenerator.java:791)
at org.eclipse.jetty.server.HttpConnection.commitResponse(HttpConnection.java:701)
at org.eclipse.jetty.server.HttpConnection$Output.close(HttpConnection.java:1077)
at org.eclipse.jetty.server.HttpWriter.close(HttpWriter.java:80)
at org.eclipse.jetty.io.UncheckedPrintWriter.close(UncheckedPrintWriter.java:133)
... 4 more
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
...
at org.eclipse.jetty.http.ssl.SslSelectChannelEndPoint.flush(SslSelectChannelEndPoint.java:438)
at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:831)
... 9 more
2010-05-17 14:47:08,425[Qi4j - Transaction : ] ERROR org.infinispan.interceptors.InvocationContextInterceptor  - Execution error: 
org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [60 seconds] on key [User/w916obtfCq3D0a7W2l5KrQAAABKg] for requestor [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:99]! Lock held by [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:98]
at org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:213)
at org.infinispan.container.EntryFactoryImpl.wrapEntryForWriting(EntryFactoryImpl.java:121)
...
at org.qi4j.entitystore.map.MapEntityStoreMixin$1$1.visitMap(MapEntityStoreMixin.java:132)
at org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:157)
Exception in thread "Qi4j - Transaction : " org.qi4j.spi.entitystore.EntityStoreException: Exception during transaction - rollback true - 
at org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:190)
Caused by: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [60 seconds] on key [User/w916obtfCq3D0a7W2l5KrQAAABKg] for requestor [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:99]! Lock held by [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:98]
at org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:213)
at org.infinispan.container.EntryFactoryImpl.wrapEntryForWriting(EntryFactoryImpl.java:121)
...
at org.qi4j.entitystore.map.MapEntityStoreMixin$1$1.visitMap(MapEntityStoreMixin.java:132)
at org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:157)
2010-05-17 14:47:09,001[Qi4j - Transaction : ] ERROR org.infinispan.interceptors.InvocationContextInterceptor  - Execution error: 
org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [60 seconds] on key [User/w9oM7pOrl2_D3N0KMIbWQwAAAAng] for requestor [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:100]! Lock held by [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:98]
at org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:213)
at org.infinispan.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:119)
at org.infinispan.interceptors.CacheLoaderInterceptor.visitPutKeyValueCommand(CacheLoaderInterceptor.java:77)
...
at org.qi4j.entitystore.map.MapEntityStoreMixin$1$1.visitMap(MapEntityStoreMixin.java:126)
at org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:157)
Exception in thread "Qi4j - Transaction : " org.qi4j.spi.entitystore.EntityStoreException: Exception during transaction - rollback true - 
at org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:190)
Caused by: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [60 seconds] on key [User/w9oM7pOrl2_D3N0KMIbWQwAAAAng] for requestor [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:100]! Lock held by [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:98]
at org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:213)
at org.infinispan.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:119)
at org.infinispan.interceptors.CacheLoaderInterceptor.visitPutKeyValueCommand(CacheLoaderInterceptor.java
...
at org.qi4j.entitystore.map.MapEntityStoreMixin$1$1.visitMap(MapEntityStoreMixin.java:126)
at org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:157)

cheers,

phil
 


 
Cheers
Manik

[1] http://fisheye.jboss.org/changelog/Infinispan/trunk?cs=1794
[2] http://fisheye.jboss.org/browse/Infinispan/trunk/core/src/main/java/org/infinispan/interceptors/InvocationContextInterceptor.java?r=269#l47

--
Manik Surtani
manik@jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org





_______________________________________________
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev