<br><br><div class="gmail_quote">On Mon, May 17, 2010 at 4:39 PM, Manik Surtani <span dir="ltr"><<a href="mailto:manik@jboss.org">manik@jboss.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im"><br>
On 17 May 2010, at 12:37, Philippe Van Dyck wrote:<br>
<br>
><br>
</div><div class="im">> On Mon, May 17, 2010 at 11:47 AM, Manik Surtani <<a href="mailto:manik@jboss.org">manik@jboss.org</a>> wrote:<br>
><br>
> On 16 May 2010, at 18:35, Philippe Van Dyck wrote:<br>
><br>
> > Hi all,<br>
> ><br>
> > in a servlet environment, you sometimes have to claim back a thread for your thread pool.<br>
> > Jetty has a nice "denial of service attack" filter and a setting (maxIdleTime) interrupting servlet threads if nothing comes out (nothing is written in the pipe).<br>
> ><br>
> > Well it does not play well with infinispan. Everybody knows that when you open a stream, the best way to reclaim scarce resources is to close it in a 'finally' statement. What about applying it to infinispan locking mechanism ? ;-)<br>
> ><br>
> > Calls to acquireLock should be guarded by a finally {releaseLock} (at least to catch an interruptedException)<br>
><br>
> This is a good point, in the event of a failure (for whatever reason) the lock may still be acquired and should be cleaned up. Care to create a JIRA for this?<br>
><br>
> Done - ISPN-444<br>
<br>
</div>Hmm, about this - I suppose it really isn't enough to wrap the tryLock call since a thread could be interrupted doing a multitude of other things in addition to just acquiring locks - e.g., waiting to write state to a cache store, network comms, etc.<br>
<br>
So while I have checked in a fix for ISPN-444 [1], I presume what we really need is a finally block around the InvocationContextInterceptor where all invocations come in [2], to clean up any locks acquired within the scope of the current call.<br>
<br>
Thoughts?<br>
<br></blockquote><div><br></div><div>Sound like a better idea indeed!</div><div>Do you have a plan to test it ?</div><div><br></div><div>BTW, this it what it looks like in the logs... when jetty lose the connection on the TCP level (connection reset by peer) it seems to interrupt the thread... (it is happening right now and ... it is quite problematic!)</div>
<div><br></div><div><div>2010-05-17 14:47:08,129[Thread-13] ERROR org.eclipse.jetty.util.log - EXCEPTION </div><div>org.eclipse.jetty.io.RuntimeIOException: org.eclipse.jetty.io.EofException</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.eclipse.jetty.io.UncheckedPrintWriter.close(UncheckedPrintWriter.java:139)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at com.netzep.common.frontend.application.DoSFilter.closeConnection(DoSFilter.java:426)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at com.netzep.common.frontend.application.DoSFilter$3.expired(DoSFilter.java:394)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.eclipse.jetty.util.thread.Timeout.tick(Timeout.java:136)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at com.netzep.common.frontend.application.DoSFilter$2.run(DoSFilter.java:242)</div>
<div>Caused by: org.eclipse.jetty.io.EofException</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:892)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.eclipse.jetty.http.HttpGenerator.complete(HttpGenerator.java:791)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.eclipse.jetty.server.HttpConnection.commitResponse(HttpConnection.java:701)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.eclipse.jetty.server.HttpConnection$Output.close(HttpConnection.java:1077)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.eclipse.jetty.server.HttpWriter.close(HttpWriter.java:80)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.eclipse.jetty.io.UncheckedPrintWriter.close(UncheckedPrintWriter.java:133)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>... 4 more</div><div>Caused by: java.io.IOException: Connection reset by peer</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at sun.nio.ch.FileDispatcher.write0(Native Method)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)</div><div>...</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.eclipse.jetty.http.ssl.SslSelectChannelEndPoint.flush(SslSelectChannelEndPoint.java:438)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:831)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>... 9 more</div>
<div>2010-05-17 14:47:08,425[Qi4j - Transaction : ] ERROR org.infinispan.interceptors.InvocationContextInterceptor - Execution error: </div><div>org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [60 seconds] on key [User/w916obtfCq3D0a7W2l5KrQAAABKg] for requestor [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:99]! Lock held by [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:98]</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:213)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.infinispan.container.EntryFactoryImpl.wrapEntryForWriting(EntryFactoryImpl.java:121)</div>
<div>...</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.qi4j.entitystore.map.MapEntityStoreMixin$1$1.visitMap(MapEntityStoreMixin.java:132)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:157)</div>
<div>Exception in thread "Qi4j - Transaction : " org.qi4j.spi.entitystore.EntityStoreException: Exception during transaction - rollback true - </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:190)</div>
<div>Caused by: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [60 seconds] on key [User/w916obtfCq3D0a7W2l5KrQAAABKg] for requestor [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:99]! Lock held by [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:98]</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:213)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.infinispan.container.EntryFactoryImpl.wrapEntryForWriting(EntryFactoryImpl.java:121)</div>
<div>...</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.qi4j.entitystore.map.MapEntityStoreMixin$1$1.visitMap(MapEntityStoreMixin.java:132)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:157)</div>
<div>2010-05-17 14:47:09,001[Qi4j - Transaction : ] ERROR org.infinispan.interceptors.InvocationContextInterceptor - Execution error: </div><div>org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [60 seconds] on key [User/w9oM7pOrl2_D3N0KMIbWQwAAAAng] for requestor [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:100]! Lock held by [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:98]</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:213)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.infinispan.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:119)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.infinispan.interceptors.CacheLoaderInterceptor.visitPutKeyValueCommand(CacheLoaderInterceptor.java:77)</div><div>...</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.qi4j.entitystore.map.MapEntityStoreMixin$1$1.visitMap(MapEntityStoreMixin.java:126)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:157)</div><div>Exception in thread "Qi4j - Transaction : " org.qi4j.spi.entitystore.EntityStoreException: Exception during transaction - rollback true - </div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:190)</div><div>Caused by: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [60 seconds] on key [User/w9oM7pOrl2_D3N0KMIbWQwAAAAng] for requestor [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:100]! Lock held by [GlobalTransaction:<domU-12-31-39-09-9C-54-54012>:98]</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.infinispan.container.EntryFactoryImpl.acquireLock(EntryFactoryImpl.java:213)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.infinispan.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:119)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.infinispan.interceptors.CacheLoaderInterceptor.visitPutKeyValueCommand(CacheLoaderInterceptor.java</div><div>...</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.qi4j.entitystore.map.MapEntityStoreMixin$1$1.visitMap(MapEntityStoreMixin.java:126)</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>at org.qi4j.entitystore.s3jclouds.S3jcloudsMapEntityStoreMixin$1.run(S3jcloudsMapEntityStoreMixin.java:157)</div></div><div><br></div><div>cheers,</div><div>
<br></div><div>phil</div><div> </div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Cheers<br>
Manik<br>
<br>
[1] <a href="http://fisheye.jboss.org/changelog/Infinispan/trunk?cs=1794" target="_blank">http://fisheye.jboss.org/changelog/Infinispan/trunk?cs=1794</a><br>
[2] <a href="http://fisheye.jboss.org/browse/Infinispan/trunk/core/src/main/java/org/infinispan/interceptors/InvocationContextInterceptor.java?r=269#l47" target="_blank">http://fisheye.jboss.org/browse/Infinispan/trunk/core/src/main/java/org/infinispan/interceptors/InvocationContextInterceptor.java?r=269#l47</a><br>
<div><div></div><div class="h5"><br>
--<br>
Manik Surtani<br>
<a href="mailto:manik@jboss.org">manik@jboss.org</a><br>
Lead, Infinispan<br>
Lead, JBoss Cache<br>
<a href="http://www.infinispan.org" target="_blank">http://www.infinispan.org</a><br>
<a href="http://www.jbosscache.org" target="_blank">http://www.jbosscache.org</a><br>
<br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
</div></div></blockquote></div><br>