Actually, I can't modify ObjectTable.readObject() signature since it's JBoss Marshalling's class. All we can really do is change the signature of StreamingMarshaller.objectFromObjectStream so that it can throw InterruptedException. In turn, COT.readObject() could throw an IE wrapped inside an IOE. VAM could then catch the IOE and if it contains an IE, throw it as is. This would force cache stores to deal with interrupted exceptions.

The correct vehicle for such work is the InterruptedException. So EvictionManager.stop() interrupting and cancelling the EvictionTask is correct behaviour. What happens here? The processEviction() method gets interrupted? We just need to make sure we handle this properly.

So the first bit in processEviction() is probably wrong: http://fisheye.jboss.org/browse/Infinispan/branches/4.2.x/core/src/main/j... if startLatch.await() is interrupted, we shouldn't just continue with the method, but rather return. It won't *prevent* the problem of the thread being interrupted while purging the cache store, but it will still prevent unnecessary purging from taking place. Further, the block that purges the cache store is wrapped in a try-catch block. Don't you see an interrupted exception here?

On 20 Oct 2010, at 14:13, Galder Zamarreño wrote: > Hi, > > Re: https://jira.jboss.org/browse/ISPN-699 > > I'm trying to figure out what the best way to solve this issue is. Basically, the problem is that when cache manager is stopped, EvictionManagerImpl cancels with interruption the evictionTask and I'm seeing issues with cacheStore.purgeExpired() not responding to cancellation properly. This results in Marshaller being stopped and then eviction thread trying to purge the cache store. Obviously, once the marshaller is stopped, nothing can be read any more. > > I've tried to simply protect cacheStore.purgeExpired() call around a Thread.currentThread().isInterrupted() call but this is not enough because we could have hundreds of buckets to check for purging, and the interruption could happen while looping through them. Now, I don't see the point of plaguing the code with Thread.currentThread().isInterrupted() checks, it'd be pointless. Instead, I wanted to share other ideas to solve this issue: > > 1. EvictionManagerImpl could wait for any ongoing eviction task to finished. This could be potentially lengthy if for example the cache store has hundreds or thousands of buckets, and we don't want for stop requests to block at all. > > 2. The main problem comes from the fact that the marshaller is being requested to read something but it can't do it anymore since it's shutting down. An alternative would be for the ConstantObjectTable to return null under the situation that is stopped and the thread is interrupted. This might work fine since if the bucket read returns null, it skips the bucket, but it's not optimal. If you have 1000s of buckets, it is going to continue looping through them, so you'd need something else, and this logic would need to be replicated in all cache stores. > > 3. COT returning null might not be a good idea. Instead, ObjectTable readObject() method could be changed to declare that it can throw an InterruptedException. This would force any caller to deal with this situation, including all current cache stores. I think this is the only way we can enforce cache stores to behave properly to interruption situations like the one mentioned above. Otherwise, we have to start doing esoteric things in all cache stores to check what a null return from unmarshalling means, or try to double guess that an IOException might be wrapping an IE. > > I'm currently leaning towards 3. > > Thoughts? > -- > Galder Zamarreño > Sr. Software Engineer > Infinispan, JBoss Cache > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 21 Oct 2010, at 09:28, Galder Zamarreño wrote: > This is what I'm trying to solve right now, a way for the FCS to get our of purging internal properly if interrupted. May be more than just the FCS. Need to check whether all of the cache stores handle being interrupted properly. Not just during a purge, but during any read/write operation. I suspect this may be your problem. A further problem - wrt purging - may be that the FCS supports async purging, where the purge operation is handed off to an executor. If this is enabled, then this task would have to be cancelled as well. See: http://fisheye.jboss.org/browse/Infinispan/branches/4.2.x/core/src/main/j...

-- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev