[infinispan-issues] [JBoss JIRA] (ISPN-2340) AsyncStore.remove() load the bucket from store inducing performance problem

[Galder Zamarreño (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721740#comment-12721740 ] 

Galder Zamarreño commented on ISPN-2340:
----------------------------------------

Hmm, this might be leftover from a previous attempt to solve the issue. Removing the load() call causes no issues.
                
> AsyncStore.remove() load the bucket from store inducing performance problem
> ---------------------------------------------------------------------------
>
>                 Key: ISPN-2340
>                 URL: https://issues.jboss.org/browse/ISPN-2340
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Loaders and Stores
>    Affects Versions: 5.2.0.Alpha4
>            Reporter: Christophe Domas
>            Assignee: Galder Zamarreño
>             Fix For: 5.2.0.CR1
>
>
> I'm currently using infinispan-5.1.5 as a local persistent cache (like a persistent map) with no eviction and write-behind for maximum performance.
> I tried the 5.2.0alpha4 version and item processing mean decreased from 4ms to 60 ms. 
> So I profiled the application and discovered that the Async.remove(Object) implementation differs from 5.2 to 5.1:
> +5.1+:
> {code}
>    @Override
>    public boolean remove(Object key) {
>       enqueue(new Remove(key));
>       return true;
>    }
> {code}
> +5.2+:
> {code}
>    @Override
>    public boolean remove(Object key) {
>       try {
>          InternalCacheEntry load = load(key);
>          if (load != null) {
>             enqueue(new Remove(key));
>             return true;
>          }
>          return false;
>       } catch (CacheLoaderException e) {
>          throw new CacheException("Could not load key/value entries from cacheloader", e);
>       }
>    }
> {code}
> So each time I remove an entry from my cache, I pay the cost of a bucket load from the store:
> {code}
> Thread [pool-9-thread-1] (Suspended)	
> 	ExtendedRiverUnmarshaller(RiverUnmarshaller).doReadObject(boolean) line: 209	
> 	ExtendedRiverUnmarshaller(AbstractObjectInput).readObject() line: 37	
> 	JBossMarshaller(AbstractJBossMarshaller).objectFromObjectStream(ObjectInput) line: 163	
> 	VersionAwareMarshaller.objectFromObjectStream(ObjectInput) line: 190	
> 	CacheMarshaller(AbstractDelegatingMarshaller).objectFromObjectStream(ObjectInput) line: 79	
> 	FileCacheStore.objectFromInputStreamInReentrantMode(InputStream) line: 433	
> 	FileCacheStore.loadBucket(File) line: 306	
> 	FileCacheStore.loadBucket(Integer) line: 280	
> 	FileCacheStore(BucketBasedCacheStore).loadLockSafe(Object, Integer) line: 60	
> 	FileCacheStore(BucketBasedCacheStore).loadLockSafe(Object, Object) line: 49	
> 	FileCacheStore(LockSupportCacheStore<L>).load(Object) line: 142	
> 	AsyncStore(AbstractDelegatingStore).load(Object) line: 120	
> 	AsyncStore.remove(Object) line: 163	
> 	CacheStoreInterceptor.visitRemoveCommand(InvocationContext, RemoveCommand) line: 221	
> 	RemoveCommand.acceptVisitor(InvocationContext, Visitor) line: 72	
> 	CacheLoaderInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext, VisitableCommand) line: 118	
> 	CacheLoaderInterceptor.visitRemoveCommand(InvocationContext, RemoveCommand) line: 138	
> 	RemoveCommand.acceptVisitor(InvocationContext, Visitor) line: 72	
> 	EntryWrappingInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext, VisitableCommand) line: 118	
> 	EntryWrappingInterceptor.invokeNextAndApplyChanges(InvocationContext, FlagAffectedCommand) line: 211	
> 	EntryWrappingInterceptor.visitRemoveCommand(InvocationContext, RemoveCommand) line: 161	
> 	RemoveCommand.acceptVisitor(InvocationContext, Visitor) line: 72	
> 	NonTransactionalLockingInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext, VisitableCommand) line: 118	
> 	NonTransactionalLockingInterceptor.visitRemoveCommand(InvocationContext, RemoveCommand) line: 104	
> 	RemoveCommand.acceptVisitor(InvocationContext, Visitor) line: 72	
> 	NotificationInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext, VisitableCommand) line: 118	
> 	NotificationInterceptor(CommandInterceptor).handleDefault(InvocationContext, VisitableCommand) line: 132	
> 	NotificationInterceptor(AbstractVisitor).visitRemoveCommand(InvocationContext, RemoveCommand) line: 68	
> 	RemoveCommand.acceptVisitor(InvocationContext, Visitor) line: 72	
> 	IsMarshallableInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext, VisitableCommand) line: 118	
> 	IsMarshallableInterceptor(CommandInterceptor).handleDefault(InvocationContext, VisitableCommand) line: 132	
> 	IsMarshallableInterceptor(AbstractVisitor).visitRemoveCommand(InvocationContext, RemoveCommand) line: 68	
> 	IsMarshallableInterceptor.visitRemoveCommand(InvocationContext, RemoveCommand) line: 125	
> 	RemoveCommand.acceptVisitor(InvocationContext, Visitor) line: 72	
> 	InvocationContextInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext, VisitableCommand) line: 118	
> 	InvocationContextInterceptor.handleAll(InvocationContext, VisitableCommand) line: 129	
> 	InvocationContextInterceptor.handleDefault(InvocationContext, VisitableCommand) line: 93	
> 	InvocationContextInterceptor(AbstractVisitor).visitRemoveCommand(InvocationContext, RemoveCommand) line: 68	
> 	RemoveCommand.acceptVisitor(InvocationContext, Visitor) line: 72	
> 	InterceptorChain.invoke(InvocationContext, VisitableCommand) line: 347	
> 	CacheImpl<K,V>.executeCommandAndCommitIfNeeded(InvocationContext, VisitableCommand) line: 999	
> 	CacheImpl<K,V>.remove(Object, EnumSet<Flag>, ClassLoader) line: 290	
> 	DecoratedCache<K,V>.remove(Object) line: 324	
> 	CacheBidiMap<K,V>(AbstractDualBidiMap<K,V>).remove(Object) line: 199	
> 	OrderCache.removePendingByClientId(String) line: 168	
> 	OrderEventCacheManager(EventCacheManager).receiveAck(OrderId) line: 166	
> 	OrderEventCacheManager(EventCacheManager).cacheEventNotification(IamapiEvent, OrderId) line: 101	
> 	OrderMarketFixService.onOrderEvent(OrderEvent) line: 88	
> 	OrderEventDispatcher.fireEvent(OrderEvent) line: 52	
> 	OrderEventDispatcher.run() line: 67	
> 	ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) line: 1110	
> 	ThreadPoolExecutor$Worker.run() line: 603	
> 	Thread.run() line: 722	
> {code}
> My config file is: 
> {code}
> 	<default>
> 		<loaders passivation="false" preload="true" shared="false">
> 			<loader class="org.infinispan.loaders.file.FileCacheStore">
> 				<properties>
> 					<property name="location" value="cache" />
> 					<property name="fsyncMode" value="perWrite" />
> 				</properties>
> 				<async enabled="true" />
> 			</loader>
> 		</loaders>
> 		<expiration reaperEnabled="false" />
> 	</default>
> {code}
> (I use perWrite for fsyncMode because we reached linux open files ulimit maximum)
> We cannot consider that as a bug, but this performance problem is a show-stopper for us to upgrade to 5.2.
> Best regards,
> Christophe

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira