InvocationContext classloader is not used when calling store, creating problems in BdbjeCacheStore -------------------------------------------------------------------------------------------------- Key: ISPN-3057 URL: https://issues.jboss.org/browse/ISPN-3057 Project: Infinispan Issue Type: Bug Components: Loaders and Stores Affects Versions: 5.1.6.FINAL Environment: OSGI (Fuse ServiceMix) Reporter: Vitalii Tymchyshyn Assignee: Pedro Ruivo Fix For: 9.0.0.Alpha4 I am working with DecoratingCache (.with()) to specify classloader to use to deserialize value. Nevertheless, I am getting next exception: {code} org.infinispan.CacheException: org.infinispan.loaders.CacheLoaderException: error removing key FIDESSA.21.0 at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:353) at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:174)[245:org.infinispan.core:5.1.6.FINAL] at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:121) at org.apache.geronimo.transaction.manager.TransactionImpl.commitResource(TransactionImpl.java:622) at org.apache.geronimo.transaction.manager.TransactionImpl.commit(TransactionImpl.java:305) at org.apache.geronimo.transaction.manager.TransactionManagerImpl.commit(TransactionManagerImpl.java:252) at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1009)[116:org.springframework.transaction:3.0.7.RELEASE] at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)[116:org.springframework.transaction:3.0.7.RELEASE] at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)[116:org.springframework.transaction:3.0.7.RELEASE] at org.apache.aries.transaction.GeronimoPlatformTransactionManager.commit(GeronimoPlatformTransactionManager.java:76) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)[:1.6.0_26] at java.lang.reflect.Method.invoke(Method.java:597)[:1.6.0_26] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)[108:org.springframework.aop:3.0.7.RELEASE] at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:58)[113:org.springframework.osgi.core:1.2.1] at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:62)[113:org.springframework.osgi.core:1.2.1] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)[108:org.springframework.aop:3.0.7.RELEASE] at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)[108:org.springframework.aop:3.0.7.RELEASE] at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)[108:org.springframework.aop:3.0.7.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)[108:org.springframework.aop:3.0.7.RELEASE] at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:56)[113:org.springframework.osgi.core:1.2.1] at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:39)[113:org.springframework.osgi.core:1.2.1] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)[108:org.springframework.aop:3.0.7.RELEASE] at org.springframework.osgi.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:59)[113:org.springframework.osgi.core:1.2.1] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)[108:org.springframework.aop:3.0.7.RELEASE] at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)[108:org.springframework.aop:3.0.7.RELEASE] at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)[108:org.springframework.aop:3.0.7.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)[108:org.springframework.aop:3.0.7.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)[108:org.springframework.aop:3.0.7.RELEASE] at $Proxy205.commit(Unknown Source)[:] at com.ubs.infinispan.utils.queue.InfinispanQueueConsumer.callInTransaction(InfinispanQueueConsumer.java:155)[240:com.ubs.ace.infinispan.utils:1.0.0] at com.ubs.infinispan.utils.queue.InfinispanQueueConsumer.doRun(InfinispanQueueConsumer.java:78)[240:com.ubs.ace.infinispan.utils:1.0.0] at org.apache.camel.component.seda.SedaConsumer.run(SedaConsumer.java:139)[122:org.apache.camel.camel-core:2.10.0.fuse-71-047] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)[:1.6.0_26] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)[:1.6.0_26] at java.lang.Thread.run(Thread.java:662)[:1.6.0_26] Caused by: org.infinispan.loaders.CacheLoaderException: error removing key FIDESSA.21.0 at org.infinispan.loaders.bdbje.BdbjeCacheStore.convertToCacheLoaderException(BdbjeCacheStore.java:589) at org.infinispan.loaders.bdbje.BdbjeCacheStore.remove(BdbjeCacheStore.java:402) at org.infinispan.loaders.bdbje.ModificationsTransactionWorker.doWork(ModificationsTransactionWorker.java:73) at com.sleepycat.collections.TransactionRunner.run(TransactionRunner.java:227) at org.infinispan.loaders.bdbje.BdbjeCacheStore.applyModifications(BdbjeCacheStore.java:289) at org.infinispan.loaders.bdbje.BdbjeCacheStore.prepare(BdbjeCacheStore.java:272) at org.infinispan.interceptors.CacheStoreInterceptor.prepareCacheLoader(CacheStoreInterceptor.java:289) at org.infinispan.interceptors.CacheStoreInterceptor.visitPrepareCommand(CacheStoreInterceptor.java:199) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130) at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:93) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:120) at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitPrepareCommand(PessimisticLockingInterceptor.java:100) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:106) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.infinispan.interceptors.StateTransferLockInterceptor.handleWithRetries(StateTransferLockInterceptor.java:213) at org.infinispan.interceptors.StateTransferLockInterceptor.visitPrepareCommand(StateTransferLockInterceptor.java:85) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130) at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:132) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:91) at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:345) ... 35 more Caused by: java.lang.ClassNotFoundException: com.ubs.ace.fmc.entity.FmcEntity not found from bundle [com.ubs.ace.quickfix] at org.springframework.osgi.util.BundleDelegatingClassLoader.findClass(BundleDelegatingClassLoader.java:103) at org.springframework.osgi.util.BundleDelegatingClassLoader.loadClass(BundleDelegatingClassLoader.java:156) at java.lang.ClassLoader.loadClass(ClassLoader.java:247)[:1.6.0_26] at java.lang.Class.forName0(Native Method)[:1.6.0_26] at java.lang.Class.forName(Class.java:247)[:1.6.0_26] at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:135) at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:116) at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:892) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1204) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) at org.infinispan.container.entries.ImmortalCacheEntry$Externalizer.readObject(ImmortalCacheEntry.java:160) at org.infinispan.container.entries.ImmortalCacheEntry$Externalizer.readObject(ImmortalCacheEntry.java:150) at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:395) at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:224) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:163) at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:114) at org.infinispan.marshall.AbstractMarshaller.objectFromByteBuffer(AbstractMarshaller.java:106) at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:99) at org.infinispan.loaders.bdbje.InternalCacheEntryBinding.entryToObject(InternalCacheEntryBinding.java:43) at org.infinispan.loaders.bdbje.InternalCacheEntryBinding.entryToObject(InternalCacheEntryBinding.java:33) at com.sleepycat.collections.DataView.makeValue(DataView.java:596) at com.sleepycat.collections.DataCursor.getCurrentValue(DataCursor.java:350) at com.sleepycat.collections.StoredContainer.removeKey(StoredContainer.java:344) at com.sleepycat.collections.StoredMap.remove(StoredMap.java:351) at org.infinispan.loaders.bdbje.BdbjeCacheStore.remove(BdbjeCacheStore.java:397) ... 73 more Caused by: java.lang.ClassNotFoundException: com.ubs.ace.fmc.entity.FmcEntity not found by com.ubs.ace.quickfix [249] at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1499) at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:75) at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1882) at java.lang.ClassLoader.loadClass(ClassLoader.java:247)[:1.6.0_26] at org.apache.felix.framework.Felix.loadBundleClass(Felix.java:1814) at org.apache.felix.framework.BundleImpl.loadClass(BundleImpl.java:929) at org.springframework.osgi.util.BundleDelegatingClassLoader.findClass(BundleDelegatingClassLoader.java:99) ... 102 more {code} It indicates that my classloader is not used. It seems that classloader from invocation context is not used during store calls. By design, it looks like store should not need to deserialize anything in store or remove calls, but BdbdjeCacheStore use BDB Map representation and map's remove or put calls return previous value that must be deserialized. As of remove (my problem), I've changed Store code to look like: {code} @Override public boolean remove(Object key) throws CacheLoaderException { try { return this.cacheMap.keySet().remove(key); } catch (RuntimeException caught) { throw convertToCacheLoaderException("error removing key " + key, caught); } } {code} Using keySet prevents the problem as keySet does not return old value. Also it should make the call a little faster. Unfortunately, map's put operation does not have any equivalent without old value return. I am not sure on workaround if I will see same problem with store (may be remove key beforehand?). So, I see two options: 1) Change BbdjeCacheStore for not to use map view for store and use keySet for remove. 2) Call stores with invocation context in generic (as I see this would require changes in org.infinispan.loaders.modifications.Modification).