[infinispan-issues] [JBoss JIRA] (ISPN-8503) Error executing MassIndexer using off-heap
Gustavo Fernandes (JIRA)
issues at jboss.org
Tue Nov 7 04:03:01 EST 2017
Gustavo Fernandes created ISPN-8503:
---------------------------------------
Summary: Error executing MassIndexer using off-heap
Key: ISPN-8503
URL: https://issues.jboss.org/browse/ISPN-8503
Project: Infinispan
Issue Type: Bug
Affects Versions: 9.2.0.Alpha1
Reporter: Lucas Ponce
Assignee: Gustavo Fernandes
Fix For: 9.2.0.Alpha2
Invoking a re-index operation on startup from the API
{code}
SearchManager searchResourceManager = Search.getSearchManager(resource);
CompletableFuture<Void> reindexResource = searchResourceManager.getMassIndexer().startAsync();
SearchManager searchResourceTypeManager = Search.getSearchManager(resourceType);
CompletableFuture<Void> reindexResourceType = searchResourceTypeManager.getMassIndexer().startAsync();
CompletableFuture.allOf(reindexResource, reindexResourceType).get();
{code}
Throws the following Exception
{code}
14:01:23,537 ERROR [org.hawkular.inventory.service.InventoryConfig] (ServerService Thread Pool -- 62) HAWKINV100005: Error reindexing caches: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: ISPN014018: Error executing MassIndexer
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at org.hawkular.inventory.service.InventoryConfig.init(InventoryConfig.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:96)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doLifecycleInterception(Jsr299BindingsInterceptor.java:114)
at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:103)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)
at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:53)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.as.ee.component.AroundConstructInterceptorFactory$1.processInvocation(AroundConstructInterceptorFactory.java:28)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.as.weld.injection.WeldInterceptorInjectionInterceptor.processInvocation(WeldInterceptorInjectionInterceptor.java:56)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.as.weld.ejb.Jsr299BindingsCreateInterceptor.processInvocation(Jsr299BindingsCreateInterceptor.java:100)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:275)
at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:349)
at org.jboss.as.ejb3.tx.LifecycleCMTTxInterceptor.processInvocation(LifecycleCMTTxInterceptor.java:68)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.as.weld.injection.WeldInjectionContextInterceptor.processInvocation(WeldInjectionContextInterceptor.java:43)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.as.ejb3.component.singleton.StartupCountDownInterceptor.processInvocation(StartupCountDownInterceptor.java:25)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161)
at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:134)
at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:88)
at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:124)
at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:138)
at org.jboss.as.ee.component.ComponentStartService$1.run(ComponentStartService.java:54)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at org.jboss.threads.JBossThread.run(JBossThread.java:320)
Caused by: org.infinispan.commons.CacheException: ISPN014018: Error executing MassIndexer
at org.infinispan.query.impl.massindex.DistributedExecutorMassIndexer.lambda$null$0(DistributedExecutorMassIndexer.java:73)
at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:870)
at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:852)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at org.infinispan.distexec.DefaultExecutorService$LocalDistributedTaskPart.lambda$execute$1(DefaultExecutorService.java:1086)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Indexing only works with entries keyed on Strings, primitives and classes that have the @Transformable annotation - you passed in a class org.infinispan.commons.marshall.WrappedByteArray. Alternatively, see org.infinispan.query.SearchManager#registerKeyTransformer
at org.infinispan.query.backend.KeyTransformationHandler.keyToString(KeyTransformationHandler.java:154)
at org.infinispan.query.impl.massindex.IndexUpdater.updateIndex(IndexUpdater.java:67)
at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:101)
at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:34)
at org.infinispan.commands.read.DistributedExecuteCommand.invokeAsync(DistributedExecuteCommand.java:99)
at org.infinispan.distexec.DefaultExecutorService$LocalDistributedTaskPart.lambda$execute$1(DefaultExecutorService.java:1074)
... 3 more
{code}
Infinispan 9.2.0.Alpha1 is running in library mode embedded in a .war application with the following configuration:
{code}
<cache-container name="hawkular-inventory">
<jmx duplicate-domains="true" />
<local-cache name="resource" statistics="true">
<transaction mode="NON_XA"/>
<persistence>
<file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
<write-behind thread-pool-size="5" modification-queue-size="10000" />
</file-store>
</persistence>
<memory>
<!--
<binary eviction="COUNT" size="10000" />
-->
<!--
<object size="5000" />
-->
<off-heap eviction="COUNT" size="100000" />
<!--
<off-heap eviction="COUNT" size="100000" />
-->
</memory>
<indexing index="LOCAL">
<indexed-entities>
<indexed-entity>org.hawkular.inventory.service.ispn.IspnResource</indexed-entity>
</indexed-entities>
<property name="default.indexmanager">near-real-time</property>
<property name="default.directory_provider">infinispan</property>
<property name="default.chunk_size">128000</property>
<property name="default.locking_cachename">resource_indexes_locking</property>
<property name="default.data_cachename">resource_indexes_data</property>
<property name="default.metadata_cachename">resource_indexes_metadata</property>
<!-- The default is 10, but we don't want to waste many cycles in merging
(tune for writes at cost of reader fragmentation) -->
<property name="default.indexwriter.merge_factor">30</property>
<!-- Never create segments larger than 1GB -->
<property name="default.indexwriter.merge_max_size">1024</property>
<!-- IndexWriter flush buffer size in MB -->
<property name="default.indexwriter.ram_buffer_size">64</property>
<!-- Enable sharding on writers -->
<property name="default.sharding_strategy.nbr_of_shards">6</property>
<property name="lucene_version">LUCENE_CURRENT</property>
</indexing>
</local-cache>
<local-cache name="resource_indexes_metadata">
<persistence passivation="false">
<file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
<write-behind thread-pool-size="5" />
</file-store>
</persistence>
<indexing index="NONE"/>
</local-cache>
<local-cache name="resource_indexes_data">
<persistence passivation="false">
<file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
<write-behind thread-pool-size="5" />
</file-store>
</persistence>
<indexing index="NONE" />
</local-cache>
<local-cache name="resource_indexes_locking">
<persistence passivation="false">
<file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
<write-behind thread-pool-size="5" />
</file-store>
</persistence>
<indexing index="NONE" />
</local-cache>
<local-cache name="resource_type" statistics="true">
<transaction mode="NON_XA"/>
<persistence>
<file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
<write-behind thread-pool-size="5" modification-queue-size="10000" />
</file-store>
</persistence>
<indexing index="LOCAL">
<indexed-entities>
<indexed-entity>org.hawkular.inventory.service.ispn.IspnResourceType</indexed-entity>
</indexed-entities>
<property name="default.indexmanager">near-real-time</property>
<property name="default.directory_provider">infinispan</property>
<property name="default.chunk_size">128000</property>
<property name="default.locking_cachename">resource_type_indexes_locking</property>
<property name="default.data_cachename">resource_type_indexes_data</property>
<property name="default.metadata_cachename">resource_type_indexes_metadata</property>
<!-- The default is 10, but we don't want to waste many cycles in merging
(tune for writes at cost of reader fragmentation) -->
<property name="default.indexwriter.merge_factor">30</property>
<!-- Never create segments larger than 1GB -->
<property name="default.indexwriter.merge_max_size">1024</property>
<!-- IndexWriter flush buffer size in MB -->
<property name="default.indexwriter.ram_buffer_size">64</property>
<!-- Enable sharding on writers -->
<property name="default.sharding_strategy.nbr_of_shards">6</property>
<property name="lucene_version">LUCENE_CURRENT</property>
</indexing>
</local-cache>
<local-cache name="resource_type_indexes_metadata">
<persistence passivation="false">
<file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
<write-behind thread-pool-size="5" />
</file-store>
</persistence>
<indexing index="NONE"/>
</local-cache>
<local-cache name="resource_type_indexes_data">
<persistence passivation="false">
<file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
<write-behind thread-pool-size="5" />
</file-store>
</persistence>
<indexing index="NONE" />
</local-cache>
<local-cache name="resource_type_indexes_locking">
<persistence passivation="false">
<file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
<write-behind thread-pool-size="5" />
</file-store>
</persistence>
<indexing index="NONE" />
</local-cache>
</cache-container>
{code}
At initialization, it tries to re-index existing resources*.dat files for data and lucene indexes.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
More information about the infinispan-issues
mailing list