[JBoss JIRA] (ISPN-8310) LockedStream invokeAll
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-8310?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-8310:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> LockedStream invokeAll
> ----------------------
>
> Key: ISPN-8310
> URL: https://issues.jboss.org/browse/ISPN-8310
> Project: Infinispan
> Issue Type: Feature Request
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 9.2.0.Alpha2
>
>
> We need a way to run a command for all data in the cache (while under a lock) but also return a value for the data processed.
> There are a few different APIs we could do for this having a {{Cache<K, V>}}
> blocking variant
> {code}
> <R> Map<K, R> evalAll(BiFunction<Cache<K, V>, ? super CacheEntry<K, V>, R> biFunction)
> {code}
> back pressure aware variant - this would have to be a hot observer on subscribe
> {code}
> <R> Publisher<R> evalAll(BiFunction<Cache<K, V>, ? super CacheEntry<K, V> R> biFunction)
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 6 months
[JBoss JIRA] (ISPN-8430) Create Online Service Configuration Templates
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-8430?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-8430:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.2.0.Alpha2
Resolution: Done
> Create Online Service Configuration Templates
> ---------------------------------------------
>
> Key: ISPN-8430
> URL: https://issues.jboss.org/browse/ISPN-8430
> Project: Infinispan
> Issue Type: Bug
> Reporter: Ryan Emerson
> Assignee: Ryan Emerson
> Fix For: 9.2.0.Alpha2
>
>
> We should create a standalone configuration file that only contains the templates that will be used for the shared-memory and caching services. This will reduce the amount of cli operations required for our dataservices images and will also allow non-openshift users to utilise the same configurations.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 6 months
[JBoss JIRA] (ISPN-8416) Error executing MassIndexer using off-heap
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-8416?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-8416:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.2.0.Alpha2
Resolution: Done
> Error executing MassIndexer using off-heap
> ------------------------------------------
>
> Key: ISPN-8416
> URL: https://issues.jboss.org/browse/ISPN-8416
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 9.2.0.Alpha1
> Reporter: Lucas Ponce
> Assignee: Gustavo Fernandes
> Fix For: 9.2.0.Alpha2
>
>
> Invoking a re-index operation on startup from the API
> {code}
> SearchManager searchResourceManager = Search.getSearchManager(resource);
> CompletableFuture<Void> reindexResource = searchResourceManager.getMassIndexer().startAsync();
> SearchManager searchResourceTypeManager = Search.getSearchManager(resourceType);
> CompletableFuture<Void> reindexResourceType = searchResourceTypeManager.getMassIndexer().startAsync();
> CompletableFuture.allOf(reindexResource, reindexResourceType).get();
> {code}
> Throws the following Exception
> {code}
> 14:01:23,537 ERROR [org.hawkular.inventory.service.InventoryConfig] (ServerService Thread Pool -- 62) HAWKINV100005: Error reindexing caches: java.util.concurrent.ExecutionException: org.infinispan.commons.CacheException: ISPN014018: Error executing MassIndexer
> at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at org.hawkular.inventory.service.InventoryConfig.init(InventoryConfig.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:96)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doLifecycleInterception(Jsr299BindingsInterceptor.java:114)
> at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:103)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
> at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)
> at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:53)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ee.component.AroundConstructInterceptorFactory$1.processInvocation(AroundConstructInterceptorFactory.java:28)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.weld.injection.WeldInterceptorInjectionInterceptor.processInvocation(WeldInterceptorInjectionInterceptor.java:56)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.weld.ejb.Jsr299BindingsCreateInterceptor.processInvocation(Jsr299BindingsCreateInterceptor.java:100)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:275)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:349)
> at org.jboss.as.ejb3.tx.LifecycleCMTTxInterceptor.processInvocation(LifecycleCMTTxInterceptor.java:68)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.weld.injection.WeldInjectionContextInterceptor.processInvocation(WeldInjectionContextInterceptor.java:43)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.as.ejb3.component.singleton.StartupCountDownInterceptor.processInvocation(StartupCountDownInterceptor.java:25)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
> at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
> at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
> at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161)
> at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:134)
> at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:88)
> at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:124)
> at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:138)
> at org.jboss.as.ee.component.ComponentStartService$1.run(ComponentStartService.java:54)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> at org.jboss.threads.JBossThread.run(JBossThread.java:320)
> Caused by: org.infinispan.commons.CacheException: ISPN014018: Error executing MassIndexer
> at org.infinispan.query.impl.massindex.DistributedExecutorMassIndexer.lambda$null$0(DistributedExecutorMassIndexer.java:73)
> at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:870)
> at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:852)
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
> at org.infinispan.distexec.DefaultExecutorService$LocalDistributedTaskPart.lambda$execute$1(DefaultExecutorService.java:1086)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalArgumentException: Indexing only works with entries keyed on Strings, primitives and classes that have the @Transformable annotation - you passed in a class org.infinispan.commons.marshall.WrappedByteArray. Alternatively, see org.infinispan.query.SearchManager#registerKeyTransformer
> at org.infinispan.query.backend.KeyTransformationHandler.keyToString(KeyTransformationHandler.java:154)
> at org.infinispan.query.impl.massindex.IndexUpdater.updateIndex(IndexUpdater.java:67)
> at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:101)
> at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:34)
> at org.infinispan.commands.read.DistributedExecuteCommand.invokeAsync(DistributedExecuteCommand.java:99)
> at org.infinispan.distexec.DefaultExecutorService$LocalDistributedTaskPart.lambda$execute$1(DefaultExecutorService.java:1074)
> ... 3 more
> {code}
> Infinispan 9.2.0.Alpha1 is running in library mode embedded in a .war application with the following configuration:
> {code}
> <cache-container name="hawkular-inventory">
> <jmx duplicate-domains="true" />
> <local-cache name="resource" statistics="true">
> <transaction mode="NON_XA"/>
> <persistence>
> <file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
> <write-behind thread-pool-size="5" modification-queue-size="10000" />
> </file-store>
> </persistence>
> <memory>
> <!--
> <binary eviction="COUNT" size="10000" />
> -->
> <!--
> <object size="5000" />
> -->
> <off-heap eviction="COUNT" size="100000" />
> <!--
> <off-heap eviction="COUNT" size="100000" />
> -->
> </memory>
> <indexing index="LOCAL">
> <indexed-entities>
> <indexed-entity>org.hawkular.inventory.service.ispn.IspnResource</indexed-entity>
> </indexed-entities>
> <property name="default.indexmanager">near-real-time</property>
> <property name="default.directory_provider">infinispan</property>
> <property name="default.chunk_size">128000</property>
> <property name="default.locking_cachename">resource_indexes_locking</property>
> <property name="default.data_cachename">resource_indexes_data</property>
> <property name="default.metadata_cachename">resource_indexes_metadata</property>
> <!-- The default is 10, but we don't want to waste many cycles in merging
> (tune for writes at cost of reader fragmentation) -->
> <property name="default.indexwriter.merge_factor">30</property>
> <!-- Never create segments larger than 1GB -->
> <property name="default.indexwriter.merge_max_size">1024</property>
> <!-- IndexWriter flush buffer size in MB -->
> <property name="default.indexwriter.ram_buffer_size">64</property>
> <!-- Enable sharding on writers -->
> <property name="default.sharding_strategy.nbr_of_shards">6</property>
> <property name="lucene_version">LUCENE_CURRENT</property>
> </indexing>
> </local-cache>
> <local-cache name="resource_indexes_metadata">
> <persistence passivation="false">
> <file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
> <write-behind thread-pool-size="5" />
> </file-store>
> </persistence>
> <indexing index="NONE"/>
> </local-cache>
> <local-cache name="resource_indexes_data">
> <persistence passivation="false">
> <file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
> <write-behind thread-pool-size="5" />
> </file-store>
> </persistence>
> <indexing index="NONE" />
> </local-cache>
> <local-cache name="resource_indexes_locking">
> <persistence passivation="false">
> <file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
> <write-behind thread-pool-size="5" />
> </file-store>
> </persistence>
> <indexing index="NONE" />
> </local-cache>
> <local-cache name="resource_type" statistics="true">
> <transaction mode="NON_XA"/>
> <persistence>
> <file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
> <write-behind thread-pool-size="5" modification-queue-size="10000" />
> </file-store>
> </persistence>
> <indexing index="LOCAL">
> <indexed-entities>
> <indexed-entity>org.hawkular.inventory.service.ispn.IspnResourceType</indexed-entity>
> </indexed-entities>
> <property name="default.indexmanager">near-real-time</property>
> <property name="default.directory_provider">infinispan</property>
> <property name="default.chunk_size">128000</property>
> <property name="default.locking_cachename">resource_type_indexes_locking</property>
> <property name="default.data_cachename">resource_type_indexes_data</property>
> <property name="default.metadata_cachename">resource_type_indexes_metadata</property>
> <!-- The default is 10, but we don't want to waste many cycles in merging
> (tune for writes at cost of reader fragmentation) -->
> <property name="default.indexwriter.merge_factor">30</property>
> <!-- Never create segments larger than 1GB -->
> <property name="default.indexwriter.merge_max_size">1024</property>
> <!-- IndexWriter flush buffer size in MB -->
> <property name="default.indexwriter.ram_buffer_size">64</property>
> <!-- Enable sharding on writers -->
> <property name="default.sharding_strategy.nbr_of_shards">6</property>
> <property name="lucene_version">LUCENE_CURRENT</property>
> </indexing>
> </local-cache>
> <local-cache name="resource_type_indexes_metadata">
> <persistence passivation="false">
> <file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
> <write-behind thread-pool-size="5" />
> </file-store>
> </persistence>
> <indexing index="NONE"/>
> </local-cache>
> <local-cache name="resource_type_indexes_data">
> <persistence passivation="false">
> <file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
> <write-behind thread-pool-size="5" />
> </file-store>
> </persistence>
> <indexing index="NONE" />
> </local-cache>
> <local-cache name="resource_type_indexes_locking">
> <persistence passivation="false">
> <file-store preload="true" fetch-state="true" read-only="false" purge="false" path="${jboss.server.data.dir}/hawkular-inventory">
> <write-behind thread-pool-size="5" />
> </file-store>
> </persistence>
> <indexing index="NONE" />
> </local-cache>
> </cache-container>
> {code}
> At initialization, it tries to re-index existing resources*.dat files for data and lucene indexes.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 6 months
[JBoss JIRA] (ISPN-8093) Remove operation for counters
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-8093?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-8093:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.2.0.Alpha2
Resolution: Done
> Remove operation for counters
> -----------------------------
>
> Key: ISPN-8093
> URL: https://issues.jboss.org/browse/ISPN-8093
> Project: Infinispan
> Issue Type: Feature Request
> Components: Clustered Counter
> Reporter: Pedro Ruivo
> Assignee: Pedro Ruivo
> Fix For: 9.2.0.Alpha2
>
>
> Allow counters to be removed. If an operation finds out that the counter doesn't exists, it is recreated with its initial value and the operation is replied.
> If a getValue is invoked and the counter doesn't exist, it returns its initial value without recreating the counter.
> New methods to the API:
> {code:java}
> CounterManager.removeCounter(String counterName) //remove the counter with this name
> StrongCounter.remove() //removes the counter represent by this instance
> WeakCounter.remove() //same as above
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 6 months
[JBoss JIRA] (ISPN-7418) Add per-cache key/value type information
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-7418?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-7418:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.2.0.Alpha2
Resolution: Done
> Add per-cache key/value type information
> ----------------------------------------
>
> Key: ISPN-7418
> URL: https://issues.jboss.org/browse/ISPN-7418
> Project: Infinispan
> Issue Type: Sub-task
> Reporter: Galder Zamarreño
> Assignee: Gustavo Fernandes
> Fix For: 9.2.0.Alpha2
>
>
> This part of the transcoding enhancement is related to the changes required to the core cache code to force the cache to store keys and values of a certain type.
> The type information should be defined via a MIME type, and should be configured by the user.
> A cache can only have a single key+value type combination. A single cache will not support storing data of different key and/or value types. Amongst other reasons, this is done for space efficiency.
> Once a cache can be configured with a specific key and value type, the user should be able to retrieve the value with an alternative MIME type. Behind the scenes, the cache should be able to transform the value from the give type to target type on the fly.
> As suggested in the wiki, there should be the possibility for the key/value type to be implicitly determined by the key+value type passed in during the first cache write. The aim of this is to enable remote clients to decide what the key+value type are upon writing to the cache for the first time, and hence avoid cache pre-configuration.
> Some transcoders might require additional configuration. The cache needs to be able to be configured with the necessary options to do on the fly transcoding for clients. E.g. schema for protobuf...etc.
> An important aspect here is that consistent hashing should not be affected by transcoding.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 6 months
[JBoss JIRA] (ISPN-6879) Calculate (and expose) minimum number of nodes for data in Infinispan
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-6879?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec commented on ISPN-6879:
-------------------------------------------
{quote}
So is this more so that we can tell them to never go below a certain number of nodes because otherwise we would have memory issues with all nodes storing so much?
I am just trying to understand the purpose of this.
{quote}
Yes, you got that right! We can do a quick chat tomorrow to make sure we are on the same page...
> Calculate (and expose) minimum number of nodes for data in Infinispan
> ---------------------------------------------------------------------
>
> Key: ISPN-6879
> URL: https://issues.jboss.org/browse/ISPN-6879
> Project: Infinispan
> Issue Type: Feature Request
> Components: Cloud Integrations, Server
> Reporter: Sebastian Łaskawiec
> Assignee: William Burns
>
> With Kubernetes autoscaling we need to be able to tell what is the minimum amount of nodes necessary for hosting data (probably some sort of size + number of nodes estimation).
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 6 months