[JBoss JIRA] (ISPN-11875) JDBC Loader
by Tristan Tarrant (Jira)
[ https://issues.redhat.com/browse/ISPN-11875?page=com.atlassian.jira.plugi... ]
Tristan Tarrant moved JDGPL-305 to ISPN-11875:
----------------------------------------------
Project: Infinispan (was: JDG planning)
Key: ISPN-11875 (was: JDGPL-305)
Issue Type: Feature Request (was: Requirement)
Workflow: GIT Pull Request with Triage workflow (was: RHMAP Workflow)
Component/s: Loaders and Stores
(was: Persistence)
> JDBC Loader
> -----------
>
> Key: ISPN-11875
> URL: https://issues.redhat.com/browse/ISPN-11875
> Project: Infinispan
> Issue Type: Feature Request
> Components: Loaders and Stores
> Reporter: Tristan Tarrant
> Assignee: Ryan Emerson
> Priority: Optional
>
> As a application developer or data grid administrator, I want to preload a cache from a relational database accessed by JDBC - via configuration without writing boilerplate code.
>
> *Solution*
> A read-only cache loader will be implemented to allow users achieve that goal by following the steps.
> The cache loader is configured per cache.
> *Step 1. Configuration* - The required set of parameters are:
> _Data source configuration_:
> _SQL query to retrieve_ I.e “select isin, title as t, description from books”
>
> A query that will retrieve all the keys and values to put into the cache, with the following structure and conventions:
> * First column, will map to a key of type String
> * Rest of columns, will map to a value of type: String, Byte[] or a complext Protobuf object
> Mapping of every row to a cache key/value will work as follows:
>
> * Simple key and values: If the query returns only 2 columns, and the second one is either String or Byte[], the value will be the content of the second column. No protobuf schema is needed.
> Example, 'select code, country from country_list"
> * Simple key and complex value. If query returns 2+ columns, a Protobuf schema will be automatically generated and used to encode all the columns' values.
> Example, "select code, country, region, total_sales from sales_results"
>
> *Step 2. Execution on cache initialization*
> The cache loader will:
> # Flush the cache
> # Run the SQL query
> # Generate Protobuf schema, if needed. Register it in the Protobuf schemas cache.
> # Transform every row of the resultset into a key and a simple value, or an object composed of simple types.
> # Put(key,value) into the cache
> # The cache will not available during the initialization process
> *Optional, Step 3. Additional refresh*
> To refresh the data after the startup, several options will exist:
> # server restart
> # Invoke a JMX operation
> # Use CLI to invoke same operation
> # Use console to invoke same operation
> *Other usability considerations.*
> Generated Protobuf schemas will be available for usage by remote clients by:
> - Download from console, for administration.
> - Printed out in console during generation, for copy & paste, for regular development.
> - Access the protobuf cache directly, for programmatic usage.
> - Console will be updated with this capability
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 10 months
[JBoss JIRA] (ISPN-11875) JDBC Loader
by Tristan Tarrant (Jira)
[ https://issues.redhat.com/browse/ISPN-11875?page=com.atlassian.jira.plugi... ]
Tristan Tarrant reassigned ISPN-11875:
--------------------------------------
Assignee: (was: Ryan Emerson)
> JDBC Loader
> -----------
>
> Key: ISPN-11875
> URL: https://issues.redhat.com/browse/ISPN-11875
> Project: Infinispan
> Issue Type: Feature Request
> Components: Loaders and Stores
> Reporter: Tristan Tarrant
> Priority: Optional
>
> As a application developer or data grid administrator, I want to preload a cache from a relational database accessed by JDBC - via configuration without writing boilerplate code.
>
> *Solution*
> A read-only cache loader will be implemented to allow users achieve that goal by following the steps.
> The cache loader is configured per cache.
> *Step 1. Configuration* - The required set of parameters are:
> _Data source configuration_:
> _SQL query to retrieve_ I.e “select isin, title as t, description from books”
>
> A query that will retrieve all the keys and values to put into the cache, with the following structure and conventions:
> * First column, will map to a key of type String
> * Rest of columns, will map to a value of type: String, Byte[] or a complext Protobuf object
> Mapping of every row to a cache key/value will work as follows:
>
> * Simple key and values: If the query returns only 2 columns, and the second one is either String or Byte[], the value will be the content of the second column. No protobuf schema is needed.
> Example, 'select code, country from country_list"
> * Simple key and complex value. If query returns 2+ columns, a Protobuf schema will be automatically generated and used to encode all the columns' values.
> Example, "select code, country, region, total_sales from sales_results"
>
> *Step 2. Execution on cache initialization*
> The cache loader will:
> # Flush the cache
> # Run the SQL query
> # Generate Protobuf schema, if needed. Register it in the Protobuf schemas cache.
> # Transform every row of the resultset into a key and a simple value, or an object composed of simple types.
> # Put(key,value) into the cache
> # The cache will not available during the initialization process
> *Optional, Step 3. Additional refresh*
> To refresh the data after the startup, several options will exist:
> # server restart
> # Invoke a JMX operation
> # Use CLI to invoke same operation
> # Use console to invoke same operation
> *Other usability considerations.*
> Generated Protobuf schemas will be available for usage by remote clients by:
> - Download from console, for administration.
> - Printed out in console during generation, for copy & paste, for regular development.
> - Access the protobuf cache directly, for programmatic usage.
> - Console will be updated with this capability
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 10 months
[JBoss JIRA] (ISPN-11853) Error and Log handling in REST
by Will Burns (Jira)
[ https://issues.redhat.com/browse/ISPN-11853?page=com.atlassian.jira.plugi... ]
Will Burns commented on ISPN-11853:
-----------------------------------
I am not 100% on the safety, however, I would be very surprised that there isn't some sort of happens before occurring between there. I believe the BlockingQueue put and take would satisfy this, the more I think about it.
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/package-su...
"Actions in a thread prior to placing an object into any concurrent collection happen-before actions subsequent to the access or removal of that element from the collection in another thread."
> Error and Log handling in REST
> ------------------------------
>
> Key: ISPN-11853
> URL: https://issues.redhat.com/browse/ISPN-11853
> Project: Infinispan
> Issue Type: Enhancement
> Components: REST
> Affects Versions: 10.1.8.Final
> Reporter: Katia Aresti
> Assignee: Gustavo Fernandes
> Priority: Major
> Labels: rest
>
> The consistency of the errors handling in the REST endpoint is not even in all the resources, and the completion stage should deal with exceptions raised instead of throwing them directly from the method
> Logs are missing in the REST endpoints, specially trace logs, which makes debugging in production hard
> Finally the NettyRestResponse.Builder apparently is not thread safe, so would be better to make it thread safe
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 10 months
[JBoss JIRA] (ISPN-11874) ClusterPublisher can get NPE during topology change with ScatteredCache
by Will Burns (Jira)
Will Burns created ISPN-11874:
---------------------------------
Summary: ClusterPublisher can get NPE during topology change with ScatteredCache
Key: ISPN-11874
URL: https://issues.redhat.com/browse/ISPN-11874
Project: Infinispan
Issue Type: Bug
Affects Versions: 11.0.0.Dev05
Reporter: Will Burns
Scattered cache can have topologies that don't have a primary owner for a segment. Currently ClusterPublisher assumes every segment has a primary owner. We need to wait for the next topology again if a segment doesn't yet have a primary owner.
Here is a sample stack trace of it occurring.
{code}
java.lang.NullPointerException
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeCommand(JGroupsTransport.java:876)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeCommand(RpcManagerImpl.java:149)
at org.infinispan.reactive.publisher.impl.ClusterPublisherManagerImpl$SubscriberHandler.sendInitialCommand(ClusterPublisherManagerImpl.java:1034)
at org.infinispan.reactive.publisher.impl.InnerPublisherSubscription.accept(InnerPublisherSubscription.java:136)
at io.reactivex.rxjava3.internal.operators.flowable.FlowableDoOnLifecycle$SubscriptionLambdaSubscriber.request(FlowableDoOnLifecycle.java:101)
at io.reactivex.rxjava3.internal.subscriptions.SubscriptionArbiter.setSubscription(SubscriptionArbiter.java:99)
at io.reactivex.rxjava3.internal.operators.flowable.FlowableRepeatUntil$RepeatSubscriber.onSubscribe(FlowableRepeatUntil.java:61)
at io.reactivex.rxjava3.internal.operators.flowable.FlowableDoOnLifecycle$SubscriptionLambdaSubscriber.onSubscribe(FlowableDoOnLifecycle.java:73)
at io.reactivex.rxjava3.processors.UnicastProcessor.subscribeActual(UnicastProcessor.java:481)
at io.reactivex.rxjava3.core.Flowable.subscribe(Flowable.java:15750)
at io.reactivex.rxjava3.internal.operators.flowable.FlowableDoOnLifecycle.subscribeActual(FlowableDoOnLifecycle.java:38)
at io.reactivex.rxjava3.core.Flowable.subscribe(Flowable.java:15750)
at io.reactivex.rxjava3.core.Flowable.subscribe(Flowable.java:15696)
at io.reactivex.rxjava3.internal.operators.flowable.FlowableScalarXMap$ScalarXMapFlowable.subscribeActual(FlowableScalarXMap.java:160)
at io.reactivex.rxjava3.core.Flowable.subscribe(Flowable.java:15750)
at io.reactivex.rxjava3.core.Flowable.subscribe(Flowable.java:15696)
at io.reactivex.rxjava3.internal.operators.flowable.FlowableScalarXMap$ScalarXMapFlowable.subscribeActual(FlowableScalarXMap.java:160)
at io.reactivex.rxjava3.core.Flowable.subscribe(Flowable.java:15750)
at io.reactivex.rxjava3.core.Flowable.subscribe(Flowable.java:15696)
at io.reactivex.rxjava3.internal.operators.flowable.FlowableRepeatUntil$RepeatSubscriber.subscribeNext(FlowableRepeatUntil.java:109)
at io.reactivex.rxjava3.internal.operators.flowable.FlowableRepeatUntil$RepeatSubscriber.onComplete(FlowableRepeatUntil.java:88)
at io.reactivex.rxjava3.internal.operators.flowable.FlowableFromCompletable$FromCompletableObserver.onComplete(FlowableFromCompletable.java:77)
at io.reactivex.rxjava3.internal.jdk8.CompletableFromCompletionStage$CompletionStageHandler.accept(CompletableFromCompletionStage.java:64)
at io.reactivex.rxjava3.internal.jdk8.CompletableFromCompletionStage$CompletionStageHandler.accept(CompletableFromCompletionStage.java:47)
at io.reactivex.rxjava3.internal.jdk8.FlowableFromCompletionStage$BiConsumerAtomicReference.accept(FlowableFromCompletionStage.java:90)
at io.reactivex.rxjava3.internal.jdk8.FlowableFromCompletionStage$BiConsumerAtomicReference.accept(FlowableFromCompletionStage.java:81)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
at org.infinispan.statetransfer.StateTransferLockImpl.notifyTopologyInstalled(StateTransferLockImpl.java:137)
at org.infinispan.statetransfer.StateConsumerImpl.lambda$onTopologyUpdate$1(StateConsumerImpl.java:342)
at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106)
at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235)
at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:143)
at org.infinispan.statetransfer.StateConsumerImpl.onTopologyUpdate(StateConsumerImpl.java:322)
at org.infinispan.scattered.impl.ScatteredStateConsumerImpl.onTopologyUpdate(ScatteredStateConsumerImpl.java:100)
at org.infinispan.statetransfer.StateTransferManagerImpl.updateProviderAndConsumer(StateTransferManagerImpl.java:206)
at org.infinispan.statetransfer.StateTransferManagerImpl.lambda$doTopologyUpdate$0(StateTransferManagerImpl.java:193)
at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106)
at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235)
at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:143)
at org.infinispan.statetransfer.StateTransferManagerImpl.doTopologyUpdate(StateTransferManagerImpl.java:192)
at org.infinispan.statetransfer.StateTransferManagerImpl.access$000(StateTransferManagerImpl.java:68)
at org.infinispan.statetransfer.StateTransferManagerImpl$1.updateConsistentHash(StateTransferManagerImpl.java:123)
at org.infinispan.topology.LocalTopologyManagerImpl.lambda$doHandleTopologyUpdate$12(LocalTopologyManagerImpl.java:399)
at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106)
at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235)
at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:143)
at org.infinispan.topology.LocalTopologyManagerImpl.doHandleTopologyUpdate(LocalTopologyManagerImpl.java:388)
at org.infinispan.topology.LocalTopologyManagerImpl.lambda$handleTopologyUpdate$8(LocalTopologyManagerImpl.java:312)
at org.infinispan.topology.LocalTopologyManagerImpl.lambda$orderOnCache$24(LocalTopologyManagerImpl.java:737)
at org.infinispan.util.concurrent.ActionSequencer.safeNonBlockingCall(ActionSequencer.java:57)
at org.infinispan.util.concurrent.ActionSequencer.access$400(ActionSequencer.java:32)
at org.infinispan.util.concurrent.ActionSequencer$SequenceEntry.run(ActionSequencer.java:226)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
{code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 10 months
[JBoss JIRA] (ISPN-9466) org.infinispan.query.clustered.QueryBox does not implement eviction for zombie iterators
by Gustavo Fernandes (Jira)
[ https://issues.redhat.com/browse/ISPN-9466?page=com.atlassian.jira.plugin... ]
Gustavo Fernandes commented on ISPN-9466:
-----------------------------------------
The QueryBox component is only used when doing CacheQuery with the LAZY param, such as
{code:java}
cacheQuery.iterator(new FetchOptions().fetchMode(FetchOptions.FetchMode.LAZY)
{code}
Since lazy iterators only makes sense with distributed Lucene Queries, and CacheQuery is being deprecated in ISPN 11 (replaced by Query that is index agnostic), I am going to close this issue.
> org.infinispan.query.clustered.QueryBox does not implement eviction for zombie iterators
> ----------------------------------------------------------------------------------------
>
> Key: ISPN-9466
> URL: https://issues.redhat.com/browse/ISPN-9466
> Project: Infinispan
> Issue Type: Task
> Components: Embedded Querying
> Reporter: Nistor Adrian
> Assignee: Gustavo Fernandes
> Priority: Major
> Labels: query-ispn11
>
> Although the javadoc hints to some eviction mechanism this is not actually implemented.
> A linked list 'ageOrderedQueries' is maintained so that active queries move to the head of the list and the list is limited to 3000 items. Still, there is no mechanism that actually removes the zombie queries from the 'queries' ConcurrentMap.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 10 months