[JBoss JIRA] (ISPN-11687) Infinispan not able to start after CR configuration change
by Marian Macik (Jira)
[ https://issues.redhat.com/browse/ISPN-11687?page=com.atlassian.jira.plugi... ]
Marian Macik commented on ISPN-11687:
-------------------------------------
[~NadirX] I attached the whole exception log, the link is in the description of this JIRA.
Which operations over HR/REST do you mean?
For this part "I even tried to store something to the Infinispan via Infinispan REST API" I used normal REST API.
For HotRod we use [this class|https://github.com/kiegroup/kogito-runtimes/blob/master/addons/pers...] which uses RemoteCacheManager injected by Quarkus.
> Infinispan not able to start after CR configuration change
> ----------------------------------------------------------
>
> Key: ISPN-11687
> URL: https://issues.redhat.com/browse/ISPN-11687
> Project: Infinispan
> Issue Type: Bug
> Components: OpenShift, Operator
> Affects Versions: 10.1.2.Final
> Reporter: Marian Macik
> Assignee: Vittorio Rigamonti
> Priority: Major
>
> Hi guys, as you may know, Kogito Operator uses Infinispan Operator to create a predefined Infinispan instance running on Openshift. Quite often we encounter an issue reproducible with these steps:
> 1. Install the Kogito Operator (this will also install Infinispan Operator).
> 2. Create a KogitoApp custom resource (CR) with this YAML:
> {code:yaml}
> apiVersion: app.kiegroup.org/v1alpha1
> kind: KogitoApp
> metadata:
> name: example-quarkus
> spec:
> enablePersistence: true
> build:
> envs:
> # enable persistence
> - name: MAVEN_ARGS_APPEND
> value: "-Ppersistence"
> gitSource:
> contextDir: process-quarkus-example
> uri: 'https://github.com/kiegroup/kogito-examples'
> reference: master
> {code}
> This will create a KogitoApp CR and will tell the Kogito Operator to provision Infinispan with one replica. Kogito application runs on Quarkus which makes use of RemoteCacheManager of Quarkus Infinispan Client Extension. Up to this point, everything works, application is deployed.
> 3. Try change Infinispan config by editing Infinispan CR a few times and Infinispan won't be able to start properly. By editing I mean - change one of these 3 parameters:
> {code:yaml}
> ...
> spec:
> container:
> cpu: ''
> extraJvmOpts: ''
> memory: ''
> ...
> {code}
> I generally want to change the cpu and memory as defaults are too low and I am also specifying `-Xmx2G` to extraJvmOpts so Infinispan has more heap than 200 MB which is default.
> Anyway, if you do this change a couple of times and after each change you wait until Infinispan pod is restarted, after ~5 times you will see [java.nio.channels.OverlappingFileLockException|https://gist.github.com/Ma...] in the Infinispan pod log.
> There is also another issue attached at the bottom of the Gist which was observed in Openshift events logs.
> What I have found is that if I create only KogitoInfra CR, which will create Infinispan CR and won't run any KogitoApp, so there is nothing connected to Infinispan, I can restart it how many times I want and it will work without any issues. I even tried to store something to the Infinispan via Infinispan REST API from the pod and tried changing Infinispan configuration then, and it worked like a charm after each restart.
> However, as soon as I deploy KogitoApp so it is connected to Infinispan using HotRod client and change Infinispan CR a few times (after each change waiting for Infinispan pod to restart), it will break with the linked exception present in the logs.
> To me it seems that this stops working once there is actual connection to Infinispan using HotRod client. I am not sure how this client works internally, but I would think that in addition to real user data there is some sort of exchange of "control data" let's say in this protocol between client and Infinispan which might break if Infinispan is suddenly restarted? Not sure, but with pushing there data using REST API (so without HotRod client) where the connection is maintained only for the time of the request, the exception didn't occur.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (ISPN-11710) MassIndexer timeouts after 4 minutes
by Gustavo Fernandes (Jira)
[ https://issues.redhat.com/browse/ISPN-11710?page=com.atlassian.jira.plugi... ]
Gustavo Fernandes updated ISPN-11710:
-------------------------------------
Description:
The MassIndexer on a clustered cache follows the Transport timeout configuration that is 4 minutes by default. On Local caches, it always uses 4 minutes (hardcoded).
Until it is possible to cancel the MassIndexer (see ISPN-11735), it should not timeout.
was:
The MassIndexer on a clustered cache follows the Transport timeout configuration that is 4 minutes by default. On Local caches, it always uses 4 minutes (hardcoded).
Until it is possible to cancel the MassIndexer (see ISPN-11735)
> MassIndexer timeouts after 4 minutes
> ------------------------------------
>
> Key: ISPN-11710
> URL: https://issues.redhat.com/browse/ISPN-11710
> Project: Infinispan
> Issue Type: Enhancement
> Components: Embedded Querying
> Affects Versions: 11.0.0.Dev04, 10.1.6.Final
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Priority: Major
>
> The MassIndexer on a clustered cache follows the Transport timeout configuration that is 4 minutes by default. On Local caches, it always uses 4 minutes (hardcoded).
> Until it is possible to cancel the MassIndexer (see ISPN-11735), it should not timeout.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (ISPN-11710) MassIndexer timeouts after 4 minutes
by Gustavo Fernandes (Jira)
[ https://issues.redhat.com/browse/ISPN-11710?page=com.atlassian.jira.plugi... ]
Gustavo Fernandes updated ISPN-11710:
-------------------------------------
Description:
The MassIndexer on a clustered cache follows the Transport timeout configuration that is 4 minutes by default. On Local caches, it always uses 4 minutes (hardcoded).
Until it is possible to cancel the MassIndexer (see ISPN-11735)
was:
The MassIndexer on a clustered cache follows the Transport timeout configuration that is 4 minutes by default. On Local caches, it always uses 4 minutes (hardcoded).
Until it is possible to cancel the MassI
> MassIndexer timeouts after 4 minutes
> ------------------------------------
>
> Key: ISPN-11710
> URL: https://issues.redhat.com/browse/ISPN-11710
> Project: Infinispan
> Issue Type: Enhancement
> Components: Embedded Querying
> Affects Versions: 11.0.0.Dev04, 10.1.6.Final
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Priority: Major
>
> The MassIndexer on a clustered cache follows the Transport timeout configuration that is 4 minutes by default. On Local caches, it always uses 4 minutes (hardcoded).
> Until it is possible to cancel the MassIndexer (see ISPN-11735)
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (ISPN-11735) Supporting cancellation in the ClusterExecutor
by Gustavo Fernandes (Jira)
Gustavo Fernandes created ISPN-11735:
----------------------------------------
Summary: Supporting cancellation in the ClusterExecutor
Key: ISPN-11735
URL: https://issues.redhat.com/browse/ISPN-11735
Project: Infinispan
Issue Type: Enhancement
Components: Core
Affects Versions: 11.0.0.Dev04
Reporter: Gustavo Fernandes
Assignee: Will Burns
Currently the ClusterExecutor can timeout and leave commands running in the server, without the possibility of signalling a cancel to them.
One idea is to implement the cancellation logic in the {{CompletableFuture.cancel(boolean ignored)}}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (ISPN-11710) MassIndexer timeouts after 4 minutes
by Gustavo Fernandes (Jira)
[ https://issues.redhat.com/browse/ISPN-11710?page=com.atlassian.jira.plugi... ]
Gustavo Fernandes updated ISPN-11710:
-------------------------------------
Description:
The MassIndexer on a clustered cache follows the Transport timeout configuration that is 4 minutes by default. On Local caches, it always uses 4 minutes (hardcoded).
Until it is possible to cancel the MassI
> MassIndexer timeouts after 4 minutes
> ------------------------------------
>
> Key: ISPN-11710
> URL: https://issues.redhat.com/browse/ISPN-11710
> Project: Infinispan
> Issue Type: Enhancement
> Components: Embedded Querying
> Affects Versions: 11.0.0.Dev04, 10.1.6.Final
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Priority: Major
>
> The MassIndexer on a clustered cache follows the Transport timeout configuration that is 4 minutes by default. On Local caches, it always uses 4 minutes (hardcoded).
> Until it is possible to cancel the MassI
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (ISPN-11202) Infinispan BOM is missing an entry for 'infinispan-jcache-commons'
by Tristan Tarrant (Jira)
[ https://issues.redhat.com/browse/ISPN-11202?page=com.atlassian.jira.plugi... ]
Tristan Tarrant commented on ISPN-11202:
----------------------------------------
I've read the description and it is not clear.
It is a dependency management problem ONLY if you explicitly depend on infinispan-jcache-commons.
> Infinispan BOM is missing an entry for 'infinispan-jcache-commons'
> ------------------------------------------------------------------
>
> Key: ISPN-11202
> URL: https://issues.redhat.com/browse/ISPN-11202
> Project: Infinispan
> Issue Type: Bug
> Components: Build
> Affects Versions: 10.1.1.Final
> Reporter: Stéphane Nicoll
> Assignee: Tristan Tarrant
> Priority: Major
> Fix For: 10.1.7.Final, 11.0.0.Dev05
>
>
> The BOM currently misses an entry for {{infinispan-jcache-commons}} so there is a chance for that module to get out-of-sync even if the bom is used.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (ISPN-11734) Store endpoint topology info for all protocols
by Tristan Tarrant (Jira)
[ https://issues.redhat.com/browse/ISPN-11734?page=com.atlassian.jira.plugi... ]
Tristan Tarrant reassigned ISPN-11734:
--------------------------------------
Assignee: Tristan Tarrant
> Store endpoint topology info for all protocols
> ----------------------------------------------
>
> Key: ISPN-11734
> URL: https://issues.redhat.com/browse/ISPN-11734
> Project: Infinispan
> Issue Type: Enhancement
> Components: Remote Protocols, Server
> Reporter: Tristan Tarrant
> Assignee: Tristan Tarrant
> Priority: Major
>
> Currently, only the Hot Rod server holds a topology cache containing the public addresses of the endpoint.
> This code should be moved to the core protocol server so it can be shared by all protocols.
> This would enable retrieving all of the addresses of the other protocols (i.e. REST) to aid with intelligent clients
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months
[JBoss JIRA] (ISPN-11734) Store endpoint topology info for all protocols
by Tristan Tarrant (Jira)
Tristan Tarrant created ISPN-11734:
--------------------------------------
Summary: Store endpoint topology info for all protocols
Key: ISPN-11734
URL: https://issues.redhat.com/browse/ISPN-11734
Project: Infinispan
Issue Type: Enhancement
Components: Remote Protocols, Server
Reporter: Tristan Tarrant
Currently, only the Hot Rod server holds a topology cache containing the public addresses of the endpoint.
This code should be moved to the core protocol server so it can be shared by all protocols.
This would enable retrieving all of the addresses of the other protocols (i.e. REST) to aid with intelligent clients
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 11 months