[JBoss JIRA] (ISPN-3450) Jon plugin - failing RebuildIndex operation on [MassIndexer] component
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-3450?page=com.atlassian.jira.plugin.... ]
Adrian Nistor updated ISPN-3450:
--------------------------------
Fix Version/s: 7.0.0.CR2
> Jon plugin - failing RebuildIndex operation on [MassIndexer] component
> ----------------------------------------------------------------------
>
> Key: ISPN-3450
> URL: https://issues.jboss.org/browse/ISPN-3450
> Project: Infinispan
> Issue Type: Bug
> Components: JMX, reporting and management
> Reporter: Tomas Sykora
> Assignee: Adrian Nistor
> Fix For: 7.0.0.CR2
>
>
> RebuildIndex operation on [MassIndexer] component is failing while issuing in JON with this error:
> java.lang.NullPointerException
> at org.infinispan.rhq.CacheComponent.invokeOperation(CacheComponent.java:157)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:634)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> I reused configuration from ISPN test suite + I was able to issue this operation successfully via jconsole tool. That's why I suspect that problems are somewhere in JON-ISPN interaction or plugin.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-3450) Jon plugin - failing RebuildIndex operation on [MassIndexer] component
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-3450?page=com.atlassian.jira.plugin.... ]
Work on ISPN-3450 started by Adrian Nistor.
-------------------------------------------
> Jon plugin - failing RebuildIndex operation on [MassIndexer] component
> ----------------------------------------------------------------------
>
> Key: ISPN-3450
> URL: https://issues.jboss.org/browse/ISPN-3450
> Project: Infinispan
> Issue Type: Bug
> Components: JMX, reporting and management
> Reporter: Tomas Sykora
> Assignee: Adrian Nistor
>
> RebuildIndex operation on [MassIndexer] component is failing while issuing in JON with this error:
> java.lang.NullPointerException
> at org.infinispan.rhq.CacheComponent.invokeOperation(CacheComponent.java:157)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:634)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> I reused configuration from ISPN test suite + I was able to issue this operation successfully via jconsole tool. That's why I suspect that problems are somewhere in JON-ISPN interaction or plugin.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-3450) Jon plugin - failing RebuildIndex operation on [MassIndexer] component
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-3450?page=com.atlassian.jira.plugin.... ]
Adrian Nistor updated ISPN-3450:
--------------------------------
Status: Pull Request Sent (was: Coding In Progress)
Git Pull Request: https://github.com/infinispan/infinispan/pull/2960
> Jon plugin - failing RebuildIndex operation on [MassIndexer] component
> ----------------------------------------------------------------------
>
> Key: ISPN-3450
> URL: https://issues.jboss.org/browse/ISPN-3450
> Project: Infinispan
> Issue Type: Bug
> Components: JMX, reporting and management
> Reporter: Tomas Sykora
> Assignee: Adrian Nistor
>
> RebuildIndex operation on [MassIndexer] component is failing while issuing in JON with this error:
> java.lang.NullPointerException
> at org.infinispan.rhq.CacheComponent.invokeOperation(CacheComponent.java:157)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:634)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> I reused configuration from ISPN test suite + I was able to issue this operation successfully via jconsole tool. That's why I suspect that problems are somewhere in JON-ISPN interaction or plugin.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4828) Increasing default internal thread pool size
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4828?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-4828:
-------------------------------
Component/s: Core
> Increasing default internal thread pool size
> --------------------------------------------
>
> Key: ISPN-4828
> URL: https://issues.jboss.org/browse/ISPN-4828
> Project: Infinispan
> Issue Type: Enhancement
> Components: Configuration, Core
> Affects Versions: 7.0.0.CR1
> Reporter: Matej Čimbora
>
> Using synchronous replication with high number of concurrent clients doing put() operations over a shared set of keys, lock-acquisition timeouts occur when various thread pools (internal, jgroups oob) do not have appropriate size.
> org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [3 seconds] on key [key_00000000000003B4] for requestor [Thread[OOB-66,default,node03-12795,5,main]]! Lock held by [Thread[OOB-314,default,node03-12795,5,main]]
> [org.infinispan.interceptors.InvocationContextInterceptor] (Stressor-1) ISPN000136: Execution error
> org.infinispan.util.concurrent.TimeoutException: org.infinispan.util.concurrent.TimeoutException: Node node04-24454 timed out
> This applies to both transactional and non-transactional configuration. The problem can be mitigated by increasing Infinispan's internal thread pool size (defined for remoteCommandsExecutor, blockingBoundedQueueThreadPool). In order to improve user experience either:
> a) When needed, the size of the thread pool should be increased as the load increases
> b) The default values should be high enough to handle even significant load (in terms of number of concurrent clients per node)
> c) The documentation should describe how the end user should size the thread pools based on expected load on the system
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4828) Increasing default internal thread pool size
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4828?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-4828:
-------------------------------
Assignee: (was: Mircea Markus)
> Increasing default internal thread pool size
> --------------------------------------------
>
> Key: ISPN-4828
> URL: https://issues.jboss.org/browse/ISPN-4828
> Project: Infinispan
> Issue Type: Enhancement
> Components: Configuration
> Affects Versions: 7.0.0.CR1
> Reporter: Matej Čimbora
>
> Using synchronous replication with high number of concurrent clients doing put() operations over a shared set of keys, lock-acquisition timeouts occur when various thread pools (internal, jgroups oob) do not have appropriate size.
> org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [3 seconds] on key [key_00000000000003B4] for requestor [Thread[OOB-66,default,node03-12795,5,main]]! Lock held by [Thread[OOB-314,default,node03-12795,5,main]]
> [org.infinispan.interceptors.InvocationContextInterceptor] (Stressor-1) ISPN000136: Execution error
> org.infinispan.util.concurrent.TimeoutException: org.infinispan.util.concurrent.TimeoutException: Node node04-24454 timed out
> This applies to both transactional and non-transactional configuration. The problem can be mitigated by increasing Infinispan's internal thread pool size (defined for remoteCommandsExecutor, blockingBoundedQueueThreadPool). In order to improve user experience either:
> a) When needed, the size of the thread pool should be increased as the load increases
> b) The default values should be high enough to handle even significant load (in terms of number of concurrent clients per node)
> c) The documentation should describe how the end user should size the thread pools based on expected load on the system
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4828) Increasing default internal thread pool size
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4828?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-4828:
-------------------------------
Component/s: Configuration
> Increasing default internal thread pool size
> --------------------------------------------
>
> Key: ISPN-4828
> URL: https://issues.jboss.org/browse/ISPN-4828
> Project: Infinispan
> Issue Type: Enhancement
> Components: Configuration
> Affects Versions: 7.0.0.CR1
> Reporter: Matej Čimbora
> Assignee: Mircea Markus
>
> Using synchronous replication with high number of concurrent clients doing put() operations over a shared set of keys, lock-acquisition timeouts occur when various thread pools (internal, jgroups oob) do not have appropriate size.
> org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [3 seconds] on key [key_00000000000003B4] for requestor [Thread[OOB-66,default,node03-12795,5,main]]! Lock held by [Thread[OOB-314,default,node03-12795,5,main]]
> [org.infinispan.interceptors.InvocationContextInterceptor] (Stressor-1) ISPN000136: Execution error
> org.infinispan.util.concurrent.TimeoutException: org.infinispan.util.concurrent.TimeoutException: Node node04-24454 timed out
> This applies to both transactional and non-transactional configuration. The problem can be mitigated by increasing Infinispan's internal thread pool size (defined for remoteCommandsExecutor, blockingBoundedQueueThreadPool). In order to improve user experience either:
> a) When needed, the size of the thread pool should be increased as the load increases
> b) The default values should be high enough to handle even significant load (in terms of number of concurrent clients per node)
> c) The documentation should describe how the end user should size the thread pools based on expected load on the system
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4828) Increasing default internal thread pool size
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4828?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-4828:
------------------------------------
a) The size of the thread pool already increase and decrease with load, but it is limited to {{max-threads}}.
b) The default {{max-threads}} for the remote executor is indeed quite small (32). We should increase that to at least 200, but keep a small {{core-threads}}.
c) I'm not sure we can do that at this point, but thread usage should become much more predictable with ISPN-2849.
> Increasing default internal thread pool size
> --------------------------------------------
>
> Key: ISPN-4828
> URL: https://issues.jboss.org/browse/ISPN-4828
> Project: Infinispan
> Issue Type: Enhancement
> Components: Configuration
> Affects Versions: 7.0.0.CR1
> Reporter: Matej Čimbora
> Assignee: Mircea Markus
>
> Using synchronous replication with high number of concurrent clients doing put() operations over a shared set of keys, lock-acquisition timeouts occur when various thread pools (internal, jgroups oob) do not have appropriate size.
> org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [3 seconds] on key [key_00000000000003B4] for requestor [Thread[OOB-66,default,node03-12795,5,main]]! Lock held by [Thread[OOB-314,default,node03-12795,5,main]]
> [org.infinispan.interceptors.InvocationContextInterceptor] (Stressor-1) ISPN000136: Execution error
> org.infinispan.util.concurrent.TimeoutException: org.infinispan.util.concurrent.TimeoutException: Node node04-24454 timed out
> This applies to both transactional and non-transactional configuration. The problem can be mitigated by increasing Infinispan's internal thread pool size (defined for remoteCommandsExecutor, blockingBoundedQueueThreadPool). In order to improve user experience either:
> a) When needed, the size of the thread pool should be increased as the load increases
> b) The default values should be high enough to handle even significant load (in terms of number of concurrent clients per node)
> c) The documentation should describe how the end user should size the thread pools based on expected load on the system
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4851) Make SyncConsistentHashFactory the default CH factory
by Dan Berindei (JIRA)
Dan Berindei created ISPN-4851:
----------------------------------
Summary: Make SyncConsistentHashFactory the default CH factory
Key: ISPN-4851
URL: https://issues.jboss.org/browse/ISPN-4851
Project: Infinispan
Issue Type: Feature Request
Components: Configuration, Core
Affects Versions: 7.0.0.CR1
Reporter: Dan Berindei
Fix For: 7.0.0.CR2, 7.0.0.Final
With ISPN-4682 fixed, SyncConsistentHashFactory should be good enough to be the default. It still allows for more variation in the number of owned segments per node (+/-10% owned segments and +/-20% for primary-owned segments), but that should be acceptable for most purposes.
The major advantage of SCHF is that it depends only on the cache members and not on the order they joined. Users expect a key to map to the same node in all caches (as long as the caches have the same members).
One downside of SCHF, especially for testing, is that the segment ownership differs between test runs (being based on the random address assigned to each node). However, most tests that depend on key ownership should use {{ControlledConsistentHashFactory}} anyway.
We also need to verify that the number of segments moved by SCHF is comparable to the number of segments moved by DefaultConsistentHashFactory (ISPN-3729).
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months