[JBoss JIRA] (ISPN-3403) NullPointerException when invoking JMX operation with debug logging enabled
by Fabian Stäber (JIRA)
[ https://issues.jboss.org/browse/ISPN-3403?page=com.atlassian.jira.plugin.... ]
Fabian Stäber updated ISPN-3403:
--------------------------------
Description:
When the CLI client runs the "cache ___defaultcache" command to select a cache, the first things that happens is that the "createSessionId" call is invoked via JMX on the server.
When the command is run for the first time, there is no previously selected cache. Therefore, the "active cache" parameter for the "createSessionId" command is null.
On the server side, in ResourceDMBean.invoke(), there is a debug log message that would log the class of the arguments received using args[i].getClass(). As args[i] is null for the "active cache" parameter the call fails with a NullPointerException.
As a result, it is not possible to select a cache with the CLI client when debug logging is enabled in ResourceDMBean.
---
I will create a very small pull request for this on GitHub.
was:
When the CLI client runs the "cache ___defaultcache" command to select a cache, the first things that happens is that the "createSessionId" call is invoked via JMX on the server.
When the command is run for the first time, there is no previously selected cache. Therefore, the "active cache" parameter for the "createSessionId" command is null.
On the server side, in ResourceDMBean.invoke(), there is a debug logging message that would log the class of the arguments received using args[i].getClass(). As args[i] is null for the "active cache" parameter the call fails with a NullPointerException.
As a result, it is not possible to select a cache with the CLI client when debug logging is enabled in ResourceDMBean.
---
I will create a very small pull request for this on GitHub.
> NullPointerException when invoking JMX operation with debug logging enabled
> ---------------------------------------------------------------------------
>
> Key: ISPN-3403
> URL: https://issues.jboss.org/browse/ISPN-3403
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 6.0.0.Alpha2
> Reporter: Fabian Stäber
> Assignee: Mircea Markus
>
> When the CLI client runs the "cache ___defaultcache" command to select a cache, the first things that happens is that the "createSessionId" call is invoked via JMX on the server.
> When the command is run for the first time, there is no previously selected cache. Therefore, the "active cache" parameter for the "createSessionId" command is null.
> On the server side, in ResourceDMBean.invoke(), there is a debug log message that would log the class of the arguments received using args[i].getClass(). As args[i] is null for the "active cache" parameter the call fails with a NullPointerException.
> As a result, it is not possible to select a cache with the CLI client when debug logging is enabled in ResourceDMBean.
> ---
> I will create a very small pull request for this on GitHub.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 10 months
[JBoss JIRA] (ISPN-3403) NullPointerException when invoking JMX operation with debug logging enabled
by Fabian Stäber (JIRA)
[ https://issues.jboss.org/browse/ISPN-3403?page=com.atlassian.jira.plugin.... ]
Fabian Stäber updated ISPN-3403:
--------------------------------
Description:
When the CLI client runs the "cache ___defaultcache" command to select a cache, the first things that happens is that the "createSessionId" call is invoked via JMX on the server.
When the command is run for the first time, there is no previously selected cache. Therefore, the "active cache" parameter for the "createSessionId" command is null.
On the server side, in ResourceDMBean.invoke(), there is a debug log message that would log the class of the arguments received using args[i].getClass(). As args[i] is null for the "active cache" parameter, the call fails with a NullPointerException.
As a result, it is not possible to select a cache with the CLI client when debug logging is enabled in ResourceDMBean.
---
I will create a very small pull request for this on GitHub.
was:
When the CLI client runs the "cache ___defaultcache" command to select a cache, the first things that happens is that the "createSessionId" call is invoked via JMX on the server.
When the command is run for the first time, there is no previously selected cache. Therefore, the "active cache" parameter for the "createSessionId" command is null.
On the server side, in ResourceDMBean.invoke(), there is a debug log message that would log the class of the arguments received using args[i].getClass(). As args[i] is null for the "active cache" parameter the call fails with a NullPointerException.
As a result, it is not possible to select a cache with the CLI client when debug logging is enabled in ResourceDMBean.
---
I will create a very small pull request for this on GitHub.
> NullPointerException when invoking JMX operation with debug logging enabled
> ---------------------------------------------------------------------------
>
> Key: ISPN-3403
> URL: https://issues.jboss.org/browse/ISPN-3403
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 6.0.0.Alpha2
> Reporter: Fabian Stäber
> Assignee: Mircea Markus
>
> When the CLI client runs the "cache ___defaultcache" command to select a cache, the first things that happens is that the "createSessionId" call is invoked via JMX on the server.
> When the command is run for the first time, there is no previously selected cache. Therefore, the "active cache" parameter for the "createSessionId" command is null.
> On the server side, in ResourceDMBean.invoke(), there is a debug log message that would log the class of the arguments received using args[i].getClass(). As args[i] is null for the "active cache" parameter, the call fails with a NullPointerException.
> As a result, it is not possible to select a cache with the CLI client when debug logging is enabled in ResourceDMBean.
> ---
> I will create a very small pull request for this on GitHub.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 10 months
[JBoss JIRA] (ISPN-3366) Data loss when entry forwarding to primary owner and primary owner shutdown
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-3366?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-3366:
------------------------------------
The missing backup entry is probably related to my fix: if state transfer starts just before we issue a put command, and we send the entries to the new owner just before we commit the entry in the put command, that new owner won't have a backup copy of the key.
State transfer used to automatically forward commands to all the new owners if the topology changed during the execution of the command, but I tried to remove it because the forwarded commands were executed without holding a lock on the primary, so they could lead to inconsistencies.
I'm still trying to come up with a better solution - perhaps sending the command to the backup owners only after the entries are committed on the primary owner will work. But in the meantime I'll just re-add the forwarding in the state transfer interceptor, as locking is not reliable in non-tx caches during state transfer anyway (two nodes could both think they are the primary owner for a key at the same time).
> Data loss when entry forwarding to primary owner and primary owner shutdown
> ---------------------------------------------------------------------------
>
> Key: ISPN-3366
> URL: https://issues.jboss.org/browse/ISPN-3366
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 5.2.4.Final, 6.0.0.Alpha1
> Reporter: Takayoshi Kimura
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 5.2.8.Final, 6.0.0.Alpha3, 6.0.0.Final
>
> Attachments: ISPN-3366-full-logs-3rd.zip, ISPN-3366-full-logs-4th.zip, ISPN-3366-logs.zip
>
>
> Looks like a problem in entry forwarding.
> Here is test scenario:
> * DIST numOwners=2, start with 4 nodes cluster then normal shutdown 1 node during load
> * HotRod putIfAbsent accesses from 40 threads (1 process, 1 remote cache instance), 40000 entries total
> After the test run, the numberOfEntries on each node are:
> * node1: 26608
> * node2: 26622
> * node3: 26746
> * node4: 0
> Total is 79976 and HotRod client received 11 errors, so 79976 + (11 * 2) = 79998. It means 1 entry is completely missing.
> Let's take a look at the missing entry, hash(thread16key59) = 574ff563.
> Current CH: owners(574ff563) are [node4, node1]
> The events sequence is:
> * hotrod -> node1
> * node1 forwarding it to primary owner node4
> * node4 doesn't process the forwarded entry, shutdown
> Result owners(7c29bccb) is [] empty. This entry is completely lost without any errors.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 10 months
[JBoss JIRA] (ISPN-3402) Add JDBC Cache Store config to RHQ plugin
by William Burns (JIRA)
William Burns created ISPN-3402:
-----------------------------------
Summary: Add JDBC Cache Store config to RHQ plugin
Key: ISPN-3402
URL: https://issues.jboss.org/browse/ISPN-3402
Project: Infinispan
Issue Type: Enhancement
Components: Loaders and Stores, Server
Reporter: William Burns
Assignee: Tristan Tarrant
Fix For: 6.0.0.Alpha2
ISPN-3350 was added to enhance some of the RHQ endpoints to allow for better runtime configuration support. However ISPN-3290 is also concurrently changing cache stores. Some changes for ISPN-3290 were mentioned to be possibly changing how the JDBC cache stores are configured and as such this has been excluded from ISPN-3350 to not implement it twice.
This is just to add in the support JDBC cache stores as they are missing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 10 months
[JBoss JIRA] (ISPN-3401) The default map/reduce cache configuration should have state transfer enabled
by Dan Berindei (JIRA)
Dan Berindei created ISPN-3401:
----------------------------------
Summary: The default map/reduce cache configuration should have state transfer enabled
Key: ISPN-3401
URL: https://issues.jboss.org/browse/ISPN-3401
Project: Infinispan
Issue Type: Bug
Components: Distributed Execution and Map/Reduce
Affects Versions: 6.0.0.Alpha2
Reporter: Dan Berindei
Assignee: Mircea Markus
Fix For: 6.0.0.Final
Map/Reduce uses a CreateCacheCommand to create an intermediate cache for the mapper/combiner results. The default configuration for the intermediate cache has fetchInMemoryState disabled, which is probably not a good idea.
CreateCacheCommand also seems too tightly linked to M/R, it contains both the default intermediate cache configuration and a mechanism to check that the cache has started that adds random keys to the cache (and so it can't be re-purposed for creating regular caches).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 10 months
[JBoss JIRA] (ISPN-3366) Data loss when entry forwarding to primary owner and primary owner shutdown
by Takayoshi Kimura (JIRA)
[ https://issues.jboss.org/browse/ISPN-3366?page=com.atlassian.jira.plugin.... ]
Takayoshi Kimura commented on ISPN-3366:
----------------------------------------
Tested 40 times, no complete data loss and missing backup entry 4 times.
If the missing backup issue is not related to ISPN-3366 fix we can close this issue and create another one for the missing backup issue.
> Data loss when entry forwarding to primary owner and primary owner shutdown
> ---------------------------------------------------------------------------
>
> Key: ISPN-3366
> URL: https://issues.jboss.org/browse/ISPN-3366
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 5.2.4.Final, 6.0.0.Alpha1
> Reporter: Takayoshi Kimura
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 5.2.8.Final, 6.0.0.Alpha3, 6.0.0.Final
>
> Attachments: ISPN-3366-full-logs-3rd.zip, ISPN-3366-full-logs-4th.zip, ISPN-3366-logs.zip
>
>
> Looks like a problem in entry forwarding.
> Here is test scenario:
> * DIST numOwners=2, start with 4 nodes cluster then normal shutdown 1 node during load
> * HotRod putIfAbsent accesses from 40 threads (1 process, 1 remote cache instance), 40000 entries total
> After the test run, the numberOfEntries on each node are:
> * node1: 26608
> * node2: 26622
> * node3: 26746
> * node4: 0
> Total is 79976 and HotRod client received 11 errors, so 79976 + (11 * 2) = 79998. It means 1 entry is completely missing.
> Let's take a look at the missing entry, hash(thread16key59) = 574ff563.
> Current CH: owners(574ff563) are [node4, node1]
> The events sequence is:
> * hotrod -> node1
> * node1 forwarding it to primary owner node4
> * node4 doesn't process the forwarded entry, shutdown
> Result owners(7c29bccb) is [] empty. This entry is completely lost without any errors.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months
[JBoss JIRA] (ISPN-3366) Data loss when entry forwarding to primary owner and primary owner shutdown
by Takayoshi Kimura (JIRA)
[ https://issues.jboss.org/browse/ISPN-3366?page=com.atlassian.jira.plugin.... ]
Takayoshi Kimura updated ISPN-3366:
-----------------------------------
Attachment: ISPN-3366-full-logs-4th.zip
Tested with https://github.com/danberindei/infinispan/tree/t_3366_52 5.2.8-SNAPSHOT.
Ran the test 20 times and found 1 missing backup entry but it's not a complete data loss.
hash(thread17key76)=5904ce3d
* hotrod -> node3
* node3 -> node1
* node1 removed this entry due to rebalance
See ISPN-3366-full-logs-4th.zip.
> Data loss when entry forwarding to primary owner and primary owner shutdown
> ---------------------------------------------------------------------------
>
> Key: ISPN-3366
> URL: https://issues.jboss.org/browse/ISPN-3366
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 5.2.4.Final, 6.0.0.Alpha1
> Reporter: Takayoshi Kimura
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 5.2.8.Final, 6.0.0.Alpha3, 6.0.0.Final
>
> Attachments: ISPN-3366-full-logs-3rd.zip, ISPN-3366-full-logs-4th.zip, ISPN-3366-logs.zip
>
>
> Looks like a problem in entry forwarding.
> Here is test scenario:
> * DIST numOwners=2, start with 4 nodes cluster then normal shutdown 1 node during load
> * HotRod putIfAbsent accesses from 40 threads (1 process, 1 remote cache instance), 40000 entries total
> After the test run, the numberOfEntries on each node are:
> * node1: 26608
> * node2: 26622
> * node3: 26746
> * node4: 0
> Total is 79976 and HotRod client received 11 errors, so 79976 + (11 * 2) = 79998. It means 1 entry is completely missing.
> Let's take a look at the missing entry, hash(thread16key59) = 574ff563.
> Current CH: owners(574ff563) are [node4, node1]
> The events sequence is:
> * hotrod -> node1
> * node1 forwarding it to primary owner node4
> * node4 doesn't process the forwarded entry, shutdown
> Result owners(7c29bccb) is [] empty. This entry is completely lost without any errors.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months
[JBoss JIRA] (ISPN-3400) Upgrade to JCache 0.9
by Galder Zamarreño (JIRA)
Galder Zamarreño created ISPN-3400:
--------------------------------------
Summary: Upgrade to JCache 0.9
Key: ISPN-3400
URL: https://issues.jboss.org/browse/ISPN-3400
Project: Infinispan
Issue Type: Component Upgrade
Components: JCache
Reporter: Galder Zamarreño
Assignee: Galder Zamarreño
Fix For: 6.0.0.Alpha3
JCache 0.9 represents the Second Public Draft of JSR-107.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months
[JBoss JIRA] (ISPN-3366) Data loss when entry forwarding to primary owner and primary owner shutdown
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-3366?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-3366:
------------------------------------
[~tkimura], could you try to build again from branch https://github.com/danberindei/infinispan/tree/t_3366_m and see if you can still reproduce the issue?
The fix there is still incomplete in that it breaks some putForExternalRead tests, but it should fix the data loss as it retries the command when the topology changes on the primary owner. BTW, this is the approach that I said wouldn't work in the previous comment - I realized the topology id is set on the originator, so checking for changes on the primary owner should work. The challenge now is to limit the cases where we retry the command, because putForExternalRead is asynchronous and retrying doesn't really work in that case.
> Data loss when entry forwarding to primary owner and primary owner shutdown
> ---------------------------------------------------------------------------
>
> Key: ISPN-3366
> URL: https://issues.jboss.org/browse/ISPN-3366
> Project: Infinispan
> Issue Type: Bug
> Components: Distributed Cache
> Affects Versions: 5.2.4.Final, 6.0.0.Alpha1
> Reporter: Takayoshi Kimura
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 5.2.8.Final, 6.0.0.Alpha3, 6.0.0.Final
>
> Attachments: ISPN-3366-full-logs-3rd.zip, ISPN-3366-logs.zip
>
>
> Looks like a problem in entry forwarding.
> Here is test scenario:
> * DIST numOwners=2, start with 4 nodes cluster then normal shutdown 1 node during load
> * HotRod putIfAbsent accesses from 40 threads (1 process, 1 remote cache instance), 40000 entries total
> After the test run, the numberOfEntries on each node are:
> * node1: 26608
> * node2: 26622
> * node3: 26746
> * node4: 0
> Total is 79976 and HotRod client received 11 errors, so 79976 + (11 * 2) = 79998. It means 1 entry is completely missing.
> Let's take a look at the missing entry, hash(thread16key59) = 574ff563.
> Current CH: owners(574ff563) are [node4, node1]
> The events sequence is:
> * hotrod -> node1
> * node1 forwarding it to primary owner node4
> * node4 doesn't process the forwarded entry, shutdown
> Result owners(7c29bccb) is [] empty. This entry is completely lost without any errors.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months
[JBoss JIRA] (ISPN-1540) Refactor distribution interceptor
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-1540?page=com.atlassian.jira.plugin.... ]
Work on ISPN-1540 stopped by William Burns.
> Refactor distribution interceptor
> ---------------------------------
>
> Key: ISPN-1540
> URL: https://issues.jboss.org/browse/ISPN-1540
> Project: Infinispan
> Issue Type: Feature Request
> Components: Distributed Cache
> Affects Versions: 5.1.0.BETA5
> Reporter: Mircea Markus
> Assignee: William Burns
> Fix For: 6.0.0.Final
>
>
> DistributionInterceptor, as it looks now is unnecessary complex. Before adding more functionality on top of it (i.e. ISPN-1539) it should be refactored:
> - extract L1 logic into a different interceptor
> - this would require moving the StateTransferLock logic into another interceptor as well
> - now that we have separation between tx and non-tx caches, we can extract the remaining logic into TransactionalDistributionInterceptor and NonTransactional...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 11 months