[JBoss JIRA] Created: (JGRP-594) Intermittently, a Null pointer exception is thrown when trying to remove non-existent entry in ReplicatedHashMap
10:34 a.m.
Intermittently, a Null pointer exception is thrown when trying to remove non-existent
entry in ReplicatedHashMap
----------------------------------------------------------------------------------------------------------------
Key: JGRP-594
URL: http://jira.jboss.com/jira/browse/JGRP-594
Project: JGroups
Issue Type: Bug
Affects Versions: 2.5
Environment: Linux
Reporter: Dipak Kothari
Assigned To: Bela Ban
Intermittently, when an entry is removed from a ReplicatedHashMap (where the entry does
not exist) the following exception is thrown:
java.lang.RuntimeException: remove(APMExample.Services.examples.ServerA09) failed
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:405)
at
com.ubs.apm.control.service.nameservice.jgroup.JGroupNameService.unRegisterService(JGroupNameService.java:132)
at
com.ubs.apm.control.sensors.ControlSensorManager.cleanup(ControlSensorManager.java:468)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:125)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:106)
at com.ubs.apm.control.example.ManagedServer.init(ManagedServer.java:20)
at com.ubs.apm.control.example.ManagedServer.main(ManagedServer.java:45)
Caused by: java.lang.RuntimeException: failed executing request [req_id=1189617222436
caller=14.64.61.201:6825
14.64.61.201:6838: sender=14.64.61.201:6838, retval=null, received=false, suspected=false
.... many such lines ...
14.64.61.201:6860: sender=14.64.61.201:6860, retval=null, received=false, suspected=false
14.64.61.201:6847: sender=14.64.61.201:6847, retval=null, received=false, suspected=false
request_msg: [dst: <null>, src: 14.64.61.201:6825 (2 headers), size=143 bytes]
rsp_mode: GET_NONE
done: true
timeout: 5000
expected_mbrs: 0 ([14.64.61.201:6815, 14.64.61.201:6816, 14.64.61.201:6824,
14.64.61.201:6825, 14.64.61.201:6826, 14.64.61.201:6827, 14.64.61.201:6828,
14.64.61.201:6829, 14.64.61.201:6830, 14.64.61.201:6831, 14.64.61.201:6833,
14.64.61.201:6834, 14.64.61.201:6835, 14.64.61.201:6836, 14.64.61.201:6837,
14.64.61.201:6838, 14.64.61.201:6839, 14.64.61.201:6842, 14.64.61.201:6843,
14.64.61.201:6844, 14.64.61.201:6845, 14.64.61.201:6846, 14.64.61.201:6847,
14.64.61.201:6848, 14.64.61.201:6849, 14.64.61.201:6850, 14.64.61.201:6851,
14.64.61.201:6852, 14.64.61.201:6853, 14.64.61.201:6854, 14.64.61.201:6855,
14.64.61.201:6856, 14.64.61.201:6857, 14.64.61.201:6858, 14.64.61.201:6859,
14.64.61.201:6860, 14.64.61.201:6861, 14.64.61.201:6862, 14.64.61.201:6863,
14.64.61.201:6864, 14.64.61.201:6865, 14.64.61.201:6866])]
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:433)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:199)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:167)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:163)
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:402)
... 6 more
Caused by: java.lang.RuntimeException: failure adding msg [dst: <null>, src:
14.64.61.201:6825 (2 headers), size=143 bytes] to the retransmit table for
14.64.61.201:6825
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:636)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:438)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:317)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:782)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:221)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:339)
at org.jgroups.JChannel.downcall(JChannel.java:1240)
at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:752)
at org.jgroups.blocks.RequestCorrelator.sendRequest(RequestCorrelator.java:301)
at org.jgroups.blocks.GroupRequest.doExecute(GroupRequest.java:440)
at org.jgroups.blocks.GroupRequest.execute(GroupRequest.java:190)
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:430)
... 10 more
Caused by: java.lang.NullPointerException
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:632)
... 21 more
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:08 a.m.
New subject: [JBoss JIRA] Updated: (JGRP-594) Intermittently, a Null pointer exception is thrown when trying to remove non-existent entry in ReplicatedHashMap
[ http://jira.jboss.com/jira/browse/JGRP-594?page=all ]
Bela Ban updated JGRP-594:
--------------------------
Fix Version/s: 2.6
Intermittently, a Null pointer exception is thrown when trying to
remove non-existent entry in ReplicatedHashMap
----------------------------------------------------------------------------------------------------------------
Key: JGRP-594
URL: http://jira.jboss.com/jira/browse/JGRP-594
Project: JGroups
Issue Type: Bug
Affects Versions: 2.5
Environment: Linux
Reporter: Dipak Kothari
Assigned To: Bela Ban
Fix For: 2.6
Intermittently, when an entry is removed from a ReplicatedHashMap (where the entry does
not exist) the following exception is thrown:
java.lang.RuntimeException: remove(APMExample.Services.examples.ServerA09) failed
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:405)
at
com.ubs.apm.control.service.nameservice.jgroup.JGroupNameService.unRegisterService(JGroupNameService.java:132)
at
com.ubs.apm.control.sensors.ControlSensorManager.cleanup(ControlSensorManager.java:468)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:125)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:106)
at com.ubs.apm.control.example.ManagedServer.init(ManagedServer.java:20)
at com.ubs.apm.control.example.ManagedServer.main(ManagedServer.java:45)
Caused by: java.lang.RuntimeException: failed executing request [req_id=1189617222436
caller=14.64.61.201:6825
14.64.61.201:6838: sender=14.64.61.201:6838, retval=null, received=false,
suspected=false
.... many such lines ...
14.64.61.201:6860: sender=14.64.61.201:6860, retval=null, received=false,
suspected=false
14.64.61.201:6847: sender=14.64.61.201:6847, retval=null, received=false,
suspected=false
request_msg: [dst: <null>, src: 14.64.61.201:6825 (2 headers), size=143 bytes]
rsp_mode: GET_NONE
done: true
timeout: 5000
expected_mbrs: 0 ([14.64.61.201:6815, 14.64.61.201:6816, 14.64.61.201:6824,
14.64.61.201:6825, 14.64.61.201:6826, 14.64.61.201:6827, 14.64.61.201:6828,
14.64.61.201:6829, 14.64.61.201:6830, 14.64.61.201:6831, 14.64.61.201:6833,
14.64.61.201:6834, 14.64.61.201:6835, 14.64.61.201:6836, 14.64.61.201:6837,
14.64.61.201:6838, 14.64.61.201:6839, 14.64.61.201:6842, 14.64.61.201:6843,
14.64.61.201:6844, 14.64.61.201:6845, 14.64.61.201:6846, 14.64.61.201:6847,
14.64.61.201:6848, 14.64.61.201:6849, 14.64.61.201:6850, 14.64.61.201:6851,
14.64.61.201:6852, 14.64.61.201:6853, 14.64.61.201:6854, 14.64.61.201:6855,
14.64.61.201:6856, 14.64.61.201:6857, 14.64.61.201:6858, 14.64.61.201:6859,
14.64.61.201:6860, 14.64.61.201:6861, 14.64.61.201:6862, 14.64.61.201:6863,
14.64.61.201:6864, 14.64.61.201:6865, 14.64.61.201:6866])]
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:433)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:199)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:167)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:163)
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:402)
... 6 more
Caused by: java.lang.RuntimeException: failure adding msg [dst: <null>, src:
14.64.61.201:6825 (2 headers), size=143 bytes] to the retransmit table for
14.64.61.201:6825
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:636)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:438)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:317)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:782)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:221)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:339)
at org.jgroups.JChannel.downcall(JChannel.java:1240)
at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:752)
at org.jgroups.blocks.RequestCorrelator.sendRequest(RequestCorrelator.java:301)
at org.jgroups.blocks.GroupRequest.doExecute(GroupRequest.java:440)
at org.jgroups.blocks.GroupRequest.execute(GroupRequest.java:190)
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:430)
... 10 more
Caused by: java.lang.NullPointerException
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:632)
... 21 more
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:40 a.m.
New subject: [JBoss JIRA] Commented: (JGRP-594) Intermittently, a Null pointer exception is thrown when trying to remove non-existent entry in ReplicatedHashMap
[ http://jira.jboss.com/jira/browse/JGRP-594?page=comments#action_12376802 ]
Bela Ban commented on JGRP-594:
-------------------------------
This can only happen when the sender of a message is not part of the retransmission table,
on JOIN and state transfer.
- Can you describe exactly when this happened ? Was it directly after a member joined (or
did a state transfer) ?
- Can you attach your config XML to this case ?
- I added some code to the client side JOIN logic to throw an exception if this happens.
Can you try this with CVS head (2.6) ?
Intermittently, a Null pointer exception is thrown when trying to
remove non-existent entry in ReplicatedHashMap
----------------------------------------------------------------------------------------------------------------
Key: JGRP-594
URL: http://jira.jboss.com/jira/browse/JGRP-594
Project: JGroups
Issue Type: Bug
Affects Versions: 2.5
Environment: Linux
Reporter: Dipak Kothari
Assigned To: Bela Ban
Fix For: 2.6
Intermittently, when an entry is removed from a ReplicatedHashMap (where the entry does
not exist) the following exception is thrown:
java.lang.RuntimeException: remove(APMExample.Services.examples.ServerA09) failed
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:405)
at
com.ubs.apm.control.service.nameservice.jgroup.JGroupNameService.unRegisterService(JGroupNameService.java:132)
at
com.ubs.apm.control.sensors.ControlSensorManager.cleanup(ControlSensorManager.java:468)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:125)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:106)
at com.ubs.apm.control.example.ManagedServer.init(ManagedServer.java:20)
at com.ubs.apm.control.example.ManagedServer.main(ManagedServer.java:45)
Caused by: java.lang.RuntimeException: failed executing request [req_id=1189617222436
caller=14.64.61.201:6825
14.64.61.201:6838: sender=14.64.61.201:6838, retval=null, received=false,
suspected=false
.... many such lines ...
14.64.61.201:6860: sender=14.64.61.201:6860, retval=null, received=false,
suspected=false
14.64.61.201:6847: sender=14.64.61.201:6847, retval=null, received=false,
suspected=false
request_msg: [dst: <null>, src: 14.64.61.201:6825 (2 headers), size=143 bytes]
rsp_mode: GET_NONE
done: true
timeout: 5000
expected_mbrs: 0 ([14.64.61.201:6815, 14.64.61.201:6816, 14.64.61.201:6824,
14.64.61.201:6825, 14.64.61.201:6826, 14.64.61.201:6827, 14.64.61.201:6828,
14.64.61.201:6829, 14.64.61.201:6830, 14.64.61.201:6831, 14.64.61.201:6833,
14.64.61.201:6834, 14.64.61.201:6835, 14.64.61.201:6836, 14.64.61.201:6837,
14.64.61.201:6838, 14.64.61.201:6839, 14.64.61.201:6842, 14.64.61.201:6843,
14.64.61.201:6844, 14.64.61.201:6845, 14.64.61.201:6846, 14.64.61.201:6847,
14.64.61.201:6848, 14.64.61.201:6849, 14.64.61.201:6850, 14.64.61.201:6851,
14.64.61.201:6852, 14.64.61.201:6853, 14.64.61.201:6854, 14.64.61.201:6855,
14.64.61.201:6856, 14.64.61.201:6857, 14.64.61.201:6858, 14.64.61.201:6859,
14.64.61.201:6860, 14.64.61.201:6861, 14.64.61.201:6862, 14.64.61.201:6863,
14.64.61.201:6864, 14.64.61.201:6865, 14.64.61.201:6866])]
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:433)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:199)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:167)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:163)
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:402)
... 6 more
Caused by: java.lang.RuntimeException: failure adding msg [dst: <null>, src:
14.64.61.201:6825 (2 headers), size=143 bytes] to the retransmit table for
14.64.61.201:6825
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:636)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:438)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:317)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:782)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:221)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:339)
at org.jgroups.JChannel.downcall(JChannel.java:1240)
at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:752)
at org.jgroups.blocks.RequestCorrelator.sendRequest(RequestCorrelator.java:301)
at org.jgroups.blocks.GroupRequest.doExecute(GroupRequest.java:440)
at org.jgroups.blocks.GroupRequest.execute(GroupRequest.java:190)
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:430)
... 10 more
Caused by: java.lang.NullPointerException
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:632)
... 21 more
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:19 p.m.
New subject: [JBoss JIRA] Commented: (JGRP-594) Intermittently, a Null pointer exception is thrown when trying to remove non-existent entry in ReplicatedHashMap
[ http://jira.jboss.com/jira/browse/JGRP-594?page=comments#action_12376813 ]
Dipak Kothari commented on JGRP-594:
------------------------------------
The initialisation of my service intialises a number of items including joing JGroup. If
an exception is thrown then it does some clean up including removing it from the
replicatedHashMap. So my code looked something like this:
catch(Exception ex) {
cleanup();
throw ex;
}
Unfortunately, I could see what caused the initial exception because I got a runtime
exception with the cleanup. However, I can confirm that it did not register (join the
gorup).
My configuration:
<config>
<TCP
start_port="${start_port}"
bind_addr="${bind_addr}"
loopback="true"
discard_incompatible_packets="true"
use_send_queues="false"
max_bundle_size="64000"
max_bundle_timeout="30"
enable_bundling="false"
sock_conn_timeout="300"
skip_suspected_members="true"
enable_diagnostics="false"
use_concurrent_stack="true"
thread_pool.enabled="true"
thread_pool.min_threads="1"
thread_pool.max_threads="8"
thread_pool.keep_alive_time="5000"
thread_pool.queue_enabled="true"
thread_pool.queue_max_size="100"
thread_pool.rejection_policy="Run"
oob_thread_pool.enabled="true"
oob_thread_pool.min_threads="1"
oob_thread_pool.max_threads="8"
oob_thread_pool.keep_alive_time="5000"
oob_thread_pool.queue_enabled="true"
oob_thread_pool.queue_max_size="100"
oob_thread_pool.rejection_policy="Run"
/>
<TCPPING
timeout="3000"
initial_hosts="${initial_hosts}"
num_initial_members="1"
port_range="${port_range}"/>
<MERGE2
max_interval="30000"
min_interval="10000"/>
<FD_SOCK/>
<VERIFY_SUSPECT
timeout="1000"/>
<pbcast.NAKACK
gc_lag="50"
retransmit_timeout="300,600,1200,2400,4800"
use_mcast_xmit="false"
max_xmit_size="64000"
discard_delivered_msgs="false"/>
<pbcast.STABLE
stability_delay="1000"
desired_avg_gossip="20000"/>
<pbcast.GMS
print_local_addr="true"
join_timeout="3000"
join_retry_timeout="5000"
view_bundling="true"
shun="false"/>
<pbcast.STATE_TRANSFER/>
</config>
I will try and use the code 2.6 head and will check - however this may take some time.
Intermittently, a Null pointer exception is thrown when trying to
remove non-existent entry in ReplicatedHashMap
----------------------------------------------------------------------------------------------------------------
Key: JGRP-594
URL: http://jira.jboss.com/jira/browse/JGRP-594
Project: JGroups
Issue Type: Bug
Affects Versions: 2.5
Environment: Linux
Reporter: Dipak Kothari
Assigned To: Bela Ban
Fix For: 2.6
Intermittently, when an entry is removed from a ReplicatedHashMap (where the entry does
not exist) the following exception is thrown:
java.lang.RuntimeException: remove(APMExample.Services.examples.ServerA09) failed
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:405)
at
com.ubs.apm.control.service.nameservice.jgroup.JGroupNameService.unRegisterService(JGroupNameService.java:132)
at
com.ubs.apm.control.sensors.ControlSensorManager.cleanup(ControlSensorManager.java:468)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:125)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:106)
at com.ubs.apm.control.example.ManagedServer.init(ManagedServer.java:20)
at com.ubs.apm.control.example.ManagedServer.main(ManagedServer.java:45)
Caused by: java.lang.RuntimeException: failed executing request [req_id=1189617222436
caller=14.64.61.201:6825
14.64.61.201:6838: sender=14.64.61.201:6838, retval=null, received=false,
suspected=false
.... many such lines ...
14.64.61.201:6860: sender=14.64.61.201:6860, retval=null, received=false,
suspected=false
14.64.61.201:6847: sender=14.64.61.201:6847, retval=null, received=false,
suspected=false
request_msg: [dst: <null>, src: 14.64.61.201:6825 (2 headers), size=143 bytes]
rsp_mode: GET_NONE
done: true
timeout: 5000
expected_mbrs: 0 ([14.64.61.201:6815, 14.64.61.201:6816, 14.64.61.201:6824,
14.64.61.201:6825, 14.64.61.201:6826, 14.64.61.201:6827, 14.64.61.201:6828,
14.64.61.201:6829, 14.64.61.201:6830, 14.64.61.201:6831, 14.64.61.201:6833,
14.64.61.201:6834, 14.64.61.201:6835, 14.64.61.201:6836, 14.64.61.201:6837,
14.64.61.201:6838, 14.64.61.201:6839, 14.64.61.201:6842, 14.64.61.201:6843,
14.64.61.201:6844, 14.64.61.201:6845, 14.64.61.201:6846, 14.64.61.201:6847,
14.64.61.201:6848, 14.64.61.201:6849, 14.64.61.201:6850, 14.64.61.201:6851,
14.64.61.201:6852, 14.64.61.201:6853, 14.64.61.201:6854, 14.64.61.201:6855,
14.64.61.201:6856, 14.64.61.201:6857, 14.64.61.201:6858, 14.64.61.201:6859,
14.64.61.201:6860, 14.64.61.201:6861, 14.64.61.201:6862, 14.64.61.201:6863,
14.64.61.201:6864, 14.64.61.201:6865, 14.64.61.201:6866])]
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:433)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:199)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:167)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:163)
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:402)
... 6 more
Caused by: java.lang.RuntimeException: failure adding msg [dst: <null>, src:
14.64.61.201:6825 (2 headers), size=143 bytes] to the retransmit table for
14.64.61.201:6825
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:636)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:438)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:317)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:782)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:221)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:339)
at org.jgroups.JChannel.downcall(JChannel.java:1240)
at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:752)
at org.jgroups.blocks.RequestCorrelator.sendRequest(RequestCorrelator.java:301)
at org.jgroups.blocks.GroupRequest.doExecute(GroupRequest.java:440)
at org.jgroups.blocks.GroupRequest.execute(GroupRequest.java:190)
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:430)
... 10 more
Caused by: java.lang.NullPointerException
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:632)
... 21 more
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:16 a.m.
New subject: [JBoss JIRA] Updated: (JGRP-594) Intermittently, a Null pointer exception is thrown when trying to remove non-existent entry in ReplicatedHashMap
[ http://jira.jboss.com/jira/browse/JGRP-594?page=all ]
Bela Ban updated JGRP-594:
--------------------------
Fix Version/s: 2.7
(was: 2.6)
Okay, I added some code to 2.6 to (hopefully !) catch the situation when a local member is
itself not part of the digest (in set/mergeDigest() in NAKACK). This is essentially log
warnings, so if you run into this again, you should see WARN messages (run with log4j at
the WARN level for org.jgroups).
I'll move this issue to 2.7, and keep it open. Please add new comments when you run
into the issue again.
Intermittently, a Null pointer exception is thrown when trying to
remove non-existent entry in ReplicatedHashMap
----------------------------------------------------------------------------------------------------------------
Key: JGRP-594
URL: http://jira.jboss.com/jira/browse/JGRP-594
Project: JGroups
Issue Type: Bug
Affects Versions: 2.5
Environment: Linux
Reporter: Dipak Kothari
Assigned To: Bela Ban
Fix For: 2.7
Intermittently, when an entry is removed from a ReplicatedHashMap (where the entry does
not exist) the following exception is thrown:
java.lang.RuntimeException: remove(APMExample.Services.examples.ServerA09) failed
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:405)
at
com.ubs.apm.control.service.nameservice.jgroup.JGroupNameService.unRegisterService(JGroupNameService.java:132)
at
com.ubs.apm.control.sensors.ControlSensorManager.cleanup(ControlSensorManager.java:468)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:125)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:106)
at com.ubs.apm.control.example.ManagedServer.init(ManagedServer.java:20)
at com.ubs.apm.control.example.ManagedServer.main(ManagedServer.java:45)
Caused by: java.lang.RuntimeException: failed executing request [req_id=1189617222436
caller=14.64.61.201:6825
14.64.61.201:6838: sender=14.64.61.201:6838, retval=null, received=false,
suspected=false
.... many such lines ...
14.64.61.201:6860: sender=14.64.61.201:6860, retval=null, received=false,
suspected=false
14.64.61.201:6847: sender=14.64.61.201:6847, retval=null, received=false,
suspected=false
request_msg: [dst: <null>, src: 14.64.61.201:6825 (2 headers), size=143 bytes]
rsp_mode: GET_NONE
done: true
timeout: 5000
expected_mbrs: 0 ([14.64.61.201:6815, 14.64.61.201:6816, 14.64.61.201:6824,
14.64.61.201:6825, 14.64.61.201:6826, 14.64.61.201:6827, 14.64.61.201:6828,
14.64.61.201:6829, 14.64.61.201:6830, 14.64.61.201:6831, 14.64.61.201:6833,
14.64.61.201:6834, 14.64.61.201:6835, 14.64.61.201:6836, 14.64.61.201:6837,
14.64.61.201:6838, 14.64.61.201:6839, 14.64.61.201:6842, 14.64.61.201:6843,
14.64.61.201:6844, 14.64.61.201:6845, 14.64.61.201:6846, 14.64.61.201:6847,
14.64.61.201:6848, 14.64.61.201:6849, 14.64.61.201:6850, 14.64.61.201:6851,
14.64.61.201:6852, 14.64.61.201:6853, 14.64.61.201:6854, 14.64.61.201:6855,
14.64.61.201:6856, 14.64.61.201:6857, 14.64.61.201:6858, 14.64.61.201:6859,
14.64.61.201:6860, 14.64.61.201:6861, 14.64.61.201:6862, 14.64.61.201:6863,
14.64.61.201:6864, 14.64.61.201:6865, 14.64.61.201:6866])]
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:433)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:199)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:167)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:163)
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:402)
... 6 more
Caused by: java.lang.RuntimeException: failure adding msg [dst: <null>, src:
14.64.61.201:6825 (2 headers), size=143 bytes] to the retransmit table for
14.64.61.201:6825
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:636)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:438)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:317)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:782)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:221)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:339)
at org.jgroups.JChannel.downcall(JChannel.java:1240)
at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:752)
at org.jgroups.blocks.RequestCorrelator.sendRequest(RequestCorrelator.java:301)
at org.jgroups.blocks.GroupRequest.doExecute(GroupRequest.java:440)
at org.jgroups.blocks.GroupRequest.execute(GroupRequest.java:190)
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:430)
... 10 more
Caused by: java.lang.NullPointerException
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:632)
... 21 more
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:33 p.m.
New subject: [JBoss JIRA] Commented: (JGRP-594) Intermittently, a Null pointer exception is thrown when trying to remove non-existent entry in ReplicatedHashMap
[ http://jira.jboss.com/jira/browse/JGRP-594?page=comments#action_12391935 ]
Dipak Kothari commented on JGRP-594:
------------------------------------
Hi,
I have worked on this issue again. As this problem is intermitten, i created a test which
ran over 12-24 hours, capturing the output.
Details of test:
Started 60 services, each adding (put) entries into the ReplicatedHashMap<String,
String>. After, 3 minutes, the logs were checked for a null pointer exception. if the
exception was found, it would take a back up of the logs. The services would be killed
using "kill -9) and the process repeated.
Snippet of code:
registry = new ReplicatedHashMap(GROUP, factory, properties, false, 10*1000);
LOGGER.debug("Group memebership = " +
registry.getChannel().getView().getMembers().size());
localAddress = registry.getLocalAddress();
LOGGER.debug("Local address = " + localAddress);
channel = (JChannel)registry.getChannel();
MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
JmxConfigurator.registerChannel(channel, mbs, "JGroups.channel",
channel.getChannelName(), true);
registry.addNotifier(new RHMListener());
dumpMap();
registry.put(service, "STARTING");
LOGGER.debug("Successfully put value");
double delay = Math.random();
Thread.sleep((long)(delay*5000));
registry.put(service, "RUNNING");
LOGGER.debug("Updated state of service ...");
while(true) {
try {
Thread.sleep(10*1000);
LOGGER.debug("Group membership = " +
registry.getChannel().getView().getMembers().size());
dumpMap();
}
catch(Exception ex) {
LOGGER.debug("Failed to get size details", ex);
}
}
This was done for JGroup 2.5.0 (our current version) and with JGroup 2.6.1. With 2.5.0,
we got 2 occurences in a 12 hour run. With 2.6.1 I got it to occur once in a 24 hour run
(although it happened after 4-5 hours). I also get a number of Errors reported.
I have attached the log file for 2.6.1 run.
Let me know if you require any other information.
Thanks.
Intermittently, a Null pointer exception is thrown when trying to
remove non-existent entry in ReplicatedHashMap
----------------------------------------------------------------------------------------------------------------
Key: JGRP-594
URL: http://jira.jboss.com/jira/browse/JGRP-594
Project: JGroups
Issue Type: Bug
Affects Versions: 2.5
Environment: Linux
Reporter: Dipak Kothari
Assigned To: Bela Ban
Fix For: 2.7
Intermittently, when an entry is removed from a ReplicatedHashMap (where the entry does
not exist) the following exception is thrown:
java.lang.RuntimeException: remove(APMExample.Services.examples.ServerA09) failed
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:405)
at
com.ubs.apm.control.service.nameservice.jgroup.JGroupNameService.unRegisterService(JGroupNameService.java:132)
at
com.ubs.apm.control.sensors.ControlSensorManager.cleanup(ControlSensorManager.java:468)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:125)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:106)
at com.ubs.apm.control.example.ManagedServer.init(ManagedServer.java:20)
at com.ubs.apm.control.example.ManagedServer.main(ManagedServer.java:45)
Caused by: java.lang.RuntimeException: failed executing request [req_id=1189617222436
caller=14.64.61.201:6825
14.64.61.201:6838: sender=14.64.61.201:6838, retval=null, received=false,
suspected=false
.... many such lines ...
14.64.61.201:6860: sender=14.64.61.201:6860, retval=null, received=false,
suspected=false
14.64.61.201:6847: sender=14.64.61.201:6847, retval=null, received=false,
suspected=false
request_msg: [dst: <null>, src: 14.64.61.201:6825 (2 headers), size=143 bytes]
rsp_mode: GET_NONE
done: true
timeout: 5000
expected_mbrs: 0 ([14.64.61.201:6815, 14.64.61.201:6816, 14.64.61.201:6824,
14.64.61.201:6825, 14.64.61.201:6826, 14.64.61.201:6827, 14.64.61.201:6828,
14.64.61.201:6829, 14.64.61.201:6830, 14.64.61.201:6831, 14.64.61.201:6833,
14.64.61.201:6834, 14.64.61.201:6835, 14.64.61.201:6836, 14.64.61.201:6837,
14.64.61.201:6838, 14.64.61.201:6839, 14.64.61.201:6842, 14.64.61.201:6843,
14.64.61.201:6844, 14.64.61.201:6845, 14.64.61.201:6846, 14.64.61.201:6847,
14.64.61.201:6848, 14.64.61.201:6849, 14.64.61.201:6850, 14.64.61.201:6851,
14.64.61.201:6852, 14.64.61.201:6853, 14.64.61.201:6854, 14.64.61.201:6855,
14.64.61.201:6856, 14.64.61.201:6857, 14.64.61.201:6858, 14.64.61.201:6859,
14.64.61.201:6860, 14.64.61.201:6861, 14.64.61.201:6862, 14.64.61.201:6863,
14.64.61.201:6864, 14.64.61.201:6865, 14.64.61.201:6866])]
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:433)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:199)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:167)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:163)
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:402)
... 6 more
Caused by: java.lang.RuntimeException: failure adding msg [dst: <null>, src:
14.64.61.201:6825 (2 headers), size=143 bytes] to the retransmit table for
14.64.61.201:6825
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:636)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:438)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:317)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:782)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:221)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:339)
at org.jgroups.JChannel.downcall(JChannel.java:1240)
at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:752)
at org.jgroups.blocks.RequestCorrelator.sendRequest(RequestCorrelator.java:301)
at org.jgroups.blocks.GroupRequest.doExecute(GroupRequest.java:440)
at org.jgroups.blocks.GroupRequest.execute(GroupRequest.java:190)
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:430)
... 10 more
Caused by: java.lang.NullPointerException
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:632)
... 21 more
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:35 p.m.
New subject: [JBoss JIRA] Updated: (JGRP-594) Intermittently, a Null pointer exception is thrown when trying to remove non-existent entry in ReplicatedHashMap
[ http://jira.jboss.com/jira/browse/JGRP-594?page=all ]
Dipak Kothari updated JGRP-594:
-------------------------------
Attachment: Server2.log
Logfile associated with the failed server.
Intermittently, a Null pointer exception is thrown when trying to
remove non-existent entry in ReplicatedHashMap
----------------------------------------------------------------------------------------------------------------
Key: JGRP-594
URL: http://jira.jboss.com/jira/browse/JGRP-594
Project: JGroups
Issue Type: Bug
Affects Versions: 2.5
Environment: Linux
Reporter: Dipak Kothari
Assigned To: Bela Ban
Fix For: 2.7
Attachments: Server2.log
Intermittently, when an entry is removed from a ReplicatedHashMap (where the entry does
not exist) the following exception is thrown:
java.lang.RuntimeException: remove(APMExample.Services.examples.ServerA09) failed
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:405)
at
com.ubs.apm.control.service.nameservice.jgroup.JGroupNameService.unRegisterService(JGroupNameService.java:132)
at
com.ubs.apm.control.sensors.ControlSensorManager.cleanup(ControlSensorManager.java:468)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:125)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:106)
at com.ubs.apm.control.example.ManagedServer.init(ManagedServer.java:20)
at com.ubs.apm.control.example.ManagedServer.main(ManagedServer.java:45)
Caused by: java.lang.RuntimeException: failed executing request [req_id=1189617222436
caller=14.64.61.201:6825
14.64.61.201:6838: sender=14.64.61.201:6838, retval=null, received=false,
suspected=false
.... many such lines ...
14.64.61.201:6860: sender=14.64.61.201:6860, retval=null, received=false,
suspected=false
14.64.61.201:6847: sender=14.64.61.201:6847, retval=null, received=false,
suspected=false
request_msg: [dst: <null>, src: 14.64.61.201:6825 (2 headers), size=143 bytes]
rsp_mode: GET_NONE
done: true
timeout: 5000
expected_mbrs: 0 ([14.64.61.201:6815, 14.64.61.201:6816, 14.64.61.201:6824,
14.64.61.201:6825, 14.64.61.201:6826, 14.64.61.201:6827, 14.64.61.201:6828,
14.64.61.201:6829, 14.64.61.201:6830, 14.64.61.201:6831, 14.64.61.201:6833,
14.64.61.201:6834, 14.64.61.201:6835, 14.64.61.201:6836, 14.64.61.201:6837,
14.64.61.201:6838, 14.64.61.201:6839, 14.64.61.201:6842, 14.64.61.201:6843,
14.64.61.201:6844, 14.64.61.201:6845, 14.64.61.201:6846, 14.64.61.201:6847,
14.64.61.201:6848, 14.64.61.201:6849, 14.64.61.201:6850, 14.64.61.201:6851,
14.64.61.201:6852, 14.64.61.201:6853, 14.64.61.201:6854, 14.64.61.201:6855,
14.64.61.201:6856, 14.64.61.201:6857, 14.64.61.201:6858, 14.64.61.201:6859,
14.64.61.201:6860, 14.64.61.201:6861, 14.64.61.201:6862, 14.64.61.201:6863,
14.64.61.201:6864, 14.64.61.201:6865, 14.64.61.201:6866])]
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:433)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:199)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:167)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:163)
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:402)
... 6 more
Caused by: java.lang.RuntimeException: failure adding msg [dst: <null>, src:
14.64.61.201:6825 (2 headers), size=143 bytes] to the retransmit table for
14.64.61.201:6825
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:636)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:438)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:317)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:782)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:221)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:339)
at org.jgroups.JChannel.downcall(JChannel.java:1240)
at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:752)
at org.jgroups.blocks.RequestCorrelator.sendRequest(RequestCorrelator.java:301)
at org.jgroups.blocks.GroupRequest.doExecute(GroupRequest.java:440)
at org.jgroups.blocks.GroupRequest.execute(GroupRequest.java:190)
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:430)
... 10 more
Caused by: java.lang.NullPointerException
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:632)
... 21 more
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:37 p.m.
New subject: [JBoss JIRA] Commented: (JGRP-594) Intermittently, a Null pointer exception is thrown when trying to remove non-existent entry in ReplicatedHashMap
[ http://jira.jboss.com/jira/browse/JGRP-594?page=comments#action_12391937 ]
Dipak Kothari commented on JGRP-594:
------------------------------------
The error mentioned above:
2007-12-12 13:26:58,695 [Timer,null,null] ERROR org.jgroups.protocols.pbcast.NAKACK -
sender 14.64.61.201:9810 not found in xmit_table. Is this serious?
Intermittently, a Null pointer exception is thrown when trying to
remove non-existent entry in ReplicatedHashMap
----------------------------------------------------------------------------------------------------------------
Key: JGRP-594
URL: http://jira.jboss.com/jira/browse/JGRP-594
Project: JGroups
Issue Type: Bug
Affects Versions: 2.5
Environment: Linux
Reporter: Dipak Kothari
Assigned To: Bela Ban
Fix For: 2.7
Attachments: Server2.log
Intermittently, when an entry is removed from a ReplicatedHashMap (where the entry does
not exist) the following exception is thrown:
java.lang.RuntimeException: remove(APMExample.Services.examples.ServerA09) failed
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:405)
at
com.ubs.apm.control.service.nameservice.jgroup.JGroupNameService.unRegisterService(JGroupNameService.java:132)
at
com.ubs.apm.control.sensors.ControlSensorManager.cleanup(ControlSensorManager.java:468)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:125)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:106)
at com.ubs.apm.control.example.ManagedServer.init(ManagedServer.java:20)
at com.ubs.apm.control.example.ManagedServer.main(ManagedServer.java:45)
Caused by: java.lang.RuntimeException: failed executing request [req_id=1189617222436
caller=14.64.61.201:6825
14.64.61.201:6838: sender=14.64.61.201:6838, retval=null, received=false,
suspected=false
.... many such lines ...
14.64.61.201:6860: sender=14.64.61.201:6860, retval=null, received=false,
suspected=false
14.64.61.201:6847: sender=14.64.61.201:6847, retval=null, received=false,
suspected=false
request_msg: [dst: <null>, src: 14.64.61.201:6825 (2 headers), size=143 bytes]
rsp_mode: GET_NONE
done: true
timeout: 5000
expected_mbrs: 0 ([14.64.61.201:6815, 14.64.61.201:6816, 14.64.61.201:6824,
14.64.61.201:6825, 14.64.61.201:6826, 14.64.61.201:6827, 14.64.61.201:6828,
14.64.61.201:6829, 14.64.61.201:6830, 14.64.61.201:6831, 14.64.61.201:6833,
14.64.61.201:6834, 14.64.61.201:6835, 14.64.61.201:6836, 14.64.61.201:6837,
14.64.61.201:6838, 14.64.61.201:6839, 14.64.61.201:6842, 14.64.61.201:6843,
14.64.61.201:6844, 14.64.61.201:6845, 14.64.61.201:6846, 14.64.61.201:6847,
14.64.61.201:6848, 14.64.61.201:6849, 14.64.61.201:6850, 14.64.61.201:6851,
14.64.61.201:6852, 14.64.61.201:6853, 14.64.61.201:6854, 14.64.61.201:6855,
14.64.61.201:6856, 14.64.61.201:6857, 14.64.61.201:6858, 14.64.61.201:6859,
14.64.61.201:6860, 14.64.61.201:6861, 14.64.61.201:6862, 14.64.61.201:6863,
14.64.61.201:6864, 14.64.61.201:6865, 14.64.61.201:6866])]
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:433)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:199)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:167)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:163)
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:402)
... 6 more
Caused by: java.lang.RuntimeException: failure adding msg [dst: <null>, src:
14.64.61.201:6825 (2 headers), size=143 bytes] to the retransmit table for
14.64.61.201:6825
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:636)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:438)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:317)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:782)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:221)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:339)
at org.jgroups.JChannel.downcall(JChannel.java:1240)
at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:752)
at org.jgroups.blocks.RequestCorrelator.sendRequest(RequestCorrelator.java:301)
at org.jgroups.blocks.GroupRequest.doExecute(GroupRequest.java:440)
at org.jgroups.blocks.GroupRequest.execute(GroupRequest.java:190)
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:430)
... 10 more
Caused by: java.lang.NullPointerException
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:632)
... 21 more
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:47 a.m.
New subject: [JBoss JIRA] Commented: (JGRP-594) Intermittently, a Null pointer exception is thrown when trying to remove non-existent entry in ReplicatedHashMap
[ http://jira.jboss.com/jira/browse/JGRP-594?page=comments#action_12395928 ]
Dipak Kothari commented on JGRP-594:
------------------------------------
I have been investigating this (and NPE in JGRP-594) further and have found the
following:
1) At start up, infrequently, i get members starting in different groups due to initial
coordinator not responding in time.
2) The MergeTask identifies the sub groups and initiates the merge.
3) New view and digest get installed. Also, the application gets a call back of View
Change.
4) The application, on identifing that its a mergeView request a getState from one of the
members (Say X) of the subgroup.
5) X returns its view of the View and digest before it had received the MergeView event
(can see in the log that this events arrives afterwards).
6) STATE_TRANSFER protocol handles the state Response
7) As flush is not in the protocol, it calls sends a SET_DIGEST event down the stack which
is picked up by NAKACK. It resets its digest (the digest it had was the correct merged
version) and updates with the one that arrived with state. However, this digest
doesn't have itself in it and so start getting the ERROR messages as mentioned above.
8) Subsequent put in ReplicatedHashMap fails with NPE in the NAKACK as it tries to add
message to the NakReceiverWindow associated with local address which now isn't there.
So I tried adding FLUSH protocol and this avoids resetting the digest and the NPE but the
same tests results in state being different between members and never seem to get to the
point where the state is the same across all members of the group.
I have looked at the bug list between 2.5 and 2.6.1 and have noticed some work that has
been done in the area of flush, gms and state transfer and so was thinking of working on
2.6.1 now rather than 2.5.0. One point though, without the flush I had the same issue with
2.6.1 as well though it only happended once in a 24 hour soak test.
Intermittently, a Null pointer exception is thrown when trying to
remove non-existent entry in ReplicatedHashMap
----------------------------------------------------------------------------------------------------------------
Key: JGRP-594
URL: http://jira.jboss.com/jira/browse/JGRP-594
Project: JGroups
Issue Type: Bug
Affects Versions: 2.5
Environment: Linux
Reporter: Dipak Kothari
Assigned To: Bela Ban
Fix For: 2.7
Attachments: Server2.log
Intermittently, when an entry is removed from a ReplicatedHashMap (where the entry does
not exist) the following exception is thrown:
java.lang.RuntimeException: remove(APMExample.Services.examples.ServerA09) failed
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:405)
at
com.ubs.apm.control.service.nameservice.jgroup.JGroupNameService.unRegisterService(JGroupNameService.java:132)
at
com.ubs.apm.control.sensors.ControlSensorManager.cleanup(ControlSensorManager.java:468)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:125)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:106)
at com.ubs.apm.control.example.ManagedServer.init(ManagedServer.java:20)
at com.ubs.apm.control.example.ManagedServer.main(ManagedServer.java:45)
Caused by: java.lang.RuntimeException: failed executing request [req_id=1189617222436
caller=14.64.61.201:6825
14.64.61.201:6838: sender=14.64.61.201:6838, retval=null, received=false,
suspected=false
.... many such lines ...
14.64.61.201:6860: sender=14.64.61.201:6860, retval=null, received=false,
suspected=false
14.64.61.201:6847: sender=14.64.61.201:6847, retval=null, received=false,
suspected=false
request_msg: [dst: <null>, src: 14.64.61.201:6825 (2 headers), size=143 bytes]
rsp_mode: GET_NONE
done: true
timeout: 5000
expected_mbrs: 0 ([14.64.61.201:6815, 14.64.61.201:6816, 14.64.61.201:6824,
14.64.61.201:6825, 14.64.61.201:6826, 14.64.61.201:6827, 14.64.61.201:6828,
14.64.61.201:6829, 14.64.61.201:6830, 14.64.61.201:6831, 14.64.61.201:6833,
14.64.61.201:6834, 14.64.61.201:6835, 14.64.61.201:6836, 14.64.61.201:6837,
14.64.61.201:6838, 14.64.61.201:6839, 14.64.61.201:6842, 14.64.61.201:6843,
14.64.61.201:6844, 14.64.61.201:6845, 14.64.61.201:6846, 14.64.61.201:6847,
14.64.61.201:6848, 14.64.61.201:6849, 14.64.61.201:6850, 14.64.61.201:6851,
14.64.61.201:6852, 14.64.61.201:6853, 14.64.61.201:6854, 14.64.61.201:6855,
14.64.61.201:6856, 14.64.61.201:6857, 14.64.61.201:6858, 14.64.61.201:6859,
14.64.61.201:6860, 14.64.61.201:6861, 14.64.61.201:6862, 14.64.61.201:6863,
14.64.61.201:6864, 14.64.61.201:6865, 14.64.61.201:6866])]
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:433)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:199)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:167)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:163)
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:402)
... 6 more
Caused by: java.lang.RuntimeException: failure adding msg [dst: <null>, src:
14.64.61.201:6825 (2 headers), size=143 bytes] to the retransmit table for
14.64.61.201:6825
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:636)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:438)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:317)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:782)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:221)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:339)
at org.jgroups.JChannel.downcall(JChannel.java:1240)
at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:752)
at org.jgroups.blocks.RequestCorrelator.sendRequest(RequestCorrelator.java:301)
at org.jgroups.blocks.GroupRequest.doExecute(GroupRequest.java:440)
at org.jgroups.blocks.GroupRequest.execute(GroupRequest.java:190)
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:430)
... 10 more
Caused by: java.lang.NullPointerException
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:632)
... 21 more
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:16 a.m.
New subject: [JBoss JIRA] Closed: (JGRP-594) Intermittently, a Null pointer exception is thrown when trying to remove non-existent entry in ReplicatedHashMap
[
https://jira.jboss.org/jira/browse/JGRP-594?page=com.atlassian.jira.plugi...
]
Bela Ban closed JGRP-594.
-------------------------
Resolution: Cannot Reproduce Bug
Intermittently, a Null pointer exception is thrown when trying to
remove non-existent entry in ReplicatedHashMap
----------------------------------------------------------------------------------------------------------------
Key: JGRP-594
URL: https://jira.jboss.org/jira/browse/JGRP-594
Project: JGroups
Issue Type: Bug
Affects Versions: 2.5
Environment: Linux
Reporter: Dipak Kothari
Assignee: Bela Ban
Fix For: 2.7
Attachments: Server2.log
Intermittently, when an entry is removed from a ReplicatedHashMap (where the entry does
not exist) the following exception is thrown:
java.lang.RuntimeException: remove(APMExample.Services.examples.ServerA09) failed
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:405)
at
com.ubs.apm.control.service.nameservice.jgroup.JGroupNameService.unRegisterService(JGroupNameService.java:132)
at
com.ubs.apm.control.sensors.ControlSensorManager.cleanup(ControlSensorManager.java:468)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:125)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:106)
at com.ubs.apm.control.example.ManagedServer.init(ManagedServer.java:20)
at com.ubs.apm.control.example.ManagedServer.main(ManagedServer.java:45)
Caused by: java.lang.RuntimeException: failed executing request [req_id=1189617222436
caller=14.64.61.201:6825
14.64.61.201:6838: sender=14.64.61.201:6838, retval=null, received=false,
suspected=false
.... many such lines ...
14.64.61.201:6860: sender=14.64.61.201:6860, retval=null, received=false,
suspected=false
14.64.61.201:6847: sender=14.64.61.201:6847, retval=null, received=false,
suspected=false
request_msg: [dst: <null>, src: 14.64.61.201:6825 (2 headers), size=143 bytes]
rsp_mode: GET_NONE
done: true
timeout: 5000
expected_mbrs: 0 ([14.64.61.201:6815, 14.64.61.201:6816, 14.64.61.201:6824,
14.64.61.201:6825, 14.64.61.201:6826, 14.64.61.201:6827, 14.64.61.201:6828,
14.64.61.201:6829, 14.64.61.201:6830, 14.64.61.201:6831, 14.64.61.201:6833,
14.64.61.201:6834, 14.64.61.201:6835, 14.64.61.201:6836, 14.64.61.201:6837,
14.64.61.201:6838, 14.64.61.201:6839, 14.64.61.201:6842, 14.64.61.201:6843,
14.64.61.201:6844, 14.64.61.201:6845, 14.64.61.201:6846, 14.64.61.201:6847,
14.64.61.201:6848, 14.64.61.201:6849, 14.64.61.201:6850, 14.64.61.201:6851,
14.64.61.201:6852, 14.64.61.201:6853, 14.64.61.201:6854, 14.64.61.201:6855,
14.64.61.201:6856, 14.64.61.201:6857, 14.64.61.201:6858, 14.64.61.201:6859,
14.64.61.201:6860, 14.64.61.201:6861, 14.64.61.201:6862, 14.64.61.201:6863,
14.64.61.201:6864, 14.64.61.201:6865, 14.64.61.201:6866])]
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:433)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:199)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:167)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:163)
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:402)
... 6 more
Caused by: java.lang.RuntimeException: failure adding msg [dst: <null>, src:
14.64.61.201:6825 (2 headers), size=143 bytes] to the retransmit table for
14.64.61.201:6825
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:636)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:438)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:317)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:782)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:221)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:339)
at org.jgroups.JChannel.downcall(JChannel.java:1240)
at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:752)
at org.jgroups.blocks.RequestCorrelator.sendRequest(RequestCorrelator.java:301)
at org.jgroups.blocks.GroupRequest.doExecute(GroupRequest.java:440)
at org.jgroups.blocks.GroupRequest.execute(GroupRequest.java:190)
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:430)
... 10 more
Caused by: java.lang.NullPointerException
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:632)
... 21 more
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:16 a.m.
New subject: [JBoss JIRA] Commented: (JGRP-594) Intermittently, a Null pointer exception is thrown when trying to remove non-existent entry in ReplicatedHashMap
[
https://jira.jboss.org/jira/browse/JGRP-594?page=com.atlassian.jira.plugi...
]
Bela Ban commented on JGRP-594:
-------------------------------
This should be fixed in 2.7 (CVS head) and 2.6. As a workaround, use FLUSH. Please try
with the latest version of 2.7 or 2.6 (2.6.5) and let us know if the issue still persists.
In the meantime I'm going to close this issue, but feel free to re-open if the issue
pops up again. In that case, I'd like to have precise instructions how to reproduce
the issue (I can't start 60 instances)...
Intermittently, a Null pointer exception is thrown when trying to
remove non-existent entry in ReplicatedHashMap
----------------------------------------------------------------------------------------------------------------
Key: JGRP-594
URL: https://jira.jboss.org/jira/browse/JGRP-594
Project: JGroups
Issue Type: Bug
Affects Versions: 2.5
Environment: Linux
Reporter: Dipak Kothari
Assignee: Bela Ban
Fix For: 2.7
Attachments: Server2.log
Intermittently, when an entry is removed from a ReplicatedHashMap (where the entry does
not exist) the following exception is thrown:
java.lang.RuntimeException: remove(APMExample.Services.examples.ServerA09) failed
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:405)
at
com.ubs.apm.control.service.nameservice.jgroup.JGroupNameService.unRegisterService(JGroupNameService.java:132)
at
com.ubs.apm.control.sensors.ControlSensorManager.cleanup(ControlSensorManager.java:468)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:125)
at
com.ubs.apm.control.sensors.ControlSensorManager.<init>(ControlSensorManager.java:106)
at com.ubs.apm.control.example.ManagedServer.init(ManagedServer.java:20)
at com.ubs.apm.control.example.ManagedServer.main(ManagedServer.java:45)
Caused by: java.lang.RuntimeException: failed executing request [req_id=1189617222436
caller=14.64.61.201:6825
14.64.61.201:6838: sender=14.64.61.201:6838, retval=null, received=false,
suspected=false
.... many such lines ...
14.64.61.201:6860: sender=14.64.61.201:6860, retval=null, received=false,
suspected=false
14.64.61.201:6847: sender=14.64.61.201:6847, retval=null, received=false,
suspected=false
request_msg: [dst: <null>, src: 14.64.61.201:6825 (2 headers), size=143 bytes]
rsp_mode: GET_NONE
done: true
timeout: 5000
expected_mbrs: 0 ([14.64.61.201:6815, 14.64.61.201:6816, 14.64.61.201:6824,
14.64.61.201:6825, 14.64.61.201:6826, 14.64.61.201:6827, 14.64.61.201:6828,
14.64.61.201:6829, 14.64.61.201:6830, 14.64.61.201:6831, 14.64.61.201:6833,
14.64.61.201:6834, 14.64.61.201:6835, 14.64.61.201:6836, 14.64.61.201:6837,
14.64.61.201:6838, 14.64.61.201:6839, 14.64.61.201:6842, 14.64.61.201:6843,
14.64.61.201:6844, 14.64.61.201:6845, 14.64.61.201:6846, 14.64.61.201:6847,
14.64.61.201:6848, 14.64.61.201:6849, 14.64.61.201:6850, 14.64.61.201:6851,
14.64.61.201:6852, 14.64.61.201:6853, 14.64.61.201:6854, 14.64.61.201:6855,
14.64.61.201:6856, 14.64.61.201:6857, 14.64.61.201:6858, 14.64.61.201:6859,
14.64.61.201:6860, 14.64.61.201:6861, 14.64.61.201:6862, 14.64.61.201:6863,
14.64.61.201:6864, 14.64.61.201:6865, 14.64.61.201:6866])]
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:433)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:199)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:167)
at org.jgroups.blocks.RpcDispatcher.callRemoteMethods(RpcDispatcher.java:163)
at org.jgroups.blocks.ReplicatedHashMap.remove(ReplicatedHashMap.java:402)
... 6 more
Caused by: java.lang.RuntimeException: failure adding msg [dst: <null>, src:
14.64.61.201:6825 (2 headers), size=143 bytes] to the retransmit table for
14.64.61.201:6825
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:636)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:438)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:317)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:782)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:221)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:339)
at org.jgroups.JChannel.downcall(JChannel.java:1240)
at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:752)
at org.jgroups.blocks.RequestCorrelator.sendRequest(RequestCorrelator.java:301)
at org.jgroups.blocks.GroupRequest.doExecute(GroupRequest.java:440)
at org.jgroups.blocks.GroupRequest.execute(GroupRequest.java:190)
at org.jgroups.blocks.MessageDispatcher.castMessage(MessageDispatcher.java:430)
... 10 more
Caused by: java.lang.NullPointerException
at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:632)
... 21 more
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
5814
days inactive
6217
days old
10 comments
2 participants
participants (2)
-
Bela Ban (JIRA) -
Dipak Kothari (JIRA)