2:33 p.m.
ConcurrentModificationException in BasicConnectionTable.retainAll
-----------------------------------------------------------------
Key: JGRP-1383
URL: https://issues.jboss.org/browse/JGRP-1383
Project: JGroups
Issue Type: Bug
Affects Versions: 2.6.20
Reporter: Dennis Reed
Assignee: Dennis Reed
Fix For: 2.6.21
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at java.util.AbstractSet.removeAll(AbstractSet.java:154)
at org.jgroups.blocks.BasicConnectionTable.retainAll(BasicConnectionTable.java:344)
at org.jgroups.protocols.TCP.retainAll(TCP.java:59)
at org.jgroups.protocols.BasicTCP.handleDownEvent(BasicTCP.java:265)
at org.jgroups.protocols.TP.down(TP.java:1145)
...
BasicTCP is missing a synchoronize on "members" when it passes it to
TCP.retainAll.
If members is modified by another thread (in TP.handleDownEvent) while
BasicConnectionTable's Set.removeAll
call is iterating over it, it triggers a ConcurrentModificationException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:14 p.m.
[
https://issues.jboss.org/browse/JGRP-1383?page=com.atlassian.jira.plugin....
]
Dennis Reed updated JGRP-1383:
------------------------------
Attachment: JGRP-1383-test.tgz
ConcurrentModificationException in BasicConnectionTable.retainAll
-----------------------------------------------------------------
Key: JGRP-1383
URL: https://issues.jboss.org/browse/JGRP-1383
Project: JGroups
Issue Type: Bug
Affects Versions: 2.6.20
Reporter: Dennis Reed
Assignee: Dennis Reed
Fix For: 2.6.21
Attachments: JGRP-1383-test.tgz
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at java.util.AbstractSet.removeAll(AbstractSet.java:154)
at org.jgroups.blocks.BasicConnectionTable.retainAll(BasicConnectionTable.java:344)
at org.jgroups.protocols.TCP.retainAll(TCP.java:59)
at org.jgroups.protocols.BasicTCP.handleDownEvent(BasicTCP.java:265)
at org.jgroups.protocols.TP.down(TP.java:1145)
...
BasicTCP is missing a synchoronize on "members" when it passes it to
TCP.retainAll.
If members is modified by another thread (in TP.handleDownEvent) while
BasicConnectionTable's Set.removeAll
call is iterating over it, it triggers a ConcurrentModificationException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:16 p.m.
[
https://issues.jboss.org/browse/JGRP-1383?page=com.atlassian.jira.plugin....
]
Dennis Reed commented on JGRP-1383:
-----------------------------------
See attached JGRP-1383-test.tgz for test case.
It contains a README with details.
Since this is a race condition, it's difficult to trigger reliably, so I used Byteman
to trigger it.
Because it uses Byteman, I did not check it in the JGroups source.
ConcurrentModificationException in BasicConnectionTable.retainAll
-----------------------------------------------------------------
Key: JGRP-1383
URL: https://issues.jboss.org/browse/JGRP-1383
Project: JGroups
Issue Type: Bug
Affects Versions: 2.6.20
Reporter: Dennis Reed
Assignee: Dennis Reed
Fix For: 2.6.21
Attachments: JGRP-1383-test.tgz
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at java.util.AbstractSet.removeAll(AbstractSet.java:154)
at org.jgroups.blocks.BasicConnectionTable.retainAll(BasicConnectionTable.java:344)
at org.jgroups.protocols.TCP.retainAll(TCP.java:59)
at org.jgroups.protocols.BasicTCP.handleDownEvent(BasicTCP.java:265)
at org.jgroups.protocols.TP.down(TP.java:1145)
...
BasicTCP is missing a synchoronize on "members" when it passes it to
TCP.retainAll.
If members is modified by another thread (in TP.handleDownEvent) while
BasicConnectionTable's Set.removeAll
call is iterating over it, it triggers a ConcurrentModificationException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:40 a.m.
[
https://issues.jboss.org/browse/JGRP-1383?page=com.atlassian.jira.plugin....
]
Bela Ban commented on JGRP-1383:
--------------------------------
Dennis, has this issue been resolved ? Looking at the code in the 2.6 branch, I see that
there is indeed a synchronized statement around access to 'members'...
The way you use byteman here is great, I have long wanted to introduce byteman
systematically into more unit tests in JGroups, e.g. tests of the FLUSH or MERGE2/3
protocols, but have failed miserably due to lack of time !
ConcurrentModificationException in BasicConnectionTable.retainAll
-----------------------------------------------------------------
Key: JGRP-1383
URL: https://issues.jboss.org/browse/JGRP-1383
Project: JGroups
Issue Type: Bug
Affects Versions: 2.6.20
Reporter: Dennis Reed
Assignee: Dennis Reed
Fix For: 2.6.21
Attachments: JGRP-1383-test.tgz
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at java.util.AbstractSet.removeAll(AbstractSet.java:154)
at org.jgroups.blocks.BasicConnectionTable.retainAll(BasicConnectionTable.java:344)
at org.jgroups.protocols.TCP.retainAll(TCP.java:59)
at org.jgroups.protocols.BasicTCP.handleDownEvent(BasicTCP.java:265)
at org.jgroups.protocols.TP.down(TP.java:1145)
...
BasicTCP is missing a synchoronize on "members" when it passes it to
TCP.retainAll.
If members is modified by another thread (in TP.handleDownEvent) while
BasicConnectionTable's Set.removeAll
call is iterating over it, it triggers a ConcurrentModificationException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:15 a.m.
[
https://issues.jboss.org/browse/JGRP-1383?page=com.atlassian.jira.plugin....
]
Dennis Reed commented on JGRP-1383:
-----------------------------------
It is resolved for 2.6.
I left it open until I check master to see if it is an issue there also
(and have been very busy, hopefully I'll get to it in the next week or so).
ConcurrentModificationException in BasicConnectionTable.retainAll
-----------------------------------------------------------------
Key: JGRP-1383
URL: https://issues.jboss.org/browse/JGRP-1383
Project: JGroups
Issue Type: Bug
Affects Versions: 2.6.20
Reporter: Dennis Reed
Assignee: Dennis Reed
Fix For: 2.6.21
Attachments: JGRP-1383-test.tgz
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at java.util.AbstractSet.removeAll(AbstractSet.java:154)
at org.jgroups.blocks.BasicConnectionTable.retainAll(BasicConnectionTable.java:344)
at org.jgroups.protocols.TCP.retainAll(TCP.java:59)
at org.jgroups.protocols.BasicTCP.handleDownEvent(BasicTCP.java:265)
at org.jgroups.protocols.TP.down(TP.java:1145)
...
BasicTCP is missing a synchoronize on "members" when it passes it to
TCP.retainAll.
If members is modified by another thread (in TP.handleDownEvent) while
BasicConnectionTable's Set.removeAll
call is iterating over it, it triggers a ConcurrentModificationException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:27 a.m.
[
https://issues.jboss.org/browse/JGRP-1383?page=com.atlassian.jira.plugin....
]
Bela Ban commented on JGRP-1383:
--------------------------------
No, it isn't an issue in master, as master uses a CopyOnWriteArraySet.
I'll close the issue then, thanks,
ConcurrentModificationException in BasicConnectionTable.retainAll
-----------------------------------------------------------------
Key: JGRP-1383
URL: https://issues.jboss.org/browse/JGRP-1383
Project: JGroups
Issue Type: Bug
Affects Versions: 2.6.20
Reporter: Dennis Reed
Assignee: Dennis Reed
Fix For: 2.6.21
Attachments: JGRP-1383-test.tgz
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at java.util.AbstractSet.removeAll(AbstractSet.java:154)
at org.jgroups.blocks.BasicConnectionTable.retainAll(BasicConnectionTable.java:344)
at org.jgroups.protocols.TCP.retainAll(TCP.java:59)
at org.jgroups.protocols.BasicTCP.handleDownEvent(BasicTCP.java:265)
at org.jgroups.protocols.TP.down(TP.java:1145)
...
BasicTCP is missing a synchoronize on "members" when it passes it to
TCP.retainAll.
If members is modified by another thread (in TP.handleDownEvent) while
BasicConnectionTable's Set.removeAll
call is iterating over it, it triggers a ConcurrentModificationException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:27 a.m.
[
https://issues.jboss.org/browse/JGRP-1383?page=com.atlassian.jira.plugin....
]
Bela Ban resolved JGRP-1383.
----------------------------
Resolution: Done
ConcurrentModificationException in BasicConnectionTable.retainAll
-----------------------------------------------------------------
Key: JGRP-1383
URL: https://issues.jboss.org/browse/JGRP-1383
Project: JGroups
Issue Type: Bug
Affects Versions: 2.6.20
Reporter: Dennis Reed
Assignee: Dennis Reed
Fix For: 2.6.21
Attachments: JGRP-1383-test.tgz
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at java.util.AbstractSet.removeAll(AbstractSet.java:154)
at org.jgroups.blocks.BasicConnectionTable.retainAll(BasicConnectionTable.java:344)
at org.jgroups.protocols.TCP.retainAll(TCP.java:59)
at org.jgroups.protocols.BasicTCP.handleDownEvent(BasicTCP.java:265)
at org.jgroups.protocols.TP.down(TP.java:1145)
...
BasicTCP is missing a synchoronize on "members" when it passes it to
TCP.retainAll.
If members is modified by another thread (in TP.handleDownEvent) while
BasicConnectionTable's Set.removeAll
call is iterating over it, it triggers a ConcurrentModificationException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:07 a.m.
[
https://issues.jboss.org/browse/JGRP-1383?page=com.atlassian.jira.plugin....
]
Bela Ban commented on JGRP-1383:
--------------------------------
Do you have a patch for this ? Do you suggest increasing the scope of synchronized to
include both retainAll() and removeAll() ?
ConcurrentModificationException in BasicConnectionTable.retainAll
-----------------------------------------------------------------
Key: JGRP-1383
URL: https://issues.jboss.org/browse/JGRP-1383
Project: JGroups
Issue Type: Bug
Affects Versions: 2.6.20
Reporter: Dennis Reed
Assignee: Dennis Reed
Fix For: 2.6.21
Attachments: JGRP-1383-test.tgz
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at java.util.AbstractSet.removeAll(AbstractSet.java:154)
at org.jgroups.blocks.BasicConnectionTable.retainAll(BasicConnectionTable.java:344)
at org.jgroups.protocols.TCP.retainAll(TCP.java:59)
at org.jgroups.protocols.BasicTCP.handleDownEvent(BasicTCP.java:265)
at org.jgroups.protocols.TP.down(TP.java:1145)
...
BasicTCP is missing a synchoronize on "members" when it passes it to
TCP.retainAll.
If members is modified by another thread (in TP.handleDownEvent) while
BasicConnectionTable's Set.removeAll
call is iterating over it, it triggers a ConcurrentModificationException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:08 a.m.
[
https://issues.jboss.org/browse/JGRP-1383?page=com.atlassian.jira.plugin....
]
Dennis Reed commented on JGRP-1383:
-----------------------------------
The patch already in 2.6.21 covers this other use case, since it makes a private copy of
the set inside a synchronized block before passing it to BasicConnectionTable.
(The symptoms were completely different than the original, so I wanted to document the new
symptoms).
It looks like master should be ok also, since BasicTCP#handleDownEvent is already sending
a private copy of the set as a side effect of looking up the physical addresses.
ConcurrentModificationException in BasicConnectionTable.retainAll
-----------------------------------------------------------------
Key: JGRP-1383
URL: https://issues.jboss.org/browse/JGRP-1383
Project: JGroups
Issue Type: Bug
Affects Versions: 2.6.20
Reporter: Dennis Reed
Assignee: Dennis Reed
Fix For: 2.6.21
Attachments: JGRP-1383-test.tgz
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at java.util.AbstractSet.removeAll(AbstractSet.java:154)
at org.jgroups.blocks.BasicConnectionTable.retainAll(BasicConnectionTable.java:344)
at org.jgroups.protocols.TCP.retainAll(TCP.java:59)
at org.jgroups.protocols.BasicTCP.handleDownEvent(BasicTCP.java:265)
at org.jgroups.protocols.TP.down(TP.java:1145)
...
BasicTCP is missing a synchoronize on "members" when it passes it to
TCP.retainAll.
If members is modified by another thread (in TP.handleDownEvent) while
BasicConnectionTable's Set.removeAll
call is iterating over it, it triggers a ConcurrentModificationException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:10 a.m.
[
https://issues.jboss.org/browse/JGRP-1383?page=com.atlassian.jira.plugin....
]
Dennis Reed edited comment on JGRP-1383 at 12/7/11 11:09 AM:
-------------------------------------------------------------
Another manifestation of this race condition:
One or more nodes in a cluster could not create a connection (to a higher IP node)
because
the connection is immediately closed.
The root cause appears to be the other node thinks there's already a connection open,
while
this node has forgotten about that connection, but not closed it (due to this race
condition
happening just before the removeAll call in BasicConnectionTable.retainAll).
ConcurrentModificationException in BasicConnectionTable.retainAll
-----------------------------------------------------------------
Key: JGRP-1383
URL: https://issues.jboss.org/browse/JGRP-1383
Project: JGroups
Issue Type: Bug
Affects Versions: 2.6.20
Reporter: Dennis Reed
Assignee: Dennis Reed
Fix For: 2.6.21
Attachments: JGRP-1383-test.tgz
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at java.util.AbstractSet.removeAll(AbstractSet.java:154)
at org.jgroups.blocks.BasicConnectionTable.retainAll(BasicConnectionTable.java:344)
at org.jgroups.protocols.TCP.retainAll(TCP.java:59)
at org.jgroups.protocols.BasicTCP.handleDownEvent(BasicTCP.java:265)
at org.jgroups.protocols.TP.down(TP.java:1145)
...
BasicTCP is missing a synchoronize on "members" when it passes it to
TCP.retainAll.
If members is modified by another thread (in TP.handleDownEvent) while
BasicConnectionTable's Set.removeAll
call is iterating over it, it triggers a ConcurrentModificationException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
5334
days inactive
5369
days old
9 comments
6 participants
participants (6)
-
Bela Ban (Commented) (JIRA) -
Bela Ban (Resolved) (JIRA)
-
Dennis Reed (Commented) (JIRA)
-
Dennis Reed (Created) (JIRA)
-
Dennis Reed (Issue Comment Edited) (JIRA)
-
Dennis Reed (Updated) (JIRA)