[jboss-jira] [JBoss JIRA] Created: (JGRP-876) ConcurrentModificationException during startup of JBoss AS 5.0.0.GA

[Paul Ferraro (JIRA)]

ConcurrentModificationException during startup of JBoss AS 5.0.0.GA
-------------------------------------------------------------------

                 Key: JGRP-876
                 URL: https://jira.jboss.org/jira/browse/JGRP-876
             Project: JGroups
          Issue Type: Bug
    Affects Versions: 2.6.7
         Environment: JBoss AS 5.0.0.GA
            Reporter: Paul Ferraro
            Assignee: Bela Ban


I encountered a rare ConcurrentModificationException while starting the all profile of JBoss AS 5.0.0.GA:

10:34:45,077 ERROR [AbstractKernelController] Error installing to Start: name=HAPartition state=Create
org.jgroups.ChannelException: connect() failed
	at org.jgroups.JChannel.connect(JChannel.java:374)
	at org.jboss.ha.framework.server.ClusterPartition$ChannelConnectTask.run(ClusterPartition.java:167)
	at org.jboss.util.threadpool.RunnableTaskWrapper.run(RunnableTaskWrapper.java:147)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
	at java.lang.Thread.run(Thread.java:595)
Caused by: java.util.ConcurrentModificationException
	at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:449)
	at java.util.AbstractList$Itr.next(AbstractList.java:420)
	at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1010)
	at java.util.AbstractCollection.addAll(AbstractCollection.java:318)
	at org.jgroups.protocols.TP.handleDownEvent(TP.java:1575)
	at org.jgroups.protocols.TP.down(TP.java:1148)
	at org.jgroups.protocols.TP$ProtocolAdapter.down(TP.java:2308)
	at org.jgroups.protocols.Discovery.down(Discovery.java:354)
	at org.jgroups.protocols.MERGE2.down(MERGE2.java:153)
	at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:370)
	at org.jgroups.protocols.FD.down(FD.java:294)
	at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:95)
	at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:672)
	at org.jgroups.protocols.UNICAST.down(UNICAST.java:462)
	at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:299)
	at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:545)
	at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:453)
	at org.jgroups.protocols.pbcast.ClientGmsImpl.becomeSingletonMember(ClientGmsImpl.java:356)
	at org.jgroups.protocols.pbcast.ClientGmsImpl.join(ClientGmsImpl.java:86)
	at org.jgroups.protocols.pbcast.ClientGmsImpl.join(ClientGmsImpl.java:39)
	at org.jgroups.protocols.pbcast.GMS.down(GMS.java:832)
	at org.jgroups.protocols.FC.down(FC.java:430)
	at org.jgroups.protocols.FRAG2.down(FRAG2.java:158)
	at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:200)
	at org.jgroups.protocols.pbcast.FLUSH.down(FLUSH.java:272)
	at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:475)
	at org.jgroups.JChannel.downcall(JChannel.java:1478)
	at org.jgroups.JChannel.connect(JChannel.java:370)

Looking at the source, here are the problematic lines:

1573: ProtocolAdapter ad=(ProtocolAdapter)prot;
1574: List<Address> tmp=ad.getMembers();
1575: members.addAll(tmp);

Looking now at ProtocolAdapter.getMembers(), I see that it does this:
public List<Address> getMembers() {
    return Collections.unmodifiableList(members);
}

While Collections.unmodifiableList(...) ensures that the caller of getMembers() cannot modify the list, it can still iterate over it (e.g. line 1575).  The offending mutation code is probably in ProtocolAdapter.down(Event):

case Event.VIEW_CHANGE:
    View view=(View)evt.getArg();
    Vector<Address> tmp=view.getMembers();
    members.clear();
    members.addAll(tmp);
    break;

There are several ways fix this, including, but not limited to:
* Make ProtocolAdapter.members volatile, and create a new member list on view change
* Make ProcotolAdapter.members a CopyOnWriteArrayList

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira