[jboss-jira] [JBoss JIRA] (JGRP-1667) OutOfMemoryError - messages are piling up

[Bela Ban (JIRA)]

    [ https://issues.jboss.org/browse/JGRP-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792683#comment-12792683 ] 

Bela Ban commented on JGRP-1667:
--------------------------------

No, the sender will block until there is more space, as I use a BlockingQueue for this
                
> OutOfMemoryError - messages are piling up
> -----------------------------------------
>
>                 Key: JGRP-1667
>                 URL: https://issues.jboss.org/browse/JGRP-1667
>             Project: JGroups
>          Issue Type: Bug
>    Affects Versions: 2.12.3
>         Environment: Windows 7
>            Reporter: Matthew Lowe
>            Assignee: Bela Ban
>            Priority: Critical
>
> One of our customers encountered OOME in their production running 5 node Infinispan cluster. Crash happened after ~month of runtime. 
> Stacktrace:
> Timer-19,_threadNameOmmitted_32726 tid=188 [RUNNABLE] [DAEMON] <--- OutOfMemoryError happened in this thread
> java.lang.OutOfMemoryError.<init>()
> org.jgroups.blocks.TCPConnectionMap$TCPConnection.send(byte[], int, int)
> org.jgroups.blocks.TCPConnectionMap$TCPConnection.access$100(TCPConnectionMap$TCPConnection, byte[], int, int)
> org.jgroups.blocks.TCPConnectionMap.send(Address, byte[], int, int)
> org.jgroups.protocols.TCP.send(Address, byte[], int, int)
> org.jgroups.protocols.BasicTCP.sendUnicast(PhysicalAddress, byte[], int, int)
> org.jgroups.protocols.TP.sendToSingleMember(Address, byte[], int, int)
> org.jgroups.protocols.TP.doSend(Buffer, Address, boolean)
> org.jgroups.protocols.TP.send(Message, Address, boolean)
> org.jgroups.protocols.TP.down(Event)
> org.jgroups.protocols.Discovery.down(Event)
> org.jgroups.protocols.TCPPING.down(Event)
> org.jgroups.protocols.MERGE2.down(Event)
> org.jgroups.protocols.FD_SOCK.down(Event)
> org.jgroups.protocols.FD.down(Event)
> org.jgroups.protocols.VERIFY_SUSPECT.down(Event)
> org.jgroups.protocols.pbcast.NAKACK.down(Event)
> org.jgroups.protocols.UNICAST.retransmit(long, Message)
> org.jgroups.stack.AckSenderWindow.retransmit(long, long, Address)
> org.jgroups.stack.DefaultRetransmitter$SeqnoTask.callRetransmissionCommand()
> org.jgroups.stack.Retransmitter$Task.run()
> org.jgroups.util.TimeScheduler2$MyTask.run()
> org.jgroups.util.TimeScheduler2$Entry.execute()
> org.jgroups.util.TimeScheduler2$1.run()
> java.lang.Thread.run()
> When I take a look into memory dump, I can see that there is ~470MB of retained heap space held by instance "org.jgroups.blocks.TCPConnectionMap$TCPConnection$Sender". Most memory is reatained by instances of "byte[]". Therefore, I'm assuming that messages are somehow piling up and eventually causing OOME, but I don't know what conditions might trigger such behavior.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira