5:13 a.m.
Peter Nerg created JGRP-1432:
--------------------------------
Summary: OutOfMemoryError in GMS
Key: JGRP-1432
URL: https://issues.jboss.org/browse/JGRP-1432
Project: JGroups
Issue Type: Bug
Affects Versions: 2.12.2
Environment: Modified SLES
Reporter: Peter Nerg
Assignee: Bela Ban
When running in a cluster with only two nodes we every now and then see issues that
JGroups fails to start a thread due to OOM.
The stack trace always points to the same place hence so it should rule out any other part
of the application.
Also taking a heap dump immediately after the OOM yields no obvious cause to the OOM.
It makes we wonder if there is a scenario where JGroups goes wild and starts to create
lots of threads.
The stack trace looks like this (often a number of OOM exceptions in a row)
2012-02-21 08:56:52,679 [ OOB-1,null] ERROR [org.jgroups.protocols.TCP] failed
handling incoming message
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:640)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.start(GMS.java:1297)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.add(GMS.java:1260)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:801)
at org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:170)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:246)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:703)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:101)
at org.jgroups.protocols.FD.up(FD.java:275)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:210)
at org.jgroups.protocols.Discovery.up(Discovery.java:294)
at org.jgroups.stack.Protocol.up(Protocol.java:413)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1109)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1665)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1647)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
The above stack trace is often preceded by the following printout:
2012-02-21 04:39:28,949 [ Timer-2,<ADDR>] WARN [org.jgroups.protocols.FILE_PING]
failed reading 9875802e-272a-0bcc-d1db-466d80f188b2.node: removing it
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
5:25 a.m.
[
https://issues.jboss.org/browse/JGRP-1432?page=com.atlassian.jira.plugin....
]
Bela Ban commented on JGRP-1432:
--------------------------------
This report is close to useless unless you can provide me more details.
Failure to create a thread can be caused by a small stack size (-Xss, IIRC) or no memory
left. I suggest run the program in a memory profiler to see if it leaks memory.
OutOfMemoryError in GMS
-----------------------
Key: JGRP-1432
URL: https://issues.jboss.org/browse/JGRP-1432
Project: JGroups
Issue Type: Bug
Affects Versions: 2.12.2
Environment: Modified SLES
Reporter: Peter Nerg
Assignee: Bela Ban
When running in a cluster with only two nodes we every now and then see issues that
JGroups fails to start a thread due to OOM.
The stack trace always points to the same place hence so it should rule out any other
part of the application.
Also taking a heap dump immediately after the OOM yields no obvious cause to the OOM.
It makes we wonder if there is a scenario where JGroups goes wild and starts to create
lots of threads.
The stack trace looks like this (often a number of OOM exceptions in a row)
2012-02-21 08:56:52,679 [ OOB-1,null] ERROR [org.jgroups.protocols.TCP] failed
handling incoming message
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:640)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.start(GMS.java:1297)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.add(GMS.java:1260)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:801)
at org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:170)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:246)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:703)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:101)
at org.jgroups.protocols.FD.up(FD.java:275)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:210)
at org.jgroups.protocols.Discovery.up(Discovery.java:294)
at org.jgroups.stack.Protocol.up(Protocol.java:413)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1109)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1665)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1647)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
The above stack trace is often preceded by the following printout:
2012-02-21 04:39:28,949 [ Timer-2,<ADDR>] WARN [org.jgroups.protocols.FILE_PING]
failed reading 9875802e-272a-0bcc-d1db-466d80f188b2.node: removing it
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
5:49 a.m.
[
https://issues.jboss.org/browse/JGRP-1432?page=com.atlassian.jira.plugin....
]
Peter Nerg commented on JGRP-1432:
----------------------------------
I have performed a heap dump just after the OOM and there is nothing to be found in the
dump.
The issue is always on the same spot (same stack trace) so it cannot so it cannot be a
coincidence.
Since the stack trace always seems to be on the same spot, i.e. where GMS receives an UP
event I'm wondering if this possibly has something do with the intermittent printouts
we see:
2012-02-21 04:39:28,949 [ Timer-2,<ADDR>] WARN [org.jgroups.protocols.FILE_PING]
failed reading 9875802e-272a-0bcc-d1db-466d80f188b2.node: removing it
It seems the members loose contact and then re-join/merge and it can take quite some time
to stabilize.
I can't figure out why the nodes loose contact every now and then. We're running
blades in the same chassi with Gb ethernet so the network connection should not be the
cause of these warnings.
Also we have a number (5-6) of Jgroups stacks that share the same TCP connection using
singleton config on the TCP.
I've added the config file we use to setup JGroups.
OutOfMemoryError in GMS
-----------------------
Key: JGRP-1432
URL: https://issues.jboss.org/browse/JGRP-1432
Project: JGroups
Issue Type: Bug
Affects Versions: 2.12.2
Environment: Modified SLES
Reporter: Peter Nerg
Assignee: Bela Ban
Attachments: tcp-fileping.xml
When running in a cluster with only two nodes we every now and then see issues that
JGroups fails to start a thread due to OOM.
The stack trace always points to the same place hence so it should rule out any other
part of the application.
Also taking a heap dump immediately after the OOM yields no obvious cause to the OOM.
It makes we wonder if there is a scenario where JGroups goes wild and starts to create
lots of threads.
The stack trace looks like this (often a number of OOM exceptions in a row)
2012-02-21 08:56:52,679 [ OOB-1,null] ERROR [org.jgroups.protocols.TCP] failed
handling incoming message
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:640)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.start(GMS.java:1297)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.add(GMS.java:1260)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:801)
at org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:170)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:246)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:703)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:101)
at org.jgroups.protocols.FD.up(FD.java:275)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:210)
at org.jgroups.protocols.Discovery.up(Discovery.java:294)
at org.jgroups.stack.Protocol.up(Protocol.java:413)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1109)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1665)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1647)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
The above stack trace is often preceded by the following printout:
2012-02-21 04:39:28,949 [ Timer-2,<ADDR>] WARN [org.jgroups.protocols.FILE_PING]
failed reading 9875802e-272a-0bcc-d1db-466d80f188b2.node: removing it
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
5:49 a.m.
[
https://issues.jboss.org/browse/JGRP-1432?page=com.atlassian.jira.plugin....
]
Peter Nerg updated JGRP-1432:
-----------------------------
Attachment: tcp-fileping.xml
The JGroups configuration file
OutOfMemoryError in GMS
-----------------------
Key: JGRP-1432
URL: https://issues.jboss.org/browse/JGRP-1432
Project: JGroups
Issue Type: Bug
Affects Versions: 2.12.2
Environment: Modified SLES
Reporter: Peter Nerg
Assignee: Bela Ban
Attachments: tcp-fileping.xml
When running in a cluster with only two nodes we every now and then see issues that
JGroups fails to start a thread due to OOM.
The stack trace always points to the same place hence so it should rule out any other
part of the application.
Also taking a heap dump immediately after the OOM yields no obvious cause to the OOM.
It makes we wonder if there is a scenario where JGroups goes wild and starts to create
lots of threads.
The stack trace looks like this (often a number of OOM exceptions in a row)
2012-02-21 08:56:52,679 [ OOB-1,null] ERROR [org.jgroups.protocols.TCP] failed
handling incoming message
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:640)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.start(GMS.java:1297)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.add(GMS.java:1260)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:801)
at org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:170)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:246)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:703)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:101)
at org.jgroups.protocols.FD.up(FD.java:275)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:210)
at org.jgroups.protocols.Discovery.up(Discovery.java:294)
at org.jgroups.stack.Protocol.up(Protocol.java:413)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1109)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1665)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1647)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
The above stack trace is often preceded by the following printout:
2012-02-21 04:39:28,949 [ Timer-2,<ADDR>] WARN [org.jgroups.protocols.FILE_PING]
failed reading 9875802e-272a-0bcc-d1db-466d80f188b2.node: removing it
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:33 a.m.
[
https://issues.jboss.org/browse/JGRP-1432?page=com.atlassian.jira.plugin....
]
Bela Ban commented on JGRP-1432:
--------------------------------
So are you saying that there is no memory issue; the heap isn't exhausted ? In that
case, there's a problem creating new threads, either because the ulimit for the
process was exhausted or because of some other restrictions (modified SLES?).
Re your config: STABLE.max_bytes should not be 0 or else you'll run out of memory if
you send a lot of messages, VIEW_SYNC can be removed, too
OutOfMemoryError in GMS
-----------------------
Key: JGRP-1432
URL: https://issues.jboss.org/browse/JGRP-1432
Project: JGroups
Issue Type: Bug
Affects Versions: 2.12.2
Environment: Modified SLES
Reporter: Peter Nerg
Assignee: Bela Ban
Attachments: tcp-fileping.xml
When running in a cluster with only two nodes we every now and then see issues that
JGroups fails to start a thread due to OOM.
The stack trace always points to the same place hence so it should rule out any other
part of the application.
Also taking a heap dump immediately after the OOM yields no obvious cause to the OOM.
It makes we wonder if there is a scenario where JGroups goes wild and starts to create
lots of threads.
The stack trace looks like this (often a number of OOM exceptions in a row)
2012-02-21 08:56:52,679 [ OOB-1,null] ERROR [org.jgroups.protocols.TCP] failed
handling incoming message
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:640)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.start(GMS.java:1297)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.add(GMS.java:1260)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:801)
at org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:170)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:246)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:703)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:101)
at org.jgroups.protocols.FD.up(FD.java:275)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:210)
at org.jgroups.protocols.Discovery.up(Discovery.java:294)
at org.jgroups.stack.Protocol.up(Protocol.java:413)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1109)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1665)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1647)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
The above stack trace is often preceded by the following printout:
2012-02-21 04:39:28,949 [ Timer-2,<ADDR>] WARN [org.jgroups.protocols.FILE_PING]
failed reading 9875802e-272a-0bcc-d1db-466d80f188b2.node: removing it
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:46 a.m.
[
https://issues.jboss.org/browse/JGRP-1432?page=com.atlassian.jira.plugin....
]
Peter Nerg commented on JGRP-1432:
----------------------------------
Sorry for not answering sooner on your post.
By accident we triggered a new error message stating an issue with creating a native
thread, after which we started to look into file descriptors.
Just as you seem to have concluded too.
As it seems we're bordering the limit of FD's which may be due to upgrade of the
application server (it's eating FD's as it they were cookies).
The unfortunate in the situation is that it always was JGroups that reported the OOM and
we didn't immediately do the connection to the file descriptors.
Largely because it has not been a problem earlier.
Sorry for the inconvenience but thanks for the tips on the configuration.
OutOfMemoryError in GMS
-----------------------
Key: JGRP-1432
URL: https://issues.jboss.org/browse/JGRP-1432
Project: JGroups
Issue Type: Bug
Affects Versions: 2.12.2
Environment: Modified SLES
Reporter: Peter Nerg
Assignee: Bela Ban
Attachments: tcp-fileping.xml
When running in a cluster with only two nodes we every now and then see issues that
JGroups fails to start a thread due to OOM.
The stack trace always points to the same place hence so it should rule out any other
part of the application.
Also taking a heap dump immediately after the OOM yields no obvious cause to the OOM.
It makes we wonder if there is a scenario where JGroups goes wild and starts to create
lots of threads.
The stack trace looks like this (often a number of OOM exceptions in a row)
2012-02-21 08:56:52,679 [ OOB-1,null] ERROR [org.jgroups.protocols.TCP] failed
handling incoming message
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:640)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.start(GMS.java:1297)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.add(GMS.java:1260)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:801)
at org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:170)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:246)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:703)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:101)
at org.jgroups.protocols.FD.up(FD.java:275)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:210)
at org.jgroups.protocols.Discovery.up(Discovery.java:294)
at org.jgroups.stack.Protocol.up(Protocol.java:413)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1109)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1665)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1647)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
The above stack trace is often preceded by the following printout:
2012-02-21 04:39:28,949 [ Timer-2,<ADDR>] WARN [org.jgroups.protocols.FILE_PING]
failed reading 9875802e-272a-0bcc-d1db-466d80f188b2.node: removing it
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:48 a.m.
[
https://issues.jboss.org/browse/JGRP-1432?page=com.atlassian.jira.plugin....
]
Bela Ban commented on JGRP-1432:
--------------------------------
So I assume this issue is moot then ? If so, can you close it ?
OutOfMemoryError in GMS
-----------------------
Key: JGRP-1432
URL: https://issues.jboss.org/browse/JGRP-1432
Project: JGroups
Issue Type: Bug
Affects Versions: 2.12.2
Environment: Modified SLES
Reporter: Peter Nerg
Assignee: Bela Ban
Attachments: tcp-fileping.xml
When running in a cluster with only two nodes we every now and then see issues that
JGroups fails to start a thread due to OOM.
The stack trace always points to the same place hence so it should rule out any other
part of the application.
Also taking a heap dump immediately after the OOM yields no obvious cause to the OOM.
It makes we wonder if there is a scenario where JGroups goes wild and starts to create
lots of threads.
The stack trace looks like this (often a number of OOM exceptions in a row)
2012-02-21 08:56:52,679 [ OOB-1,null] ERROR [org.jgroups.protocols.TCP] failed
handling incoming message
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:640)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.start(GMS.java:1297)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.add(GMS.java:1260)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:801)
at org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:170)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:246)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:703)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:101)
at org.jgroups.protocols.FD.up(FD.java:275)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:210)
at org.jgroups.protocols.Discovery.up(Discovery.java:294)
at org.jgroups.stack.Protocol.up(Protocol.java:413)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1109)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1665)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1647)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
The above stack trace is often preceded by the following printout:
2012-02-21 04:39:28,949 [ Timer-2,<ADDR>] WARN [org.jgroups.protocols.FILE_PING]
failed reading 9875802e-272a-0bcc-d1db-466d80f188b2.node: removing it
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:16 a.m.
[
https://issues.jboss.org/browse/JGRP-1432?page=com.atlassian.jira.plugin....
]
Peter Nerg closed JGRP-1432.
----------------------------
Resolution: Rejected
Not a bug in JGroups, issues with lacking file descriptors
OutOfMemoryError in GMS
-----------------------
Key: JGRP-1432
URL: https://issues.jboss.org/browse/JGRP-1432
Project: JGroups
Issue Type: Bug
Affects Versions: 2.12.2
Environment: Modified SLES
Reporter: Peter Nerg
Assignee: Bela Ban
Attachments: tcp-fileping.xml
When running in a cluster with only two nodes we every now and then see issues that
JGroups fails to start a thread due to OOM.
The stack trace always points to the same place hence so it should rule out any other
part of the application.
Also taking a heap dump immediately after the OOM yields no obvious cause to the OOM.
It makes we wonder if there is a scenario where JGroups goes wild and starts to create
lots of threads.
The stack trace looks like this (often a number of OOM exceptions in a row)
2012-02-21 08:56:52,679 [ OOB-1,null] ERROR [org.jgroups.protocols.TCP] failed
handling incoming message
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:640)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.start(GMS.java:1297)
at org.jgroups.protocols.pbcast.GMS$ViewHandler.add(GMS.java:1260)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:801)
at org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:170)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:246)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:703)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:101)
at org.jgroups.protocols.FD.up(FD.java:275)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:210)
at org.jgroups.protocols.Discovery.up(Discovery.java:294)
at org.jgroups.stack.Protocol.up(Protocol.java:413)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1109)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1665)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1647)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
The above stack trace is often preceded by the following printout:
2012-02-21 04:39:28,949 [ Timer-2,<ADDR>] WARN [org.jgroups.protocols.FILE_PING]
failed reading 9875802e-272a-0bcc-d1db-466d80f188b2.node: removing it
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
4699
days inactive
4699
days old
7 comments
2 participants
participants (2)
-
Bela Ban (JIRA) -
Peter Nerg (JIRA)