[JBoss JIRA] (JGRP-1675) Threads stuck in FlowControl.decrementIfEnoughCredits
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1675?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-1675:
--------------------------------
[~dan.berindei] Yes, it is possible that an INT or OOB thread will process messages with different flags in UNICAST or NAKACK. This is as designed, and the probability of this happening is proportional to the thread pool sizes and the types of messages received. So if we have a large OOB pool and receive a lot of OOB messages and only a few regular messages, then chances are that the regular messages will be batched together with the OOB messages in UNICAST or NAKACK and will get passed up to the application via an OOB thread.
> Threads stuck in FlowControl.decrementIfEnoughCredits
> -----------------------------------------------------
>
> Key: JGRP-1675
> URL: https://issues.jboss.org/browse/JGRP-1675
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 3.4
> Reporter: Radim Vansa
> Assignee: Bela Ban
> Fix For: 3.5
>
> Attachments: jgroups-udp-radim.xml, RemoteGetStressTest.java, UPerf2.java
>
>
> I have recently observed a repeated situation where many (or all) threads have been stuck waiting for credits in FlowControl protocol.
> The credit request was not handled on the other node as this is non-oob message and some (actually many of them - cause unknown) messages before the request have been lost - therefore the request was waiting for them to be re-sent.
> However, these have not been re-sent properly as the retransmission request was not received - all OOB threads were stuck in the FlowControl protocol as these handled some other request and tried to send a response - but the response could not be sent until FlowControl gets the credits.
> The probability of such situation could be lowered by tagging the credit request to be OOB - then it would be handled immediately. If the credit replenish message would then be processed in regular OOB pool, this could get already depleted by many requests, but setting up the internal thread pool would solve the problem.
> Other consideration would be to allow releasing thread from FlowControl (let it send the message even without credits) if it waits there for too long.
> h3. Workaround
> It appears that setting MFC and UFC max_credits to 10M or removing these protocols at all is a workaround for this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (JGRP-1675) Threads stuck in FlowControl.decrementIfEnoughCredits
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1675?page=com.atlassian.jira.plugin.... ]
Bela Ban updated JGRP-1675:
---------------------------
Fix Version/s: (was: 3.4.1)
Internal pool doesn't exist in 3.2, fix is only applicable to 3.5
> Threads stuck in FlowControl.decrementIfEnoughCredits
> -----------------------------------------------------
>
> Key: JGRP-1675
> URL: https://issues.jboss.org/browse/JGRP-1675
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 3.4
> Reporter: Radim Vansa
> Assignee: Bela Ban
> Fix For: 3.5
>
> Attachments: jgroups-udp-radim.xml, RemoteGetStressTest.java, UPerf2.java
>
>
> I have recently observed a repeated situation where many (or all) threads have been stuck waiting for credits in FlowControl protocol.
> The credit request was not handled on the other node as this is non-oob message and some (actually many of them - cause unknown) messages before the request have been lost - therefore the request was waiting for them to be re-sent.
> However, these have not been re-sent properly as the retransmission request was not received - all OOB threads were stuck in the FlowControl protocol as these handled some other request and tried to send a response - but the response could not be sent until FlowControl gets the credits.
> The probability of such situation could be lowered by tagging the credit request to be OOB - then it would be handled immediately. If the credit replenish message would then be processed in regular OOB pool, this could get already depleted by many requests, but setting up the internal thread pool would solve the problem.
> Other consideration would be to allow releasing thread from FlowControl (let it send the message even without credits) if it waits there for too long.
> h3. Workaround
> It appears that setting MFC and UFC max_credits to 10M or removing these protocols at all is a workaround for this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (JBREM-1307) TimedOutputStream OutputTimerTask call to close on SSL OutputStream blocks
by Ron Sigal (JIRA)
[ https://issues.jboss.org/browse/JBREM-1307?page=com.atlassian.jira.plugin... ]
Ron Sigal closed JBREM-1307.
----------------------------
> TimedOutputStream OutputTimerTask call to close on SSL OutputStream blocks
> --------------------------------------------------------------------------
>
> Key: JBREM-1307
> URL: https://issues.jboss.org/browse/JBREM-1307
> Project: JBoss Remoting
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: transport
> Affects Versions: 2.2.3.SP2, 2.5.4.SP4
> Environment: JBoss EAP 4.3.0 CP08 and CP10
> Reporter: Doug Grove
> Assignee: Ron Sigal
> Labels: messaging
> Fix For: 2.5.4.SP5
>
> Attachments: jboss-remoting-src.jar, jboss-remoting.jar
>
>
> In order to provide a mechanism for timing out a network/socket write operation, the org.jboss.remoting.transport.socket.TimedOutputStream class starts a timer. If the duration of the write operation exceeds the timer interval, then the timer will call close() on the output stream. This interrupts the output stream write() operation and causes the write operation to throw an IOException.
> When the output stream uses secure sockets (SSL), however, the call to close() on the output stream can block indefinitely. The threads associated with the write() and close() operation will only clear when the tcp keepalive timeout mechanism in the operation system is invoked.
> This appears to be related to a know bug in the java runtime: http://bugs.sun.com/view_bug.do?bug_id=6358629
> Specifically, in a thread dump you will see:
> "Timer-9" daemon prio=10 tid=0x5016c400 nid=0x2638 waiting on condition [0x4cbad000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x585021a0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114)
> at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:720)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.sendAlert(SSLSocketImpl.java:1720)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.warning(SSLSocketImpl.java:1564)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1362)
> - locked <0x58502030> (a com.sun.net.ssl.internal.ssl.SSLSocketImpl)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1301)
> at com.sun.net.ssl.internal.ssl.AppOutputStream.close(AppOutputStream.java:80)
> at org.jboss.remoting.transport.socket.TimedOutputStream.close(TimedOutputStream.java:57)
> at org.jboss.remoting.transport.socket.TimedOutputStream$OutputTimerTask.run(TimedOutputStream.java:145)
> at java.util.TimerThread.mainLoop(Timer.java:512)
> at java.util.TimerThread.run(Timer.java:462)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (JBREM-1307) TimedOutputStream OutputTimerTask call to close on SSL OutputStream blocks
by Ron Sigal (JIRA)
[ https://issues.jboss.org/browse/JBREM-1307?page=com.atlassian.jira.plugin... ]
Ron Sigal updated JBREM-1307:
-----------------------------
Fix Version/s: 2.5.4.SP5
Affects Version/s: 2.5.4.SP4
> TimedOutputStream OutputTimerTask call to close on SSL OutputStream blocks
> --------------------------------------------------------------------------
>
> Key: JBREM-1307
> URL: https://issues.jboss.org/browse/JBREM-1307
> Project: JBoss Remoting
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: transport
> Affects Versions: 2.2.3.SP2, 2.5.4.SP4
> Environment: JBoss EAP 4.3.0 CP08 and CP10
> Reporter: Doug Grove
> Assignee: Ron Sigal
> Labels: messaging
> Fix For: 2.5.4.SP5
>
> Attachments: jboss-remoting-src.jar, jboss-remoting.jar
>
>
> In order to provide a mechanism for timing out a network/socket write operation, the org.jboss.remoting.transport.socket.TimedOutputStream class starts a timer. If the duration of the write operation exceeds the timer interval, then the timer will call close() on the output stream. This interrupts the output stream write() operation and causes the write operation to throw an IOException.
> When the output stream uses secure sockets (SSL), however, the call to close() on the output stream can block indefinitely. The threads associated with the write() and close() operation will only clear when the tcp keepalive timeout mechanism in the operation system is invoked.
> This appears to be related to a know bug in the java runtime: http://bugs.sun.com/view_bug.do?bug_id=6358629
> Specifically, in a thread dump you will see:
> "Timer-9" daemon prio=10 tid=0x5016c400 nid=0x2638 waiting on condition [0x4cbad000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x585021a0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114)
> at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:720)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.sendAlert(SSLSocketImpl.java:1720)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.warning(SSLSocketImpl.java:1564)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1362)
> - locked <0x58502030> (a com.sun.net.ssl.internal.ssl.SSLSocketImpl)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1301)
> at com.sun.net.ssl.internal.ssl.AppOutputStream.close(AppOutputStream.java:80)
> at org.jboss.remoting.transport.socket.TimedOutputStream.close(TimedOutputStream.java:57)
> at org.jboss.remoting.transport.socket.TimedOutputStream$OutputTimerTask.run(TimedOutputStream.java:145)
> at java.util.TimerThread.mainLoop(Timer.java:512)
> at java.util.TimerThread.run(Timer.java:462)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (JBREM-1307) TimedOutputStream OutputTimerTask call to close on SSL OutputStream blocks
by Ron Sigal (JIRA)
[ https://issues.jboss.org/browse/JBREM-1307?page=com.atlassian.jira.plugin... ]
Ron Sigal resolved JBREM-1307.
------------------------------
Resolution: Done
> TimedOutputStream OutputTimerTask call to close on SSL OutputStream blocks
> --------------------------------------------------------------------------
>
> Key: JBREM-1307
> URL: https://issues.jboss.org/browse/JBREM-1307
> Project: JBoss Remoting
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: transport
> Affects Versions: 2.2.3.SP2, 2.5.4.SP4
> Environment: JBoss EAP 4.3.0 CP08 and CP10
> Reporter: Doug Grove
> Assignee: Ron Sigal
> Labels: messaging
> Fix For: 2.5.4.SP5
>
> Attachments: jboss-remoting-src.jar, jboss-remoting.jar
>
>
> In order to provide a mechanism for timing out a network/socket write operation, the org.jboss.remoting.transport.socket.TimedOutputStream class starts a timer. If the duration of the write operation exceeds the timer interval, then the timer will call close() on the output stream. This interrupts the output stream write() operation and causes the write operation to throw an IOException.
> When the output stream uses secure sockets (SSL), however, the call to close() on the output stream can block indefinitely. The threads associated with the write() and close() operation will only clear when the tcp keepalive timeout mechanism in the operation system is invoked.
> This appears to be related to a know bug in the java runtime: http://bugs.sun.com/view_bug.do?bug_id=6358629
> Specifically, in a thread dump you will see:
> "Timer-9" daemon prio=10 tid=0x5016c400 nid=0x2638 waiting on condition [0x4cbad000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x585021a0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114)
> at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:720)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.sendAlert(SSLSocketImpl.java:1720)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.warning(SSLSocketImpl.java:1564)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1362)
> - locked <0x58502030> (a com.sun.net.ssl.internal.ssl.SSLSocketImpl)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1301)
> at com.sun.net.ssl.internal.ssl.AppOutputStream.close(AppOutputStream.java:80)
> at org.jboss.remoting.transport.socket.TimedOutputStream.close(TimedOutputStream.java:57)
> at org.jboss.remoting.transport.socket.TimedOutputStream$OutputTimerTask.run(TimedOutputStream.java:145)
> at java.util.TimerThread.mainLoop(Timer.java:512)
> at java.util.TimerThread.run(Timer.java:462)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months
[JBoss JIRA] (WFLY-2438) Missing/unavailable dependencie jboss.ejb.view-method-security-attributes.foo.bar when deploy ejb type webservice
by Jim Ma (JIRA)
[ https://issues.jboss.org/browse/WFLY-2438?page=com.atlassian.jira.plugin.... ]
Jim Ma resolved WFLY-2438.
--------------------------
Resolution: Done
> Missing/unavailable dependencie jboss.ejb.view-method-security-attributes.foo.bar when deploy ejb type webservice
> -----------------------------------------------------------------------------------------------------------------
>
> Key: WFLY-2438
> URL: https://issues.jboss.org/browse/WFLY-2438
> Project: WildFly
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Web Services
> Affects Versions: 8.0.0.Beta1
> Reporter: Jim Ma
> Assignee: Jim Ma
> Fix For: 8.0.0.CR1
>
>
> Exception is thrown when deploy an ejb webservice without the security domain defined :
> 11:34:18,075 ERROR [org.jboss.as.server] (DeploymentScanner-threads - 1) JBAS015870: Deploy of deployment "WSJarDeploy.jar" was rolled back with the following failure message: {"JBAS014771: Services with missing/unavailable dependencies" => ["jboss.ws.endpoint.\"WSJar.jar\".WSJar is missing [jboss.ejb.view-method-security-attributes.WSJar_component.WSJar_component.WSJar.\"org.jboss.ws.HelloWsBean\"]"]}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
11 years, 2 months