[JBoss JIRA] (JBWEB-287) Websocket application working with multiple clients (e.g. chat style applications) fails due NullPointerException
by Radim Hatlapatka (JIRA)
[ https://issues.jboss.org/browse/JBWEB-287?page=com.atlassian.jira.plugin.... ]
Radim Hatlapatka closed JBWEB-287.
----------------------------------
Verified that the fix done in revision r2334 on 7.4.x branch fixes the issue
> Websocket application working with multiple clients (e.g. chat style applications) fails due NullPointerException
> -----------------------------------------------------------------------------------------------------------------
>
> Key: JBWEB-287
> URL: https://issues.jboss.org/browse/JBWEB-287
> Project: JBoss Web
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Core
> Reporter: Radim Hatlapatka
> Assignee: Remy Maucherat
> Priority: Critical
> Fix For: JBossWeb-7.4.0.GA
>
> Attachments: byteslounge.war
>
>
> When trying to run websocket application with jbossweb 7.4.0.Beta1 (with EAP 6.2.0), websocket connection is successfully established but upon a message sent NullPointerException is thrown:
> {noformat}
> 11:47:27,176 ERROR [org.apache.catalina.core.StandardWrapperValve] (http-localhost.localdomain/127.0.0.1:8080-25) JBWEB000374: IO listener processing for servlet default threw exception: java.lang.NullPointerException
> at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.onWritePossible(WsRemoteEndpointImplServer.java:92) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.tomcat.websocket.server.WsHttpUpgradeHandler$WsWriteListener.onWritePossible(WsHttpUpgradeHandler.java:232) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardWrapperValve.async(StandardWrapperValve.java:603) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardWrapperValve.event(StandardWrapperValve.java:350) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardContextValve.event(StandardContextValve.java:171) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.valves.ValveBase.event(ValveBase.java:185) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardHostValve.event(StandardHostValve.java:247) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.valves.ValveBase.event(ValveBase.java:185) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardEngineValve.event(StandardEngineValve.java:121) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.connector.CoyoteAdapter.event(CoyoteAdapter.java:228) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.coyote.http11.Http11NioProcessor.event(Http11NioProcessor.java:230) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.event(Http11NioProtocol.java:818) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.tomcat.util.net.NioEndpoint$ChannelProcessor.run(NioEndpoint.java:917) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
> 11:47:27,177 ERROR [org.apache.tomcat.websocket.pojo.PojoEndpointBase] (http-localhost.localdomain/127.0.0.1:8080-25) JBWEB008809: No error handling configured for [com.byteslounge.websockets.WebSocketTest] and the following error occurred: java.io.EOFException
> at org.apache.catalina.core.StandardWrapperValve.async(StandardWrapperValve.java:553) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardWrapperValve.event(StandardWrapperValve.java:350) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardContextValve.event(StandardContextValve.java:171) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.valves.ValveBase.event(ValveBase.java:185) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardHostValve.event(StandardHostValve.java:247) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.valves.ValveBase.event(ValveBase.java:185) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardEngineValve.event(StandardEngineValve.java:121) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.connector.CoyoteAdapter.event(CoyoteAdapter.java:237) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.coyote.http11.Http11NioProcessor.event(Http11NioProcessor.java:230) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.event(Http11NioProtocol.java:818) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.tomcat.util.net.NioEndpoint$ChannelProcessor.run(NioEndpoint.java:917) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
> 11:47:27,178 ERROR [org.apache.tomcat.websocket.pojo.PojoEndpointBase] (http-localhost.localdomain/127.0.0.1:8080-25) JBWEB008809: No error handling configured for [com.byteslounge.websockets.WebSocketTest] and the following error occurred: java.io.EOFException
> at org.apache.catalina.core.StandardWrapperValve.async(StandardWrapperValve.java:553) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardWrapperValve.event(StandardWrapperValve.java:350) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardContextValve.event(StandardContextValve.java:171) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.valves.ValveBase.event(ValveBase.java:185) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardHostValve.event(StandardHostValve.java:247) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.valves.ValveBase.event(ValveBase.java:185) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.core.StandardEngineValve.event(StandardEngineValve.java:121) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.catalina.connector.CoyoteAdapter.event(CoyoteAdapter.java:237) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.coyote.http11.Http11NioProcessor.event(Http11NioProcessor.java:230) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.event(Http11NioProtocol.java:818) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at org.apache.tomcat.util.net.NioEndpoint$ChannelProcessor.run(NioEndpoint.java:917) [jbossweb-7.4.0.Beta1.jar:7.4.0.Beta1]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
> {noformat}
> The same application correctly works with Tomcat 7
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 5 months
[JBoss JIRA] (JGRP-1732) UNICAST/NAKACK: threads from the internal thread pool should not do work stealing
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1732?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-1732:
--------------------------------
Possible solutions:
# Ack a message only when it has been *delivered* not *received*. The problem here is that delivery might take a while and so the sender would keep resending the message, which leads to unnecessary traffic.
# After delivery of an INTERNAL|OOB message, check if there are more messages in the table. If so, pick a thread from the regular thread pool to remove and deliver them.
> UNICAST/NAKACK: threads from the internal thread pool should not do work stealing
> ---------------------------------------------------------------------------------
>
> Key: JGRP-1732
> URL: https://issues.jboss.org/browse/JGRP-1732
> Project: JGroups
> Issue Type: Enhancement
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 3.5
>
>
> In NAKACK\{2\} and UNICAST\{2,3\}, threads from all thread pools (regular, OOB and internal) add messages to the table, then grab as many (ordered) messages as possible from the table and pass them up.
> This could lead to the case where an internal thread passes up regular or OOB messages, which might block. There's a (small) chance that this exhausts the internal thread pool.
> Internal threads should therefore never block, and never steal work from other thread pools.
> SOLUTION:
> * An internal thread only adds the message to the table and passes it up if in order
> * If the internal message is OOB, it is passed up and then the thread returns
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 5 months
[JBoss JIRA] (JGRP-1732) UNICAST/NAKACK: threads from the internal thread pool should not do work stealing
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1732?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-1732:
--------------------------------
This can be reproduced with UPerf and 1 sender thread, sending messages to itself: every now and then the sender thread times out on the RPC because the request is not delivered and so no response is received.
When using multiple sender threads, the problem is less likely to occur as different threads will pick up 'leftover' messages and deliver them.
> UNICAST/NAKACK: threads from the internal thread pool should not do work stealing
> ---------------------------------------------------------------------------------
>
> Key: JGRP-1732
> URL: https://issues.jboss.org/browse/JGRP-1732
> Project: JGroups
> Issue Type: Enhancement
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 3.5
>
>
> In NAKACK\{2\} and UNICAST\{2,3\}, threads from all thread pools (regular, OOB and internal) add messages to the table, then grab as many (ordered) messages as possible from the table and pass them up.
> This could lead to the case where an internal thread passes up regular or OOB messages, which might block. There's a (small) chance that this exhausts the internal thread pool.
> Internal threads should therefore never block, and never steal work from other thread pools.
> SOLUTION:
> * An internal thread only adds the message to the table and passes it up if in order
> * If the internal message is OOB, it is passed up and then the thread returns
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 5 months
[JBoss JIRA] (JGRP-1732) UNICAST/NAKACK: threads from the internal thread pool should not do work stealing
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1732?page=com.atlassian.jira.plugin.... ]
Bela Ban reopened JGRP-1732:
----------------------------
There's a problem with not doing work stealing:
* We have A sending unicast messages to B
* A sends 5, 6(OOB|INTERNAL), 7
* B receives regular message 5 and delivers it
* B receives regular message 7 and queues it
* B receives OOB|INTERNAL message 6 and delivers it
** Since message 6 is marked as OOB|INTERNAL, the thread returns and does not perform any work stealing, ie. it does not deliver message 7
* B sends an ACK for 8 to A
** The problem is that now A does *not retransmit A:8*, so it could be picked up by a new thread and get delivered !
==> Message A:8 will not get delivered until A sends another message !
> UNICAST/NAKACK: threads from the internal thread pool should not do work stealing
> ---------------------------------------------------------------------------------
>
> Key: JGRP-1732
> URL: https://issues.jboss.org/browse/JGRP-1732
> Project: JGroups
> Issue Type: Enhancement
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 3.5
>
>
> In NAKACK\{2\} and UNICAST\{2,3\}, threads from all thread pools (regular, OOB and internal) add messages to the table, then grab as many (ordered) messages as possible from the table and pass them up.
> This could lead to the case where an internal thread passes up regular or OOB messages, which might block. There's a (small) chance that this exhausts the internal thread pool.
> Internal threads should therefore never block, and never steal work from other thread pools.
> SOLUTION:
> * An internal thread only adds the message to the table and passes it up if in order
> * If the internal message is OOB, it is passed up and then the thread returns
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 5 months
[JBoss JIRA] (JGRP-1732) UNICAST/NAKACK: threads from the internal thread pool should not do work stealing
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1732?page=com.atlassian.jira.plugin.... ]
Bela Ban edited comment on JGRP-1732 at 1/7/14 2:46 AM:
--------------------------------------------------------
OK, the reason is that UNICAST3 corrects the above scenario quickly (within xmit_interval ms), whereas NAKACK2 has to wait until the next stability message (STABLE), which is seconds by default.
Reverting the change for NAKACK2.
was (Author: belaban):
OK, the reason is that UNICAST3 corrects the above scenario quickly (within xmit_interval ms), whereas NAKACK2 has to wait until the next stability message (STABLE), which is seconds by default.
Reverting the changr for NAKACK2.
> UNICAST/NAKACK: threads from the internal thread pool should not do work stealing
> ---------------------------------------------------------------------------------
>
> Key: JGRP-1732
> URL: https://issues.jboss.org/browse/JGRP-1732
> Project: JGroups
> Issue Type: Enhancement
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 3.5
>
>
> In NAKACK\{2\} and UNICAST\{2,3\}, threads from all thread pools (regular, OOB and internal) add messages to the table, then grab as many (ordered) messages as possible from the table and pass them up.
> This could lead to the case where an internal thread passes up regular or OOB messages, which might block. There's a (small) chance that this exhausts the internal thread pool.
> Internal threads should therefore never block, and never steal work from other thread pools.
> SOLUTION:
> * An internal thread only adds the message to the table and passes it up if in order
> * If the internal message is OOB, it is passed up and then the thread returns
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 5 months