[JBoss JIRA] (JGRP-1675) Threads stuck in FlowControl.decrementIfEnoughCredits
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/JGRP-1675?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on JGRP-1675:
------------------------------------
There seems to be another scenario where the deadlock happens right at the beginning of the test and the originators don't receive any complete responses:
UFC:REPLENISH messages are usually sent after a message has been delivered to the application. If the application also wants to send a response message, and the response message can't be sent because of insufficient credits, the node won't send REPLENISH messages.
The thread that is trying to send the response will periodically wake up and send a CREDIT_REQUEST message. But the CREDIT_REQUEST message doesn't have the OOB flag, so it will be blocked in UNICAST3 if the recipient is also overloaded and already dropped some OOB messages.
> Threads stuck in FlowControl.decrementIfEnoughCredits
> -----------------------------------------------------
>
> Key: JGRP-1675
> URL: https://issues.jboss.org/browse/JGRP-1675
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 3.4
> Reporter: Radim Vansa
> Assignee: Bela Ban
> Fix For: 3.5
>
> Attachments: jgroups-udp-radim.xml, RemoteGetStressTest.java, UPerf2.java
>
>
> I have recently observed a repeated situation where many (or all) threads have been stuck waiting for credits in FlowControl protocol.
> The credit request was not handled on the other node as this is non-oob message and some (actually many of them - cause unknown) messages before the request have been lost - therefore the request was waiting for them to be re-sent.
> However, these have not been re-sent properly as the retransmission request was not received - all OOB threads were stuck in the FlowControl protocol as these handled some other request and tried to send a response - but the response could not be sent until FlowControl gets the credits.
> The probability of such situation could be lowered by tagging the credit request to be OOB - then it would be handled immediately. If the credit replenish message would then be processed in regular OOB pool, this could get already depleted by many requests, but setting up the internal thread pool would solve the problem.
> Other consideration would be to allow releasing thread from FlowControl (let it send the message even without credits) if it waits there for too long.
> h3. Workaround
> It appears that setting MFC and UFC max_credits to 10M or removing these protocols at all is a workaround for this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (WFLY-2439) Arquillian Wildfly Managed - support exploded mode
by Ondrej Zizka (JIRA)
Ondrej Zizka created WFLY-2439:
----------------------------------
Summary: Arquillian Wildfly Managed - support exploded mode
Key: WFLY-2439
URL: https://issues.jboss.org/browse/WFLY-2439
Project: WildFly
Issue Type: Feature Request
Security Level: Public (Everyone can see)
Components: Test Suite
Affects Versions: 8.0.0.Beta1
Reporter: Ondrej Zizka
Arquillian WildFly managed container currently doesn't support exploded mode deployment. JBDS supports it through moving the files to the standalone/deployments dir. It would be nice if we could add some tests covering this to the WildFly testsuite.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (WFLY-2391) Wildfly caches content in exploded mode, breaking developer productivity
by Xavier Coulon (JIRA)
[ https://issues.jboss.org/browse/WFLY-2391?page=com.atlassian.jira.plugin.... ]
Xavier Coulon commented on WFLY-2391:
-------------------------------------
Following [~goot]'s idea, I configured the buffer-cache to its minimal size as a workaround and it works:
{code}
<buffer-caches>
<buffer-cache name="default" buffer-size="1" buffers-per-region="1" max-regions="1"/>
</buffer-caches>
{code}
You should probably load an unrelated page (like /index.html) 5 times in a row after server startup to fill the cache before working on your application, since the minimal acceptable cache size is 1, not 0.
Granted, this is just a workaround ;-)
> Wildfly caches content in exploded mode, breaking developer productivity
> ------------------------------------------------------------------------
>
> Key: WFLY-2391
> URL: https://issues.jboss.org/browse/WFLY-2391
> Project: WildFly
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Web (Undertow)
> Affects Versions: 8.0.0.Beta1
> Environment: MacOSX with JBossTools 4.1.1.Beta1
> Reporter: Xavier Coulon
> Assignee: Stuart Douglas
> Attachments: jboss-as-kitchensink-html5-mobile.war.zip
>
>
> While I was building a sample application with JBoss Tools 4.1.1.beta1 on WildFly 8.0.Beta1, I noticed that after a few minutes (or a few browser requests), the content of my index.html file seemed to be cached by the server, although I used the "exploded content" deployment mode (since I published the content using the WildFly Server Adapter in JBoss Tools).
>
> I checked the actual content of the index.html in the deployments folder and it contained the latest changes, which means that the JBoss Tools Server Adapter is doing its job well ;-)
> I also tried to edit the index.html file directly in the deployments folder, and once again, I got no update in both Chrome and Firefox browsers.
>
> I checked in the "Network" tab of the browsers and could see that the server response for the index.html page had a "200 OK" status, which means that there's no browser caching involved.
> I tried to get the index.html page with cUrl and got the same old version, which definitely excludes a browser caching issue.
> At the end of the dat, this means that after a few changes, my browsers keep getting an old version of the deployed resources, which in turns means that I have to stop and restart the server to get the new content, and this is pretty bad in term of dev productivity.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (WFLY-2427) Launcher API
by Andre Dietisheim (JIRA)
[ https://issues.jboss.org/browse/WFLY-2427?page=com.atlassian.jira.plugin.... ]
Andre Dietisheim commented on WFLY-2427:
----------------------------------------
I can offer my help to tackle this. Looks like a low hanging fruit to get into things. WDYT?
> Launcher API
> ------------
>
> Key: WFLY-2427
> URL: https://issues.jboss.org/browse/WFLY-2427
> Project: WildFly
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Components: Server
> Reporter: Brian Stansberry
>
> 1) The AS should have some sort of API for launching our processes so tools that want a process have a clear contract instead of having to guess at what's relevant in our ever-changing scripts.
> 2) We want the main class in our process launch to be what's invoked by java -jar jboss-modules.jar. We don't want java -jar jboss-as-launcher.jar which does some stuff and then calls org.jboss.modules.Main.
> 3) JBoss Modules itself shouldn't have a lot of the stuff in it that's relevant to an AS launcher API, because many of those things are not relevant to JBoss Modules in a generic sense.
> What we could do though is provide a launcher lib that isn't involved at all in our normal boot. Something that would only be used by tools that want to launch a separate, i.e. non-embedded, AS process.
> So, some sort of stable configuration API and then a simple
> java.lang.Process launch()
> Basically, a utility that does the ProcessBuilder stuff that everybody is doing themselves now.
> HOWEVER...
> Eclipse-based tools like JBDS use Eclipse APIs for launch and would not use the above launch() method.
> So, besides that launch method, look into adding some methods to give the necessary inputs to the Eclipse API be useful. So Eclipse-based tools don't ask it for the process but can still get a standard launch configuration.
> I'd only want to do that if those methods would return something generally understandable, but a String or List<String> for classpath, List<String>s for vm/program args, some representation that "-jar jboss-modules.jar" is the way to get the main class -- those all seem generic enough.
> Any "which VM" stuff is consider out of scope; choosing the VM is the responsibility of the tool. Options that are not universally supported across VMs and are those a function of VM choice, like whether to use -server, are also out of scope.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (JGRP-1675) Threads stuck in FlowControl.decrementIfEnoughCredits
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/JGRP-1675?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on JGRP-1675:
------------------------------------
I was able to reproduce the deadlock again with an updated test, here: https://github.com/danberindei/JGroups/blob/JGRP-1675/tests/stress/org/jg...
It looks like a regular message can be processed on the INT thread pool:
{noformat}
"INT-2,cluster,D@1395" prio=5 tid=0x32 nid=NA waiting
java.lang.Thread.State: WAITING
at java.lang.Object.wait(Object.java:-1)
at org.jgroups.protocols.FlowControl$Credit.decrementIfEnoughCredits(FlowControl.java:567)
at org.jgroups.protocols.UFC.handleDownMessage(UFC.java:121)
at org.jgroups.protocols.FlowControl.down(FlowControl.java:330)
at org.jgroups.protocols.FlowControl.down(FlowControl.java:340)
at org.jgroups.protocols.FRAG2.fragment(FRAG2.java:243)
at org.jgroups.protocols.FRAG2.down(FRAG2.java:122)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:1024)
at org.jgroups.JChannel.down(JChannel.java:760)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.down(MessageDispatcher.java:684)
at org.jgroups.blocks.RequestCorrelator.sendResponse(RequestCorrelator.java:516)
at org.jgroups.blocks.RequestCorrelator.sendReply(RequestCorrelator.java:506)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:476)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:374)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:666)
at org.jgroups.JChannel.up(JChannel.java:730)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1019)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:182)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:434)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:434)
at org.jgroups.stack.Protocol.up(Protocol.java:409)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294)
at org.jgroups.protocols.UNICAST3.removeAndDeliver(UNICAST3.java:796)
at org.jgroups.protocols.UNICAST3.handleBatchReceived(UNICAST3.java:752)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:466)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:662)
at org.jgroups.stack.Protocol.up(Protocol.java:409)
at org.jgroups.protocols.TP.passBatchUp(TP.java:1422)
at org.jgroups.protocols.TP$BatchHandler.run(TP.java:1574)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
While debugging, I was able to look at the message batch in UNICAST3.handleBatchReceived, and see that it still has a REPLENISH message waiting to be delivered.
> Threads stuck in FlowControl.decrementIfEnoughCredits
> -----------------------------------------------------
>
> Key: JGRP-1675
> URL: https://issues.jboss.org/browse/JGRP-1675
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 3.4
> Reporter: Radim Vansa
> Assignee: Bela Ban
> Fix For: 3.5
>
> Attachments: jgroups-udp-radim.xml, RemoteGetStressTest.java, UPerf2.java
>
>
> I have recently observed a repeated situation where many (or all) threads have been stuck waiting for credits in FlowControl protocol.
> The credit request was not handled on the other node as this is non-oob message and some (actually many of them - cause unknown) messages before the request have been lost - therefore the request was waiting for them to be re-sent.
> However, these have not been re-sent properly as the retransmission request was not received - all OOB threads were stuck in the FlowControl protocol as these handled some other request and tried to send a response - but the response could not be sent until FlowControl gets the credits.
> The probability of such situation could be lowered by tagging the credit request to be OOB - then it would be handled immediately. If the credit replenish message would then be processed in regular OOB pool, this could get already depleted by many requests, but setting up the internal thread pool would solve the problem.
> Other consideration would be to allow releasing thread from FlowControl (let it send the message even without credits) if it waits there for too long.
> h3. Workaround
> It appears that setting MFC and UFC max_credits to 10M or removing these protocols at all is a workaround for this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (JGRP-1675) Threads stuck in FlowControl.decrementIfEnoughCredits
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1675?page=com.atlassian.jira.plugin.... ]
Bela Ban updated JGRP-1675:
---------------------------
Attachment: UPerf2.java
The test to see if this blocks or not is Dan's UPerf2 (attached)
> Threads stuck in FlowControl.decrementIfEnoughCredits
> -----------------------------------------------------
>
> Key: JGRP-1675
> URL: https://issues.jboss.org/browse/JGRP-1675
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 3.4
> Reporter: Radim Vansa
> Assignee: Bela Ban
> Fix For: 3.5
>
> Attachments: jgroups-udp-radim.xml, RemoteGetStressTest.java, UPerf2.java
>
>
> I have recently observed a repeated situation where many (or all) threads have been stuck waiting for credits in FlowControl protocol.
> The credit request was not handled on the other node as this is non-oob message and some (actually many of them - cause unknown) messages before the request have been lost - therefore the request was waiting for them to be re-sent.
> However, these have not been re-sent properly as the retransmission request was not received - all OOB threads were stuck in the FlowControl protocol as these handled some other request and tried to send a response - but the response could not be sent until FlowControl gets the credits.
> The probability of such situation could be lowered by tagging the credit request to be OOB - then it would be handled immediately. If the credit replenish message would then be processed in regular OOB pool, this could get already depleted by many requests, but setting up the internal thread pool would solve the problem.
> Other consideration would be to allow releasing thread from FlowControl (let it send the message even without credits) if it waits there for too long.
> h3. Workaround
> It appears that setting MFC and UFC max_credits to 10M or removing these protocols at all is a workaround for this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (JGRP-1675) Threads stuck in FlowControl.decrementIfEnoughCredits
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1675?page=com.atlassian.jira.plugin.... ]
Bela Ban updated JGRP-1675:
---------------------------
Attachment: jgroups-udp-radim.xml
Works (= doesn't block) with jgroups-udp.radim.xml. The different is the larger thread pools in Radim's config.
> Threads stuck in FlowControl.decrementIfEnoughCredits
> -----------------------------------------------------
>
> Key: JGRP-1675
> URL: https://issues.jboss.org/browse/JGRP-1675
> Project: JGroups
> Issue Type: Bug
> Affects Versions: 3.4
> Reporter: Radim Vansa
> Assignee: Bela Ban
> Fix For: 3.5
>
> Attachments: jgroups-udp-radim.xml, RemoteGetStressTest.java
>
>
> I have recently observed a repeated situation where many (or all) threads have been stuck waiting for credits in FlowControl protocol.
> The credit request was not handled on the other node as this is non-oob message and some (actually many of them - cause unknown) messages before the request have been lost - therefore the request was waiting for them to be re-sent.
> However, these have not been re-sent properly as the retransmission request was not received - all OOB threads were stuck in the FlowControl protocol as these handled some other request and tried to send a response - but the response could not be sent until FlowControl gets the credits.
> The probability of such situation could be lowered by tagging the credit request to be OOB - then it would be handled immediately. If the credit replenish message would then be processed in regular OOB pool, this could get already depleted by many requests, but setting up the internal thread pool would solve the problem.
> Other consideration would be to allow releasing thread from FlowControl (let it send the message even without credits) if it waits there for too long.
> h3. Workaround
> It appears that setting MFC and UFC max_credits to 10M or removing these protocols at all is a workaround for this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (WFLY-2383) resource-adapter id attribute has to match ra name otherwise MDB is not deployed
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/WFLY-2383?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on WFLY-2383:
-----------------------------------------------
Ondrej Chaloupka <ochaloup(a)redhat.com> made a comment on [bug 1012044|https://bugzilla.redhat.com/show_bug.cgi?id=1012044]
This was not fixed. When I use the same rar name for archive and id and mdb link then I get exception [1] (I'm working with activemq). The only way how to really load the RAR I have to define names in the following way:
<subsystem xmlns="urn:jboss:domain:ejb3:1.4">
<mdb>
<resource-adapter-ref resource-adapter-name="activemq.rar"/>
<bean-instance-pool-ref pool-name="mdb-strict-max-pool"/>
</mdb>
...
<subsystem xmlns="urn:jboss:domain:resource-adapters:1.1">
<resource-adapters>
<resource-adapter id="activemq">
<archive>activemq.rar</archive>
...
which means that archive has to be archive.rar and the id just archive.
[1]
ERROR [org.jboss.msc.service.fail] (MSC service thread 1-1) MSC000001: Failed to start service jboss.deployment.unit."jbossts_crashrecovery-jms-mdb_crashrecovery-jms-mdb.jar".component.mdb.CREATE: org.jboss.msc.service.StartException in service jboss.deployment.unit."jbossts_crashrecovery-jms-mdb_crashrecovery-jms-mdb.jar".component.mdb.CREATE: Failed to start service
at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1767) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
Caused by: java.lang.IllegalStateException: JBAS014331: No resource adapter registered with resource adapter name activemq
at org.jboss.as.ejb3.component.EJBUtilities.createActivationSpecs(EJBUtilities.java:85)
at org.jboss.as.ejb3.component.messagedriven.MessageDrivenComponentCreateService.createComponent(MessageDrivenComponentCreateService.java:93)
at org.jboss.as.ee.component.BasicComponentCreateService.start(BasicComponentCreateService.java:91)
at org.jboss.as.ejb3.component.messagedriven.MessageDrivenComponentCreateService.start(MessageDrivenComponentCreateService.java:79)
at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.4.GA-redhat-1.jar:1.0.4.GA-redhat-1]
... 3 more
> resource-adapter id attribute has to match ra name otherwise MDB is not deployed
> --------------------------------------------------------------------------------
>
> Key: WFLY-2383
> URL: https://issues.jboss.org/browse/WFLY-2383
> Project: WildFly
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Reporter: Stefano Maestri
> Assignee: Stefano Maestri
>
> The resource-adapter id has to have the same name as the ra archive has. Otherwise the the MDB is not deployed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months