[JBoss JIRA] (WFLY-6926) TimeoutException: Replication timeout when handling request with DIST cache
by Michal Vinkler (JIRA)
[ https://issues.jboss.org/browse/WFLY-6926?page=com.atlassian.jira.plugin.... ]
Michal Vinkler commented on WFLY-6926:
--------------------------------------
[~pferraro] You are right, the issue is gone when using -Dinfinispan.stagger.delay=0.
Link to run: http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-failo...
> TimeoutException: Replication timeout when handling request with DIST cache
> ---------------------------------------------------------------------------
>
> Key: WFLY-6926
> URL: https://issues.jboss.org/browse/WFLY-6926
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 10.1.0.CR1
> Reporter: Michal Vinkler
> Assignee: Paul Ferraro
> Priority: Blocker
> Fix For: 10.1.0.Final
>
>
> Seen in http-session scenarios with *DIST* cache only. REPL cache scenarios seem not to be affected.
> TimeoutExceptions occur right away after sending first request and definitely result in client getting 500:
> {code}
> [JBossINF] [0m[31m04:31:39,033 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-1) ISPN000136: Error executing command GetKeyValueCommand, writing keys []: org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf19
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:801)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:642)
> [JBossINF] at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> [JBossINF] at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> [JBossINF] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> [JBossINF] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.staggeredProcessNext(CommandAwareRpcDispatcher.java:375)
> [JBossINF] at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.lambda$processCallsStaggered$3(CommandAwareRpcDispatcher.java:357)
> [JBossINF] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [JBossINF] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> [JBossINF] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [JBossINF] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [JBossINF] at java.lang.Thread.run(Thread.java:745)
> {code}
> Client:
> {code}
> 2016/08/08 04:31:39:068 EDT [WARN ][Runner - 4] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error sampling data: Internal Server Error>
> org.jboss.smartfrog.loaddriver.RequestProcessingException: Invalid response code: 500 Content: Internal Server Error
> at org.jboss.smartfrog.loaddriver.http.HttpRequestProcessorFactoryImpl$HttpRequestProcessor.processRequest(HttpRequestProcessorFactoryImpl.java:163)
> at org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:52)
> at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:103)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> I was able to reproduce it even with a very small load of 30 sessions.
> Server link (run with 30 sessions):
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-failo...
> Client link (run with 30 sessions):
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-failo...
> This issue occurs under different conditions than JBEAP-3779 (which is similar), that's why I opened new issue.
> Number of occurrences of TimeoutExceptions is very high. The server logs are full of them.
> This issue makes clustering with *DIST* cache unusable, thus giving blocker priority.
> Also, there is a issue with shutting down the servers in the end of the test, I don't know if it can be related, but I am hitting it constantly in the http-session DIST scenarios.
> The server shutdown usually gets stuck on this command:
> {code}
> [JBossINF] [0m[0m04:38:51,694 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-7) ISPN000082: Stopping the RpcDispatcher for channel web
> {code}
> *EDIT:*
> Also ejb-remote tests with *DIST* cache are affected (even if only 20 sessions are created).
> Server log for ejb-remote run:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-failo...
> Client log for ejb-remote run:
> http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/mvinkler_eap-7x-failo...
> *EDIT #2:*
> Can be very easily reproduced locally. Just start a cluster of at least 3 servers with clusterbench deployed. Then do few requests to various nodes until you get 500 as a response. Also notice, when shutting down a node, which logged TimeoutException before, the node won't stop and get stuck during the shutdown.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFCORE-897) IPv6 address in security realm using Kerberos
by Bartosz Baranowski (JIRA)
[ https://issues.jboss.org/browse/WFCORE-897?page=com.atlassian.jira.plugin... ]
Bartosz Baranowski reassigned WFCORE-897:
-----------------------------------------
Assignee: Darran Lofthouse (was: Bartosz Baranowski)
> IPv6 address in security realm using Kerberos
> ---------------------------------------------
>
> Key: WFCORE-897
> URL: https://issues.jboss.org/browse/WFCORE-897
> Project: WildFly Core
> Issue Type: Bug
> Components: Security
> Reporter: Martin Choma
> Assignee: Darran Lofthouse
> Priority: Optional
>
> When kerberos in realm is configured to use IPv6 address with square brackets, eg. [2620:52:0:2804:56ee:75ff:fe34:630e], EAP generates TGS-REQ for remote/2620:52:0:2804:56ee:75ff:fe34:630e instead of remote/[2620:52:0:2804:56ee:75ff:fe34:630e]. It cause failures when remote/[2620:52:0:2804:56ee:75ff:fe34:630e]@JBOSS.ORG is used in keytab.
> This happens when such realm secures CLI or EJB remoting. It doesnt happen when used for securing management console."
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFCORE-1408) Cannot obtain DOMImplementationRegistry instance
by Bartosz Baranowski (JIRA)
[ https://issues.jboss.org/browse/WFCORE-1408?page=com.atlassian.jira.plugi... ]
Bartosz Baranowski reassigned WFCORE-1408:
------------------------------------------
Assignee: Jason Greene (was: Bartosz Baranowski)
> Cannot obtain DOMImplementationRegistry instance
> ------------------------------------------------
>
> Key: WFCORE-1408
> URL: https://issues.jboss.org/browse/WFCORE-1408
> Project: WildFly Core
> Issue Type: Bug
> Reporter: Bartosz Baranowski
> Assignee: Jason Greene
>
> testDOMImplementationRegistry(org.jboss.as.test.smoke.xml.DOMImplementationRegistryTestCase) Time elapsed: 0.09 sec <<< ERROR!
> java.lang.ClassNotFoundException: com.sun.org.apache.xerces.internal.dom.DOMXSImplementationSourceImpl from [Module "deployment.dom-registry-test:main" from Service Module Loader]
> at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:213)
> at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:459)
> at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:408)
> at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:389)
> at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:134)
> at org.w3c.dom.bootstrap.DOMImplementationRegistry.newInstance(DOMImplementationRegistry.java:182)
> at org.jboss.as.test.smoke.xml.DOMImplementationRegistryTestCase.testDOMImplementationRegistry(DOMImplementationRegistryTestCase.java:52)
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFCORE-1717) ExtensionXml incorrectly handles duplicate module names
by Brian Stansberry (JIRA)
Brian Stansberry created WFCORE-1717:
----------------------------------------
Summary: ExtensionXml incorrectly handles duplicate module names
Key: WFCORE-1717
URL: https://issues.jboss.org/browse/WFCORE-1717
Project: WildFly Core
Issue Type: Bug
Components: Domain Management
Affects Versions: 3.0.0.Alpha5, 2.2.0.Final
Reporter: Toby Crawley
Fix For: 3.0.0.CR1
Toby Crawley reports:
- adding a duplicate extension under server > extensions causes an
IllegalStateException that occurs when trying to generate the
XMLStreamException
(https://gist.github.com/tobias/59d155afe0c88e268b83cb75734353eb)
The problem happens because before the parser calls the invalidAttributeValue method to throw an exception, it has already consumed the element, preventing invalidAttributeValue working properly.
This should really be a custom exception that more clearly explains the problem, which is that the 'module' attribute value must be unique across the extension elements.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFLY-6958) Use more intuitive values for consistent-hash-strategy
by Paul Ferraro (JIRA)
Paul Ferraro created WFLY-6958:
----------------------------------
Summary: Use more intuitive values for consistent-hash-strategy
Key: WFLY-6958
URL: https://issues.jboss.org/browse/WFLY-6958
Project: WildFly
Issue Type: Enhancement
Components: Clustering
Affects Versions: 10.0.0.Final
Reporter: Paul Ferraro
Assignee: Paul Ferraro
Fix For: 11.0.0.Alpha1
Currently consistent-hash-strategy can have on of 2 values: INTRA_CACHE vs INTER_CACHE.
This was initially meant to describe a consistent hash that would be consistent within a cache vs across multiple caches within a cache container. In retrospect, these value are not very descriptive.
Better would be consistent-hash-strategy="AGE|ADDRESS" or "AGE|NAME", where the strategy uses the cache view sorted by either age, or name/address.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFCORE-1716) JMX domains jboss.as and jboss.as.expr do not always correctly handle propertly list patters in queryMBeans and queryNames
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFCORE-1716?page=com.atlassian.jira.plugi... ]
Brian Stansberry updated WFCORE-1716:
-------------------------------------
Fix Version/s: 3.0.0.Alpha6
> JMX domains jboss.as and jboss.as.expr do not always correctly handle propertly list patters in queryMBeans and queryNames
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: WFCORE-1716
> URL: https://issues.jboss.org/browse/WFCORE-1716
> Project: WildFly Core
> Issue Type: Bug
> Components: JMX
> Reporter: Brian Stansberry
> Assignee: Brian Stansberry
> Fix For: 3.0.0.Alpha6
>
>
> The ObjectName filtering logic added for WFLY-3161 does not handle queryMBeans and queryNames correctly in some cases where the ObjectName param to the method is a property list pattern (i.e. one that includes a simple '*' at the end of the list of properties, meaning "all other properties besides previous ones whose keys were specifically enumerated, match".)
> The problem occurs when the the ObjectName does specifically include some keys, and those keys don't correspond to the final elements of the related management resource's PathAddress. As the RootResourceIterator walks the management resource tree, ModelControllerMBeanHelper.ObjectNameMatchResourceAction will not identify the parent resources of children that should match as matching, with the result that the iterator will not descend into that part of the tree and the children will not be matched. For example, this query will return an empty set because the /socket-binding-group=standard-sockets parent will not be matched, preventing checks of the socket-binding-group children.
> {code}
> Set<ObjectInstance> instances = connection.queryMBeans(createObjectName("jboss.as:socket-binding=*,*"), null);
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFCORE-1716) JMX domains jboss.as and jboss.as.expr do not always correctly handle propertly list patters in queryMBeans and queryNames
by Brian Stansberry (JIRA)
Brian Stansberry created WFCORE-1716:
----------------------------------------
Summary: JMX domains jboss.as and jboss.as.expr do not always correctly handle propertly list patters in queryMBeans and queryNames
Key: WFCORE-1716
URL: https://issues.jboss.org/browse/WFCORE-1716
Project: WildFly Core
Issue Type: Bug
Components: JMX
Reporter: Brian Stansberry
Assignee: Brian Stansberry
The ObjectName filtering logic added for WFLY-3161 does not handle queryMBeans and queryNames correctly in some cases where the ObjectName param to the method is a property list pattern (i.e. one that includes a simple '*' at the end of the list of properties, meaning "all other properties besides previous ones whose keys were specifically enumerated, match".)
The problem occurs when the the ObjectName does specifically include some keys, and those keys don't correspond to the final elements of the related management resource's PathAddress. As the RootResourceIterator walks the management resource tree, ModelControllerMBeanHelper.ObjectNameMatchResourceAction will not identify the parent resources of children that should match as matching, with the result that the iterator will not descend into that part of the tree and the children will not be matched. For example, this query will return an empty set because the /socket-binding-group=standard-sockets parent will not be matched, preventing checks of the socket-binding-group children.
{code}
Set<ObjectInstance> instances = connection.queryMBeans(createObjectName("jboss.as:socket-binding=*,*"), null);
{code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFCORE-1164) Warning about private module use issued twice
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/WFCORE-1164?page=com.atlassian.jira.plugi... ]
RH Bugzilla Integration commented on WFCORE-1164:
-------------------------------------------------
Aaron Ogburn <aogburn(a)redhat.com> changed the Status of [bug 1365668|https://bugzilla.redhat.com/show_bug.cgi?id=1365668] from ASSIGNED to POST
> Warning about private module use issued twice
> ---------------------------------------------
>
> Key: WFCORE-1164
> URL: https://issues.jboss.org/browse/WFCORE-1164
> Project: WildFly Core
> Issue Type: Bug
> Components: Server
> Reporter: Frank Langelage
> Assignee: Stuart Douglas
> Priority: Minor
> Fix For: 2.0.4.Final
>
> Attachments: test.war
>
>
> My web app is using some modules of WildFly AS which are not public.
> The warning about using private modules is issued twice for every module:
> 09.11. 22:07:34,088 WARN [org.jboss.as.dependency.private#start] WFLYSRV0018: Deployment "deployment.web-maj2e-langfr-dev.war" is using a private module ("org.jboss.common-core:main") which may be changed or removed in future versions without notice.
> 09.11. 22:07:34,090 WARN [org.jboss.as.dependency.private#start] WFLYSRV0018: Deployment "deployment.web-maj2e-langfr-dev.war" is using a private module ("org.jboss.common-core:main") which may be changed or removed in future versions without notice.
> 09.11. 22:07:34,091 WARN [org.jboss.as.dependency.private#start] WFLYSRV0018: Deployment "deployment.web-maj2e-langfr-dev.war" is using a private module ("org.apache.commons.collections:main") which may be changed or removed in future versions without notice.
> 09.11. 22:07:34,091 WARN [org.jboss.as.dependency.private#start] WFLYSRV0018: Deployment "deployment.web-maj2e-langfr-dev.war" is using a private module ("org.apache.commons.collections:main") which may be changed or removed in future versions without notice.
> 09.11. 22:07:34,092 WARN [org.jboss.as.dependency.private#start] WFLYSRV0018: Deployment "deployment.web-maj2e-langfr-dev.war" is using a private module ("org.apache.cxf.impl:main") which may be changed or removed in future versions without notice.
> 09.11. 22:07:34,094 WARN [org.jboss.as.dependency.private#start] WFLYSRV0018: Deployment "deployment.web-maj2e-langfr-dev.war" is using a private module ("org.apache.cxf.impl:main") which may be changed or removed in future versions without notice.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months