[JBoss JIRA] (WFCORE-40) Deadlock while logging
by Andreas Kaster (JIRA)
[ https://issues.jboss.org/browse/WFCORE-40?page=com.atlassian.jira.plugin.... ]
Andreas Kaster updated WFCORE-40:
---------------------------------
Attachment: stack_140901_2254
Attached the stack file from the last crash
> Deadlock while logging
> ----------------------
>
> Key: WFCORE-40
> URL: https://issues.jboss.org/browse/WFCORE-40
> Project: WildFly Core
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Logging
> Environment: CentOS 6.5 64bit, java7u45 64bit (and 32 bit, the same behavior)
> Reporter: Stefan Schueffler
> Assignee: James Perkins
> Priority: Critical
> Attachments: stack_140901_2254
>
>
> We hit really often a deadlock? in org.jboss.stdio.StdioContext$DelegatingPrintStream.println(StdioContext.java:474)
> Even if the "StdioContext" belongs to Jboss-Logging, the problem occurs in our production wildfly installation twice to 4 times a day - all threads are deadlocking while trying to log.debug, log.error, or (sometimes) System.out.println from our application code, and wildfly does not respond anymore...
> The partial stacktrace always is similar to this one:
> {code}
> "default task-64" prio=10 tid=0x4c539c00 nid=0x5ef9 waiting for monitor entry [0x495e0000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.io.PrintStream.println(PrintStream.java:806)
> - waiting to lock <0x5ee0adf8> (a java.io.PrintStream)
> at org.jboss.stdio.StdioContext$DelegatingPrintStream.println(StdioContext.java:474)
> at jsp.communications.statuschange.selectStatus_jsp._jspService(selectStatus_jsp.java:413)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:69)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
> at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:82)
> {code}
> While investigating the StdioContext - class, i really wondered whether the used "locking/checking by using a threadlocal" could have worked in a multi-threaded environment (it should have the very same problems as every "double checking" algorithm without proper synchronization).
> If all threads are hanging in this particular lock, only a full wildfly-restart recovers in our case.
> My preferred solution would be a rework of the used org.jboss.stdio. classes, as the used idioms of ThreadLocals for reentrant-checks are at least highly unusual?
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 10 months
[JBoss JIRA] (WFLY-3464) Nonexistent ldap group causes authentication to fail in security-realm
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/WFLY-3464?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on WFLY-3464:
-----------------------------------------------
Josef Cacek <jcacek(a)redhat.com> changed the Status of [bug 1128176|https://bugzilla.redhat.com/show_bug.cgi?id=1128176] from ON_QA to VERIFIED
> Nonexistent ldap group causes authentication to fail in security-realm
> -----------------------------------------------------------------------
>
> Key: WFLY-3464
> URL: https://issues.jboss.org/browse/WFLY-3464
> Project: WildFly
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Domain Management
> Affects Versions: 8.1.0.Final
> Reporter: Derek Horton
> Assignee: Darran Lofthouse
>
> The LdapGroupSearcher code will fail if it tries to lookup a group that
> does not exist on the local ldap server.
> This can happen when the ldap systems are configured as trusted domains.
> Even though the security-realm is not configured to use the trusted domain
> (it is configured to only look at a single ldap server), the
> user's entry on one ldap server could point at a group that exists on
> the other (trusted) ldap server.
> The LdapGroupSearcher code attempts to lookup this role and it fails. This
> failure is sent back to the http server which results in an HTTP 500 error
> and leaves the user with no way to authenticate/login.
> There is currently not a way to tell the group searcher code to ignore the
> group/role that cannot be found.
> 2014-06-06 12:44:39,819 TRACE [org.jboss.as.domain.management.security] (XNIO-1 task-1) Group found with distinguishedName=cn=TestManagedRole,ou=People,dc=my-ds-domain,dc=com
> 2014-06-06 12:44:39,821 TRACE [org.jboss.as.domain.management.security] (XNIO-1 task-1) Failure supplementing Subject: javax.naming.NameNotFoundException: [LDAP: error code 32 - No Such Object]; remaining name 'cn=TestManagedRole,ou=People,dc=my-ds-domain,dc=com'
> at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3112) [rt.jar:1.7.0_45]
> at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3033) [rt.jar:1.7.0_45]
> at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2840) [rt.jar:1.7.0_45]
> at com.sun.jndi.ldap.LdapCtx.c_getAttributes(LdapCtx.java:1332) [rt.jar:1.7.0_45]
> at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_getAttributes(ComponentDirContext.java:231) [rt.jar:1.7.0_45]
> at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:139) [rt.jar:1.7.0_45]
> at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:127) [rt.jar:1.7.0_45]
> at javax.naming.directory.InitialDirContext.getAttributes(InitialDirContext.java:142) [rt.jar:1.7.0_45]
> at javax.naming.directory.InitialDirContext.getAttributes(InitialDirContext.java:142) [rt.jar:1.7.0_45]
> at org.jboss.as.domain.management.security.LdapGroupSearcherFactory$PrincipalToGroupSearcher.search(LdapGroupSearcherFactory.java:256) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.management.security.LdapGroupSearcherFactory$PrincipalToGroupSearcher.search(LdapGroupSearcherFactory.java:191) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.management.security.LdapCacheService$NoCacheCache.search(LdapCacheService.java:223) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.management.security.LdapSubjectSupplementalService$LdapSubjectSupplemental.loadGroupEntries(LdapSubjectSupplementalService.java:218) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.management.security.LdapSubjectSupplementalService$LdapSubjectSupplemental.loadGroups(LdapSubjectSupplementalService.java:195) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.management.security.LdapSubjectSupplementalService$LdapSubjectSupplemental.loadGroups(LdapSubjectSupplementalService.java:188) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.management.security.LdapSubjectSupplementalService$LdapSubjectSupplemental.supplementSubject(LdapSubjectSupplementalService.java:163) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.management.security.SecurityRealmService$1.createSubjectUserInfo(SecurityRealmService.java:200) [wildfly-domain-management-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.http.server.security.RealmIdentityManager.verify(RealmIdentityManager.java:155) [wildfly-domain-http-interface-8.1.0.Final.jar:8.1.0.Final]
> at org.jboss.as.domain.http.server.security.RealmIdentityManager.verify(RealmIdentityManager.java:120) [wildfly-domain-http-interface-8.1.0.Final.jar:8.1.0.Final]
> at io.undertow.security.impl.BasicAuthenticationMechanism.authenticate(BasicAuthenticationMechanism.java:110) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
> at org.jboss.as.domain.http.server.security.AuthenticationMechanismWrapper.authenticate(AuthenticationMechanismWrapper.java:57) [wildfly-domain-http-interface-8.1.0.Final.jar:8.1.0.Final]
> at io.undertow.security.impl.SecurityContextImpl$AuthAttempter.transition(SecurityContextImpl.java:281) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
> at io.undertow.security.impl.SecurityContextImpl$AuthAttempter.transition(SecurityContextImpl.java:298) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
> at io.undertow.security.impl.SecurityContextImpl$AuthAttempter.access$100(SecurityContextImpl.java:268) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
> at io.undertow.security.impl.SecurityContextImpl.attemptAuthentication(SecurityContextImpl.java:131) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
> at io.undertow.security.impl.SecurityContextImpl.authTransition(SecurityContextImpl.java:106) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
> at io.undertow.security.impl.SecurityContextImpl.authenticate(SecurityContextImpl.java:99) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
> at io.undertow.security.handlers.AuthenticationCallHandler.handleRequest(AuthenticationCallHandler.java:50) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
> at io.undertow.server.Connectors.executeRootHandler(Connectors.java:177) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
> at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:727) [undertow-core-1.0.15.Final.jar:1.0.15.Final]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 10 months
[JBoss JIRA] (WFCORE-40) Deadlock while logging
by Stefan Schueffler (JIRA)
[ https://issues.jboss.org/browse/WFCORE-40?page=com.atlassian.jira.plugin.... ]
Stefan Schueffler commented on WFCORE-40:
-----------------------------------------
Unfortunately i'm not able to provide an easy to reproduce kickstart project. None of my test-projects (using jmeter for load, different loggings, System.out, Stacktraces and so on) leads to the same locking/blocking of threads.
Apart from this, in our live production environment the error is always the same: either we reduce logging to an absolute minimum (i.e. no logging at all), or we will run in the locking/waiting for lock situation sooner or later (at least once a day).
I found this issue in jira, which sounds the same in terms of "problem description/behavior" : endless loop in fillInStackTrace. (Un)fortunately, this bug is solved (according to the jira ticket), and the fixed version is installed in wildfly - so, we do not have the exact same problem here, but a very similar problem behavior...
https://issues.jboss.org/browse/XNIO-215
> Deadlock while logging
> ----------------------
>
> Key: WFCORE-40
> URL: https://issues.jboss.org/browse/WFCORE-40
> Project: WildFly Core
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Logging
> Environment: CentOS 6.5 64bit, java7u45 64bit (and 32 bit, the same behavior)
> Reporter: Stefan Schueffler
> Assignee: James Perkins
> Priority: Critical
>
> We hit really often a deadlock? in org.jboss.stdio.StdioContext$DelegatingPrintStream.println(StdioContext.java:474)
> Even if the "StdioContext" belongs to Jboss-Logging, the problem occurs in our production wildfly installation twice to 4 times a day - all threads are deadlocking while trying to log.debug, log.error, or (sometimes) System.out.println from our application code, and wildfly does not respond anymore...
> The partial stacktrace always is similar to this one:
> {code}
> "default task-64" prio=10 tid=0x4c539c00 nid=0x5ef9 waiting for monitor entry [0x495e0000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.io.PrintStream.println(PrintStream.java:806)
> - waiting to lock <0x5ee0adf8> (a java.io.PrintStream)
> at org.jboss.stdio.StdioContext$DelegatingPrintStream.println(StdioContext.java:474)
> at jsp.communications.statuschange.selectStatus_jsp._jspService(selectStatus_jsp.java:413)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:69)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
> at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:82)
> {code}
> While investigating the StdioContext - class, i really wondered whether the used "locking/checking by using a threadlocal" could have worked in a multi-threaded environment (it should have the very same problems as every "double checking" algorithm without proper synchronization).
> If all threads are hanging in this particular lock, only a full wildfly-restart recovers in our case.
> My preferred solution would be a rework of the used org.jboss.stdio. classes, as the used idioms of ThreadLocals for reentrant-checks are at least highly unusual?
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 10 months
[JBoss JIRA] (WFLY-3803) Missing -jandex.jar libs with build using feature-pack
by Alessio Soldano (JIRA)
[ https://issues.jboss.org/browse/WFLY-3803?page=com.atlassian.jira.plugin.... ]
Alessio Soldano updated WFLY-3803:
----------------------------------
Description: After the recent changes to have feature-packs, the build is not creating {noformat}-jandex.jar{noformat} libs anymore. We used to have two of them in the org.apache.cxf.impl module. Those are strictly required for WS-* functionalities. (was: After the recent changes to have feature-packs, the build is not creating "*-jandex.jar" libs anymore. We used to have two of them in the org.apache.cxf.impl module. Those are strictly required for WS-* functionalities.)
> Missing -jandex.jar libs with build using feature-pack
> ------------------------------------------------------
>
> Key: WFLY-3803
> URL: https://issues.jboss.org/browse/WFLY-3803
> Project: WildFly
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Build System
> Reporter: Alessio Soldano
> Assignee: Tomaz Cerar
> Priority: Blocker
> Fix For: 9.0.0.Beta1
>
>
> After the recent changes to have feature-packs, the build is not creating {noformat}-jandex.jar{noformat} libs anymore. We used to have two of them in the org.apache.cxf.impl module. Those are strictly required for WS-* functionalities.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 10 months
[JBoss JIRA] (WFLY-3803) Missing -jandex.jar libs with build using feature-pack
by Alessio Soldano (JIRA)
[ https://issues.jboss.org/browse/WFLY-3803?page=com.atlassian.jira.plugin.... ]
Alessio Soldano updated WFLY-3803:
----------------------------------
Description: After the recent changes to have feature-packs, the build is not creating "*-jandex.jar" libs anymore. We used to have two of them in the org.apache.cxf.impl module. Those are strictly required for WS-* functionalities. (was: After the recent changes to have feature-packs, the build is not creating *-jandex.jar libs anymore. We used to have two of them in the org.apache.cxf.impl module. Those are strictly required for WS-* functionalities.)
> Missing -jandex.jar libs with build using feature-pack
> ------------------------------------------------------
>
> Key: WFLY-3803
> URL: https://issues.jboss.org/browse/WFLY-3803
> Project: WildFly
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Build System
> Reporter: Alessio Soldano
> Assignee: Tomaz Cerar
> Priority: Blocker
> Fix For: 9.0.0.Beta1
>
>
> After the recent changes to have feature-packs, the build is not creating "*-jandex.jar" libs anymore. We used to have two of them in the org.apache.cxf.impl module. Those are strictly required for WS-* functionalities.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 10 months
[JBoss JIRA] (WFLY-3803) Missing -jandex.jar libs with build using feature-pack
by Alessio Soldano (JIRA)
[ https://issues.jboss.org/browse/WFLY-3803?page=com.atlassian.jira.plugin.... ]
Alessio Soldano updated WFLY-3803:
----------------------------------
Description: After the recent changes to have feature-packs, the build is not creating *-jandex.jar libs anymore. We used to have two of them in the org.apache.cxf.impl module. Those are strictly required for WS-* functionalities. (was: After the recent changes to have feature-packs, the build is not creating -jandex.jar libs anymore. We used to have two of them in the org.apache.cxf.impl module. Those are strictly required for WS-* functionalities.)
> Missing -jandex.jar libs with build using feature-pack
> ------------------------------------------------------
>
> Key: WFLY-3803
> URL: https://issues.jboss.org/browse/WFLY-3803
> Project: WildFly
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Build System
> Reporter: Alessio Soldano
> Assignee: Tomaz Cerar
> Priority: Blocker
> Fix For: 9.0.0.Beta1
>
>
> After the recent changes to have feature-packs, the build is not creating *-jandex.jar libs anymore. We used to have two of them in the org.apache.cxf.impl module. Those are strictly required for WS-* functionalities.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 10 months
[JBoss JIRA] (WFLY-3803) Missing -jandex.jar libs with build using feature-pack
by Alessio Soldano (JIRA)
Alessio Soldano created WFLY-3803:
-------------------------------------
Summary: Missing -jandex.jar libs with build using feature-pack
Key: WFLY-3803
URL: https://issues.jboss.org/browse/WFLY-3803
Project: WildFly
Issue Type: Bug
Security Level: Public (Everyone can see)
Components: Build System
Reporter: Alessio Soldano
Assignee: Tomaz Cerar
Priority: Blocker
Fix For: 9.0.0.Beta1
After the recent changes to have feature-packs, the build is not creating -jandex.jar libs anymore. We used to have two of them in the org.apache.cxf.impl module. Those are strictly required for WS-* functionalities.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 10 months
[JBoss JIRA] (JBMETA-379) Missing param-name in a web.xml causes NullPointerException during deployment
by Jean-Frederic Clere (JIRA)
[ https://issues.jboss.org/browse/JBMETA-379?page=com.atlassian.jira.plugin... ]
Jean-Frederic Clere updated JBMETA-379:
---------------------------------------
Fix Version/s: 8.0.1.Final
> Missing param-name in a web.xml causes NullPointerException during deployment
> ------------------------------------------------------------------------------
>
> Key: JBMETA-379
> URL: https://issues.jboss.org/browse/JBMETA-379
> Project: JBoss Metadata
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: web
> Affects Versions: 8.0.0.Final
> Reporter: Jay Kumar SenSharma
> Assignee: Jean-Frederic Clere
> Fix For: 8.0.1.Final
>
> Attachments: ContextParamNullDemo.war
>
>
> - While deploying a WAR, If the web.xml file is used which has context <param-value> defined, However it has missing <param-name> then it causes NullPointerException as following:
> {code}
> 00:12:09,583 INFO [org.jboss.as.server.deployment] (MSC service thread 1-6) WFLYSRV0027: Starting deployment of "ContextParamNullDemo.war" (runtime-name: "ContextParamNullDemo.war")
> 00:12:09,591 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-7) MSC000001: Failed to start service jboss.deployment.unit."ContextParamNullDemo.war".PARSE: org.jboss.msc.service.StartException in service jboss.deployment.unit."ContextParamNullDemo.war".PARSE: WFLYSRV0153: Failed to process phase PARSE of deployment "ContextParamNullDemo.war"
> at org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:163) [wildfly-server-9.0.0.Alpha1-SNAPSHOT.jar:9.0.0.Alpha1-SNAPSHOT]
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_51]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_51]
> Caused by: java.lang.NullPointerException
> at org.jboss.as.jsf.deployment.JSFVersionProcessor.deploy(JSFVersionProcessor.java:91)
> at org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:156) [wildfly-server-9.0.0.Alpha1-SNAPSHOT.jar:9.0.0.Alpha1-SNAPSHOT]
> ... 5 more
> 00:12:09,598 ERROR [org.jboss.as.controller.management-operation] (DeploymentScanner-threads - 1) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "ContextParamNullDemo.war")]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.deployment.unit.\"ContextParamNullDemo.war\".PARSE" => "org.jboss.msc.service.StartException in service jboss.deployment.unit.\"ContextParamNullDemo.war\".PARSE: WFLYSRV0153: Failed to process phase PARSE of deployment \"ContextParamNullDemo.war\"
> Caused by: java.lang.NullPointerException"}}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 10 months