[JBoss JIRA] (WFLY-10991) Memory leak when deployment is redeployed multiple times
by Jan Stourac (JIRA)
Jan Stourac created WFLY-10991:
----------------------------------
Summary: Memory leak when deployment is redeployed multiple times
Key: WFLY-10991
URL: https://issues.jboss.org/browse/WFLY-10991
Project: WildFly
Issue Type: Bug
Components: MP OpenTracing
Affects Versions: 14.0.0.Beta2
Reporter: Jan Stourac
Assignee: Juraci Paixão Kröhling
Priority: Critical
Attachments: memoryLeaks.war, problematic-wf-build-heap-diff-start-end-org-filter.png, problematic-wf-build-heap-diff-start-end.png, sane-wf-build-heap-diff-start-end-org-filter.png, sane-wf-build-heap-diff-start-end.png
It looks like redeploying attached deployment [^memoryLeaks.war] multiple times (our automated test does it 100 times), there seems to be some memory leak in the WildFly. Using git bisect, the commit that started to fail is [651dc8a2f0d4ccad6a35f7b1d28626b43ebb2a5d in WildFly repository|https://github.com/wildfly/wildfly/commits/651dc8a2f0d4ccad6a3...]. That is why I selected MP OpenTracing component for this issue.
Brief description of the test:
# attached deployment is deployed
# System.gc() is called via EJB call
# it checks initial heap usage at the beginning of the test via EJB call that checks all instances of ManagementFactory.getMemoryPoolMXBeans()
# deployment is redeployed 100 times
# calls System.gc() via EJP call again and checks heap usage again via EJB call and compares the results
I tried to check manually via [visualVM|https://visualvm.github.io/] tool. You can create a heapdump at the beginning of the test and at the end of the test. Then compare them easily and check the differences. Using a filter I can see quite a huge increase of instances of the {{org.jboss.modules.*}} classes. However, biggest memory footprint is huge increase of the instances of the {{java.util.HashMap$Node}}, {{char[]}}, {{java.util.HashMap$Node[]}} and few others. I don't see such huge increase of the instances of these classes when I use older build of WildFly prior to the problematic commit. To be honest I don't see direct link with the problematic commit I linked before. As the heapdumps are too big to be attached here, I am attaching some screenshots.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 10 months
[JBoss JIRA] (WFCORE-4097) CredentialReference doesn't work with the new MSC API
by ehsavoie Hugonnet (JIRA)
ehsavoie Hugonnet created WFCORE-4097:
-----------------------------------------
Summary: CredentialReference doesn't work with the new MSC API
Key: WFCORE-4097
URL: https://issues.jboss.org/browse/WFCORE-4097
Project: WildFly Core
Issue Type: Bug
Components: Security
Affects Versions: 6.0.2.Final
Reporter: ehsavoie Hugonnet
Assignee: Darran Lofthouse
Priority: Blocker
When using the new MSC API the ServiceBuilder.addDependency can't be used, it should use ServiceBuilder.requires instead. The code should allow the correct method to be called.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 10 months
[JBoss JIRA] (JBJCA-1384) ActivationImpl doesn't seem thread safe
by Kabir Khan (JIRA)
[ https://issues.jboss.org/browse/JBJCA-1384?page=com.atlassian.jira.plugin... ]
Kabir Khan reassigned JBJCA-1384:
---------------------------------
Assignee: Stefano Maestri
> ActivationImpl doesn't seem thread safe
> ---------------------------------------
>
> Key: JBJCA-1384
> URL: https://issues.jboss.org/browse/JBJCA-1384
> Project: IronJacamar
> Issue Type: Bug
> Reporter: Kabir Khan
> Assignee: Stefano Maestri
> Priority: Critical
>
> When looking into https://issues.jboss.org/browse/WFLY-10986 I had to figure out what was modifying org.jboss.jca.common.metadata.resourceadapter.ActivationImpl.connectionDefinitions and figure out what to synchronize on. There is a subclass org.jboss.as.connector.subsystems.resourceadapters.ModifiableResourceAdapter which adds the changes.
> ModifiableResourceAdapter correctly synchronizes when adding data, which is good, but really all the fields in ActivationImpl should be final to make sure they are safely published as I don't think the fields are ever overwritten (otherwise they should be volatile).
> Also, I think it is possible to get similar ConcurrentModificationExceptions in ActivationImpl's implementations of hashCode(), equals(), and toString() as they all end up iterating things which could be modified by ModifiableResourceAdapter. So these all really ought to be synchronized. If you don't want to do it in ActivationImpl itself, you can probably do so in ModifiableResourceAdapter, e.g.:
> {code}
> @Override
> public synchronized int hashCode()
> {
> return super.hashCode();
> }
> @Override
> public synchronized boolean equals(Object obj)
> {
> return super.equals(obj);
> }
> @Override
> public synchronized String toString()
> {
> return super.toString();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 10 months
[JBoss JIRA] (JBJCA-1384) ActivationImpl doesn't seem thread safe
by Kabir Khan (JIRA)
Kabir Khan created JBJCA-1384:
---------------------------------
Summary: ActivationImpl doesn't seem thread safe
Key: JBJCA-1384
URL: https://issues.jboss.org/browse/JBJCA-1384
Project: IronJacamar
Issue Type: Bug
Reporter: Kabir Khan
Priority: Critical
When looking into https://issues.jboss.org/browse/WFLY-10986 I had to figure out what was modifying org.jboss.jca.common.metadata.resourceadapter.ActivationImpl.connectionDefinitions and figure out what to synchronize on. There is a subclass org.jboss.as.connector.subsystems.resourceadapters.ModifiableResourceAdapter which adds the changes.
ModifiableResourceAdapter correctly synchronizes when adding data, which is good, but really all the fields in ActivationImpl should be final to make sure they are safely published as I don't think the fields are ever overwritten (otherwise they should be volatile).
Also, I think it is possible to get similar ConcurrentModificationExceptions in ActivationImpl's implementations of hashCode(), equals(), and toString() as they all end up iterating things which could be modified by ModifiableResourceAdapter. So these all really ought to be synchronized. If you don't want to do it in ActivationImpl itself, you can probably do so in ModifiableResourceAdapter, e.g.:
{code}
@Override
public synchronized int hashCode()
{
return super.hashCode();
}
@Override
public synchronized boolean equals(Object obj)
{
return super.equals(obj);
}
@Override
public synchronized String toString()
{
return super.toString();
}
{code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 10 months
[JBoss JIRA] (WFLY-10986) Concurrent Modification Exception when starting server with external JMS broker
by Kabir Khan (JIRA)
[ https://issues.jboss.org/browse/WFLY-10986?page=com.atlassian.jira.plugin... ]
Kabir Khan commented on WFLY-10986:
-----------------------------------
[~mnovak] Can you try with my PR at https://github.com/wildfly/wildfly/pull/11652?
> Concurrent Modification Exception when starting server with external JMS broker
> -------------------------------------------------------------------------------
>
> Key: WFLY-10986
> URL: https://issues.jboss.org/browse/WFLY-10986
> Project: WildFly
> Issue Type: Bug
> Components: JCA, JMS
> Affects Versions: 14.0.1.Final
> Reporter: Martin Styk
> Assignee: Stefano Maestri
> Priority: Blocker
>
> Start of a WildFly with resource adapter configured to remote messaging broker sometimes fails with following error.
> {code}
> ERROR [org.jboss.msc.service.fail] (MSC service thread 1-4) MSC000001: Failed to start service jboss.resourceadapters.ra."wmq.jmsra".CF: org.jboss.msc.service.StartException in service jboss.resourceadapters.ra."wmq.jmsra".CF: Failed to start service
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.execute(ServiceControllerImpl.java:1728)
> at org.jboss.msc.service.ServiceControllerImpl$ControllerTask.run(ServiceControllerImpl.java:1556)
> at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
> at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1364)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
> at java.util.ArrayList$Itr.next(ArrayList.java:859)
> at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1042)
> at org.jboss.as.connector.metadata.api.resourceadapter.ActivationSecurityUtil.isLegacySecurityRequired(ActivationSecurityUtil.java:41)
> at org.jboss.as.connector.subsystems.resourceadapters.ConnectionDefinitionService.start(ConnectionDefinitionService.java:68)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1736)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.execute(ServiceControllerImpl.java:1698)
> ... 6 more
> {code}
> This issue was introduced in fix for WFLY-9978 in [this commit |https://github.com/ehsavoie/wildfly/commit/8d35c77130c8ba8c47008e1de1225971fa5d8c2e]
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 10 months
[JBoss JIRA] (WFLY-10990) UnsupportedOperationException thrown during start if CredentialReference is used in messaging-activemq subsystem
by Miroslav Novak (JIRA)
Miroslav Novak created WFLY-10990:
-------------------------------------
Summary: UnsupportedOperationException thrown during start if CredentialReference is used in messaging-activemq subsystem
Key: WFLY-10990
URL: https://issues.jboss.org/browse/WFLY-10990
Project: WildFly
Issue Type: Bug
Components: JMS
Affects Versions: 14.0.0.Final
Reporter: Miroslav Novak
Assignee: Jeff Mesnil
Priority: Critical
Attachments: bridge-cs001.jceks, standalone-full-ha.xml
If messaging subsystem is referencing elytron's credential store like in:
{code}
<bridge name="myBridge" queue-name="jms.queue.InQueue" forwarding-address="jms.queue.OutQueue" ha="true" reconnect-attempts="-1" use-duplicate-detection="true" user="johnOut" static-connectors="bridge-connector">
<credential-reference store="bridge-cs001" alias="john_out_alias_wrong"/>
</bridge>
{code}
then during start server throws exception:
{code}
12:49:04,367 ERROR [org.jboss.as.controller.management-operation] (ServerService Thread Pool -- 66) WFLYCTL0013: Operation ("add") failed - address: ([
("subsystem" => "messaging-activemq"),
("server" => "default")
]): java.lang.UnsupportedOperationException
at org.jboss.msc.service.ServiceBuilderImpl.addDependency(ServiceBuilderImpl.java:409)
at org.jboss.msc.service.DelegatingServiceBuilder.addDependency(DelegatingServiceBuilder.java:147)
at org.jboss.as.controller.security.CredentialReference.getCredentialSourceSupplier(CredentialReference.java:323)
at org.wildfly.extension.messaging.activemq.ServerAdd.addBridgeCredentialStoreReference(ServerAdd.java:732)
at org.wildfly.extension.messaging.activemq.ServerAdd.access$200(ServerAdd.java:175)
at org.wildfly.extension.messaging.activemq.ServerAdd$2.execute(ServerAdd.java:439)
at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:999)
at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:743)
at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:467)
at org.jboss.as.controller.ParallelBootOperationStepHandler$ParallelBootTask.run(ParallelBootOperationStepHandler.java:384)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1349)
at java.lang.Thread.run(Thread.java:748)
at org.jboss.threads.JBossThread.run(JBossThread.java:485)
{code}
Attaching standalone-full-ha.xml with which is possible to reproduce the issue.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 10 months