[JBoss JIRA] (ISPN-12325) Hot Rod server instances in tests use too much memory
by Dan Berindei (Jira)
[ https://issues.redhat.com/browse/ISPN-12325?page=com.atlassian.jira.plugi... ]
Dan Berindei updated ISPN-12325:
--------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/8690
> Hot Rod server instances in tests use too much memory
> -----------------------------------------------------
>
> Key: ISPN-12325
> URL: https://issues.redhat.com/browse/ISPN-12325
> Project: Infinispan
> Issue Type: Bug
> Components: Server, Test Suite
> Affects Versions: 12.0.0.Dev03
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Major
> Fix For: 12.0.0.Dev04
>
>
> When a test class finishes, TestNG does not discard the test instance, instead it keeps it in memory until the test suite finishes. This means test class fields that uses a lot of memory will accumulate in memory, making the test suite use a lot more heap than it should.
> ISPN-8478 added a generic way to set {{Cache}} and {{EmbeddedCacheManager}} fields to {{null}} in order to reduce memory usage, but hotrod-client tests do not have anything similar for {{HotRodServer}} and {{RemoteCacheManager}} fields.
> That might need to change: hotrod-client tests often keep references to {{HotRodServer}} instances in fields, and the servers use > 3MB, most of it in the {{SerializationContextRegistryImpl}}:
> {noformat}
> Class Name | Shallow Heap | Retained Heap | Percentage
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
> org.infinispan.client.hotrod.ReplTopologyChangeTest @ 0xc52bf018 | 96 | 10,358,136 | 1.02%
> |- org.infinispan.server.hotrod.test.HotRodTestingUtil$1 @ 0xe30cc538 | 128 | 3,420,600 | 0.34%
> | |- org.infinispan.marshall.protostream.impl.SerializationContextRegistryImpl @ 0xcd18d210 | 24 | 2,493,584 | 0.25%
> | | |- org.infinispan.marshall.protostream.impl.SerializationContextRegistryImpl$MarshallerContext @ 0xe3d18c08| 32 | 2,358,320 | 0.23%
> | | | |- org.infinispan.protostream.impl.SerializationContextImpl @ 0xe3d18e78 | 64 | 2,357,696 | 0.23%
> | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xe3d67fe0 | 72 | 967,680 | 0.10%
> | | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xe41b8798 | 72 | 690,528 | 0.07%
> | | | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xc997eaf8 | 72 | 413,376 | 0.04%
> | | | | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xc5ed3cd8 | 72 | 214,640 | 0.02%
> | | | | | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xc5ed4b60 | 72 | 183,648 | 0.02%
> | | | | | | | | | |- org.infinispan.protostream.descriptors.Descriptor @ 0xc5edabe8 | 72 | 132,104 | 0.01%
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 6 months
[JBoss JIRA] (ISPN-12325) Hot Rod server instances in tests use too much memory
by Dan Berindei (Jira)
[ https://issues.redhat.com/browse/ISPN-12325?page=com.atlassian.jira.plugi... ]
Dan Berindei updated ISPN-12325:
--------------------------------
Status: Open (was: New)
> Hot Rod server instances in tests use too much memory
> -----------------------------------------------------
>
> Key: ISPN-12325
> URL: https://issues.redhat.com/browse/ISPN-12325
> Project: Infinispan
> Issue Type: Bug
> Components: Server, Test Suite
> Affects Versions: 12.0.0.Dev03
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Major
> Fix For: 12.0.0.Dev04
>
>
> When a test class finishes, TestNG does not discard the test instance, instead it keeps it in memory until the test suite finishes. This means test class fields that uses a lot of memory will accumulate in memory, making the test suite use a lot more heap than it should.
> ISPN-8478 added a generic way to set {{Cache}} and {{EmbeddedCacheManager}} fields to {{null}} in order to reduce memory usage, but hotrod-client tests do not have anything similar for {{HotRodServer}} and {{RemoteCacheManager}} fields.
> That might need to change: hotrod-client tests often keep references to {{HotRodServer}} instances in fields, and the servers use > 3MB, most of it in the {{SerializationContextRegistryImpl}}:
> {noformat}
> Class Name | Shallow Heap | Retained Heap | Percentage
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
> org.infinispan.client.hotrod.ReplTopologyChangeTest @ 0xc52bf018 | 96 | 10,358,136 | 1.02%
> |- org.infinispan.server.hotrod.test.HotRodTestingUtil$1 @ 0xe30cc538 | 128 | 3,420,600 | 0.34%
> | |- org.infinispan.marshall.protostream.impl.SerializationContextRegistryImpl @ 0xcd18d210 | 24 | 2,493,584 | 0.25%
> | | |- org.infinispan.marshall.protostream.impl.SerializationContextRegistryImpl$MarshallerContext @ 0xe3d18c08| 32 | 2,358,320 | 0.23%
> | | | |- org.infinispan.protostream.impl.SerializationContextImpl @ 0xe3d18e78 | 64 | 2,357,696 | 0.23%
> | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xe3d67fe0 | 72 | 967,680 | 0.10%
> | | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xe41b8798 | 72 | 690,528 | 0.07%
> | | | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xc997eaf8 | 72 | 413,376 | 0.04%
> | | | | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xc5ed3cd8 | 72 | 214,640 | 0.02%
> | | | | | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xc5ed4b60 | 72 | 183,648 | 0.02%
> | | | | | | | | | |- org.infinispan.protostream.descriptors.Descriptor @ 0xc5edabe8 | 72 | 132,104 | 0.01%
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 6 months
[JBoss JIRA] (ISPN-9064) SiteManualSwitchTest.testManualClusterSwitch randomly fails
by Dan Berindei (Jira)
[ https://issues.redhat.com/browse/ISPN-9064?page=com.atlassian.jira.plugin... ]
Dan Berindei updated ISPN-9064:
-------------------------------
Attachment: ISPN-12325_hotrod_test_memory_20200914-1907_SiteManualSwitchTest-infinispan-client-hotrod.log.gz
> SiteManualSwitchTest.testManualClusterSwitch randomly fails
> -----------------------------------------------------------
>
> Key: ISPN-9064
> URL: https://issues.redhat.com/browse/ISPN-9064
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Protocols
> Reporter: Galder Zamarreño
> Priority: Major
> Labels: testsuite_stability
> Attachments: ISPN-12325_hotrod_test_memory_20200914-1907_SiteManualSwitchTest-infinispan-client-hotrod.log.gz
>
>
> {code}
> java.lang.AssertionError: expected:<1> but was:<0>
> at org.infinispan.client.hotrod.xsite.AbstractHotRodSiteFailoverTest.assertSiteHit(AbstractHotRodSiteFailoverTest.java:146)
> at org.infinispan.client.hotrod.xsite.SiteManualSwitchTest.assertSingleSiteHit(SiteManualSwitchTest.java:47)
> at org.infinispan.client.hotrod.xsite.SiteManualSwitchTest.testManualClusterSwitch(SiteManualSwitchTest.java:37)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 6 months
[JBoss JIRA] (ISPN-9064) SiteManualSwitchTest.testManualClusterSwitch randomly fails
by Dan Berindei (Jira)
[ https://issues.redhat.com/browse/ISPN-9064?page=com.atlassian.jira.plugin... ]
Dan Berindei commented on ISPN-9064:
------------------------------------
The test is still failing randomly in master, and I have a trace log (attached). The manual site switch is supposed to set the topology id of all caches to {{-2}} ({{SWITCH_CLUSTER_TOPOLOGY}}), but the test cache's topology sometimes stays valid:
{noformat}
19:15:57,496 DEBUG (testng-Test:[]) [ChannelFactory] Switching to NYC-2, servers: [127.0.0.1/<unresolved>:37451], setting topology.
19:15:57,496 INFO (testng-Test:[]) [HOTROD] ISPN004052: Manually switched to cluster 'NYC-2'
19:15:57,505 TRACE (testng-Test:[]) [TopologyInfo] Is topology id (5) valid? true
19:15:57,505 TRACE (testng-Test:[]) [SegmentConsistentHash] Find server in segment id 253 for key [B0x2803
19:15:57,522 TRACE (testng-Test:[]) [TopologyInfo] Using consistent hash for determining the server: Optional[127.0.0.1/<unresolved>:35361]
{noformat}
> SiteManualSwitchTest.testManualClusterSwitch randomly fails
> -----------------------------------------------------------
>
> Key: ISPN-9064
> URL: https://issues.redhat.com/browse/ISPN-9064
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Protocols
> Reporter: Galder Zamarreño
> Priority: Major
> Labels: testsuite_stability
> Attachments: ISPN-12325_hotrod_test_memory_20200914-1907_SiteManualSwitchTest-infinispan-client-hotrod.log.gz
>
>
> {code}
> java.lang.AssertionError: expected:<1> but was:<0>
> at org.infinispan.client.hotrod.xsite.AbstractHotRodSiteFailoverTest.assertSiteHit(AbstractHotRodSiteFailoverTest.java:146)
> at org.infinispan.client.hotrod.xsite.SiteManualSwitchTest.assertSingleSiteHit(SiteManualSwitchTest.java:47)
> at org.infinispan.client.hotrod.xsite.SiteManualSwitchTest.testManualClusterSwitch(SiteManualSwitchTest.java:37)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 6 months
[JBoss JIRA] (ISPN-12325) Hot Rod server instances in tests use too much memory
by Dan Berindei (Jira)
Dan Berindei created ISPN-12325:
-----------------------------------
Summary: Hot Rod server instances in tests use too much memory
Key: ISPN-12325
URL: https://issues.redhat.com/browse/ISPN-12325
Project: Infinispan
Issue Type: Bug
Components: Server, Test Suite
Affects Versions: 12.0.0.Dev03
Reporter: Dan Berindei
Assignee: Dan Berindei
Fix For: 12.0.0.Dev04
When a test class finishes, TestNG does not discard the test instance, instead it keeps it in memory until the test suite finishes. This means test class fields that uses a lot of memory will accumulate in memory, making the test suite use a lot more heap than it should.
ISPN-8478 added a generic way to set {{Cache}} and {{EmbeddedCacheManager}} fields to {{null}} in order to reduce memory usage, but hotrod-client tests do not have anything similar for {{HotRodServer}} and {{RemoteCacheManager}} fields.
That might need to change: hotrod-client tests often keep references to {{HotRodServer}} instances in fields, and the servers use > 3MB, most of it in the {{SerializationContextRegistryImpl}}:
{noformat}
Class Name | Shallow Heap | Retained Heap | Percentage
-------------------------------------------------------------------------------------------------------------------------------------------------------------
org.infinispan.client.hotrod.ReplTopologyChangeTest @ 0xc52bf018 | 96 | 10,358,136 | 1.02%
|- org.infinispan.server.hotrod.test.HotRodTestingUtil$1 @ 0xe30cc538 | 128 | 3,420,600 | 0.34%
| |- org.infinispan.marshall.protostream.impl.SerializationContextRegistryImpl @ 0xcd18d210 | 24 | 2,493,584 | 0.25%
| | |- org.infinispan.marshall.protostream.impl.SerializationContextRegistryImpl$MarshallerContext @ 0xe3d18c08| 32 | 2,358,320 | 0.23%
| | | |- org.infinispan.protostream.impl.SerializationContextImpl @ 0xe3d18e78 | 64 | 2,357,696 | 0.23%
| | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xe3d67fe0 | 72 | 967,680 | 0.10%
| | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xe41b8798 | 72 | 690,528 | 0.07%
| | | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xc997eaf8 | 72 | 413,376 | 0.04%
| | | | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xc5ed3cd8 | 72 | 214,640 | 0.02%
| | | | | | | | |- org.infinispan.protostream.descriptors.FileDescriptor @ 0xc5ed4b60 | 72 | 183,648 | 0.02%
| | | | | | | | | |- org.infinispan.protostream.descriptors.Descriptor @ 0xc5edabe8 | 72 | 132,104 | 0.01%
-------------------------------------------------------------------------------------------------------------------------------------------------------------
{noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 7 months
[JBoss JIRA] (IPROTO-152) Split SerializationContextInitializer into 2 separate interfaces, one to register schemas+marshallers and one to provide access to the generated schema
by Nistor Adrian (Jira)
[ https://issues.redhat.com/browse/IPROTO-152?page=com.atlassian.jira.plugi... ]
Nistor Adrian updated IPROTO-152:
---------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/protostream/pull/104
> Split SerializationContextInitializer into 2 separate interfaces, one to register schemas+marshallers and one to provide access to the generated schema
> -------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: IPROTO-152
> URL: https://issues.redhat.com/browse/IPROTO-152
> Project: Infinispan ProtoStream
> Issue Type: Enhancement
> Affects Versions: 4.3.3.Final
> Reporter: Nistor Adrian
> Assignee: Nistor Adrian
> Priority: Major
> Fix For: 4.4.0.Alpha1, 4.3.4.Final
>
>
> We need a way to support generating marshallers without always generating a schema.
> For that we can introduce a new flag in AutoProtoSchemaBuilder , marshallersOnly (default false), that will cause schema generation and registration to be skipped.
> But the current form of SerializationContextInitializer interface mixes up access to the name and contents of the generated schema and the registration of generated marshallers and schemas, and if marshallersOnly=true it's unclear what getProtoFileName() and getProtoFile() should return.
> To fix that we should move methods getProtoFileName() and getProtoFile() to a sub-interface called GeneratedSchema. All generated SerializationContextInitializer implementations will implement this interface, unless marshallersOnly=true, in which case they will implement SerializationContextInitializer only.
> The methods will not be really moved right now, they will be temporarily kept in SerializationContextInitializer also and will only be removed for good in version 5.
>
> {color:#00627a} {color}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 7 months
[JBoss JIRA] (IPROTO-152) Split SerializationContextInitializer into 2 separate interfaces, one to register schemas+marshallers and one to provide access to the generated schema
by Nistor Adrian (Jira)
[ https://issues.redhat.com/browse/IPROTO-152?page=com.atlassian.jira.plugi... ]
Nistor Adrian updated IPROTO-152:
---------------------------------
Fix Version/s: 4.4.0.Alpha1
4.3.4.Final
> Split SerializationContextInitializer into 2 separate interfaces, one to register schemas+marshallers and one to provide access to the generated schema
> -------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: IPROTO-152
> URL: https://issues.redhat.com/browse/IPROTO-152
> Project: Infinispan ProtoStream
> Issue Type: Enhancement
> Affects Versions: 4.3.3.Final
> Reporter: Nistor Adrian
> Assignee: Nistor Adrian
> Priority: Major
> Fix For: 4.4.0.Alpha1, 4.3.4.Final
>
>
> We need a way to support generating marshallers without always generating a schema.
> For that we can introduce a new flag in AutoProtoSchemaBuilder , marshallersOnly (default false), that will cause schema generation and registration to be skipped.
> But the current form of SerializationContextInitializer interface mixes up access to the name and contents of the generated schema and the registration of generated marshallers and schemas, and if marshallersOnly=true it's unclear what getProtoFileName() and getProtoFile() should return.
> To fix that we should move methods getProtoFileName() and getProtoFile() to a sub-interface called GeneratedSchema. All generated SerializationContextInitializer implementations will implement this interface, unless marshallersOnly=true, in which case they will implement SerializationContextInitializer only.
> The methods will not be really moved right now, they will be temporarily kept in SerializationContextInitializer also and will only be removed for good in version 5.
>
> {color:#00627a} {color}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 7 months
[JBoss JIRA] (IPROTO-152) Split SerializationContextInitializer into 2 separate interfaces, one to register schemas+marshallers and one to provide access to the generated schema
by Nistor Adrian (Jira)
[ https://issues.redhat.com/browse/IPROTO-152?page=com.atlassian.jira.plugi... ]
Nistor Adrian reassigned IPROTO-152:
------------------------------------
Assignee: Nistor Adrian
> Split SerializationContextInitializer into 2 separate interfaces, one to register schemas+marshallers and one to provide access to the generated schema
> -------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: IPROTO-152
> URL: https://issues.redhat.com/browse/IPROTO-152
> Project: Infinispan ProtoStream
> Issue Type: Enhancement
> Affects Versions: 4.3.3.Final
> Reporter: Nistor Adrian
> Assignee: Nistor Adrian
> Priority: Major
>
> We need a way to support generating marshallers without always generating a schema.
> For that we can introduce a new flag in AutoProtoSchemaBuilder , marshallersOnly (default false), that will cause schema generation and registration to be skipped.
> But the current form of SerializationContextInitializer interface mixes up access to the name and contents of the generated schema and the registration of generated marshallers and schemas, and if marshallersOnly=true it's unclear what getProtoFileName() and getProtoFile() should return.
> To fix that we should move methods getProtoFileName() and getProtoFile() to a sub-interface called GeneratedSchema. All generated SerializationContextInitializer implementations will implement this interface, unless marshallersOnly=true, in which case they will implement SerializationContextInitializer only.
> The methods will not be really moved right now, they will be temporarily kept in SerializationContextInitializer also and will only be removed for good in version 5.
>
> {color:#00627a} {color}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 7 months