[JBoss JIRA] (ISPN-7971) Display a warning that changes in standalone mode are not persisted
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-7971?page=com.atlassian.jira.plugin.... ]
Ryan Emerson commented on ISPN-7971:
------------------------------------
[~vblagojevic][~sebastian.laskawiec] For this to be effective, we need a way for the console to determine that the standalone server is running on openshift.
> Display a warning that changes in standalone mode are not persisted
> -------------------------------------------------------------------
>
> Key: ISPN-7971
> URL: https://issues.jboss.org/browse/ISPN-7971
> Project: Infinispan
> Issue Type: Feature Request
> Components: JMX, reporting and management
> Reporter: Sebastian Łaskawiec
> Assignee: Vladimir Blagojevic
> Attachments: Screen Shot 2017-07-26 at 9.45.27 AM.png
>
>
> The Infinispan Management Console can also operate in Standalone mode. We would like to make it accessible in OpenShift. However we should probably display a warning that the console should be used only for monitoring and none of the changes are persisted.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 5 months
[JBoss JIRA] (ISPN-8111) OffHeapBoundedSingleNodeTest.testMultiThreaded fails with trace logging enabled
by Dan Berindei (JIRA)
Dan Berindei created ISPN-8111:
----------------------------------
Summary: OffHeapBoundedSingleNodeTest.testMultiThreaded fails with trace logging enabled
Key: ISPN-8111
URL: https://issues.jboss.org/browse/ISPN-8111
Project: Infinispan
Issue Type: Bug
Components: Test Suite - Core
Affects Versions: 9.1.0.Final
Reporter: Dan Berindei
Assignee: William Burns
Priority: Critical
This is related to ISPN-8110, but it may be possible to fix ISPN-8110 and still have {{OffHeapBoundedSingleNodeTest.testMultiThreaded}} take more than 10 seconds.
10 seconds seems more than enough for a CPU-bound test, but maybe we need to allow for more time when trace logging is enabled, or maybe we can test only the data container to eliminate the unrelated logging from the (unnecessary, I think) commands and interceptors:
{noformat}
15:52:10,227 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command PutKeyValueCommand{key=WrappedByteArray{bytes=[B0x010201046B657931, hashCode=1745974967}, value=WrappedByteArray{bytes=[B0x0102010676616C75..[10], hashCode=635645735}, flags=[], commandInvocationId=, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, successful=true, topologyId=-1} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@5f3c06c1]
15:52:10,229 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command PrepareCommand {modifications=[PutKeyValueCommand{key=WrappedByteArray{bytes=[B0x010201046B657931, hashCode=1745974967}, value=WrappedByteArray{bytes=[B0x0102010676616C75..[10], hashCode=635645735}, flags=[], commandInvocationId=, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, successful=true, topologyId=-1}], onePhaseCommit=false, retried=false, gtx=GlobalTx:local:2, cacheName='___defaultcache', topologyId=-1} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@3bc20a52]
15:52:10,231 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command CommitCommand {gtx=GlobalTx:local:2, cacheName='___defaultcache', topologyId=-1} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@2af56f39]
{noformat}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 5 months
[JBoss JIRA] (ISPN-8111) OffHeapBoundedSingleNodeTest.testMultiThreaded fails with trace logging enabled
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-8111?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-8111:
-------------------------------
Status: Open (was: New)
> OffHeapBoundedSingleNodeTest.testMultiThreaded fails with trace logging enabled
> -------------------------------------------------------------------------------
>
> Key: ISPN-8111
> URL: https://issues.jboss.org/browse/ISPN-8111
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 9.1.0.Final
> Reporter: Dan Berindei
> Assignee: William Burns
> Priority: Critical
> Labels: testsuite_stability
>
> This is related to ISPN-8110, but it may be possible to fix ISPN-8110 and still have {{OffHeapBoundedSingleNodeTest.testMultiThreaded}} take more than 10 seconds.
> 10 seconds seems more than enough for a CPU-bound test, but maybe we need to allow for more time when trace logging is enabled, or maybe we can test only the data container to eliminate the unrelated logging from the (unnecessary, I think) commands and interceptors:
> {noformat}
> 15:52:10,227 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command PutKeyValueCommand{key=WrappedByteArray{bytes=[B0x010201046B657931, hashCode=1745974967}, value=WrappedByteArray{bytes=[B0x0102010676616C75..[10], hashCode=635645735}, flags=[], commandInvocationId=, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, successful=true, topologyId=-1} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@5f3c06c1]
> 15:52:10,229 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command PrepareCommand {modifications=[PutKeyValueCommand{key=WrappedByteArray{bytes=[B0x010201046B657931, hashCode=1745974967}, value=WrappedByteArray{bytes=[B0x0102010676616C75..[10], hashCode=635645735}, flags=[], commandInvocationId=, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, successful=true, topologyId=-1}], onePhaseCommit=false, retried=false, gtx=GlobalTx:local:2, cacheName='___defaultcache', topologyId=-1} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@3bc20a52]
> 15:52:10,231 TRACE (testng-Test:[]) [InvocationContextInterceptor] Invoked with command CommitCommand {gtx=GlobalTx:local:2, cacheName='___defaultcache', topologyId=-1} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@2af56f39]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 5 months
[JBoss JIRA] (ISPN-8110) OffHeapBoundedDataContainer.ensureSize() busy loop takes too long
by Dan Berindei (JIRA)
Dan Berindei created ISPN-8110:
----------------------------------
Summary: OffHeapBoundedDataContainer.ensureSize() busy loop takes too long
Key: ISPN-8110
URL: https://issues.jboss.org/browse/ISPN-8110
Project: Infinispan
Issue Type: Task
Components: Core
Affects Versions: 9.1.0.Final
Reporter: Dan Berindei
Assignee: William Burns
{{OffHeapBoundedDataContainer.ensureSize()}} needs to acquire the {{lruLock}} in order to check the container size, and then it needs to acquire the write lock of the entry pointed at by the LRU list head ({{firstAddress}}) in order to remove it.
Currently it tries to acquire the entry write lock with {{locks.getLockFromHashCode(hashCode).writeLock().tryLock()}} while holding the {{lruLock}}. But the entry's read-write lock is actually shared by a bucket of entries, so another thread reading a completely different entry fail the {{tryLock}} and will force {{ensureSize()}} to unlock {{lruLock}} and try again.
Even worse, a regular put will acquire the entry/bucket's write lock first, and then try to acquire the {{lruLock}} while holding the bucket lock. Because {{lruLock}} isn't fair, it's possible for 2 or more {{ensureSize}} threads to block a writer thread for a long time.
{{OffHeapBoundedSingleNodeTest.testMultiThreaded}} shows a different pathological scenario, one that I don't have an explanation for yet. At least with trace logging enabled, the {{ensureSize()}} loop can repeat for 100+ milliseconds, but without any other thread logging anything:
{noformat}
15:52:20,012 TRACE (ForkThread-3,Test:[]) [BoundedOffHeapDataContainer] Removing first LRU node at 140339935456768
15:52:20,012 TRACE (ForkThread-3,Test:[]) [UnsafeWrapper] Retrieved long value 140339734124528 from address 140339935456784
15:52:20,012 TRACE (ForkThread-3,Test:[]) [UnsafeWrapper] Wrote long value 0 to address 140339734124536
15:52:20,012 TRACE (ForkThread-3,Test:[]) [UnpooledOffHeapMemoryAllocator] Deallocating off heap memory at 140339935456768 with 28 bytes. Total size: 8727
15:52:20,012 TRACE (ForkThread-3,Test:[]) [UnsafeWrapper] Copying memory of object null offset by 140340069670800 to [B@38d5b85f offset by 16 with a total of 17 bytes
15:52:20,012 TRACE (ForkThread-5,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,012 TRACE (ForkThread-3,Test:[]) [UnpooledOffHeapMemoryAllocator] Deallocating off heap memory at 140340069670784 with 57 bytes. Total size: 8670
15:52:20,012 TRACE (ForkThread-5,Test:[]) [UnsafeWrapper] Retrieved long value 140339868355200 from address 140339734124528
15:52:20,012 TRACE (ForkThread-2,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,012 TRACE (ForkThread-5,Test:[]) [BoundedOffHeapDataContainer] Removing entry: 140339868355200 due to eviction due to size 102 being larger than maximum of 100
15:52:20,012 TRACE (ForkThread-2,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,012 TRACE (ForkThread-5,Test:[]) [UnsafeWrapper] Copying memory of object null offset by 140339868355216 to [B@5e23dca1 offset by 16 with a total of 17 bytes
15:52:20,012 TRACE (ForkThread-2,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,012 TRACE (ForkThread-5,Test:[]) [UnsafeWrapper] Copying memory of object null offset by 140339868355233 to [B@50382f4a offset by 16 with a total of 11 bytes
15:52:20,012 TRACE (ForkThread-2,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,012 TRACE (ForkThread-5,Test:[]) [UnsafeWrapper] Retrieved long value 0 from address 140339868355208
15:52:20,012 TRACE (ForkThread-2,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,012 TRACE (ForkThread-5,Test:[]) [UnsafeWrapper] Copying memory of object null offset by 140339868355216 to [B@3b554f41 offset by 16 with a total of 17 bytes
15:52:20,012 TRACE (ForkThread-2,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,012 TRACE (ForkThread-5,Test:[]) [UnsafeWrapper] Copying memory of object null offset by 140339868355233 to [B@472da290 offset by 16 with a total of 11 bytes
15:52:20,012 TRACE (ForkThread-2,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,012 TRACE (ForkThread-5,Test:[]) [UnsafeWrapper] Copying memory of object null offset by 140339868355244 to [B@7aba800 offset by 16 with a total of 0 bytes
15:52:20,012 TRACE (ForkThread-2,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,012 TRACE (ForkThread-5,Test:[]) [UnsafeWrapper] Copying memory of object null offset by 140339868355244 to [B@167885c0 offset by 16 with a total of 13 bytes
15:52:20,012 TRACE (ForkThread-2,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,012 TRACE (ForkThread-5,Test:[]) [UnsafeWrapper] Retrieved long value 140339734124528 from address 140339868355200
15:52:20,012 TRACE (ForkThread-2,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,012 TRACE (ForkThread-2,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,012 TRACE (ForkThread-2,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
...
15:52:20,147 TRACE (ForkThread-3,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,147 TRACE (ForkThread-3,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,147 TRACE (ForkThread-3,Test:[]) [UnsafeWrapper] Retrieved int value -1075141583 from address 140339734124552
15:52:20,147 TRACE (ForkThread-5,Test:[]) [BoundedOffHeapDataContainer] Removing first LRU node at 140339734124528
{noformat}
Off-topic, it's a bit weird that {{UnsafeWrapper}} logs every address being read from/written to, but the data container doesn't log the keys and values that it reads/writes...
I propose (after discussing this with Will) that {{ensureSize()}} should acquire the bucket lock after releasing the {{lruLock}}, so it can use {{lock()}} instead of {{tryLock()}}.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 5 months
[JBoss JIRA] (ISPN-7971) Display a warning that changes in standalone mode are not persisted
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-7971?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec commented on ISPN-7971:
-------------------------------------------
Looks awesome!
> Display a warning that changes in standalone mode are not persisted
> -------------------------------------------------------------------
>
> Key: ISPN-7971
> URL: https://issues.jboss.org/browse/ISPN-7971
> Project: Infinispan
> Issue Type: Feature Request
> Components: JMX, reporting and management
> Reporter: Sebastian Łaskawiec
> Assignee: Vladimir Blagojevic
> Attachments: Screen Shot 2017-07-26 at 9.45.27 AM.png
>
>
> The Infinispan Management Console can also operate in Standalone mode. We would like to make it accessible in OpenShift. However we should probably display a warning that the console should be used only for monitoring and none of the changes are persisted.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 5 months
[JBoss JIRA] (ISPN-7971) Display a warning that changes in standalone mode are not persisted
by Vladimir Blagojevic (JIRA)
[ https://issues.jboss.org/browse/ISPN-7971?page=com.atlassian.jira.plugin.... ]
Vladimir Blagojevic updated ISPN-7971:
--------------------------------------
Attachment: Screen Shot 2017-07-26 at 9.45.27 AM.png
> Display a warning that changes in standalone mode are not persisted
> -------------------------------------------------------------------
>
> Key: ISPN-7971
> URL: https://issues.jboss.org/browse/ISPN-7971
> Project: Infinispan
> Issue Type: Feature Request
> Components: JMX, reporting and management
> Reporter: Sebastian Łaskawiec
> Assignee: Vladimir Blagojevic
> Attachments: Screen Shot 2017-07-26 at 9.45.27 AM.png
>
>
> The Infinispan Management Console can also operate in Standalone mode. We would like to make it accessible in OpenShift. However we should probably display a warning that the console should be used only for monitoring and none of the changes are persisted.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 5 months