[ https://issues.jboss.org/browse/ISPN-9817?page=com.atlassian.jira.plugin.... ] Rakesh Vende updated ISPN-9817: ------------------------------- Steps to Reproduce: *We need to address two issue here 1. We need to control amount of data to be replicated. In ReplicationQueueImpl.java flush method has drainReplQueue() method. This method drains entire queue, may result in create large byte array which gets fail one OOM. Even though we minimize frquency as low as possible, there is possibility huge data coming in. Resolution - We need to have explicit control on how much queue is getting drained before replication begins. 2. Async replication is becoming Sync replication. This is because the way add method implemented. It calls flush method on checking max element converts async replication to sync. This also needs to be fixed. * ---- 1. Application threads frequently calling put method on replicated cache results in calling flush method of ReplicationQueueImpl.java 2. This cause application threads to wait for every 500th put call to complete the cache replication from the queue 3. This becomes kind of sync replication which blocks application threads. 4. To avoid this situation, we can increase the queue size large enough, which, apparently, does not have any side effect as queue is linked blocking queue and application threads will only get blocked when queue becomes full. 5. However this puts pressure on aysnc queue, which has to replicate entire queue at once. _replicationQueue-thread--p4-t1 tid=119 [RUNNABLE] [DAEMON] <--- OutOfMemoryError happened in this thread java.lang.OutOfMemoryError.<init>() OutOfMemoryError.java:48 org.infinispan.commons.io.ExposedByteArrayOutputStream.write(byte[], int, int) ExposedByteArrayOutputStream.java:71 _ 6. This out of memory happens when JVM fails to allocate continuations chunk of memory in the form of array of 1 or 2 GB was: 1. Application threads frequently calling put method on replicated cache results in calling flush method of ReplicationQueueImpl.java 2. This cause application threads to wait for every 500th put call to complete the cache replication from the queue 3. This becomes kind of sync replication which blocks application threads. 4. To avoid this situation, we can increase the queue size large enough, which, apparently, does not have any side effect as queue is linked blocking queue and application threads will only get blocked when queue becomes full. 5. However this puts pressure on aysnc queue, which has to replicate entire queue at once. _replicationQueue-thread--p4-t1 tid=119 [RUNNABLE] [DAEMON] <--- OutOfMemoryError happened in this thread java.lang.OutOfMemoryError.<init>() OutOfMemoryError.java:48 org.infinispan.commons.io.ExposedByteArrayOutputStream.write(byte[], int, int) ExposedByteArrayOutputStream.java:71 _ 6. This out of memory happens when JVM fails to allocate continuations chunk of memory in the form of array of 1 or 2 GB -- This message was sent by Atlassian Jira (v7.12.1#712002)