[teiid-issues] [JBoss JIRA] (TEIID-3443) wrong estimation of BufferFrontedFileStoreCache.maxMemoryBlocks

[Hisanobu Okuda (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13060026#comment-13060026 ] 

Hisanobu Okuda commented on TEIID-3443:
---------------------------------------

suggested fix:-

{code}
diff --git a/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java b/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java
index 8e2464a..e050db5 100644
--- a/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java
+++ b/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java
@@ -576,7 +576,7 @@ public class BufferFrontedFileStoreCache implements Cache<PhysicalInfo> {
                }
                memoryWritePermits = new Semaphore(blocks);
                maxMemoryBlocks = Math.min(MAX_DOUBLE_INDIRECT, blocks);
-               maxMemoryBlocks = Math.min(maxMemoryBlocks, maxStorageObjectSize>>LOG_BLOCK_SIZE + ((maxStorageObjectSize&BufferFrontedFileStoreCache.BLOCK_MASK)>0?1:0));
+               maxMemoryBlocks = Math.min(maxMemoryBlocks, maxStorageObjectSize/(1<<LOG_BLOCK_SIZE));
                //try to maintain enough freespace so that writers don't block in cleaning
                cleaningThreshold = Math.min(maxMemoryBlocks<<4, blocks>>1);
                criticalCleaningThreshold = Math.min(maxMemoryBlocks<<2, blocks>>2);
@@ -1233,4 +1233,4 @@ public class BufferFrontedFileStoreCache implements Cache<PhysicalInfo> {
                this.compactBufferFiles = compactBufferFiles;
        }
        
-}
\ No newline at end of file
+}
{code}

> wrong estimation of BufferFrontedFileStoreCache.maxMemoryBlocks
> ---------------------------------------------------------------
>
>                 Key: TEIID-3443
>                 URL: https://issues.jboss.org/browse/TEIID-3443
>             Project: Teiid
>          Issue Type: Bug
>          Components: Query Engine
>    Affects Versions: 8.4, 8.10
>         Environment: JDV 6.0.x, 6.1.0
>            Reporter: Hisanobu Okuda
>            Assignee: Steven Hawkins
>
> BufferFrontedFileStoreCache.maxMemoryBlocks is under-estimated. For example, when max-storage-object-size=8000000, maxMemoryBlocks
> is 488 at the line #575 in org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize() (after that, it is decrimented some times, but it does not matter).
> {code}
> main[1] run
> > 
> Breakpoint hit: "thread=MSC service thread 1-3", org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize(), line=573 bci=163
> 573    		maxMemoryBlocks = Math.min(maxMemoryBlocks, maxStorageObjectSize>>LOG_BLOCK_SIZE + ((maxStorageObjectSize&BufferFrontedFileStoreCache.BLOCK_MASK)>0?1:0));
> MSC service thread 1-3[1] next
> > 
> Step completed: "thread=MSC service thread 1-3", org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize(), line=575 bci=198
> 575    		cleaningThreshold = Math.min(maxMemoryBlocks<<4, blocks>>1);
> MSC service thread 1-3[1] dump this
>  this = {
>     ...snip...
>     maxStorageObjectSize: 8000000
>     ...snip...
>     maxMemoryBlocks: 488
> {code}
> An actual cache object size for maxMemoryBlocks: 488 is roughly:-
> {code}
> 8192 * 488 = 3997696
> {code}
> It is less than half of maxStorageObjectSize. It sometimes causes TEIID30001.



--
This message was sent by Atlassian JIRA
(v6.3.11#6341)