[teiid-issues] [JBoss JIRA] (TEIID-3422) Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()

Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize() -------------------------------------------------------------------------------------------------------------------------------------------------------- Key: TEIID-3422 URL: https://issues.jboss.org/browse/TEIID-3422 Project: Teiid Issue Type: Bug Components: Query Engine Affects Versions: 8.7 Environment: JBoss DV 6.0, 6.1 Reporter: hisao furuichi Assignee: Steven Hawkins Fix For: 6.2-8.7.2, 8.11 Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize(). One of our user needs to set more than 512MB for buffer-service-max-storage-object-size to avoid TEIID30001[1], this limitation becomes a critical issue for them. Additional Information: By taking look at the source code[2], if the value of maxStorageObjectSize is more than equals with 536870912, the loop becomes infinit, and will cause OOME. [1] TEIID30001 Max block number exceeded by 233,144 21,422,155. Increase the maxStorageObjectSize to support larger storage objects. Alternatively you could make the processor batch size smaller. [2]teiid/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java === public static final long MAX_ADDRESSABLE_MEMORY = 1l<<(ADDRESS_BITS+LOG_BLOCK_SIZE); ~~ static final int BLOCK_SIZE = 1 << LOG_BLOCK_SIZE; ~~ public void initialize() throws TeiidComponentException { ~~ List<BlockStore> stores = new ArrayList<BlockStore>(); int size = BLOCK_SIZE; int files = 32; //this allows us to have 64 terabytes of smaller block sizes do { stores.add(new BlockStore(this.storageManager, size, 30, files)); size <<=1; if (files > 1) { files >>= 1; } } while ((size>>1) < maxStorageObjectSize); ~~ ===