[JBoss JIRA] (TEIID-3270) Allow UDFs to call Teiid functions
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3270?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-3270:
---------------------------------------
Nearly there. We'll represent these functions as procedures and then add an entry into a function tree based upon the same conversion that is done in SQLParserUtil. We may want to add more validation of what types of statements are allowed in a function definition - for example sql server will not allow updates, try/catch, calls to other stored procedures etc.
> Allow UDFs to call Teiid functions
> ----------------------------------
>
> Key: TEIID-3270
> URL: https://issues.jboss.org/browse/TEIID-3270
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine
> Reporter: Sanjeev Gour
> Assignee: Steven Hawkins
> Fix For: 8.11
>
>
> Currently custom functions require the implementation on be in Java, there could be an alternative way to allow using Teiid expressions inside the custom functions. The related thread is here-
> https://developer.jboss.org/thread/251179
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
10 years, 12 months
[JBoss JIRA] (TEIID-3419) Add ability to set a default role across all VDBs
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3419?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-3419:
----------------------------------
Fix Version/s: 8.11
> Add ability to set a default role across all VDBs
> -------------------------------------------------
>
> Key: TEIID-3419
> URL: https://issues.jboss.org/browse/TEIID-3419
> Project: Teiid
> Issue Type: Feature Request
> Components: Server
> Reporter: Marc Shirley
> Assignee: Steven Hawkins
> Fix For: 8.11
>
>
> There should be an ability to either set a default role across the server, or to restrict access by default. For example, all VDBs have a default user role that is applied before the VDB specific roles are. Alternatively, there could be a system wide flag that toggles the behavior of VDBs when no roles are defined between accessible to all or denied to all.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
10 years, 12 months
[JBoss JIRA] (TEIID-3419) Add ability to set a default role across all VDBs
by Marc Shirley (JIRA)
[ https://issues.jboss.org/browse/TEIID-3419?page=com.atlassian.jira.plugin... ]
Marc Shirley commented on TEIID-3419:
-------------------------------------
Yes, it seems that would be acceptable to deny deployment when no roles are defined.
> Add ability to set a default role across all VDBs
> -------------------------------------------------
>
> Key: TEIID-3419
> URL: https://issues.jboss.org/browse/TEIID-3419
> Project: Teiid
> Issue Type: Feature Request
> Components: Server
> Reporter: Marc Shirley
> Assignee: Steven Hawkins
>
> There should be an ability to either set a default role across the server, or to restrict access by default. For example, all VDBs have a default user role that is applied before the VDB specific roles are. Alternatively, there could be a system wide flag that toggles the behavior of VDBs when no roles are defined between accessible to all or denied to all.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
10 years, 12 months
[JBoss JIRA] (TEIID-3422) Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3422?page=com.atlassian.jira.plugin... ]
Steven Hawkins edited comment on TEIID-3422 at 4/10/15 9:26 AM:
----------------------------------------------------------------
It is really up to the details. If there is a very specific data set, it's a large vm, etc. then it could be fine to have a high setting.
was (Author: shawkins):
It is really up to the details. If there is a very specific data set, it's a large vm, etc. then it could be fine to have a higher setting.
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
> --------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-3422
> URL: https://issues.jboss.org/browse/TEIID-3422
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.7
> Environment: JBoss DV 6.0, 6.1
> Reporter: hisao furuichi
> Assignee: Steven Hawkins
> Fix For: 6.2-8.7.2, 8.11
>
>
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize().
> One of our user needs to set more than 512MB for buffer-service-max-storage-object-size to avoid TEIID30001[1], this limitation becomes a critical issue for them.
> Additional Information:
> By taking look at the source code[2], if the value of maxStorageObjectSize is more than equals with 536870912, the loop becomes infinit, and will cause OOME.
> [1]
> TEIID30001 Max block number exceeded by 233,144 21,422,155. Increase the maxStorageObjectSize to support larger storage objects. Alternatively you could make the processor batch size smaller.
> [2]teiid/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java
> ===
> public static final long MAX_ADDRESSABLE_MEMORY = 1l<<(ADDRESS_BITS+LOG_BLOCK_SIZE);
> ~~
> static final int BLOCK_SIZE = 1 << LOG_BLOCK_SIZE;
> ~~
> public void initialize() throws TeiidComponentException {
> ~~
> List<BlockStore> stores = new ArrayList<BlockStore>();
> int size = BLOCK_SIZE;
> int files = 32; //this allows us to have 64 terabytes of smaller block sizes
> do {
> stores.add(new BlockStore(this.storageManager, size, 30, files));
> size <<=1;
> if (files > 1) {
> files >>= 1;
> }
> } while ((size>>1) < maxStorageObjectSize);
> ~~
> ===
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
10 years, 12 months
[JBoss JIRA] (TEIID-3422) Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3422?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-3422:
---------------------------------------
It is really up to the details. If there is a very specific data set, it's a large vm, etc. then it could be fine to have a higher setting.
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
> --------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-3422
> URL: https://issues.jboss.org/browse/TEIID-3422
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.7
> Environment: JBoss DV 6.0, 6.1
> Reporter: hisao furuichi
> Assignee: Steven Hawkins
> Fix For: 6.2-8.7.2, 8.11
>
>
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize().
> One of our user needs to set more than 512MB for buffer-service-max-storage-object-size to avoid TEIID30001[1], this limitation becomes a critical issue for them.
> Additional Information:
> By taking look at the source code[2], if the value of maxStorageObjectSize is more than equals with 536870912, the loop becomes infinit, and will cause OOME.
> [1]
> TEIID30001 Max block number exceeded by 233,144 21,422,155. Increase the maxStorageObjectSize to support larger storage objects. Alternatively you could make the processor batch size smaller.
> [2]teiid/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java
> ===
> public static final long MAX_ADDRESSABLE_MEMORY = 1l<<(ADDRESS_BITS+LOG_BLOCK_SIZE);
> ~~
> static final int BLOCK_SIZE = 1 << LOG_BLOCK_SIZE;
> ~~
> public void initialize() throws TeiidComponentException {
> ~~
> List<BlockStore> stores = new ArrayList<BlockStore>();
> int size = BLOCK_SIZE;
> int files = 32; //this allows us to have 64 terabytes of smaller block sizes
> do {
> stores.add(new BlockStore(this.storageManager, size, 30, files));
> size <<=1;
> if (files > 1) {
> files >>= 1;
> }
> } while ((size>>1) < maxStorageObjectSize);
> ~~
> ===
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
10 years, 12 months
[JBoss JIRA] (TEIID-3422) Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-3422?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-3422:
------------------------------------
Is there a logging message that we could put out a warning message, and maybe a suggestion with it?
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
> --------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-3422
> URL: https://issues.jboss.org/browse/TEIID-3422
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.7
> Environment: JBoss DV 6.0, 6.1
> Reporter: hisao furuichi
> Assignee: Steven Hawkins
> Fix For: 6.2-8.7.2, 8.11
>
>
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize().
> One of our user needs to set more than 512MB for buffer-service-max-storage-object-size to avoid TEIID30001[1], this limitation becomes a critical issue for them.
> Additional Information:
> By taking look at the source code[2], if the value of maxStorageObjectSize is more than equals with 536870912, the loop becomes infinit, and will cause OOME.
> [1]
> TEIID30001 Max block number exceeded by 233,144 21,422,155. Increase the maxStorageObjectSize to support larger storage objects. Alternatively you could make the processor batch size smaller.
> [2]teiid/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java
> ===
> public static final long MAX_ADDRESSABLE_MEMORY = 1l<<(ADDRESS_BITS+LOG_BLOCK_SIZE);
> ~~
> static final int BLOCK_SIZE = 1 << LOG_BLOCK_SIZE;
> ~~
> public void initialize() throws TeiidComponentException {
> ~~
> List<BlockStore> stores = new ArrayList<BlockStore>();
> int size = BLOCK_SIZE;
> int files = 32; //this allows us to have 64 terabytes of smaller block sizes
> do {
> stores.add(new BlockStore(this.storageManager, size, 30, files));
> size <<=1;
> if (files > 1) {
> files >>= 1;
> }
> } while ((size>>1) < maxStorageObjectSize);
> ~~
> ===
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
10 years, 12 months
[JBoss JIRA] (TEIID-3422) Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-3422?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration updated TEIID-3422:
-------------------------------------------
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1210577, https://bugzilla.redhat.com/show_bug.cgi?id=1210714 (was: https://bugzilla.redhat.com/show_bug.cgi?id=1210577)
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
> --------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-3422
> URL: https://issues.jboss.org/browse/TEIID-3422
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.7
> Environment: JBoss DV 6.0, 6.1
> Reporter: hisao furuichi
> Assignee: Steven Hawkins
> Fix For: 6.2-8.7.2, 8.11
>
>
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize().
> One of our user needs to set more than 512MB for buffer-service-max-storage-object-size to avoid TEIID30001[1], this limitation becomes a critical issue for them.
> Additional Information:
> By taking look at the source code[2], if the value of maxStorageObjectSize is more than equals with 536870912, the loop becomes infinit, and will cause OOME.
> [1]
> TEIID30001 Max block number exceeded by 233,144 21,422,155. Increase the maxStorageObjectSize to support larger storage objects. Alternatively you could make the processor batch size smaller.
> [2]teiid/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java
> ===
> public static final long MAX_ADDRESSABLE_MEMORY = 1l<<(ADDRESS_BITS+LOG_BLOCK_SIZE);
> ~~
> static final int BLOCK_SIZE = 1 << LOG_BLOCK_SIZE;
> ~~
> public void initialize() throws TeiidComponentException {
> ~~
> List<BlockStore> stores = new ArrayList<BlockStore>();
> int size = BLOCK_SIZE;
> int files = 32; //this allows us to have 64 terabytes of smaller block sizes
> do {
> stores.add(new BlockStore(this.storageManager, size, 30, files));
> size <<=1;
> if (files > 1) {
> files >>= 1;
> }
> } while ((size>>1) < maxStorageObjectSize);
> ~~
> ===
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
10 years, 12 months
[JBoss JIRA] (TEIID-3422) Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-3422?page=com.atlassian.jira.plugin... ]
Van Halbert updated TEIID-3422:
-------------------------------
Fix Version/s: 6.2-8.7.2
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
> --------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-3422
> URL: https://issues.jboss.org/browse/TEIID-3422
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.7
> Environment: JBoss DV 6.0, 6.1
> Reporter: hisao furuichi
> Assignee: Steven Hawkins
> Fix For: 6.2-8.7.2, 8.11
>
>
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize().
> One of our user needs to set more than 512MB for buffer-service-max-storage-object-size to avoid TEIID30001[1], this limitation becomes a critical issue for them.
> Additional Information:
> By taking look at the source code[2], if the value of maxStorageObjectSize is more than equals with 536870912, the loop becomes infinit, and will cause OOME.
> [1]
> TEIID30001 Max block number exceeded by 233,144 21,422,155. Increase the maxStorageObjectSize to support larger storage objects. Alternatively you could make the processor batch size smaller.
> [2]teiid/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java
> ===
> public static final long MAX_ADDRESSABLE_MEMORY = 1l<<(ADDRESS_BITS+LOG_BLOCK_SIZE);
> ~~
> static final int BLOCK_SIZE = 1 << LOG_BLOCK_SIZE;
> ~~
> public void initialize() throws TeiidComponentException {
> ~~
> List<BlockStore> stores = new ArrayList<BlockStore>();
> int size = BLOCK_SIZE;
> int files = 32; //this allows us to have 64 terabytes of smaller block sizes
> do {
> stores.add(new BlockStore(this.storageManager, size, 30, files));
> size <<=1;
> if (files > 1) {
> files >>= 1;
> }
> } while ((size>>1) < maxStorageObjectSize);
> ~~
> ===
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
10 years, 12 months
[JBoss JIRA] (TEIID-3422) Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3422?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-3422.
-----------------------------------
Fix Version/s: 8.11
Resolution: Done
Updated to use a long for the size variable and added a hard max of 1 GB for the max object size.
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
> --------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-3422
> URL: https://issues.jboss.org/browse/TEIID-3422
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.7
> Environment: JBoss DV 6.0, 6.1
> Reporter: hisao furuichi
> Assignee: Steven Hawkins
> Fix For: 8.11
>
>
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize().
> One of our user needs to set more than 512MB for buffer-service-max-storage-object-size to avoid TEIID30001[1], this limitation becomes a critical issue for them.
> Additional Information:
> By taking look at the source code[2], if the value of maxStorageObjectSize is more than equals with 536870912, the loop becomes infinit, and will cause OOME.
> [1]
> TEIID30001 Max block number exceeded by 233,144 21,422,155. Increase the maxStorageObjectSize to support larger storage objects. Alternatively you could make the processor batch size smaller.
> [2]teiid/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java
> ===
> public static final long MAX_ADDRESSABLE_MEMORY = 1l<<(ADDRESS_BITS+LOG_BLOCK_SIZE);
> ~~
> static final int BLOCK_SIZE = 1 << LOG_BLOCK_SIZE;
> ~~
> public void initialize() throws TeiidComponentException {
> ~~
> List<BlockStore> stores = new ArrayList<BlockStore>();
> int size = BLOCK_SIZE;
> int files = 32; //this allows us to have 64 terabytes of smaller block sizes
> do {
> stores.add(new BlockStore(this.storageManager, size, 30, files));
> size <<=1;
> if (files > 1) {
> files >>= 1;
> }
> } while ((size>>1) < maxStorageObjectSize);
> ~~
> ===
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
10 years, 12 months
[JBoss JIRA] (TEIID-3422) Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3422?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-3422:
---------------------------------------
> By taking look at the source code[2], if the value of maxStorageObjectSize is more than equals with 536870912, the loop becomes infinit, and will cause OOME.
Yes there is an issue there due to the integer overflow. That will be corrected.
> One of our user needs to set more than 512MB for buffer-service-max-storage-object-size to avoid TEIID30001[1], this limitation becomes a critical issue for them.
How big is the vm, processor batch size, and what kind of data is being serialized? Setting a value that high probably means that there is data inline in the batches that should not be.
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize()
> --------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-3422
> URL: https://issues.jboss.org/browse/TEIID-3422
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.7
> Environment: JBoss DV 6.0, 6.1
> Reporter: hisao furuichi
> Assignee: Steven Hawkins
>
> Setting more than 513MB for buffer-service-max-storage-object-size, OOME occurs at org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize().
> One of our user needs to set more than 512MB for buffer-service-max-storage-object-size to avoid TEIID30001[1], this limitation becomes a critical issue for them.
> Additional Information:
> By taking look at the source code[2], if the value of maxStorageObjectSize is more than equals with 536870912, the loop becomes infinit, and will cause OOME.
> [1]
> TEIID30001 Max block number exceeded by 233,144 21,422,155. Increase the maxStorageObjectSize to support larger storage objects. Alternatively you could make the processor batch size smaller.
> [2]teiid/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java
> ===
> public static final long MAX_ADDRESSABLE_MEMORY = 1l<<(ADDRESS_BITS+LOG_BLOCK_SIZE);
> ~~
> static final int BLOCK_SIZE = 1 << LOG_BLOCK_SIZE;
> ~~
> public void initialize() throws TeiidComponentException {
> ~~
> List<BlockStore> stores = new ArrayList<BlockStore>();
> int size = BLOCK_SIZE;
> int files = 32; //this allows us to have 64 terabytes of smaller block sizes
> do {
> stores.add(new BlockStore(this.storageManager, size, 30, files));
> size <<=1;
> if (files > 1) {
> files >>= 1;
> }
> } while ((size>>1) < maxStorageObjectSize);
> ~~
> ===
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
10 years, 12 months