April 2015 - teiid-issues - Jboss List Archives

[JBoss JIRA] (TEIID-3443) wrong estimation of BufferFrontedFileStoreCache.maxMemoryBlocks

by Hisanobu Okuda (JIRA)

[ https://issues.jboss.org/browse/TEIID-3443?page=com.atlassian.jira.plugin... ] Hisanobu Okuda commented on TEIID-3443: --------------------------------------- suggested fix:- {code} diff --git a/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java b/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java index 8e2464a..e050db5 100644 --- a/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java +++ b/engine/src/main/java/org/teiid/common/buffer/impl/BufferFrontedFileStoreCache.java @@ -576,7 +576,7 @@ public class BufferFrontedFileStoreCache implements Cache<PhysicalInfo> { } memoryWritePermits = new Semaphore(blocks); maxMemoryBlocks = Math.min(MAX_DOUBLE_INDIRECT, blocks); - maxMemoryBlocks = Math.min(maxMemoryBlocks, maxStorageObjectSize>>LOG_BLOCK_SIZE + ((maxStorageObjectSize&BufferFrontedFileStoreCache.BLOCK_MASK)>0?1:0)); + maxMemoryBlocks = Math.min(maxMemoryBlocks, maxStorageObjectSize/(1<<LOG_BLOCK_SIZE)); //try to maintain enough freespace so that writers don't block in cleaning cleaningThreshold = Math.min(maxMemoryBlocks<<4, blocks>>1); criticalCleaningThreshold = Math.min(maxMemoryBlocks<<2, blocks>>2); @@ -1233,4 +1233,4 @@ public class BufferFrontedFileStoreCache implements Cache<PhysicalInfo> { this.compactBufferFiles = compactBufferFiles; } -} \ No newline at end of file +} {code} > wrong estimation of BufferFrontedFileStoreCache.maxMemoryBlocks > --------------------------------------------------------------- > > Key: TEIID-3443 > URL: https://issues.jboss.org/browse/TEIID-3443 > Project: Teiid > Issue Type: Bug > Components: Query Engine > Affects Versions: 8.4, 8.10 > Environment: JDV 6.0.x, 6.1.0 > Reporter: Hisanobu Okuda > Assignee: Steven Hawkins > > BufferFrontedFileStoreCache.maxMemoryBlocks is under-estimated. For example, when max-storage-object-size=8000000, maxMemoryBlocks > is 488 at the line #575 in org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize() (after that, it is decrimented some times, but it does not matter). > {code} > main[1] run > > > Breakpoint hit: "thread=MSC service thread 1-3", org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize(), line=573 bci=163 > 573 maxMemoryBlocks = Math.min(maxMemoryBlocks, maxStorageObjectSize>>LOG_BLOCK_SIZE + ((maxStorageObjectSize&BufferFrontedFileStoreCache.BLOCK_MASK)>0?1:0)); > MSC service thread 1-3[1] next > > > Step completed: "thread=MSC service thread 1-3", org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize(), line=575 bci=198 > 575 cleaningThreshold = Math.min(maxMemoryBlocks<<4, blocks>>1); > MSC service thread 1-3[1] dump this > this = { > ...snip... > maxStorageObjectSize: 8000000 > ...snip... > maxMemoryBlocks: 488 > {code} > An actual cache object size for maxMemoryBlocks: 488 is roughly:- > {code} > 8192 * 488 = 3997696 > {code} > It is less than half of maxStorageObjectSize. It sometimes causes TEIID30001. -- This message was sent by Atlassian JIRA (v6.3.11#6341)

9 years, 8 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3443) wrong estimation of BufferFrontedFileStoreCache.maxMemoryBlocks

by Hisanobu Okuda (JIRA)

Hisanobu Okuda created TEIID-3443: ------------------------------------- Summary: wrong estimation of BufferFrontedFileStoreCache.maxMemoryBlocks Key: TEIID-3443 URL: https://issues.jboss.org/browse/TEIID-3443 Project: Teiid Issue Type: Bug Components: Query Engine Affects Versions: 8.10, 8.4 Environment: JDV 6.0.x, 6.1.0 Reporter: Hisanobu Okuda Assignee: Steven Hawkins BufferFrontedFileStoreCache.maxMemoryBlocks is under-estimated. For example, when max-storage-object-size=8000000, maxMemoryBlocks is 488 at the line #575 in org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize() (after that, it is decrimented some times, but it does not matter). {code} main[1] run > Breakpoint hit: "thread=MSC service thread 1-3", org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize(), line=573 bci=163 573 maxMemoryBlocks = Math.min(maxMemoryBlocks, maxStorageObjectSize>>LOG_BLOCK_SIZE + ((maxStorageObjectSize&BufferFrontedFileStoreCache.BLOCK_MASK)>0?1:0)); MSC service thread 1-3[1] next > Step completed: "thread=MSC service thread 1-3", org.teiid.common.buffer.impl.BufferFrontedFileStoreCache.initialize(), line=575 bci=198 575 cleaningThreshold = Math.min(maxMemoryBlocks<<4, blocks>>1); MSC service thread 1-3[1] dump this this = { ...snip... maxStorageObjectSize: 8000000 ...snip... maxMemoryBlocks: 488 {code} An actual cache object size for maxMemoryBlocks: 488 is roughly:- {code} 8192 * 488 = 3997696 {code} It is less than half of maxStorageObjectSize. It sometimes causes TEIID30001. -- This message was sent by Atlassian JIRA (v6.3.11#6341)

9 years, 8 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3442) Apache Spark support via SparkSQL and DataFrames

by John Muller (JIRA)

John Muller created TEIID-3442: ---------------------------------- Summary: Apache Spark support via SparkSQL and DataFrames Key: TEIID-3442 URL: https://issues.jboss.org/browse/TEIID-3442 Project: Teiid Issue Type: Feature Request Components: Misc. Connectors Affects Versions: 8.10 Reporter: John Muller Assignee: Steven Hawkins Fix For: 9.x Eliciting comments for Apache Spark support. With the release of Panda's like DataFrames, it is a little more feasible to directly translate to SparkSQL: https://spark.apache.org/docs/latest/sql-programming-guide.html Options in order of complexity: 1. Use the existing Hive connector / translator. Spark still uses the Hive metastore. 2. Thrift JDBC driver. This is what Microstrategy, Tableau, QlikView and others use, most rudimentary API for accessing Spark. 3. Native SparkSQL via building Spark jobs and submitting them to a running Spark driver. -- This message was sent by Atlassian JIRA (v6.3.11#6341)

9 years, 8 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3359) Allow more control over odata layer caching

by John Muller (JIRA)

[ https://issues.jboss.org/browse/TEIID-3359?page=com.atlassian.jira.plugin... ] John Muller commented on TEIID-3359: ------------------------------------ While it would break strict compliance with OData V2 & V4, I think the simplest option would be to allow a cache hint as a ReST query parameter. This would offer even more control over what gets cached and what doesn't; right down to the query-by-query level. Config can still serve as the defaults for odata caching. > Allow more control over odata layer caching > ------------------------------------------- > > Key: TEIID-3359 > URL: https://issues.jboss.org/browse/TEIID-3359 > Project: Teiid > Issue Type: Enhancement > Components: OData > Affects Versions: 8.7 > Reporter: Steven Hawkins > Assignee: Ramesh Reddy > Labels: Alpha1 > Fix For: 8.11 > > > The caching is always performed at a user level and for each query. Consumers may need to localize the caching affect only to paging results and scope only to that interaction, rather than for all users. -- This message was sent by Atlassian JIRA (v6.3.11#6341)

9 years, 8 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3412) Change MS Access to use jackcess, since ODBC-JDBC bridge is removed in JDK 1.8

by Kylin Soong (JIRA)

[ https://issues.jboss.org/browse/TEIID-3412?page=com.atlassian.jira.plugin... ] Kylin Soong commented on TEIID-3412: ------------------------------------ > Can you put together a pull request with their latest driver? Yes, I will base on 2.0.9.4 Release to add a ucanaccess translator > Change MS Access to use jackcess, since ODBC-JDBC bridge is removed in JDK 1.8 > ------------------------------------------------------------------------------ > > Key: TEIID-3412 > URL: https://issues.jboss.org/browse/TEIID-3412 > Project: Teiid > Issue Type: Feature Request > Components: Misc. Connectors > Affects Versions: 9.x > Reporter: Van Halbert > Assignee: Kylin Soong > > In JDK 1.8, the ODBC-JDBC bridge is removed, so accessing MS Access using this method will not be available. An alternative, open source, driver that can be used is: http://jackcess.sourceforge.net > In nexus: > <dependency> > <groupId>com.healthmarketscience.jackcess</groupId> > <artifactId>jackcess</artifactId> > <version>2.0.9</version> > </dependency> > This may require a new translator, but won't be know until a deeper look is done. -- This message was sent by Atlassian JIRA (v6.3.11#6341)

9 years, 8 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3441) XMLParse should allow varbinary as well

by Steven Hawkins (JIRA)

Steven Hawkins created TEIID-3441: ------------------------------------- Summary: XMLParse should allow varbinary as well Key: TEIID-3441 URL: https://issues.jboss.org/browse/TEIID-3441 Project: Teiid Issue Type: Quality Risk Components: Query Engine Reporter: Steven Hawkins Assignee: Steven Hawkins Priority: Minor Fix For: 8.11 XMLParse accepts blob and character types. It should also accept varbinary. -- This message was sent by Atlassian JIRA (v6.3.11#6341)

9 years, 8 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3440) add the ability to handle the existence of a Byte Order Mark (BOM) during XML File import

by Johnathon Lee (JIRA)

[ https://issues.jboss.org/browse/TEIID-3440?page=com.atlassian.jira.plugin... ] Johnathon Lee closed TEIID-3440. -------------------------------- Resolution: Done wrong project. > add the ability to handle the existence of a Byte Order Mark (BOM) during XML File import > ----------------------------------------------------------------------------------------- > > Key: TEIID-3440 > URL: https://issues.jboss.org/browse/TEIID-3440 > Project: Teiid > Issue Type: Feature Request > Reporter: Johnathon Lee > Assignee: Steven Hawkins > > existence of a BOM ( "\xEF\xBB\xBF" ) will cause an error during the import of an XML file. Can this be accounted for regardless of encoding? -- This message was sent by Atlassian JIRA (v6.3.11#6341)

9 years, 8 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3440) add the ability to handle the existence of a Byte Order Mark (BOM) during XML File import

by Johnathon Lee (JIRA)

Johnathon Lee created TEIID-3440: ------------------------------------ Summary: add the ability to handle the existence of a Byte Order Mark (BOM) during XML File import Key: TEIID-3440 URL: https://issues.jboss.org/browse/TEIID-3440 Project: Teiid Issue Type: Feature Request Reporter: Johnathon Lee Assignee: Steven Hawkins existence of a BOM ( "\xEF\xBB\xBF" ) will cause an error during the import of an XML file. Can this be accounted for regardless of encoding? -- This message was sent by Atlassian JIRA (v6.3.11#6341)

9 years, 8 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3412) Change MS Access to use jackcess, since ODBC-JDBC bridge is removed in JDK 1.8

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-3412?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-3412: --------------------------------------- Yes this would only be for ms access. That's the only general odbc source that we have left. As for general odbc source support that will have to be filled by third parties for now. Can you put together a pull request with their latest driver? > Change MS Access to use jackcess, since ODBC-JDBC bridge is removed in JDK 1.8 > ------------------------------------------------------------------------------ > > Key: TEIID-3412 > URL: https://issues.jboss.org/browse/TEIID-3412 > Project: Teiid > Issue Type: Feature Request > Components: Misc. Connectors > Affects Versions: 9.x > Reporter: Van Halbert > Assignee: Kylin Soong > > In JDK 1.8, the ODBC-JDBC bridge is removed, so accessing MS Access using this method will not be available. An alternative, open source, driver that can be used is: http://jackcess.sourceforge.net > In nexus: > <dependency> > <groupId>com.healthmarketscience.jackcess</groupId> > <artifactId>jackcess</artifactId> > <version>2.0.9</version> > </dependency> > This may require a new translator, but won't be know until a deeper look is done. -- This message was sent by Atlassian JIRA (v6.3.11#6341)

9 years, 8 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3358) Issues with entity set names

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-3358?page=com.atlassian.jira.plugin... ] Steven Hawkins updated TEIID-3358: ---------------------------------- Fix Version/s: 6.2-8.7.2 > Issues with entity set names > ---------------------------- > > Key: TEIID-3358 > URL: https://issues.jboss.org/browse/TEIID-3358 > Project: Teiid > Issue Type: Bug > Components: OData > Affects Versions: 8.7 > Reporter: Steven Hawkins > Assignee: Ramesh Reddy > Labels: Alpha1 > Fix For: 6.2-8.7.2, 8.11 > > > The entity set name used for sql generation should be the fully qualified name. If there is for example two table with the same names in schemas visible to odata, but one of the tables does not have a key, then an ambiguous name exception will be thrown if an odata url is used with only the base table name. > It also appears that the URI link in the results metadata uses the non-qualified table name, which will have issues in the scenario above. > We may also need to document or add an additional check for ambiguity as the OData4j findEdmEntitySet logic will return the first matching entity, which is generally against our approach to resolving. -- This message was sent by Atlassian JIRA (v6.3.11#6341)

9 years, 8 months

1
0
0 / 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

teiid-issues April 2015