August 2014 - teiid-issues - Jboss List Archives

[JBoss JIRA] (TEIID-3099) RejectedExecutionException for Teiid during high query load

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-3099?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-3099: --------------------------------------- After looking at this again, it would be simpler to just set the actual pool max threads to a much higher number or unbounded, with the possibility of temporarily creating too many threads rather than getting the rejection. > RejectedExecutionException for Teiid during high query load > ----------------------------------------------------------- > > Key: TEIID-3099 > URL: https://issues.jboss.org/browse/TEIID-3099 > Project: Teiid > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Query Engine, Server > Affects Versions: 8.7 > Environment: Windows (x64) and Mainframe z/OS (x64) > Reporter: Mark Ackert > Assignee: Steven Hawkins > Labels: queryengine, rejectedexecutionexception, teiid-engine, threads > > Occasionally, when a standalone Teiid server is under high load from concurrent queries, a RejectedExecutionException is thrown. The relevant part of the stacktrace is below. I investigated the ThreadReuseExecutor source, and I believe the issue's cause is based around the sycnhronized(poolLock) code - see below the stack trace for info, and psuedo-code snippet with my analysis below the stacktrace. > Caused by: java.util.concurrent.RejectedExecutionException: Task org.teiid.dqp.internal.process.ThreadReuseExecutor$3@c75a8114 rejected from org.teiid.dqp.internal.process.ThreadReuseExecutor$2@8b1166ec[Running, pool size = 128, active threads = 128, queued tasks = 0, completed tasks = 692637] > at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048) [rt.jar:1.7.0] > at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821) [rt.jar:1.7.0] > at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372) [rt.jar:1.7.0] > at org.teiid.dqp.internal.process.ThreadReuseExecutor.executeDirect(ThreadReuseExecutor.java:200) [teiid-engine-8.7.0.FinalCAFix-SNAPSHOT.jar:8.7.0.FinalCAFix-SNAPSHOT] > at org.teiid.dqp.internal.process.ThreadReuseExecutor.execute(ThreadReuseExecutor.java:177) [teiid-engine-8.7.0.FinalCAFix-SNAPSHOT.jar:8.7.0.FinalCAFix-SNAPSHOT] > In the below executeDirect, the race condition that appears plausible to me is: given "activeCount" is at the thread pool size limit and a single thread is finishing execution...Let activeCount be reduced by one in the tpe.execute runnable's synchronized block. The synchronized block exits, and goes to the warnWait/logging code - at this point a thread context switch occurs, and a new PrioritizedRunnable (one which isn't waiting on the poolLock at the beginning of the executeDirect method) comes through executeDirect and proceeds forward because "poolLock" has been released, "activeCount" is now sizeLimit-1. The tpe tries to execute a new Runnable wrapping this PrioritizedRunnable, but the previous thread which hasn't completed it's logging code yet is still present in the the threadpool, and as such we get a RejectedExecutionException due to too many threads trying to be executed in a fixed size thread pool. > private void executeDirect(final PrioritizedRunnable command) { > boolean atMaxThreads = false; > synchronized (poolLock) { > .... if activeCount!=max_limit; activeCount++ > } > ....... > tpe.execute(..... > finally { > synchronized (poolLock) { > ....... > activeCont--; > } > if (success) { > ......some log code > } > } -- This message was sent by Atlassian JIRA (v6.3.1#6329)

10 years, 4 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3098) Expose the ModeShape JCR functions in the modeshape translator metadata

by Van Halbert (JIRA)

[ https://issues.jboss.org/browse/TEIID-3098?page=com.atlassian.jira.plugin... ] Work on TEIID-3098 started by Van Halbert. ------------------------------------------ > Expose the ModeShape JCR functions in the modeshape translator metadata > ------------------------------------------------------------------------ > > Key: TEIID-3098 > URL: https://issues.jboss.org/browse/TEIID-3098 > Project: Teiid > Issue Type: Enhancement > Security Level: Public(Everyone can see) > Components: Misc. Connectors > Affects Versions: 8.7.1 > Reporter: Van Halbert > Assignee: Van Halbert > > Expose the ModeShape JCR functions in the metadata in the modeshape translator. -- This message was sent by Atlassian JIRA (v6.3.1#6329)

10 years, 4 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3099) RejectedExecutionException for Teiid during high query load

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-3099?page=com.atlassian.jira.plugin... ] Steven Hawkins commented on TEIID-3099: --------------------------------------- Yes, I agree that can occur. Even if we expand the scope of the poolLock there is still a potential for a timing issue. It seems best to not rely on a SynchronousQueue and instead use a LinkeBlockingQueue, but we'll also have to check that queue as part of the reuse logic as to keep scheduling more fair. > RejectedExecutionException for Teiid during high query load > ----------------------------------------------------------- > > Key: TEIID-3099 > URL: https://issues.jboss.org/browse/TEIID-3099 > Project: Teiid > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Query Engine, Server > Affects Versions: 8.7 > Environment: Windows (x64) and Mainframe z/OS (x64) > Reporter: Mark Ackert > Assignee: Steven Hawkins > Labels: queryengine, rejectedexecutionexception, teiid-engine, threads > > Occasionally, when a standalone Teiid server is under high load from concurrent queries, a RejectedExecutionException is thrown. The relevant part of the stacktrace is below. I investigated the ThreadReuseExecutor source, and I believe the issue's cause is based around the sycnhronized(poolLock) code - see below the stack trace for info, and psuedo-code snippet with my analysis below the stacktrace. > Caused by: java.util.concurrent.RejectedExecutionException: Task org.teiid.dqp.internal.process.ThreadReuseExecutor$3@c75a8114 rejected from org.teiid.dqp.internal.process.ThreadReuseExecutor$2@8b1166ec[Running, pool size = 128, active threads = 128, queued tasks = 0, completed tasks = 692637] > at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048) [rt.jar:1.7.0] > at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821) [rt.jar:1.7.0] > at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372) [rt.jar:1.7.0] > at org.teiid.dqp.internal.process.ThreadReuseExecutor.executeDirect(ThreadReuseExecutor.java:200) [teiid-engine-8.7.0.FinalCAFix-SNAPSHOT.jar:8.7.0.FinalCAFix-SNAPSHOT] > at org.teiid.dqp.internal.process.ThreadReuseExecutor.execute(ThreadReuseExecutor.java:177) [teiid-engine-8.7.0.FinalCAFix-SNAPSHOT.jar:8.7.0.FinalCAFix-SNAPSHOT] > In the below executeDirect, the race condition that appears plausible to me is: given "activeCount" is at the thread pool size limit and a single thread is finishing execution...Let activeCount be reduced by one in the tpe.execute runnable's synchronized block. The synchronized block exits, and goes to the warnWait/logging code - at this point a thread context switch occurs, and a new PrioritizedRunnable (one which isn't waiting on the poolLock at the beginning of the executeDirect method) comes through executeDirect and proceeds forward because "poolLock" has been released, "activeCount" is now sizeLimit-1. The tpe tries to execute a new Runnable wrapping this PrioritizedRunnable, but the previous thread which hasn't completed it's logging code yet is still present in the the threadpool, and as such we get a RejectedExecutionException due to too many threads trying to be executed in a fixed size thread pool. > private void executeDirect(final PrioritizedRunnable command) { > boolean atMaxThreads = false; > synchronized (poolLock) { > .... if activeCount!=max_limit; activeCount++ > } > ....... > tpe.execute(..... > finally { > synchronized (poolLock) { > ....... > activeCont--; > } > if (success) { > ......some log code > } > } -- This message was sent by Atlassian JIRA (v6.3.1#6329)

10 years, 4 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3099) RejectedExecutionException for Teiid during high query load

by Mark Ackert (JIRA)

Mark Ackert created TEIID-3099: ---------------------------------- Summary: RejectedExecutionException for Teiid during high query load Key: TEIID-3099 URL: https://issues.jboss.org/browse/TEIID-3099 Project: Teiid Issue Type: Bug Security Level: Public (Everyone can see) Components: Query Engine, Server Affects Versions: 8.7 Environment: Windows (x64) and Mainframe z/OS (x64) Reporter: Mark Ackert Assignee: Steven Hawkins Occasionally, when a standalone Teiid server is under high load from concurrent queries, a RejectedExecutionException is thrown. The relevant part of the stacktrace is below. I investigated the ThreadReuseExecutor source, and I believe the issue's cause is based around the sycnhronized(poolLock) code - see below the stack trace for info, and psuedo-code snippet with my analysis below the stacktrace. Caused by: java.util.concurrent.RejectedExecutionException: Task org.teiid.dqp.internal.process.ThreadReuseExecutor$3@c75a8114 rejected from org.teiid.dqp.internal.process.ThreadReuseExecutor$2@8b1166ec[Running, pool size = 128, active threads = 128, queued tasks = 0, completed tasks = 692637] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048) [rt.jar:1.7.0] at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821) [rt.jar:1.7.0] at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372) [rt.jar:1.7.0] at org.teiid.dqp.internal.process.ThreadReuseExecutor.executeDirect(ThreadReuseExecutor.java:200) [teiid-engine-8.7.0.FinalCAFix-SNAPSHOT.jar:8.7.0.FinalCAFix-SNAPSHOT] at org.teiid.dqp.internal.process.ThreadReuseExecutor.execute(ThreadReuseExecutor.java:177) [teiid-engine-8.7.0.FinalCAFix-SNAPSHOT.jar:8.7.0.FinalCAFix-SNAPSHOT] In the below executeDirect, the race condition that appears plausible to me is: given "activeCount" is at the thread pool size limit and a single thread is finishing execution...Let activeCount be reduced by one in the tpe.execute runnable's synchronized block. The synchronized block exits, and goes to the warnWait/logging code - at this point a thread context switch occurs, and a new PrioritizedRunnable (one which isn't waiting on the poolLock at the beginning of the executeDirect method) comes through executeDirect and proceeds forward because "poolLock" has been released, "activeCount" is now sizeLimit-1. The tpe tries to execute a new Runnable wrapping this PrioritizedRunnable, but the previous thread which hasn't completed it's logging code yet is still present in the the threadpool, and as such we get a RejectedExecutionException due to too many threads trying to be executed in a fixed size thread pool. private void executeDirect(final PrioritizedRunnable command) { boolean atMaxThreads = false; synchronized (poolLock) { .... if activeCount!=max_limit; activeCount++ } ....... tpe.execute(..... finally { synchronized (poolLock) { ....... activeCont--; } if (success) { ......some log code } } -- This message was sent by Atlassian JIRA (v6.3.1#6329)

10 years, 4 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3041) Add support for xml document models in ddl

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-3041?page=com.atlassian.jira.plugin... ] Steven Hawkins resolved TEIID-3041. ----------------------------------- Fix Version/s: (was: 8.9) Resolution: Deferred After discussions with Barry, we'll look to leave the existing support alone and just use index metadata for xml document models. If this becomes an inhibitor for Designer though we can revisit this issue. > Add support for xml document models in ddl > ------------------------------------------ > > Key: TEIID-3041 > URL: https://issues.jboss.org/browse/TEIID-3041 > Project: Teiid > Issue Type: Enhancement > Security Level: Public(Everyone can see) > Components: Query Engine > Reporter: Steven Hawkins > Assignee: Steven Hawkins > > xml document model support was left out of the initial ddl design, but since the feature has not yet been retargeted from designer we should add support. -- This message was sent by Atlassian JIRA (v6.3.1#6329)

10 years, 4 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3098) Expose the ModeShape JCR functions in the modeshape translator metadata

by Van Halbert (JIRA)

Van Halbert created TEIID-3098: ---------------------------------- Summary: Expose the ModeShape JCR functions in the modeshape translator metadata Key: TEIID-3098 URL: https://issues.jboss.org/browse/TEIID-3098 Project: Teiid Issue Type: Enhancement Security Level: Public (Everyone can see) Components: Misc. Connectors Affects Versions: 8.7.1 Reporter: Van Halbert Assignee: Steven Hawkins Expose the ModeShape JCR functions in the metadata in the modeshape translator. -- This message was sent by Atlassian JIRA (v6.3.1#6329)

10 years, 4 months

1
0
0 / 0

[JBoss JIRA] (TEIID-3098) Expose the ModeShape JCR functions in the modeshape translator metadata

by Van Halbert (JIRA)

[ https://issues.jboss.org/browse/TEIID-3098?page=com.atlassian.jira.plugin... ] Van Halbert reassigned TEIID-3098: ---------------------------------- Assignee: Van Halbert (was: Steven Hawkins) > Expose the ModeShape JCR functions in the modeshape translator metadata > ------------------------------------------------------------------------ > > Key: TEIID-3098 > URL: https://issues.jboss.org/browse/TEIID-3098 > Project: Teiid > Issue Type: Enhancement > Security Level: Public(Everyone can see) > Components: Misc. Connectors > Affects Versions: 8.7.1 > Reporter: Van Halbert > Assignee: Van Halbert > > Expose the ModeShape JCR functions in the metadata in the modeshape translator. -- This message was sent by Atlassian JIRA (v6.3.1#6329)

10 years, 4 months

1
0
0 / 0

[JBoss JIRA] (TEIID-1910) Teiid support for EclipseLink persistance provider

by Steven Hawkins (JIRA)

[ https://issues.jboss.org/browse/TEIID-1910?page=com.atlassian.jira.plugin... ] Steven Hawkins resolved TEIID-1910. ----------------------------------- Fix Version/s: 8.9 (was: Open To Community) Resolution: Done Marking as resolved and adding a release note. > Teiid support for EclipseLink persistance provider > -------------------------------------------------- > > Key: TEIID-1910 > URL: https://issues.jboss.org/browse/TEIID-1910 > Project: Teiid > Issue Type: Task > Security Level: Public(Everyone can see) > Components: Query Engine > Affects Versions: 8.0 > Reporter: Sunil Varma > Assignee: Kylin Soong > Fix For: 8.9 > > > We need a dialect support for Eclipselink similar to the one done for HIbernate. > See thread:https://community.jboss.org/thread/176306 > op -- This message was sent by Atlassian JIRA (v6.3.1#6329)

10 years, 4 months

1
0
0 / 0

[JBoss JIRA] (TEIID-1910) Teiid support for EclipseLink persistance provider

by Kylin Soong (JIRA)

[ https://issues.jboss.org/browse/TEIID-1910?page=com.atlassian.jira.plugin... ] Kylin Soong commented on TEIID-1910: ------------------------------------ I have done add the doc(https://docs.jboss.org/author/display/teiid88final/Using+Teiid+with+E..., eclipselink only depend on eclipselink jar it available in maven, I think no other addition need to package. > Teiid support for EclipseLink persistance provider > -------------------------------------------------- > > Key: TEIID-1910 > URL: https://issues.jboss.org/browse/TEIID-1910 > Project: Teiid > Issue Type: Task > Security Level: Public(Everyone can see) > Components: Query Engine > Affects Versions: 8.0 > Reporter: Sunil Varma > Assignee: Kylin Soong > Fix For: Open To Community > > > We need a dialect support for Eclipselink similar to the one done for HIbernate. > See thread:https://community.jboss.org/thread/176306 > op -- This message was sent by Atlassian JIRA (v6.3.1#6329)

10 years, 4 months

1
0
0 / 0

[JBoss JIRA] (TEIID-2355) BigQuery integration

by Masilamani Subramanyam (JIRA)

[ https://issues.jboss.org/browse/TEIID-2355?page=com.atlassian.jira.plugin... ] Masilamani Subramanyam commented on TEIID-2355: ----------------------------------------------- I am trying to access BigQuery using Teiid with starsschema-bigquery-jdbc driver bqjdbc-1.4.jar but it's not working in teiid. Interesting is that it works fine when i try to access using SQuirrel SQL client. Just wondering whether anyone from teiid community tried already to connect and query successfully using starsschema-bigquery-jdbc driver ? > BigQuery integration > -------------------- > > Key: TEIID-2355 > URL: https://issues.jboss.org/browse/TEIID-2355 > Project: Teiid > Issue Type: Feature Request > Security Level: Public(Everyone can see) > Components: Misc. Connectors > Reporter: Steven Hawkins > Assignee: Kylin Soong > Fix For: Open To Community > > > http://code.google.com/p/starschema-bigquery-jdbc/ looks promising for simple support of BigQuery. Some of the dependencies could already be in the google-api module. -- This message was sent by Atlassian JIRA (v6.3.1#6329)

10 years, 4 months

1
0
0 / 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

teiid-issues August 2014