8:04 p.m.
Add query hint to limit parallel execution of union members
-----------------------------------------------------------
Key: TEIID-1474
URL: https://issues.jboss.org/browse/TEIID-1474
Project: Teiid
Issue Type: Feature Request
Reporter: Howard Abrams
Assignee: Steven Hawkins
We would like a new query hint to limit the number of parallel executions of union
members. This is useful in two situations:
1. When the queries are expensive, but the overall union contains a limit that is most
likely to be fulfilled by the first member of the union
2. When the queries are backed by a common resource which may be overloaded by multiple
queries
The addition of a hint would make this manageable by the client; perhaps something like
"/*+ parallel=2*/"?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
7:17 p.m.
New subject: [JBoss JIRA] Commented: (TEIID-1474) Add query hint to limit parallel execution of union members
[
https://issues.jboss.org/browse/TEIID-1474?page=com.atlassian.jira.plugin...
]
Steven Hawkins commented on TEIID-1474:
---------------------------------------
To round out the proposal:
Would the hint also affect joins to force serial execution?
Can the hint appear anywhere above the union (through 1 or more view layers), or would it
have to be located on the union query?
What heuristics would we apply to determine the firing order of branches?
Add query hint to limit parallel execution of union members
-----------------------------------------------------------
Key: TEIID-1474
URL: https://issues.jboss.org/browse/TEIID-1474
Project: Teiid
Issue Type: Feature Request
Reporter: Howard Abrams
Assignee: Steven Hawkins
We would like a new query hint to limit the number of parallel executions of union
members. This is useful in two situations:
1. When the queries are expensive, but the overall union contains a limit that is most
likely to be fulfilled by the first member of the union
2. When the queries are backed by a common resource which may be overloaded by multiple
queries
The addition of a hint would make this manageable by the client; perhaps something like
"/*+ parallel=2*/"?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:58 p.m.
New subject: [JBoss JIRA] Commented: (TEIID-1474) Add query hint to limit parallel execution of union members
[
https://issues.jboss.org/browse/TEIID-1474?page=com.atlassian.jira.plugin...
]
Howard Abrams commented on TEIID-1474:
--------------------------------------
Good questions.
Just for some background info, our current use-case for this is that we have a union made
up of over 400 queries; bad things happen within out underlying data-sources when we try
to execute an unconstrained query against it. So for now, we're just looking to be
able to serialize (or limit the parallelism) of a specific union to a) not overload our
system, and b) if we issue a query with a "limit 10" that is fulfilled by the
first query in the union don't bother hitting up the other 399 queries.
Presumably, if possible, the hint could effect joins as well, but I haven't given that
too much thought.
I would want the hint to only effect the specif union it next to. That way if we had a
very complex query with unions within union/joins/etc, we can keep some amount of
parallelism.
Assuming there was nothing inherit in the query against the union that would indicate
ordering, executing them in the order they appear in the union would seem most natural.
Add query hint to limit parallel execution of union members
-----------------------------------------------------------
Key: TEIID-1474
URL: https://issues.jboss.org/browse/TEIID-1474
Project: Teiid
Issue Type: Feature Request
Reporter: Howard Abrams
Assignee: Steven Hawkins
We would like a new query hint to limit the number of parallel executions of union
members. This is useful in two situations:
1. When the queries are expensive, but the overall union contains a limit that is most
likely to be fulfilled by the first member of the union
2. When the queries are backed by a common resource which may be overloaded by multiple
queries
The addition of a hint would make this manageable by the client; perhaps something like
"/*+ parallel=2*/"?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
5:59 p.m.
New subject: [JBoss JIRA] Commented: (TEIID-1474) Add query hint to limit parallel execution of union members
[
https://issues.jboss.org/browse/TEIID-1474?page=com.atlassian.jira.plugin...
]
Steven Hawkins commented on TEIID-1474:
---------------------------------------
After more thought, my initial approach would be to add a setting to limit the source
concurrency of all user requests.
By default it would be set to something like min(maxThreads, max(4,
2*maxThreads/maxActivePlans))
Having it at 1 would execute the request serially, just like when transactional.
Add query hint to limit parallel execution of union members
-----------------------------------------------------------
Key: TEIID-1474
URL: https://issues.jboss.org/browse/TEIID-1474
Project: Teiid
Issue Type: Feature Request
Reporter: Howard Abrams
Assignee: Steven Hawkins
We would like a new query hint to limit the number of parallel executions of union
members. This is useful in two situations:
1. When the queries are expensive, but the overall union contains a limit that is most
likely to be fulfilled by the first member of the union
2. When the queries are backed by a common resource which may be overloaded by multiple
queries
The addition of a hint would make this manageable by the client; perhaps something like
"/*+ parallel=2*/"?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:19 p.m.
New subject: [JBoss JIRA] Updated: (TEIID-1474) Add query hint to limit parallel execution of union members
[
https://issues.jboss.org/browse/TEIID-1474?page=com.atlassian.jira.plugin...
]
Steven Hawkins updated TEIID-1474:
----------------------------------
Fix Version/s: 7.4
Component/s: Query Engine
Affects: [Documentation (Ref Guide, User Guide, etc.), Release Notes]
The config now has userRequestSourceConcurrency, which is also covered in the admin guide,
to limit parallel source execution on a per user query basis. Having a hint override this
value is now straight-forward to implement, but it would take effect for the entire user
query, rather than a specific union or join. Would that cover your usecase, or would you
need control down to a specific operation?
Add query hint to limit parallel execution of union members
-----------------------------------------------------------
Key: TEIID-1474
URL: https://issues.jboss.org/browse/TEIID-1474
Project: Teiid
Issue Type: Feature Request
Components: Query Engine
Reporter: Howard Abrams
Assignee: Steven Hawkins
Fix For: 7.4
We would like a new query hint to limit the number of parallel executions of union
members. This is useful in two situations:
1. When the queries are expensive, but the overall union contains a limit that is most
likely to be fulfilled by the first member of the union
2. When the queries are backed by a common resource which may be overloaded by multiple
queries
The addition of a hint would make this manageable by the client; perhaps something like
"/*+ parallel=2*/"?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:57 p.m.
New subject: [JBoss JIRA] Updated: (TEIID-1474) Add a limit to the number of concurrent source queries issued.
[
https://issues.jboss.org/browse/TEIID-1474?page=com.atlassian.jira.plugin...
]
Steven Hawkins updated TEIID-1474:
----------------------------------
Summary: Add a limit to the number of concurrent source queries issued. (was: Add
query hint to limit parallel execution of union members)
Description: The default of allowing an unlimited number of concurrent source queries
is not desirable in extreme situations. A limit should be added to prevent contention or
other problems with issuing a large number of source queries. (was: We would like a new
query hint to limit the number of parallel executions of union members. This is useful in
two situations:
1. When the queries are expensive, but the overall union contains a limit that is most
likely to be fulfilled by the first member of the union
2. When the queries are backed by a common resource which may be overloaded by multiple
queries
The addition of a hint would make this manageable by the client; perhaps something like
"/*+ parallel=2*/"? )
updated to split into separate issues.
Add a limit to the number of concurrent source queries issued.
--------------------------------------------------------------
Key: TEIID-1474
URL: https://issues.jboss.org/browse/TEIID-1474
Project: Teiid
Issue Type: Feature Request
Components: Query Engine
Reporter: Howard Abrams
Assignee: Steven Hawkins
Fix For: 7.4
The default of allowing an unlimited number of concurrent source queries is not desirable
in extreme situations. A limit should be added to prevent contention or other problems
with issuing a large number of source queries.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:01 p.m.
New subject: [JBoss JIRA] Resolved: (TEIID-1474) Add a limit to the number of concurrent source queries issued.
[
https://issues.jboss.org/browse/TEIID-1474?page=com.atlassian.jira.plugin...
]
Steven Hawkins resolved TEIID-1474.
-----------------------------------
Resolution: Done
resolving the major issue, which was to add UserRequestSourceConcurrency. The addition of
a hint is tbd.
Add a limit to the number of concurrent source queries issued.
--------------------------------------------------------------
Key: TEIID-1474
URL: https://issues.jboss.org/browse/TEIID-1474
Project: Teiid
Issue Type: Feature Request
Components: Query Engine
Reporter: Howard Abrams
Assignee: Steven Hawkins
Fix For: 7.4
The default of allowing an unlimited number of concurrent source queries is not desirable
in extreme situations. A limit should be added to prevent contention or other problems
with issuing a large number of source queries.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:30 p.m.
New subject: [JBoss JIRA] Closed: (TEIID-1474) Add a limit to the number of concurrent source queries issued.
[
https://issues.jboss.org/browse/TEIID-1474?page=com.atlassian.jira.plugin...
]
Steven Hawkins closed TEIID-1474.
---------------------------------
Add a limit to the number of concurrent source queries issued.
--------------------------------------------------------------
Key: TEIID-1474
URL: https://issues.jboss.org/browse/TEIID-1474
Project: Teiid
Issue Type: Feature Request
Components: Query Engine
Reporter: Howard Abrams
Assignee: Steven Hawkins
Fix For: 7.4
The default of allowing an unlimited number of concurrent source queries is not desirable
in extreme situations. A limit should be added to prevent contention or other problems
with issuing a large number of source queries.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
4717
days inactive
4883
days old
7 comments
2 participants
participants (2)
-
Howard Abrams (JIRA) -
Steven Hawkins (JIRA)