[teiid-issues] [JBoss JIRA] (TEIID-2879) Add more Resource Management

[Steven Hawkins (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12950694#comment-12950694 ] 

Steven Hawkins commented on TEIID-2879:
---------------------------------------

There is already some minimal logic to throttle cpu, which is done via the timeslicing property on the engine (if processing exceeds the timeslice, then the plan will be requeued for processing).  However this doesn't provide any hard guarantees and isn't based upon any attributes of the query. 

A problem we have with any throttling scheme performed on an active basis is that there are other resources that can be held by the plan - source connections and memory (the heap footprint of an inactive plan is typically manageable, but intermediate data spilled to disk can be significant).  We have tracking logic already to see what source connections are held by a plan, but we don't track buffer resources to that level (only down to the connection).  So without additional tracking logic to be more reactive, we'd have to apply throttling before the plan got started (similar to our max active plans setting).

Active heap throttling is somewhat manageable with the recommendation from the admin guide:

"In systems where large intermediate results are normal (scrolling cursors or sorting over millions of rows) you can consider increasing the max-processing-kb and decreasing the max-reserve-kb so that each request has access to an effectively smaller buffer space."

That is tell Teiid to use a more per plan allocation scheme rather than using a more shared memory pool.
                
> Add more Resource Management
> ----------------------------
>
>                 Key: TEIID-2879
>                 URL: https://issues.jboss.org/browse/TEIID-2879
>             Project: Teiid
>          Issue Type: Feature Request
>            Reporter: Rick Wagner
>            Assignee: Steven Hawkins
>
> To protect the system sometimes we may need to throttle the queries to ensure it does not exhaust all the existing resources on the server. Resource management is the key to play this role of monitoring and throttling queries. Oracle products offer such a feature for managing the consumption of resources. We have quite a number of (Teiid) profiles running on a big box and we want to ensure no silly queires are hogging on to all the resources and consequently causing issue to others.
> Ability to throttle the query by restricting query that uses
> 1) Too much CPU
> 2) Too much java heap
> 3) Long running query

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira