[JBoss JIRA] (TEIID-219) Add a way to limit or suppress queries that exceed a threshold Cost-Based Optimizer value
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-219?page=com.atlassian.jira.plugin.... ]
Steven Hawkins commented on TEIID-219:
--------------------------------------
We do now have active restrictions on memory / disk, but not proactive restrictions.
Possible Dimensions:
- cpu
- memory (max footprint / total usage)
- network (from a source, all sources, to client)
- wall clock time - active restriction based upon timeouts
With the current logic we are best suited to estimate memory / network based upon row counts. There again we have an active, but not proactive restrictions, on source row counts.
We don't have anything meaningful for cpu utilization as we don't have a model of consumption down to the expression level. We will likely need to introduce a very simple metric.
I don't see us making any meaningful estimate of wall clock time.
> Add a way to limit or suppress queries that exceed a threshold Cost-Based Optimizer value
> -----------------------------------------------------------------------------------------
>
> Key: TEIID-219
> URL: https://issues.jboss.org/browse/TEIID-219
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine, Server
> Affects Versions: 6.0.0
> Reporter: Marc Shirley
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 12.0
>
>
> Client requesting an ability to impose a limit on, or suppress, any query that exceeds a cost estimate value specified by the user/administrator.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 6 months
[JBoss JIRA] (TEIID-5563) Separate the Teiid WildFly distribution into their own repo or module
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5563?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5563:
---------------------------------------
> But you being one who does all the builds I do understand the pain of more work, so, I am fine whatever we decide in the end
The overriding consideration is can this be done through maven modules alone - especially initially. If yes, then is using a separate repo overkill.
> Separate lifecycles is something we should consider IMO. Otherwise, we need to set a precedent and call about teiid-thorntail and teiid-spring-boot and how to handle those with a single build.
The primary release driver is Teiid itself. When will we release a Teiid 13.1 for example and not the corresponding runtime support? In my mind this would only occur if/when we have truly fundamental divergence - core Teiid is at JRE X and runtime R is at JRE Y. Otherwise I would see a release across all runtimes.
What you are probably more thinking of is likely the scenario that a fix is applicable to only runtime T, then can we get that out "faster", correct?
> Deemphasize WildFly as main distribution, set its course if we want to do regular updates or not. I understand that is we said for 13 and beyond.
See above. Unless there is a fundamental conflict, I do not see a scenario where it makes sense to release a Teiid 13.1 for example and not do it across all supported runtimes. Can you define what you mean by irregular updates? How that would benefit us? Or are you proposing a further restructuring of the distribution - such that there could be some kind of core teiid overlay on base kit of wildfly + teiid-wildfly integration?
> Gives a better option for the community to be at set Teiid version but update the resource-adapters or WildFly version. We can sync these releases more with WF releases?
Maybe that's an extension of what I'm asking above - are you proposing new kititng / overlay / patching logic? If so we need to understand what is being proposed.
> I also would like core Teiid to move to basepom after the changes. Clear separation of modules.
> I see we will be divergent in support of "data sources" based on distribution flavour. I would not be obligated, say if I came up with source support in SB not WildFly or vice versa.
Neither of these necessitate a new repo.
> Separate the Teiid WildFly distribution into their own repo or module
> ---------------------------------------------------------------------
>
> Key: TEIID-5563
> URL: https://issues.jboss.org/browse/TEIID-5563
> Project: Teiid
> Issue Type: Task
> Components: Build/Kits, Integration Tests
> Affects Versions: 12.x
> Reporter: Van Halbert
> Assignee: Van Halbert
> Priority: Major
>
> With the different types of distributions now being built based on Teiid like
> * WildFly
> * Thorntail
> * SpringBoot
> * Embedded
> Having main `github.com/teiid` repo contain artifacts for WildFly distribution poses an unnecessary dependency burden on other distributions. For example, the resource adapters are mainly for use with WildFly and Thorntail V2. They can still be applicable to embedded and SpringBoot as either non-managed connection factories, or require the user to provide appropriate management / pooling. When they are pulled into environments like SpringBoot they will bring several wildfly dependencies / and JEE constructs that are unnecessary - specifically the dependencies jboss-jaxws-api (teiid-api), javax.activation (teiid-core), jboss-transaction-api (teiid-api), and jboss-connector-api (teiid-api).
> This proposal is to separate "teiid" core modules along with all translators into one repository and move all the resource adapters along with any feature-packs and module building into a sperate repository like "teiid-wildfly". With the goal of providing similar connectivity in SpringBoot using as much common logic from SpringBoot (Spring Data in particular) and even Fuse as possible.
> Basically, when finished, the core Teiid modules only provide Maven artifacts along with their clean pom.xml with for direct and maybe transitive dependencies. None of the WildFly based modules support will be here. At worst this may introduce duplicate dependencies into the WildFly environment as we will need to manage all core / translator dependencies independently from those provided by WildFly. Where possible the teiid-api / core modules will need to use the generic javax dependency - or even remove the dependency if possible.
> The "teiid-wildfly" module on the other hand, will exactly preserve what is in the current repo as far feature packs and module building and overall WildFly distribution building.
> A follow-on task we can task we can look into is, converting the main teiid then to use basepom rather than the jboss-parent pom. However, the recommendation is this should be done after, not simultaneously.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 6 months
[JBoss JIRA] (TEIID-5563) Separate the Teiid WildFly distribution into their own repo or module
by Ramesh Reddy (Jira)
[ https://issues.jboss.org/browse/TEIID-5563?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-5563:
-------------------------------------
[~shawkins] My thinking on separate repository as follows
1. Separate lifecycles is something we should consider IMO. Otherwise, we need to set a precedent and call about teiid-thorntail and teiid-spring-boot and how to handle those with a single build.
2. Deemphasize WildFly as main distribution, set its course if we want to do regular updates or not. I understand that is we said for 13 and beyond.
3. Gives a better option for the community to be at set Teiid version but update the resource-adapters or WildFly version. We can sync these releases more with WF releases?
4. I also would like core Teiid to move to basepom after the changes. Clear separation of modules.
5. I see we will be divergent in support of "data sources" based on distribution flavour. I would not be obligated, say if I came up with source support in SB not WildFly or vice versa.
But you being one who does all the builds I do understand the pain of more work, so, I am fine whatever we decide in the end :)
> Separate the Teiid WildFly distribution into their own repo or module
> ---------------------------------------------------------------------
>
> Key: TEIID-5563
> URL: https://issues.jboss.org/browse/TEIID-5563
> Project: Teiid
> Issue Type: Task
> Components: Build/Kits, Integration Tests
> Affects Versions: 12.x
> Reporter: Van Halbert
> Assignee: Van Halbert
> Priority: Major
>
> With the different types of distributions now being built based on Teiid like
> * WildFly
> * Thorntail
> * SpringBoot
> * Embedded
> Having main `github.com/teiid` repo contain artifacts for WildFly distribution poses an unnecessary dependency burden on other distributions. For example, the resource adapters are mainly for use with WildFly and Thorntail V2. They can still be applicable to embedded and SpringBoot as either non-managed connection factories, or require the user to provide appropriate management / pooling. When they are pulled into environments like SpringBoot they will bring several wildfly dependencies / and JEE constructs that are unnecessary - specifically the dependencies jboss-jaxws-api (teiid-api), javax.activation (teiid-core), jboss-transaction-api (teiid-api), and jboss-connector-api (teiid-api).
> This proposal is to separate "teiid" core modules along with all translators into one repository and move all the resource adapters along with any feature-packs and module building into a sperate repository like "teiid-wildfly". With the goal of providing similar connectivity in SpringBoot using as much common logic from SpringBoot (Spring Data in particular) and even Fuse as possible.
> Basically, when finished, the core Teiid modules only provide Maven artifacts along with their clean pom.xml with for direct and maybe transitive dependencies. None of the WildFly based modules support will be here. At worst this may introduce duplicate dependencies into the WildFly environment as we will need to manage all core / translator dependencies independently from those provided by WildFly. Where possible the teiid-api / core modules will need to use the generic javax dependency - or even remove the dependency if possible.
> The "teiid-wildfly" module on the other hand, will exactly preserve what is in the current repo as far feature packs and module building and overall WildFly distribution building.
> A follow-on task we can task we can look into is, converting the main teiid then to use basepom rather than the jboss-parent pom. However, the recommendation is this should be done after, not simultaneously.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 6 months
[JBoss JIRA] (TEIID-5563) Separate the Teiid WildFly distribution into their own repo or module
by Van Halbert (Jira)
[ https://issues.jboss.org/browse/TEIID-5563?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-5563:
------------------------------------
Then lets talk about this before I continue this path of resolving all these dependency issues.
> Separate the Teiid WildFly distribution into their own repo or module
> ---------------------------------------------------------------------
>
> Key: TEIID-5563
> URL: https://issues.jboss.org/browse/TEIID-5563
> Project: Teiid
> Issue Type: Task
> Components: Build/Kits, Integration Tests
> Affects Versions: 12.x
> Reporter: Van Halbert
> Assignee: Van Halbert
> Priority: Major
>
> With the different types of distributions now being built based on Teiid like
> * WildFly
> * Thorntail
> * SpringBoot
> * Embedded
> Having main `github.com/teiid` repo contain artifacts for WildFly distribution poses an unnecessary dependency burden on other distributions. For example, the resource adapters are mainly for use with WildFly and Thorntail V2. They can still be applicable to embedded and SpringBoot as either non-managed connection factories, or require the user to provide appropriate management / pooling. When they are pulled into environments like SpringBoot they will bring several wildfly dependencies / and JEE constructs that are unnecessary - specifically the dependencies jboss-jaxws-api (teiid-api), javax.activation (teiid-core), jboss-transaction-api (teiid-api), and jboss-connector-api (teiid-api).
> This proposal is to separate "teiid" core modules along with all translators into one repository and move all the resource adapters along with any feature-packs and module building into a sperate repository like "teiid-wildfly". With the goal of providing similar connectivity in SpringBoot using as much common logic from SpringBoot (Spring Data in particular) and even Fuse as possible.
> Basically, when finished, the core Teiid modules only provide Maven artifacts along with their clean pom.xml with for direct and maybe transitive dependencies. None of the WildFly based modules support will be here. At worst this may introduce duplicate dependencies into the WildFly environment as we will need to manage all core / translator dependencies independently from those provided by WildFly. Where possible the teiid-api / core modules will need to use the generic javax dependency - or even remove the dependency if possible.
> The "teiid-wildfly" module on the other hand, will exactly preserve what is in the current repo as far feature packs and module building and overall WildFly distribution building.
> A follow-on task we can task we can look into is, converting the main teiid then to use basepom rather than the jboss-parent pom. However, the recommendation is this should be done after, not simultaneously.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 6 months
[JBoss JIRA] (TEIID-4498) Generalize subquery batch processing
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-4498?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4498:
---------------------------------------
After some experimentation, it's possible to just nest the dependent set criteria when re plan the subquery. So I should have this by tomorrow at least for the simple case. The more complicated case of multiple rows will take some more work. This will introduce a new rule so that the subquery logic is consolidated and moved out of rule merge criteria.
> Generalize subquery batch processing
> ------------------------------------
>
> Key: TEIID-4498
> URL: https://issues.jboss.org/browse/TEIID-4498
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 12.0
>
>
> For subqueries that cannot be unnested, but can be modified to project their correlation, we should have at least batch processing rather than the row by row default.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 6 months
[JBoss JIRA] (TEIID-5563) Separate the Teiid WildFly distribution into their own repo or module
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5563?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5563:
---------------------------------------
> Does "client" need to be moved to teiid-wildfly, it has a dependency on org.jboss.modules in the ModuleHelper.
That can be refined to jboss-client maven module. The kitted client jar doesn't need the modules logic.
Thinking about things some more, I'm not sure about using a separate repository at this point. It seems to needlessly complicate the release process. If on every teiid release we will release a version of the wildfly integration, it would be much simpler to have this all under one repository. As is, I'd likely link to the new repo as a git submodule. If and only if the projects have different lifecycles does a separate repo / versioning seem necessary.
> Separate the Teiid WildFly distribution into their own repo or module
> ---------------------------------------------------------------------
>
> Key: TEIID-5563
> URL: https://issues.jboss.org/browse/TEIID-5563
> Project: Teiid
> Issue Type: Task
> Components: Build/Kits, Integration Tests
> Affects Versions: 12.x
> Reporter: Van Halbert
> Assignee: Van Halbert
> Priority: Major
>
> With the different types of distributions now being built based on Teiid like
> * WildFly
> * Thorntail
> * SpringBoot
> * Embedded
> Having main `github.com/teiid` repo contain artifacts for WildFly distribution poses an unnecessary dependency burden on other distributions. For example, the resource adapters are mainly for use with WildFly and Thorntail V2. They can still be applicable to embedded and SpringBoot as either non-managed connection factories, or require the user to provide appropriate management / pooling. When they are pulled into environments like SpringBoot they will bring several wildfly dependencies / and JEE constructs that are unnecessary - specifically the dependencies jboss-jaxws-api (teiid-api), javax.activation (teiid-core), jboss-transaction-api (teiid-api), and jboss-connector-api (teiid-api).
> This proposal is to separate "teiid" core modules along with all translators into one repository and move all the resource adapters along with any feature-packs and module building into a sperate repository like "teiid-wildfly". With the goal of providing similar connectivity in SpringBoot using as much common logic from SpringBoot (Spring Data in particular) and even Fuse as possible.
> Basically, when finished, the core Teiid modules only provide Maven artifacts along with their clean pom.xml with for direct and maybe transitive dependencies. None of the WildFly based modules support will be here. At worst this may introduce duplicate dependencies into the WildFly environment as we will need to manage all core / translator dependencies independently from those provided by WildFly. Where possible the teiid-api / core modules will need to use the generic javax dependency - or even remove the dependency if possible.
> The "teiid-wildfly" module on the other hand, will exactly preserve what is in the current repo as far feature packs and module building and overall WildFly distribution building.
> A follow-on task we can task we can look into is, converting the main teiid then to use basepom rather than the jboss-parent pom. However, the recommendation is this should be done after, not simultaneously.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 6 months
[JBoss JIRA] (TEIID-5563) Separate the Teiid WildFly distribution into their own repo or module
by Ramesh Reddy (Jira)
[ https://issues.jboss.org/browse/TEIID-5563?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-5563:
-------------------------------------
No, client should not be moved to teiid-wildfly but we need to find an alternative way to resolve that or move only the affected code etc. Leave dependency as is for now, either [~shawkins] and myself need to redo some refactoring there to get rid of the modules dependency altogether, it is optional even now.
BTW, eclipselink-platform is like Hibernate should be aligned with core Teiid IMO. Overall I think it looks pretty good :)
I suspect more build type changes on Teiid Core side.
> Separate the Teiid WildFly distribution into their own repo or module
> ---------------------------------------------------------------------
>
> Key: TEIID-5563
> URL: https://issues.jboss.org/browse/TEIID-5563
> Project: Teiid
> Issue Type: Task
> Components: Build/Kits, Integration Tests
> Affects Versions: 12.x
> Reporter: Van Halbert
> Assignee: Van Halbert
> Priority: Major
>
> With the different types of distributions now being built based on Teiid like
> * WildFly
> * Thorntail
> * SpringBoot
> * Embedded
> Having main `github.com/teiid` repo contain artifacts for WildFly distribution poses an unnecessary dependency burden on other distributions. For example, the resource adapters are mainly for use with WildFly and Thorntail V2. They can still be applicable to embedded and SpringBoot as either non-managed connection factories, or require the user to provide appropriate management / pooling. When they are pulled into environments like SpringBoot they will bring several wildfly dependencies / and JEE constructs that are unnecessary - specifically the dependencies jboss-jaxws-api (teiid-api), javax.activation (teiid-core), jboss-transaction-api (teiid-api), and jboss-connector-api (teiid-api).
> This proposal is to separate "teiid" core modules along with all translators into one repository and move all the resource adapters along with any feature-packs and module building into a sperate repository like "teiid-wildfly". With the goal of providing similar connectivity in SpringBoot using as much common logic from SpringBoot (Spring Data in particular) and even Fuse as possible.
> Basically, when finished, the core Teiid modules only provide Maven artifacts along with their clean pom.xml with for direct and maybe transitive dependencies. None of the WildFly based modules support will be here. At worst this may introduce duplicate dependencies into the WildFly environment as we will need to manage all core / translator dependencies independently from those provided by WildFly. Where possible the teiid-api / core modules will need to use the generic javax dependency - or even remove the dependency if possible.
> The "teiid-wildfly" module on the other hand, will exactly preserve what is in the current repo as far feature packs and module building and overall WildFly distribution building.
> A follow-on task we can task we can look into is, converting the main teiid then to use basepom rather than the jboss-parent pom. However, the recommendation is this should be done after, not simultaneously.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 6 months
[JBoss JIRA] (TEIID-5563) Separate the Teiid WildFly distribution into their own repo or module
by Van Halbert (Jira)
[ https://issues.jboss.org/browse/TEIID-5563?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-5563:
------------------------------------
Does "client" need to be moved to teiid-wildfly, it has a dependency on org.jboss.modules in the ModuleHelper.
> Separate the Teiid WildFly distribution into their own repo or module
> ---------------------------------------------------------------------
>
> Key: TEIID-5563
> URL: https://issues.jboss.org/browse/TEIID-5563
> Project: Teiid
> Issue Type: Task
> Components: Build/Kits, Integration Tests
> Affects Versions: 12.x
> Reporter: Van Halbert
> Assignee: Van Halbert
> Priority: Major
>
> With the different types of distributions now being built based on Teiid like
> * WildFly
> * Thorntail
> * SpringBoot
> * Embedded
> Having main `github.com/teiid` repo contain artifacts for WildFly distribution poses an unnecessary dependency burden on other distributions. For example, the resource adapters are mainly for use with WildFly and Thorntail V2. They can still be applicable to embedded and SpringBoot as either non-managed connection factories, or require the user to provide appropriate management / pooling. When they are pulled into environments like SpringBoot they will bring several wildfly dependencies / and JEE constructs that are unnecessary - specifically the dependencies jboss-jaxws-api (teiid-api), javax.activation (teiid-core), jboss-transaction-api (teiid-api), and jboss-connector-api (teiid-api).
> This proposal is to separate "teiid" core modules along with all translators into one repository and move all the resource adapters along with any feature-packs and module building into a sperate repository like "teiid-wildfly". With the goal of providing similar connectivity in SpringBoot using as much common logic from SpringBoot (Spring Data in particular) and even Fuse as possible.
> Basically, when finished, the core Teiid modules only provide Maven artifacts along with their clean pom.xml with for direct and maybe transitive dependencies. None of the WildFly based modules support will be here. At worst this may introduce duplicate dependencies into the WildFly environment as we will need to manage all core / translator dependencies independently from those provided by WildFly. Where possible the teiid-api / core modules will need to use the generic javax dependency - or even remove the dependency if possible.
> The "teiid-wildfly" module on the other hand, will exactly preserve what is in the current repo as far feature packs and module building and overall WildFly distribution building.
> A follow-on task we can task we can look into is, converting the main teiid then to use basepom rather than the jboss-parent pom. However, the recommendation is this should be done after, not simultaneously.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 6 months
[JBoss JIRA] (TEIID-5563) Separate the Teiid WildFly distribution into their own repo or module
by Van Halbert (Jira)
[ https://issues.jboss.org/browse/TEIID-5563?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-5563:
------------------------------------
Now I'm working on the changes to teiid/teiid.git.
> Separate the Teiid WildFly distribution into their own repo or module
> ---------------------------------------------------------------------
>
> Key: TEIID-5563
> URL: https://issues.jboss.org/browse/TEIID-5563
> Project: Teiid
> Issue Type: Task
> Components: Build/Kits, Integration Tests
> Affects Versions: 12.x
> Reporter: Van Halbert
> Assignee: Van Halbert
> Priority: Major
>
> With the different types of distributions now being built based on Teiid like
> * WildFly
> * Thorntail
> * SpringBoot
> * Embedded
> Having main `github.com/teiid` repo contain artifacts for WildFly distribution poses an unnecessary dependency burden on other distributions. For example, the resource adapters are mainly for use with WildFly and Thorntail V2. They can still be applicable to embedded and SpringBoot as either non-managed connection factories, or require the user to provide appropriate management / pooling. When they are pulled into environments like SpringBoot they will bring several wildfly dependencies / and JEE constructs that are unnecessary - specifically the dependencies jboss-jaxws-api (teiid-api), javax.activation (teiid-core), jboss-transaction-api (teiid-api), and jboss-connector-api (teiid-api).
> This proposal is to separate "teiid" core modules along with all translators into one repository and move all the resource adapters along with any feature-packs and module building into a sperate repository like "teiid-wildfly". With the goal of providing similar connectivity in SpringBoot using as much common logic from SpringBoot (Spring Data in particular) and even Fuse as possible.
> Basically, when finished, the core Teiid modules only provide Maven artifacts along with their clean pom.xml with for direct and maybe transitive dependencies. None of the WildFly based modules support will be here. At worst this may introduce duplicate dependencies into the WildFly environment as we will need to manage all core / translator dependencies independently from those provided by WildFly. Where possible the teiid-api / core modules will need to use the generic javax dependency - or even remove the dependency if possible.
> The "teiid-wildfly" module on the other hand, will exactly preserve what is in the current repo as far feature packs and module building and overall WildFly distribution building.
> A follow-on task we can task we can look into is, converting the main teiid then to use basepom rather than the jboss-parent pom. However, the recommendation is this should be done after, not simultaneously.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 6 months