[JBoss JIRA] (TEIID-2301) Programatically invalidate a single execution's cache using cache API
by Mark Addleman (JIRA)
[ https://issues.jboss.org/browse/TEIID-2301?page=com.atlassian.jira.plugin... ]
Mark Addleman commented on TEIID-2301:
--------------------------------------
I was thinking that Teiid's caching API could, somehow, return an opaque key to the cache that the execution could later use to invalidate. I'm not sure how to graft this onto the existing API, though. Perhaps a new API at the execution rather than the execution factory?
In my mind, the primary use case for this is ReusableExecutions which will likely have some asynchronous behavior. The execution could provide a result set that is cached at any scope. Through some means, it becomes aware that new data is available, so it invalidates the data is Teiid's cache.
I'm pretty sure that every one of our use cases that might take advantage of a cache invalidation feature could be implemented as a materialized view with a mechanism to call the stored procedure used to refresh the materialized view so I consider this feature a low priority.
> Programatically invalidate a single execution's cache using cache API
> ---------------------------------------------------------------------
>
> Key: TEIID-2301
> URL: https://issues.jboss.org/browse/TEIID-2301
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Mark Addleman
> Assignee: Steven Hawkins
>
> Using the execution cache API, I want to invalidate a single execution's cache entry.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years
[JBoss JIRA] (TEIID-2139) Factor out common table expressions
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2139?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-2139:
----------------------------------
Summary: Factor out common table expressions (was: CLONE - Factor out common table expressions)
Complexity: Medium
Forum Reference: https://community.jboss.org/message/742355
The request scope caching for commands was added as part of TEIID-2077, which somewhat addresses the initial issue discussed in the forum posting.
The other variations to this are:
1. suppress pushdown so that multiple table accesses can be grouped
2. caching beyond the request scope - transaction/session
3. automatically convert subplans to common table expressions for reuse (somewhat related to 1), which is the thrust of this issue.
The approach to 3) would likely be to start with multiple view/table access and replace the plan nodes with common table access - this would create a local temporary table, but perhaps would be better served with something more like the cached tuple source logic.
2) would need to be hint based and if based upon tuple buffers could backend into the common rs caching logic - otherwise I'd have to look at modifying the session temp table logic.
> Factor out common table expressions
> -----------------------------------
>
> Key: TEIID-2139
> URL: https://issues.jboss.org/browse/TEIID-2139
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Mark Addleman
> Assignee: Steven Hawkins
> Fix For: 8.3
>
>
> Query planner should factor out common table expressions within queries so they are only executed once. See https://community.jboss.org/message/742355 for more context.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years
[JBoss JIRA] (TEIID-2169) Buffer Manger's memory should be backed by Infinispan Cache
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2169?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2169:
---------------------------------------
Based upon https://community.jboss.org/thread/178102 and several INSP issues about memory pressure eviction, this is not something that we can currently do or even address in the near term. The eviction strategy is not customizable and the eviction queue is not accessible (the Eviction is not accessible from a Segment and the queue is not accessible from the Eviction).
The core issue here is whether the LFRU imposes too much overhead vs. LIRS. We should do some performance tests on hit ratios and overhead to see if we need to push this feature into INSP.
Ideally the entire strategy could be implemented with INSP, however they have already rejected the use of NIO managed direct memory with the possible inclusion of JNI accessed current maps from intel c libraries. There again it could be evaluated whether adapting Teiid's existing use of direct memory (which isn't quite complete as keys are still held on heap and would need to be adapted for chained eviction) could be beneficial.
> Buffer Manger's memory should be backed by Infinispan Cache
> -----------------------------------------------------------
>
> Key: TEIID-2169
> URL: https://issues.jboss.org/browse/TEIID-2169
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine, Server
> Reporter: Ramesh Reddy
> Assignee: Steven Hawkins
>
> Currently the buffer manager uses the memory from the local VM's heap for processing the intermediate query results, anything that overruns is saved on the disk. Using this model so far proven very efficient.
> However, when it comes to using the data grid as unlimited temp backing store where the results can be shared with other nodes in the case of fail-over of the session, this in current model does not work. By using the infinispan's local cache in standalone and replicated cache in distributed node for the buffer manager these issues can resolved.
> Issues may be, currently Buffer Manager uses efficient btree based index store, also the uses file storage in block. To make it work, Teiid needs to have access to the Infinispan's eviction queue and may also needs to implement the cache store.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years
[JBoss JIRA] (TEIID-2328) Imported vdb external mat views fail subsequent validation
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2328?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-2328.
-----------------------------------
Resolution: Done
Most of the changes for the fix were to expand testablity to the Embedded/FakeServer. The crux of the issue was in MetadataValidator to check if the materialization table was already in a resolved form. Also added general checks to the MetadataValidator to skip validation of imported schemas, since it is unnecessary to revalidate.
> Imported vdb external mat views fail subsequent validation
> ----------------------------------------------------------
>
> Key: TEIID-2328
> URL: https://issues.jboss.org/browse/TEIID-2328
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.0
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 8.3
>
>
> The metadatavalidator is setting the materialization table to a resolved form, which removes the schema from the table name and thus fails to validate a second time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years
[JBoss JIRA] (TEIID-2328) Imported vdb external mat views fail subsequent validation
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-2328:
-------------------------------------
Summary: Imported vdb external mat views fail subsequent validation
Key: TEIID-2328
URL: https://issues.jboss.org/browse/TEIID-2328
Project: Teiid
Issue Type: Bug
Components: Query Engine
Affects Versions: 8.0
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 8.3
The metadatavalidator is setting the materialization table to a resolved form, which removes the schema from the table name and thus fails to validate a second time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years
[JBoss JIRA] (TEIID-2311) Add simple row based security to data roles
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2311?page=com.atlassian.jira.plugin... ]
Steven Hawkins edited comment on TEIID-2311 at 12/17/12 3:32 PM:
-----------------------------------------------------------------
Updated the release notes as well. I expect that we'll refine this feature based upon feedback. Some other possible pieces that could still be worked before this is fully realized are allowing for the policy decider or other pluggable interface to supply the role conditions and finer control over the affect of the condition (constraint only vs. filter only, etc.).
was (Author: shawkins):
Updated the release notes as well. I expect that we'll refine this feature based upon feedback. Some other possible pieces that could still be worked before this is resolved are allowing for the policy decider or other pluggable interface to supply the role conditions and finer control over the affect of the condition (constraint only vs. filter only, etc.).
> Add simple row based security to data roles
> -------------------------------------------
>
> Key: TEIID-2311
> URL: https://issues.jboss.org/browse/TEIID-2311
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine
> Affects Versions: 8.2
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 8.3
>
>
> A common request is to implement row based security. The common workaround of modifying transformations is generally not a good solution.
> We should look at adding support for simple table filters and column masks.
> To be effective, filtering permissions however would have to act differently than normal data roles. They would need to be applied all the time - and not just against the end user queries.
> For example, for tables:
> <permission>
> <resource-name>SCHEMA.TABLE</resource-name>
> <filter>COLUMNA=2</filter>
> </permission>
> Meaning allow the CRUD of the given row only if COLUMNA has the value of 2. Any valid predicate against just the referenced table would be allowed as a filter. Each such permission would be applied as an additional predicate any time the table is referenced (in views, inserts, updates, deletes, etc.).
> Allows would not be specified here as we want the filter to always specify inclusion. Any applicable permissions in additional roles would be applied disjunctively - filter OR filter.
> We could possibly support column masks via case expressions, such as:
> <permission>
> <resource-name>SCHEMA.TABLE.COLUMN</resource-name>
> <mask>CASE WHEN ...</mask>
> </permission>
> However this is slightly more complicated. Presumably the mask would only apply to projection and makes more sense to be applied at the final output/user query (more like a data role).
> If we work the issue to specify the object type of a permission, then the name could alternatively refer to datatype or even an extension property to make the masking a little easier.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years
[JBoss JIRA] (TEIID-2311) Add simple row based security to data roles
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2311?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-2311.
-----------------------------------
Resolution: Done
Updated the release notes as well. I expect that we'll refine this feature based upon feedback. Some other possible pieces that could still be worked before this is resolved are allowing for the policy decider or other pluggable interface to supply the role conditions and finer control over the affect of the condition (constraint only vs. filter only, etc.).
> Add simple row based security to data roles
> -------------------------------------------
>
> Key: TEIID-2311
> URL: https://issues.jboss.org/browse/TEIID-2311
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine
> Affects Versions: 8.2
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 8.3
>
>
> A common request is to implement row based security. The common workaround of modifying transformations is generally not a good solution.
> We should look at adding support for simple table filters and column masks.
> To be effective, filtering permissions however would have to act differently than normal data roles. They would need to be applied all the time - and not just against the end user queries.
> For example, for tables:
> <permission>
> <resource-name>SCHEMA.TABLE</resource-name>
> <filter>COLUMNA=2</filter>
> </permission>
> Meaning allow the CRUD of the given row only if COLUMNA has the value of 2. Any valid predicate against just the referenced table would be allowed as a filter. Each such permission would be applied as an additional predicate any time the table is referenced (in views, inserts, updates, deletes, etc.).
> Allows would not be specified here as we want the filter to always specify inclusion. Any applicable permissions in additional roles would be applied disjunctively - filter OR filter.
> We could possibly support column masks via case expressions, such as:
> <permission>
> <resource-name>SCHEMA.TABLE.COLUMN</resource-name>
> <mask>CASE WHEN ...</mask>
> </permission>
> However this is slightly more complicated. Presumably the mask would only apply to projection and makes more sense to be applied at the final output/user query (more like a data role).
> If we work the issue to specify the object type of a permission, then the name could alternatively refer to datatype or even an extension property to make the masking a little easier.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years
[JBoss JIRA] (TEIID-2327) Add column masking
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-2327:
-------------------------------------
Summary: Add column masking
Key: TEIID-2327
URL: https://issues.jboss.org/browse/TEIID-2327
Project: Teiid
Issue Type: Sub-task
Components: Query Engine
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Support for column masking should be added to data roles. A typical approach would be to only affect the final projected values from user queries and have only one mask applicable at a time.
A possibly related ability would be to hide columns (much like hidden tables) to our metadata.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years