[JBoss JIRA] (TEIID-4745) Avoid loading the same file over again when using 'org.jboss.teiid.resource-adapter.file'
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4745?page=com.atlassian.jira.plugin... ]
Work on TEIID-4745 started by Steven Hawkins.
---------------------------------------------
> Avoid loading the same file over again when using 'org.jboss.teiid.resource-adapter.file'
> -----------------------------------------------------------------------------------------
>
> Key: TEIID-4745
> URL: https://issues.jboss.org/browse/TEIID-4745
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Environment: * WildFly 10
> * Teiid Server 9.1.2
> * CentOs 7
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
> Fix For: 10.3
>
>
> When using the file resource adapter _org.jboss.teiid.resource-adapter.file_, it should be possible that the system per se, by the means of a new property in the resource-adapter, or other option in the view definition, or other configuration elsewhere, to not to unnecessary load a file that is already loaded. Theoretically it should be possible, for example, to check the file date or the file checksum, to see if the file is already loaded by _Teiid_. If it is, then there is no need to parse the file again.
> This is important due to, specially, performance improvements. If there are a lot of files to be processed, and these files are big (200 MB or more), the system will be processing over and over again the same files if they are not updated for quite some time.
> If, for example, the user uses a Materialized View, then the view will be constantly being updated with the same data, which brings another performance penalty.
> *Note*: This could be extended to other resource-adapters with some sort of last data date for example, if the user whant to use this sort of mechanism. For example, if using a database like MySQL as source, then tthe user could define a last update table, and the system could do a check first against this table before tryng to load the data.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (TEIID-5285) Add high-level feature for redirection of updates
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-5285?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-5285:
-------------------------------------
Since the user is not really after any virtualization aspects, IMO this needs to be transparent. Sure the assumption is the usage of Teiid SQL is mandated. In order to be transparent, we need to generate either Entities or VDB or both with Spring Boot. I feel, Annotation based approach will not work best here as it involves work from the developer and needs to have Teiid knowledge, we need to remove that obstacle. Generating VDB will remove that. Now after generation of the VDB, we can still make Spring Boot as the execution environment if needs to be embedded with the application, or if we take sidecar approach then we can use spring-boot or swarm. Either case the user would need to generate the Entities for their domain model.
I think we should consider Annotation based code generation, only if we are going to give the developers a starting point to modify the View-based entities.
I was thinking about sidecar approach to enable this for supporting more languages, but we can progress from one approach to another. Yes, when sidecar deploys it is in the same Pod as a different container which is the same host.
> Add high-level feature for redirection of updates
> -------------------------------------------------
>
> Key: TEIID-5285
> URL: https://issues.jboss.org/browse/TEIID-5285
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine, Teiid Spring Boot
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 10.3
>
>
> In microservices testing it is desirable to test against production/live data but not commit any updates. We should offer a simple solution that can defined by extension metadata and enabled/disabled by a feature flag. We may need to make simplifying assumptions about the scope (per session, per application, etc.) and durability of the updates.
> Under the covers this will be achieved by using views, update triggers, and a store for the updates and when not enabled the expectation is that all operations should pass through. However the application will be limited to using Teiid SQL and will be required to use the Teiid or pg driver, or Teiid spring boot.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (TEIID-5285) Add high-level feature for redirection of updates
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5285?page=com.atlassian.jira.plugin... ]
Steven Hawkins edited comment on TEIID-5285 at 4/12/18 3:55 PM:
----------------------------------------------------------------
> Now we need to write a utility to generate VDB file, given the Main DB and Update DB details. Including any resulting DDL for Update DB.
Are you thinking like a code generator approach? I had assumed that this would based upon Spring Boot annotations and within the context of that vdb.
> If we take the sidecar approach with this, which I think is appropriate, as it will give a black box experience to the user
Our initial though was that since this makes the client dependent upon Teiid sql (and the Teiid or pg client), it's not really something that can be too transparent. So the initial target was just around Spring Boot integration.
As a black box / side car are you envisioning that Teiid is running on the same host and you are deploying these kind of VDBs into it?
was (Author: shawkins):
> Now we need to write a utility to generate VDB file, given the Main DB and Update DB details. Including any resulting DDL for Update DB.
> If we take the sidecar approach with this, which I think is appropriate, as it will give a black box experience to the user
Our initial though was that since this makes the client dependent upon Teiid sql (and the Teiid or pg client), it's not really something that can be too transparent. So the initial target was just around Spring Boot integration.
As a black box / side car are you envisioning that Teiid is running on the same host and you are deploying these kind of VDBs into it?
> Add high-level feature for redirection of updates
> -------------------------------------------------
>
> Key: TEIID-5285
> URL: https://issues.jboss.org/browse/TEIID-5285
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine, Teiid Spring Boot
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 10.3
>
>
> In microservices testing it is desirable to test against production/live data but not commit any updates. We should offer a simple solution that can defined by extension metadata and enabled/disabled by a feature flag. We may need to make simplifying assumptions about the scope (per session, per application, etc.) and durability of the updates.
> Under the covers this will be achieved by using views, update triggers, and a store for the updates and when not enabled the expectation is that all operations should pass through. However the application will be limited to using Teiid SQL and will be required to use the Teiid or pg driver, or Teiid spring boot.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (TEIID-5314) usage of environment properties
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-5314:
-------------------------------------
Summary: usage of environment properties
Key: TEIID-5314
URL: https://issues.jboss.org/browse/TEIID-5314
Project: Teiid
Issue Type: Enhancement
Components: Server
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 10.3
Places in Teiid that look for system properties should allow for overrides to be specified via environment properties as well. This will make it easier to configure optional behavior on openshift.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (TEIID-5285) Add high-level feature for redirection of updates
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5285?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5285:
---------------------------------------
> Now we need to write a utility to generate VDB file, given the Main DB and Update DB details. Including any resulting DDL for Update DB.
> If we take the sidecar approach with this, which I think is appropriate, as it will give a black box experience to the user
Our initial though was that since this makes the client dependent upon Teiid sql (and the Teiid or pg client), it's not really something that can be too transparent. So the initial target was just around Spring Boot integration.
As a black box / side car are you envisioning that Teiid is running on the same host and you are deploying these kind of VDBs into it?
> Add high-level feature for redirection of updates
> -------------------------------------------------
>
> Key: TEIID-5285
> URL: https://issues.jboss.org/browse/TEIID-5285
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine, Teiid Spring Boot
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 10.3
>
>
> In microservices testing it is desirable to test against production/live data but not commit any updates. We should offer a simple solution that can defined by extension metadata and enabled/disabled by a feature flag. We may need to make simplifying assumptions about the scope (per session, per application, etc.) and durability of the updates.
> Under the covers this will be achieved by using views, update triggers, and a store for the updates and when not enabled the expectation is that all operations should pass through. However the application will be limited to using Teiid SQL and will be required to use the Teiid or pg driver, or Teiid spring boot.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (TEIID-5312) NullPointerException thrown when the second time login via GSS API
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5312?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-5312.
-----------------------------------
Fix Version/s: 10.1.3
10.2.1
Resolution: Done
Based upon code review the fix will address this issue. However I wasn't able to fully get my local environment setup - I was modifying https://github.com/criteo/kerberos-docker to utilize teiid at the client and server. I'll update this issue if there is anything that comes of that.
> NullPointerException thrown when the second time login via GSS API
> ------------------------------------------------------------------
>
> Key: TEIID-5312
> URL: https://issues.jboss.org/browse/TEIID-5312
> Project: Teiid
> Issue Type: Bug
> Components: Server
> Affects Versions: 8.12.12.6_3
> Reporter: Yuming Zhu
> Assignee: Steven Hawkins
> Fix For: 10.3, 10.1.3, 10.2.1
>
>
> The error was thrown when trying to login at the second time
> ODBC:
> {code}
> 09 Apr 2018 09:14:39,941 ERROR [org.teiid.ODBC] (New I/O worker #31) TEIID40015 Unexpected error occurred: java.lang.NullPointerException
> at org.teiid.jboss.JBossSecurityHelper.buildGSSResult(JBossSecurityHelper.java:211) [teiid-jboss-integration-8.12.12.6_3-redhat-1.jar:8.12.12.6_3-redhat-1]
> at org.teiid.jboss.JBossSecurityHelper.negotiateGssLogin(JBossSecurityHelper.java:186) [teiid-jboss-integration-8.12.12.6_3-redhat-1.jar:8.12.12.6_3-redhat-1]
> at org.teiid.services.SessionServiceImpl.neogitiateGssLogin(SessionServiceImpl.java:560) [teiid-runtime-8.12.12.6_3-redhat-1.jar:8.12.12.6_3-redhat-1]
> at org.teiid.transport.LogonImpl.neogitiateGssLogin(LogonImpl.java:207) [teiid-runtime-8.12.12.6_3-redhat-1.jar:8.12.12.6_3-redhat-1]
> at org.teiid.odbc.ODBCServerRemoteImpl.logon(ODBCServerRemoteImpl.java:249) [teiid-runtime-8.12.12.6_3-redhat-1.jar:8.12.12.6_3-redhat-1]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_161]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_161]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_161]
> at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_161]
> at org.teiid.transport.ODBCClientInstance.processMessage(ODBCClientInstance.java:127) [teiid-runtime-8.12.12.6_3-redhat-1.jar:8.12.12.6_3-redhat-1]
> at org.teiid.transport.ODBCClientInstance.receivedMessage(ODBCClientInstance.java:116) [teiid-runtime-8.12.12.6_3-redhat-1.jar:8.12.12.6_3-redhat-1]
> at org.teiid.transport.SSLAwareChannelHandler.messageReceived(SSLAwareChannelHandler.java:216) [teiid-runtime-8.12.12.6_3-redhat-1.jar:8.12.12.6_3-redhat-1]
> at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:328) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_161]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161]
> {code}
> JDBC:
> {code}
> 2018-04-09 09:43:06,333 INFO [org.teiid.SECURITY] (New I/O worker #13:) TEIID40017 Unexpected exception for session null: java.lang.NullPointerException
> at org.teiid.jboss.JBossSecurityHelper.buildGSSResult(JBossSecurityHelper.java:211)
> at org.teiid.jboss.JBossSecurityHelper.negotiateGssLogin(JBossSecurityHelper.java:186)
> at org.teiid.services.SessionServiceImpl.neogitiateGssLogin(SessionServiceImpl.java:560)
> at org.teiid.transport.LogonImpl.neogitiateGssLogin(LogonImpl.java:207)
> at org.teiid.transport.LogonImpl.neogitiateGssLogin(LogonImpl.java:181)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.teiid.transport.ServerWorkItem.run(ServerWorkItem.java:87)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:284)
> at org.teiid.transport.SocketClientInstance.processMessagePacket(SocketClientInstance.java:231)
> at org.teiid.transport.SocketClientInstance.receivedMessage(SocketClientInstance.java:217)
> at org.teiid.transport.SSLAwareChannelHandler.messageReceived(SSLAwareChannelHandler.java:216)
> at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:142)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:310)
> at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
> at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:328)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> security-domain config:
> {code:xml}
> <security-domain name="REDHAT.COM" cache-type="default">
> <authentication>
> <login-module code="SPNEGO" flag="requisite">
> <module-option name="password-stacking" value="useFirstPass" />
> <module-option name="serverSecurityDomain" value="host" />
> <module-option name="removeRealmFromPrincipal" value="true"/>
> </login-module>
> <login-module code="LdapExtended" flag="optional">
> <module-option name="password-stacking" value="useFirstPass"/>
> <module-option name="java.naming.factory.initial" value="com.sun.jndi.ldap.LdapCtxFactory"/>
> <module-option name="java.naming.provider.url" value="ldap://ldap.corp.redhat.com:389/"/>
> <module-option name="java.naming.security.authentication" value="none"/>
> <module-option name="baseCtxDN" value="ou=Users,dc=redhat,dc=com"/>
> <module-option name="baseFilter" value="(&(objectClass=person)(uid={0}))"/>
> <module-option name="rolesCtxDN" value="ou=Groups,dc=redhat,dc=com"/>
> <module-option name="roleFilter" value="(&(objectClass=posixGroup)(memberUid={0}))"/>
> <module-option name="roleAttributeID" value="cn"/>
> <module-option name="searchScope" value="ONELEVEL_SCOPE"/>
> </login-module>
> <login-module code="RoleMapping" flag="optional">
> <module-option name="password-stacking" value="useFirstPass"/>
> <module-option name="rolesProperties" value="file:/opt/jboss-eap/standalone/configuration/redhat-mapping-roles.properties"/>
> </login-module>
> <login-module code="UsersRoles" flag="required">
> <module-option name="password-stacking" value="useFirstPass" />
> <module-option name="usersProperties" value="file:/opt/jboss-eap/standalone/configuration/redhat-users.properties" />
> <module-option name="rolesProperties" value="file:/opt/jboss-eap/standalone/configuration/redhat-roles.properties" />
> </login-module>
> </authentication>
> </security-domain>
> <security-domain name="host" cache-type="default">
> <authentication>
> <login-module code="Kerberos" flag="required">
> <module-option name="storeKey" value="true" />
> <module-option name="useKeyTab" value="true" />
> <module-option name="principal" value="postgres/teiid.host.dev.eng.pek2.redhat.com(a)REDHAT.COM" />
> <module-option name="keyTab" value="/opt/jboss-eap/standalone/configuration/postgres.keytab" />
> <module-option name="doNotPrompt" value="true" />
> </login-module>
> </authentication>
> </security-domain>
> {code}
> transport:
> {code:xml}
> <transport name="jdbc-gssapi" socket-binding="teiid-jdbc-gssapi" protocol="teiid">
> <authentication security-domain="REDHAT.COM" type="GSS" />
> <ssl mode="enabled" ssl-protocol="TLSv1" keymanagement-algorithm="SunX509">
> <keystore name="/opt/jboss-eap/standalone/jboss.keystore" password="xxxxxx" key-alias="teiid.host.dev.eng.pek2.redhat.com"/>
> </ssl>
> </transport>
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (TEIID-5285) Add high-level feature for redirection of updates
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5285?page=com.atlassian.jira.plugin... ]
Steven Hawkins edited comment on TEIID-5285 at 4/12/18 12:27 PM:
-----------------------------------------------------------------
Here is a proposal to redirect updates (with no later resolution/merge to the underlying source) down to row level. Given tbl with a primary key, we need to create tbl_modifications (jdg, pg?) that contains the same columns (and potentially indexes) with an additional state column. State could be one of three values (created, deleted, updated), which could just be numeric values as we don't have an enum concept.
A view will be created as the new access point for tbl:
{code}
SELECT * from tbl left outer join tbl_modifications on (pk predicates) where tbl_modifications.state is null
union all
SELECT col, ... from tbl_modifications where state <> deleted
{code}
An issue with the above is if a created row end is also then populated in the actual store. We can leave it as a TODO to resolve that.
Performance could be an issue as any joins or operations performed above the view would likely need to have federated processing as the optimizer would not generally be able to remove the union or the left outer join.
The update triggers on the view would look like:
{code}
create trigger on ... instead of insert
begin atomic
//TODO: may need to do an check against tbl
upsert into tbl_modifications (col, ..., state) values (new.col, ..., created);
end
{code}
{code}
create trigger on ... instead of delete
begin atomic
upsert into tbl_modifications (col, ..., state) values (null, ..., deleted);
end
{code}
An enhancement for delete is to differentiate between rows that were created and those that were updated - the former can be deleted completely. However the update procedure would need further refinement as it transitions rows from created to updated.
{code}
create trigger on ... instead of update
begin atomic
if (changing.pk ...)
begin
upsert into tbl_modifications (pk cols) values (old.pk, .., deleted);
//TODO: we probably need to check tbl for existence of the new pk
upsert into tbl_modifications (col, ..., state) values (new.col, .., created);
end
else
upsert into tbl_modifications (col, ..., state) values (new.col, .., updated);
end
{code}
[~rareddy] does something based upon the above seem workable?
was (Author: shawkins):
Here is a proposal to redirect updates (with no later resolution/merge to the underlying source) down to row level. Given tbl with a primary key, we need to create tbl_modifications (jdg, pg?) that contains the same columns (and potentially indexes) with an additional state column. State could be one of three values (created, deleted, updated), which could just be numeric values as we don't have an enum concept.
A view will be created as the new access point for tbl:
{code}
SELECT * from tbl left outer join tbl_modifications on (pk predicates) where tbl_modifications.state is null
union all
SELECT col, ... from tbl_modifications where state <> deleted
{code}
An issue with the above is if a created row end is also then populated in the actual store. We can leave it as a TODO to resolve that.
Performance could be an issue as any joins or operations performed above the view would likely need to have federated processing as the optimizer would not generally be able to remove the union or the left outer join.
The update triggers on the view would look like:
{code}
create trigger on ... instead of insert
begin atomic
merge into tbl_modifications (col, ..., state) values (new.col, ..., created);
end
{code}
{code}
create trigger on ... instead of delete
begin atomic
merge into tbl_modifications (col, ..., state) values (null, ..., deleted);
end
{code}
An enhancement for delete is to differentiate between rows that were created and those that were updated - the former can be deleted completely.
It would be easy to have:
{code}
create trigger on ... instead of update
begin atomic
merge into tbl_modifications (col, ..., state) values (new.col, .., updated);
end
{code}
However that update is too simplistic as a modification of the primary key would need to be treated as delete and a create.
[~rareddy] does something based upon the above seem workable?
> Add high-level feature for redirection of updates
> -------------------------------------------------
>
> Key: TEIID-5285
> URL: https://issues.jboss.org/browse/TEIID-5285
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine, Teiid Spring Boot
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 10.3
>
>
> In microservices testing it is desirable to test against production/live data but not commit any updates. We should offer a simple solution that can defined by extension metadata and enabled/disabled by a feature flag. We may need to make simplifying assumptions about the scope (per session, per application, etc.) and durability of the updates.
> Under the covers this will be achieved by using views, update triggers, and a store for the updates and when not enabled the expectation is that all operations should pass through. However the application will be limited to using Teiid SQL and will be required to use the Teiid or pg driver, or Teiid spring boot.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months
[JBoss JIRA] (TEIID-5285) Add high-level feature for redirection of updates
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-5285?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-5285:
-------------------------------------
Yes, that looks like a good start. The assumption should be placed that the Main DB's records are not being altered simultaneously by any other app.
Now we need to write a utility to generate VDB file, given the Main DB and Update DB details. Including any resulting DDL for Update DB.
If we take the sidecar approach with this, which I think is appropriate, as it will give a black box experience to the user, and will support different programming environments, we need to generate an Appliance either using Spring Boot or WildFly-Swarm that given the configuration about databases will configure rest automagically and expose a JDBC and ODBC port.
Packaging this into OpenShift, I feel there are a few more knobs to make it look seamless.
> Add high-level feature for redirection of updates
> -------------------------------------------------
>
> Key: TEIID-5285
> URL: https://issues.jboss.org/browse/TEIID-5285
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine, Teiid Spring Boot
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 10.3
>
>
> In microservices testing it is desirable to test against production/live data but not commit any updates. We should offer a simple solution that can defined by extension metadata and enabled/disabled by a feature flag. We may need to make simplifying assumptions about the scope (per session, per application, etc.) and durability of the updates.
> Under the covers this will be achieved by using views, update triggers, and a store for the updates and when not enabled the expectation is that all operations should pass through. However the application will be limited to using Teiid SQL and will be required to use the Teiid or pg driver, or Teiid spring boot.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 11 months