[JBoss JIRA] (TEIID-4738) Change JDG Materialization to use Upsert
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4738?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4738:
---------------------------------------
> That only UPSERT will be used for JDG cache sources or any other map cache that uses object translator.
As long as you are comfortable with that breaking change - both for the object model, and for the teiid metadata - then yes.
> Change JDG Materialization to use Upsert
> ----------------------------------------
>
> Key: TEIID-4738
> URL: https://issues.jboss.org/browse/TEIID-4738
> Project: Teiid
> Issue Type: Enhancement
> Components: JDG Connector
> Reporter: Van Halbert
> Assignee: Van Halbert
> Fix For: 9.3
>
>
> Change materialization to use the new Upsert feature.
> This will enable the removing of the Alias and Staging caches.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
[JBoss JIRA] (TEIID-4738) Change JDG Materialization to use Upsert
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-4738?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-4738:
------------------------------------
I just want to ask before I continue to make these massive changes are made, that moving to use UPSERT, will mean that there will no longer be support materialization using multiple caches. That only UPSERT will be used for JDG cache sources or any other map cache that uses object translator.
> Change JDG Materialization to use Upsert
> ----------------------------------------
>
> Key: TEIID-4738
> URL: https://issues.jboss.org/browse/TEIID-4738
> Project: Teiid
> Issue Type: Enhancement
> Components: JDG Connector
> Reporter: Van Halbert
> Assignee: Van Halbert
> Fix For: 9.3
>
>
> Change materialization to use the new Upsert feature.
> This will enable the removing of the Alias and Staging caches.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
[JBoss JIRA] (TEIID-4552) Missing support for connection to Facebook via OAuth 2
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4552?page=com.atlassian.jira.plugin... ]
Ramesh Reddy updated TEIID-4552:
--------------------------------
Fix Version/s: 9.3
(was: 9.2)
> Missing support for connection to Facebook via OAuth 2
> ------------------------------------------------------
>
> Key: TEIID-4552
> URL: https://issues.jboss.org/browse/TEIID-4552
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 8.12.5
> Reporter: Lucie Fabrikova
> Assignee: Kylin Soong
> Fix For: 9.3
>
> Attachments: facebook3-vdb.xml, query, server.log, standalone.xml, teiid-oauth-util.sh-output
>
>
> I would like to connect to facebook resource adapter and configure OAuth 2 security domain for it; I used teiid-oauth-util.sh to generate the security domain (see attachment). Facebook doesn't support refresh tokens and in the generated code is refresh token's value "null".
> If I try to execute VDB with source pointing to such resource adapter I get error:
> ERROR [org.teiid.CONNECTOR] (Worker0_QueryProcessorQueue13) Connector worker process failed for atomic-request=Gapuea1/NRcn.6.3.0: org.apache.cxf.rs.security.oauth2.provider.OAuthServiceException
>
> (see server.log, facebook-vdb.xml, standalone.xml)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
[JBoss JIRA] (TEIID-4627) High-level event handling
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4627?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-4627.
-----------------------------------
Resolution: Done
Marking the initial work as complete. From here we'll open new issues as needed.
> High-level event handling
> -------------------------
>
> Key: TEIID-4627
> URL: https://issues.jboss.org/browse/TEIID-4627
> Project: Teiid
> Issue Type: Enhancement
> Components: Grammar, Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 9.2
>
>
> We should look into high-level plugability of event handling at a row level. This could even be handled at the language level with new trigger semantics for source tables: create trigger on tbl on cdc (insert|update...) - which could potentially even specify the topic mapping other details. There is complexity for multi-source models here though.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
[JBoss JIRA] (TEIID-4745) Avoid loading the same file over again when using 'org.jboss.teiid.resource-adapter.file'
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4745?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4745:
---------------------------------------
In general this concern is being addressed by the change data capture feature - TEIID-4627 / TEIID-4526
An issue is that the file translator does not fit neatly into the paradigm as it's exposing files, not row level change events.
> Avoid loading the same file over again when using 'org.jboss.teiid.resource-adapter.file'
> -----------------------------------------------------------------------------------------
>
> Key: TEIID-4745
> URL: https://issues.jboss.org/browse/TEIID-4745
> Project: Teiid
> Issue Type: Enhancement
> Environment: * WildFly 10
> * Teiid Server 9.1.2
> * CentOs 7
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
>
> When using the file resource adapter _org.jboss.teiid.resource-adapter.file_, it should be possible that the system per se, by the means of a new property in the resource-adapter, or other option in the view definition, or other configuration elsewhere, to not to unnecessary load a file that is already loaded. Theoretically it should be possible, for example, to check the file date or the file checksum, to see if the file is already loaded by _Teiid_. If it is, then there is no need to parse the file again.
> This is important due to, specially, performance improvements. If there are a lot of files to be processed, and these files are big (200 MB or more), the system will be processing over and over again the same files if they are not updated for quite some time.
> If, for example, the user uses a Materialized View, then the view will be constantly being updated with the same data, which brings another performance penalty.
> *Note*: This could be extended to other resource-adapters with some sort of last data date for example, if the user whant to use this sort of mechanism. For example, if using a database like MySQL as source, then tthe user could define a last update table, and the system could do a check first against this table before tryng to load the data.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
[JBoss JIRA] (TEIID-4734) Under some circumstances after restart Teiid Server does not update Materialized views anymore
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4734?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4734:
---------------------------------------
> My comment was meant for the mvstatus based spin solution.
That was meant for an approach that would work with 9.2 and prior other than just relying on manual intervention.
> Under some circumstances after restart Teiid Server does not update Materialized views anymore
> ----------------------------------------------------------------------------------------------
>
> Key: TEIID-4734
> URL: https://issues.jboss.org/browse/TEIID-4734
> Project: Teiid
> Issue Type: Bug
> Components: Common
> Affects Versions: 9.1.2
> Environment: * Wildfly 10
> * MySQL 5.6.32
> * Teiid Server 9.1.2
> Reporter: Pedro Inácio
> Assignee: Ramesh Reddy
> Fix For: 9.3
>
> Attachments: StatusTableAfterStart.csv, StatusTableAfterStop.csv
>
>
> Under some circumstances, start and stop the server several times at different points, some Materialized Views are not being updated by the server.
> It is observable that in status table some views are no longer updated. In server log are being printed lots of INFO lines.
> Waited more than 15 minutes and the server was not capable of recovering.
> See the _StatusTableAfterStop.csv_ file to observe the status table content after stopping the server.
> See the _StatusTableAfterStart.csv_ file to observe the status table content after approximately 15 minutes after restart.
> Check _server.log_ for the Teiid Server log.
> See that hundreds of lines similar to the following occur
> {panel:title=Server.log}
> 2017-01-31 13:06:34,597 INFO [org.teiid.COMMAND_LOG] (Teiid Timer0) JfGY9cInAbrb START USER COMMAND: startTime=2017-01-31 13:06:34.597 requestID=JfGY9cInAbrb.0 txID=null sessionID=JfGY9cInAbrb applicationName=internal principal=embedded-async vdbName=CountryServiceListVDB vdbVersion=1 sql=execute SYSADMIN.matViewStatus('NumberingPlan', 'numbering_plan')
> {panel}
> The server cannot continue to load materialized views.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
[JBoss JIRA] (TEIID-4734) Under some circumstances after restart Teiid Server does not update Materialized views anymore
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4734?page=com.atlassian.jira.plugin... ]
Ramesh Reddy edited comment on TEIID-4734 at 2/13/17 12:27 PM:
---------------------------------------------------------------
My comment was meant for the mvstatus based spin solution. On the additional columns, let's go through the rest of the materialization issues in 9.3, then we can see what makes sense. I will get this into 9.3, update the documentation.
was (Author: rareddy):
My comment was meant for the mvstatus based spin solution. On the additional columns, let's go through the rest of the materialization issues in 9.3, then we can see what makes sense.
> Under some circumstances after restart Teiid Server does not update Materialized views anymore
> ----------------------------------------------------------------------------------------------
>
> Key: TEIID-4734
> URL: https://issues.jboss.org/browse/TEIID-4734
> Project: Teiid
> Issue Type: Bug
> Components: Common
> Affects Versions: 9.1.2
> Environment: * Wildfly 10
> * MySQL 5.6.32
> * Teiid Server 9.1.2
> Reporter: Pedro Inácio
> Assignee: Ramesh Reddy
> Fix For: 9.3
>
> Attachments: StatusTableAfterStart.csv, StatusTableAfterStop.csv
>
>
> Under some circumstances, start and stop the server several times at different points, some Materialized Views are not being updated by the server.
> It is observable that in status table some views are no longer updated. In server log are being printed lots of INFO lines.
> Waited more than 15 minutes and the server was not capable of recovering.
> See the _StatusTableAfterStop.csv_ file to observe the status table content after stopping the server.
> See the _StatusTableAfterStart.csv_ file to observe the status table content after approximately 15 minutes after restart.
> Check _server.log_ for the Teiid Server log.
> See that hundreds of lines similar to the following occur
> {panel:title=Server.log}
> 2017-01-31 13:06:34,597 INFO [org.teiid.COMMAND_LOG] (Teiid Timer0) JfGY9cInAbrb START USER COMMAND: startTime=2017-01-31 13:06:34.597 requestID=JfGY9cInAbrb.0 txID=null sessionID=JfGY9cInAbrb applicationName=internal principal=embedded-async vdbName=CountryServiceListVDB vdbVersion=1 sql=execute SYSADMIN.matViewStatus('NumberingPlan', 'numbering_plan')
> {panel}
> The server cannot continue to load materialized views.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
[JBoss JIRA] (TEIID-4734) Under some circumstances after restart Teiid Server does not update Materialized views anymore
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4734?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-4734:
-------------------------------------
My comment was meant for the mvstatus based spin solution. On the additional columns, let's go through the rest of the materialization issues in 9.3, then we can see what makes sense.
> Under some circumstances after restart Teiid Server does not update Materialized views anymore
> ----------------------------------------------------------------------------------------------
>
> Key: TEIID-4734
> URL: https://issues.jboss.org/browse/TEIID-4734
> Project: Teiid
> Issue Type: Bug
> Components: Common
> Affects Versions: 9.1.2
> Environment: * Wildfly 10
> * MySQL 5.6.32
> * Teiid Server 9.1.2
> Reporter: Pedro Inácio
> Assignee: Ramesh Reddy
> Fix For: 9.3
>
> Attachments: StatusTableAfterStart.csv, StatusTableAfterStop.csv
>
>
> Under some circumstances, start and stop the server several times at different points, some Materialized Views are not being updated by the server.
> It is observable that in status table some views are no longer updated. In server log are being printed lots of INFO lines.
> Waited more than 15 minutes and the server was not capable of recovering.
> See the _StatusTableAfterStop.csv_ file to observe the status table content after stopping the server.
> See the _StatusTableAfterStart.csv_ file to observe the status table content after approximately 15 minutes after restart.
> Check _server.log_ for the Teiid Server log.
> See that hundreds of lines similar to the following occur
> {panel:title=Server.log}
> 2017-01-31 13:06:34,597 INFO [org.teiid.COMMAND_LOG] (Teiid Timer0) JfGY9cInAbrb START USER COMMAND: startTime=2017-01-31 13:06:34.597 requestID=JfGY9cInAbrb.0 txID=null sessionID=JfGY9cInAbrb applicationName=internal principal=embedded-async vdbName=CountryServiceListVDB vdbVersion=1 sql=execute SYSADMIN.matViewStatus('NumberingPlan', 'numbering_plan')
> {panel}
> The server cannot continue to load materialized views.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month
[JBoss JIRA] (TEIID-4734) Under some circumstances after restart Teiid Server does not update Materialized views anymore
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4734?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4734:
---------------------------------------
> I would rather we stick to above solution rather than another one which is involves best guesses.
I'm not sure I follow you. Do you mean you don't want to yet consider other changes to the status table for 9.3? If so I'm fine with that. I just want to make sure we're thinking about future proofing at some point.
> Under some circumstances after restart Teiid Server does not update Materialized views anymore
> ----------------------------------------------------------------------------------------------
>
> Key: TEIID-4734
> URL: https://issues.jboss.org/browse/TEIID-4734
> Project: Teiid
> Issue Type: Bug
> Components: Common
> Affects Versions: 9.1.2
> Environment: * Wildfly 10
> * MySQL 5.6.32
> * Teiid Server 9.1.2
> Reporter: Pedro Inácio
> Assignee: Ramesh Reddy
> Fix For: 9.3
>
> Attachments: StatusTableAfterStart.csv, StatusTableAfterStop.csv
>
>
> Under some circumstances, start and stop the server several times at different points, some Materialized Views are not being updated by the server.
> It is observable that in status table some views are no longer updated. In server log are being printed lots of INFO lines.
> Waited more than 15 minutes and the server was not capable of recovering.
> See the _StatusTableAfterStop.csv_ file to observe the status table content after stopping the server.
> See the _StatusTableAfterStart.csv_ file to observe the status table content after approximately 15 minutes after restart.
> Check _server.log_ for the Teiid Server log.
> See that hundreds of lines similar to the following occur
> {panel:title=Server.log}
> 2017-01-31 13:06:34,597 INFO [org.teiid.COMMAND_LOG] (Teiid Timer0) JfGY9cInAbrb START USER COMMAND: startTime=2017-01-31 13:06:34.597 requestID=JfGY9cInAbrb.0 txID=null sessionID=JfGY9cInAbrb applicationName=internal principal=embedded-async vdbName=CountryServiceListVDB vdbVersion=1 sql=execute SYSADMIN.matViewStatus('NumberingPlan', 'numbering_plan')
> {panel}
> The server cannot continue to load materialized views.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 1 month