[JBoss JIRA] (TEIIDSB-147) Caching and materialization changes for JDG / OpenShift
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-147?focusedWorklogId=12450111&pa... ]
Steven Hawkins logged work on TEIIDSB-147:
------------------------------------------
Author: Steven Hawkins
Created on: 14/Feb/20 8:39 AM
Start Date: 14/Feb/20 8:39 AM
Worklog Time Spent: 2 hours
Issue Time Tracking
-------------------
Remaining Estimate: 0 minutes
Time Spent: 2 hours
Worklog Id: (was: 12450111)
> Caching and materialization changes for JDG / OpenShift
> -------------------------------------------------------
>
> Key: TEIIDSB-147
> URL: https://issues.redhat.com/browse/TEIIDSB-147
> Project: Teiid Spring Boot
> Issue Type: Enhancement
> Components: core, OpenShift
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Critical
> Fix For: 1.4.0
>
> Time Spent: 2 hours
> Remaining Estimate: 0 minutes
>
> Started this as a higher level issue in Teiid Spring Boot rather than core Teiid since the work won't align well to Teiid 13.1, but some things may need addressed there as well.
> Things break down roughly into:
> * Single-pod support
> ** Define the expectations for how we'll be configured to hit a JDG instance - will it be a user exercise to create if it doesn't exist, will we need to create, will the be a crd strategy for creation, and will it be multi-tenet? There is also the issue / assumption of cache sharing based upon vdb name - is that sufficient for now?
> ** validate external materialization to JDG likely using a transactional upsert strategy to avoid issues like TEIID-5436
> * Multi-pod support
> ** everything from single-pod support
> ** Result set caching needs to directly write batches / results to JDG rather than relying upon on demand replication.
> ** requires replacing the logic that detect failed materialization loads
> ** internal materialization replacement needs to based upon something more like external materialization, but that can be seen as a next step that is turnkey internal materialization to JDG (little to no user setup required)
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 1 month
[JBoss JIRA] (TEIIDSB-147) Caching and materialization changes for JDG / OpenShift
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-147?page=com.atlassian.jira.plug... ]
Steven Hawkins updated TEIIDSB-147:
-----------------------------------
Story Points: 2 (was: 8)
I'm not sure of the tracking approach, but changing this to just 2 story points instead of the original 8 - which is now reflected in the work that went into the individual issues. So that this can be resolved earlier, I'd vote for splitting the multi-pod concerns off to a separate enhancement.
> Caching and materialization changes for JDG / OpenShift
> -------------------------------------------------------
>
> Key: TEIIDSB-147
> URL: https://issues.redhat.com/browse/TEIIDSB-147
> Project: Teiid Spring Boot
> Issue Type: Enhancement
> Components: core, OpenShift
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Critical
> Fix For: 1.4.0
>
>
> Started this as a higher level issue in Teiid Spring Boot rather than core Teiid since the work won't align well to Teiid 13.1, but some things may need addressed there as well.
> Things break down roughly into:
> * Single-pod support
> ** Define the expectations for how we'll be configured to hit a JDG instance - will it be a user exercise to create if it doesn't exist, will we need to create, will the be a crd strategy for creation, and will it be multi-tenet? There is also the issue / assumption of cache sharing based upon vdb name - is that sufficient for now?
> ** validate external materialization to JDG likely using a transactional upsert strategy to avoid issues like TEIID-5436
> * Multi-pod support
> ** everything from single-pod support
> ** Result set caching needs to directly write batches / results to JDG rather than relying upon on demand replication.
> ** requires replacing the logic that detect failed materialization loads
> ** internal materialization replacement needs to based upon something more like external materialization, but that can be seen as a next step that is turnkey internal materialization to JDG (little to no user setup required)
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 1 month
[JBoss JIRA] (TEIIDSB-154) Create an example of materialization with JDG on openshift
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-154?focusedWorklogId=12450110&pa... ]
Steven Hawkins logged work on TEIIDSB-154:
------------------------------------------
Author: Steven Hawkins
Created on: 14/Feb/20 8:36 AM
Start Date: 14/Feb/20 8:36 AM
Worklog Time Spent: 5 hours
Issue Time Tracking
-------------------
Remaining Estimate: 6 hours (was: 1 day, 3 hours)
Time Spent: 1 day, 1 hour (was: 4 hours)
Worklog Id: (was: 12450110)
> Create an example of materialization with JDG on openshift
> ----------------------------------------------------------
>
> Key: TEIIDSB-154
> URL: https://issues.redhat.com/browse/TEIIDSB-154
> Project: Teiid Spring Boot
> Issue Type: Sub-task
> Components: documentation, examples
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 1.4.0
>
> Original Estimate: 1 day, 7 hours
> Time Spent: 1 day, 1 hour
> Remaining Estimate: 6 hours
>
> Create an example showing materialization with JDG on openshift.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 1 month
[JBoss JIRA] (TEIIDSB-154) Create an example of materialization with JDG on openshift
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-154?page=com.atlassian.jira.plug... ]
Steven Hawkins commented on TEIIDSB-154:
----------------------------------------
Added a pr with an example showing materialization to infinispan using a transactional cache for both the target and the status table. This can be refined to better highlight the performance benefit and/or cache update/refresh features.
> Create an example of materialization with JDG on openshift
> ----------------------------------------------------------
>
> Key: TEIIDSB-154
> URL: https://issues.redhat.com/browse/TEIIDSB-154
> Project: Teiid Spring Boot
> Issue Type: Sub-task
> Components: documentation, examples
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 1.4.0
>
> Original Estimate: 1 day, 7 hours
> Time Spent: 4 hours
> Remaining Estimate: 1 day, 3 hours
>
> Create an example showing materialization with JDG on openshift.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 1 month
[JBoss JIRA] (TEIID-5904) Issues with infinispan bulk operations
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5904?page=com.atlassian.jira.plugi... ]
Steven Hawkins updated TEIID-5904:
----------------------------------
Original Estimate: 2 hours
Remaining Estimate: 2 hours
> Issues with infinispan bulk operations
> --------------------------------------
>
> Key: TEIID-5904
> URL: https://issues.redhat.com/browse/TEIID-5904
> Project: Teiid
> Issue Type: Bug
> Components: Infinispan
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 13.1, 12.3.2, 13.0.3
>
> Original Estimate: 2 hours
> Remaining Estimate: 2 hours
>
> The infinispan batch size logic is off by 1 and it only returns a single update count, but does not report that in the capabilities - which means that only a single row is reported as updated.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 1 month
[JBoss JIRA] (TEIID-5904) Issues with infinispan bulk operations
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5904?page=com.atlassian.jira.plugi... ]
Steven Hawkins resolved TEIID-5904.
-----------------------------------
Assignee: Steven Hawkins (was: Ramesh Reddy)
Resolution: Done
Made the minor changes and added a bulk test.
> Issues with infinispan bulk operations
> --------------------------------------
>
> Key: TEIID-5904
> URL: https://issues.redhat.com/browse/TEIID-5904
> Project: Teiid
> Issue Type: Bug
> Components: Infinispan
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 13.1, 12.3.2, 13.0.3
>
>
> The infinispan batch size logic is off by 1 and it only returns a single update count, but does not report that in the capabilities - which means that only a single row is reported as updated.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 1 month
[JBoss JIRA] (TEIID-5904) Issues with infinispan bulk operations
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5904?focusedWorklogId=12450109&pag... ]
Steven Hawkins logged work on TEIID-5904:
-----------------------------------------
Author: Steven Hawkins
Created on: 14/Feb/20 8:32 AM
Start Date: 14/Feb/20 8:32 AM
Worklog Time Spent: 2 hours
Issue Time Tracking
-------------------
Remaining Estimate: 0 minutes (was: 2 hours)
Time Spent: 2 hours
Worklog Id: (was: 12450109)
> Issues with infinispan bulk operations
> --------------------------------------
>
> Key: TEIID-5904
> URL: https://issues.redhat.com/browse/TEIID-5904
> Project: Teiid
> Issue Type: Bug
> Components: Infinispan
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 13.1, 12.3.2, 13.0.3
>
> Original Estimate: 2 hours
> Time Spent: 2 hours
> Remaining Estimate: 0 minutes
>
> The infinispan batch size logic is off by 1 and it only returns a single update count, but does not report that in the capabilities - which means that only a single row is reported as updated.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 1 month
[JBoss JIRA] (TEIID-5904) Issues with infinispan bulk operations
by Steven Hawkins (Jira)
Steven Hawkins created TEIID-5904:
-------------------------------------
Summary: Issues with infinispan bulk operations
Key: TEIID-5904
URL: https://issues.redhat.com/browse/TEIID-5904
Project: Teiid
Issue Type: Bug
Components: Infinispan
Reporter: Steven Hawkins
Assignee: Ramesh Reddy
Fix For: 13.1, 12.3.2, 13.0.3
The infinispan batch size logic is off by 1 and it only returns a single update count, but does not report that in the capabilities - which means that only a single row is reported as updated.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 1 month
[JBoss JIRA] (TEIID-5903) Performance problems
by Renat Eskenin (Jira)
[ https://issues.redhat.com/browse/TEIID-5903?page=com.atlassian.jira.plugi... ]
Renat Eskenin commented on TEIID-5903:
--------------------------------------
We need to build by maven one fat jar with wildfly+teiid+connection pool for running in docker.
> Performance problems
> --------------------
>
> Key: TEIID-5903
> URL: https://issues.redhat.com/browse/TEIID-5903
> Project: Teiid
> Issue Type: Enhancement
> Components: Salesforce Connector
> Affects Versions: 13.0.2
> Reporter: Renat Eskenin
> Assignee: Steven Hawkins
> Priority: Major
>
> If we call SOQL
> {code:sql}
> SELECT Account.BillingCountry__c, Account.Name from Contact LIMIT 1
> {code}
> We have time in salesforce workbench (UI for SOAP): Returned records 1 - 1 of 1 total record in 0.947 seconds:
> If we call SQL over teiid spring salesforce example:
> {code:sql}
> SELECT Account.BillingCountry__c, Account.Name from Contact LEFT OUTER JOIN /*+ MAKEDEP */ Account
> on Contact.AccountId = Account.id
> LIMIT 1
> {code}
> we have request time: Time from sf:3.561s
> How we can optimize request time?
> We used
> {code}
> @Autowired
> private DataSource sfds;
> @Bean
> @Primary
> public NamedParameterJdbcTemplate getSfNamedParameterJdbcTemplate() {
> return new NamedParameterJdbcTemplate(sfds);
> }
> {code}
> And
> {code}
> @Autowired
> private NamedParameterJdbcTemplate sftemplate;
> long start = System.currentTimeMillis();
> SqlRowSet rowSet = jdbcTemplate.queryForRowSet(sql, params);
> log.info("Time from {}:{}", sourceName, (((double) (System.currentTimeMillis() - start)) / 1000.0d));
> {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 1 month