[JBoss JIRA] Created: (TEIIDDES-715) Oracle metadata import using Designer takes long time for large schemas
9:41 p.m.
Oracle metadata import using Designer takes long time for large schemas
-----------------------------------------------------------------------
Key: TEIIDDES-715
URL: https://jira.jboss.org/browse/TEIIDDES-715
Project: Teiid Designer
Issue Type: Bug
Components: Import/Export
Affects Versions: 7.1
Reporter: Ramesh Reddy
Importing metadata from Oracle database seems to take longer than any other database, when
the underlying the schema is large. This may be because the default settings with Oracle
Driver tries to get lot more metadata then needed. Should explore ways to minimize the
time to take import.
Teiid in the Dynamic VDB import uses "Table Types" and "schema" name
to limit the number of tables it imports, may be we should adopt similar thing in
Designer. It is possible that Designer is already doing such settings.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:54 p.m.
New subject: [JBoss JIRA] Updated: (TEIIDDES-715) Oracle metadata import using Designer takes long time for large schemas
[
https://jira.jboss.org/browse/TEIIDDES-715?page=com.atlassian.jira.plugin...
]
Barry LaFond updated TEIIDDES-715:
----------------------------------
Fix Version/s: 7.3
Oracle metadata import using Designer takes long time for large
schemas
-----------------------------------------------------------------------
Key: TEIIDDES-715
URL: https://jira.jboss.org/browse/TEIIDDES-715
Project: Teiid Designer
Issue Type: Bug
Components: Import/Export
Affects Versions: 7.1
Reporter: Ramesh Reddy
Fix For: 7.3
Importing metadata from Oracle database seems to take longer than any other database,
when the underlying the schema is large. This may be because the default settings with
Oracle Driver tries to get lot more metadata then needed. Should explore ways to minimize
the time to take import.
Teiid in the Dynamic VDB import uses "Table Types" and "schema" name
to limit the number of tables it imports, may be we should adopt similar thing in
Designer. It is possible that Designer is already doing such settings.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:01 p.m.
New subject: [JBoss JIRA] Assigned: (TEIIDDES-715) Oracle metadata import using Designer takes long time for large schemas
[
https://issues.jboss.org/browse/TEIIDDES-715?page=com.atlassian.jira.plug...
]
Barry LaFond reassigned TEIIDDES-715:
-------------------------------------
Assignee: Dan Florian
Oracle metadata import using Designer takes long time for large
schemas
-----------------------------------------------------------------------
Key: TEIIDDES-715
URL: https://issues.jboss.org/browse/TEIIDDES-715
Project: Teiid Designer
Issue Type: Bug
Components: Import/Export
Affects Versions: 7.1
Reporter: Ramesh Reddy
Assignee: Dan Florian
Fix For: 7.4
Importing metadata from Oracle database seems to take longer than any other database,
when the underlying the schema is large. This may be because the default settings with
Oracle Driver tries to get lot more metadata then needed. Should explore ways to minimize
the time to take import.
Teiid in the Dynamic VDB import uses "Table Types" and "schema" name
to limit the number of tables it imports, may be we should adopt similar thing in
Designer. It is possible that Designer is already doing such settings.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:38 p.m.
New subject: [JBoss JIRA] Commented: (TEIIDDES-715) Oracle metadata import using Designer takes long time for large schemas
[
https://issues.jboss.org/browse/TEIIDDES-715?page=com.atlassian.jira.plug...
]
Dan Florian commented on TEIIDDES-715:
--------------------------------------
I did some measurements using the JDBC Oracle and SQL Server importers. Imported the Books
test database using both importers. Here are some results from one of the runs:
Time to execute RelationalModelProcessorImpl.performExecute(Context, List):
- Oracle: 4953 (all results in milliseconds)
- SQL Server: 1906
- Difference: 3047
These 3 statements (each executed multiple times) account for 2389 of the 3047
difference:
- java.sql.DatabaseMetaData.getColumns(String,String,String,String) in
GetColumnsRequest.performInvocation(Response): 23% of of the 2389 difference
- java.sql.ResultSet.next() in
Response.addResults(Response,ResultSet,boolean,TupleValidator) when creating columns: 58%
of of the 2389 difference
- java.sql.DatabaseMetaData.getPrimaryKeys(String,String,String) in
GetPrimaryKeyRequest.performInvocation(Response): 19% of the 2389 difference
This indicates the time differences are attributed to the vendor's driver
implementation or in the driver default settings as Ramesh says in the description. I will
look at the driver settings next.
Oracle metadata import using Designer takes long time for large
schemas
-----------------------------------------------------------------------
Key: TEIIDDES-715
URL: https://issues.jboss.org/browse/TEIIDDES-715
Project: Teiid Designer
Issue Type: Bug
Components: Import/Export
Affects Versions: 7.1
Reporter: Ramesh Reddy
Assignee: Dan Florian
Fix For: 7.4
Importing metadata from Oracle database seems to take longer than any other database,
when the underlying the schema is large. This may be because the default settings with
Oracle Driver tries to get lot more metadata then needed. Should explore ways to minimize
the time to take import.
Teiid in the Dynamic VDB import uses "Table Types" and "schema" name
to limit the number of tables it imports, may be we should adopt similar thing in
Designer. It is possible that Designer is already doing such settings.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
5:53 p.m.
New subject: [JBoss JIRA] Commented: (TEIIDDES-715) Oracle metadata import using Designer takes long time for large schemas
[
https://issues.jboss.org/browse/TEIIDDES-715?page=com.atlassian.jira.plug...
]
Dan Florian commented on TEIIDDES-715:
--------------------------------------
Before the import wizard's finish button is clicked, the SQL Server importer is taking
*WAY* more time than the oracle importer. When you consider the total import time (going
through the wizard pages and finishing the wizard), Oracle is much faster than SQL
Server.
The major difference is the time it takes to display the JdbcImportMetadataPage. In
Oracle, the JdbcImportMetadataPage wizard page takes less than a second to display. But in
SQL Server, that same wizard page takes around 17-18 seconds. It appears the SQL Server
importer is fetching much more metadata up front while Oracle much more at the end of the
wizard. For instance, prior to finishing the wizard, the SQL Server importer calls
DatabaseMetaData.getSchemas() *LOTS* of times while the Oracle importer only calls it
once. One last thing, and not sure if this is important, but the execution code paths of
the wizard does diverge based on if the DB "supports catalogs." SQL Server does
"supportCatalogs" and Oracle does not.
Oracle metadata import using Designer takes long time for large
schemas
-----------------------------------------------------------------------
Key: TEIIDDES-715
URL: https://issues.jboss.org/browse/TEIIDDES-715
Project: Teiid Designer
Issue Type: Bug
Components: Import/Export
Affects Versions: 7.1
Reporter: Ramesh Reddy
Assignee: Dan Florian
Fix For: 7.4
Importing metadata from Oracle database seems to take longer than any other database,
when the underlying the schema is large. This may be because the default settings with
Oracle Driver tries to get lot more metadata then needed. Should explore ways to minimize
the time to take import.
Teiid in the Dynamic VDB import uses "Table Types" and "schema" name
to limit the number of tables it imports, may be we should adopt similar thing in
Designer. It is possible that Designer is already doing such settings.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
7:20 p.m.
New subject: [JBoss JIRA] Commented: (TEIIDDES-715) Oracle metadata import using Designer takes long time for large schemas
[
https://issues.jboss.org/browse/TEIIDDES-715?page=com.atlassian.jira.plug...
]
Barry LaFond commented on TEIIDDES-715:
---------------------------------------
Also note that our DB's (Oracle and SQL Server, whatever) probably don't have the
same "Schema" on them. So if one DB has many more schema and we're fetching
ALL of them (3rd JDBC import page), then it would probably take more time.
Oracle metadata import using Designer takes long time for large
schemas
-----------------------------------------------------------------------
Key: TEIIDDES-715
URL: https://issues.jboss.org/browse/TEIIDDES-715
Project: Teiid Designer
Issue Type: Bug
Components: Import/Export
Affects Versions: 7.1
Reporter: Ramesh Reddy
Assignee: Dan Florian
Fix For: 7.4
Importing metadata from Oracle database seems to take longer than any other database,
when the underlying the schema is large. This may be because the default settings with
Oracle Driver tries to get lot more metadata then needed. Should explore ways to minimize
the time to take import.
Teiid in the Dynamic VDB import uses "Table Types" and "schema" name
to limit the number of tables it imports, may be we should adopt similar thing in
Designer. It is possible that Designer is already doing such settings.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:16 p.m.
New subject: [JBoss JIRA] Updated: (TEIIDDES-715) Oracle metadata import using Designer takes long time for large schemas
[
https://issues.jboss.org/browse/TEIIDDES-715?page=com.atlassian.jira.plug...
]
Barry LaFond updated TEIIDDES-715:
----------------------------------
Assignee: (was: Dan Florian)
Fix Version/s: 7.5
(was: 7.4)
Need to set up very similar DB Schema for Oracle + other DB's before we can really
address this issue. Pushing to 7.5
Oracle metadata import using Designer takes long time for large
schemas
-----------------------------------------------------------------------
Key: TEIIDDES-715
URL: https://issues.jboss.org/browse/TEIIDDES-715
Project: Teiid Designer
Issue Type: Bug
Components: Import/Export
Affects Versions: 7.1
Reporter: Ramesh Reddy
Fix For: 7.5
Importing metadata from Oracle database seems to take longer than any other database,
when the underlying the schema is large. This may be because the default settings with
Oracle Driver tries to get lot more metadata then needed. Should explore ways to minimize
the time to take import.
Teiid in the Dynamic VDB import uses "Table Types" and "schema" name
to limit the number of tables it imports, may be we should adopt similar thing in
Designer. It is possible that Designer is already doing such settings.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:06 p.m.
New subject: [JBoss JIRA] Updated: (TEIIDDES-715) Oracle metadata import using Designer takes long time for large schemas
[
https://issues.jboss.org/browse/TEIIDDES-715?page=com.atlassian.jira.plug...
]
Barry LaFond updated TEIIDDES-715:
----------------------------------
Fix Version/s: 8.0
(was: 7.5)
Oracle metadata import using Designer takes long time for large
schemas
-----------------------------------------------------------------------
Key: TEIIDDES-715
URL: https://issues.jboss.org/browse/TEIIDDES-715
Project: Teiid Designer
Issue Type: Bug
Components: Import/Export
Affects Versions: 7.1
Reporter: Ramesh Reddy
Fix For: 8.0
Importing metadata from Oracle database seems to take longer than any other database,
when the underlying the schema is large. This may be because the default settings with
Oracle Driver tries to get lot more metadata then needed. Should explore ways to minimize
the time to take import.
Teiid in the Dynamic VDB import uses "Table Types" and "schema" name
to limit the number of tables it imports, may be we should adopt similar thing in
Designer. It is possible that Designer is already doing such settings.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
4938
days inactive
5156
days old
teiid-designer-issues@lists.jboss.org
7 comments
3 participants
participants (3)
-
Barry LaFond (JIRA) -
Dan Florian (JIRA)
-
Ramesh Reddy (JIRA)