1:37 p.m.
Max, hi.
I'd like to discuss JBIDE-2173 & JBIDE-2162.
From our JBIDE-2162 discussion:
"I try debug DatabaseMetaData code and seems I find the reason. The main
problem here - for each table extracted all foreing keys for all tables:"
This is of course JBIDE-2173 and it seems here are the problems only with
MySQL - I have check Postrgre and got better performance (Postrgre has no
problems with performance).
Also I try to study the problem to find common workaround for it - and seems
it exist!
Now to get foreing keys we are using
getMetaDataDialect().getExportedKeys
but it possible to use
getMetaDataDialect().getImportedKeys
and we will get much better performance - I have check it - it possible and
we will get performance O(n) instead of O(n^2) for MySQL.
Seems this have a sense, isn't it?
About JDBCReader - when I try to modify it I saw some potential problem here
-
This is code to add table into dbs:
Table table = dbs.addTable(quote(getSchemaForModel(schemaName)),
getCatalogForModel(catalogName), quote(tableName));
This is code to get table from dbs:
dbs.getTable(schemaName, catalogName, tableName)
obvious that here will be a problem with quotes.
I think the better way incorporate quotes process into DatabaseCollector.
Best regards,
Vitali
4:38 a.m.
Hi Vitali,
"I try debug DatabaseMetaData code and seems I find the reason.
The main
problem here - for each table extracted all foreing keys for all tables:"
This is of course JBIDE-2173 and it seems here are the problems only with
MySQL - I have check Postrgre and got better performance (Postrgre has no
problems with performance).
ok.
Also I try to study the problem to find common workaround for it -
and seems
it exist!
Now to get foreing keys we are using
getMetaDataDialect().getExportedKeys
but it possible to use
getMetaDataDialect().getImportedKeys
and we will get much better performance - I have check it - it possible and
we will get performance O(n) instead of O(n^2) for MySQL.
Seems this have a sense, isn't it?
Did you create a patch for this ? Does all unittests passes ? I assume alot of logic
has to change now that foreignkeys are in "reverse" order ?
About JDBCReader - when I try to modify it I saw some potential
problem here
-
This is code to add table into dbs:
Table table = dbs.addTable(quote(getSchemaForModel(schemaName)),
getCatalogForModel(catalogName), quote(tableName));
This is code to get table from dbs:
dbs.getTable(schemaName, catalogName, tableName)
obvious that here will be a problem with quotes.
sorry, but I don't see what is obvious in that ?
I think the better way incorporate quotes process into
DatabaseCollector.
Why ?
Please explain.
-max
12:41 p.m.
Max, hi.
I was on vacations.
> Did you create a patch for this ? Does all unittests passes ? I
assume
alot of logic
> has to change now that foreignkeys are in "reverse"
order ?
Yes I have created patch for this ->
https://jira.jboss.org/jira/browse/JBIDE-2173 -> JBIDE-2173_20080630.zip.
I added some info about process time before and after the patch.
I compare unittests before patch and after - it had the same number of
Runs/Errors/Failures in both cases.
You could check the patch to find possible problems - as I expect it should
work -
may be it's logic not so smooth - we could discuss it to find better
solution.
I'll compare all unittests once more for better confidence and report you
about all problems.
> sorry, but I don't see what is obvious in that ?
You could see this changes in JBIDE-2173_20080630.zip ->
org.hibernate.cfg.reveng_patch.txt ->
>
- Table fkTable = dbs.getTable(fkSchema,
fkCatalog, fkTableName);
+ Table fkTable =
dbs.getTable(quote(fkSchema), quote(fkCatalog), quote(fkTableName));
>
I find this error in some unittest for quotes (when I
tested
JBIDE-2173_20080630 patch with unittestes) ->
This is a way to add table into dbs:
Table table = dbs.addTable(quote(getSchemaForModel(schemaName)),
getCatalogForModel(catalogName), quote(tableName));
You see - we add quotes for schema & table (if it's necessary to have quotes
added),
but then we try to get table -> dbs.getTable we have fkSchema without quotes
- and get an error.
In the case without my patch this situation was no reproducible.
> > I think the better way incorporate quotes process into
DatabaseCollector.
> Why ?
> Please explain.
My statement is disputable - " incorporate quotes process into
DatabaseCollector " means
We can use
Table fkTable = dbs.getTable(fkSchema, fkCatalog, fkTableName);
&
Table table = dbs.addTable(getSchemaForModel(schemaName),
getCatalogForModel(catalogName), tableName);
And forget about quotes outside DatabaseCollector,
But seems we can't easy and beautiful transfer the function quote() from
JDBCReader to DatabaseCollector offsprings.
So statement is just disputable.
Vitali
-----Original Message-----
From: Max Rydahl Andersen [mailto:max.andersen@redhat.com]
Sent: Monday, July 14, 2008 11:39 AM
To: Vitali Yemialyanchyk
Cc: jbosstools-dev(a)lists.jboss.org
Subject: Re: JBIDE-2173 & JBIDE-2162 - performance improvement (for MySQL?)
Hi Vitali,
"I try debug DatabaseMetaData code and seems I find the reason.
The main
problem here - for each table extracted all foreing keys for all tables:"
This is of course JBIDE-2173 and it seems here are the problems only with
MySQL - I have check Postrgre and got better performance (Postrgre has no
problems with performance).
ok.
Also I try to study the problem to find common workaround for it -
and
seems
it exist!
Now to get foreing keys we are using
getMetaDataDialect().getExportedKeys
but it possible to use
getMetaDataDialect().getImportedKeys
and we will get much better performance - I have check it - it possible
and
we will get performance O(n) instead of O(n^2) for MySQL.
Seems this have a sense, isn't it?
Did you create a patch for this ? Does all unittests passes ? I assume alot
of logic
has to change now that foreignkeys are in "reverse" order ?
About JDBCReader - when I try to modify it I saw some potential
problem
here
-
This is code to add table into dbs:
Table table = dbs.addTable(quote(getSchemaForModel(schemaName)),
getCatalogForModel(catalogName), quote(tableName));
This is code to get table from dbs:
dbs.getTable(schemaName, catalogName, tableName)
obvious that here will be a problem with quotes.
sorry, but I don't see what is obvious in that ?
I think the better way incorporate quotes process into
DatabaseCollector.
Why ?
Please explain.
-max
7:05 a.m.
>> Did you create a patch for this ? Does all unittests passes ?
I assume
alot of logic
>> has to change now that foreignkeys are in "reverse" order ?
Yes I have created patch for this ->
https://jira.jboss.org/jira/browse/JBIDE-2173 -> JBIDE-2173_20080630.zip.
I added some info about process time before and after the patch.
I compare unittests before patch and after - it had the same number of
Runs/Errors/Failures in both cases.
You could check the patch to find possible problems - as I expect it
should
work -
may be it's logic not so smooth - we could discuss it to find better
solution.
I'll compare all unittests once more for better confidence and report you
about all problems.
Any news ?
>> sorry, but I don't see what is obvious in that ?
You could see this changes in JBIDE-2173_20080630.zip ->
org.hibernate.cfg.reveng_patch.txt ->
>>
- Table fkTable = dbs.getTable(fkSchema,
fkCatalog, fkTableName);
+ Table fkTable =
dbs.getTable(quote(fkSchema), quote(fkCatalog), quote(fkTableName));
>>
I find this error in some unittest for quotes (when I tested
JBIDE-2173_20080630 patch with unittestes) ->
This is a way to add table into dbs:
Table table = dbs.addTable(quote(getSchemaForModel(schemaName)),
getCatalogForModel(catalogName), quote(tableName));
You see - we add quotes for schema & table (if it's necessary to have
quotes
added),
but then we try to get table -> dbs.getTable we have fkSchema without
quotes
- and get an error.
In the case without my patch this situation was no reproducible.
>> > I think the better way incorporate quotes process into
DatabaseCollector.
>> Why ?
>> Please explain.
My statement is disputable - " incorporate quotes process into
DatabaseCollector " means
We can use
Table fkTable = dbs.getTable(fkSchema, fkCatalog, fkTableName);
&
Table table = dbs.addTable(getSchemaForModel(schemaName),
getCatalogForModel(catalogName), tableName);
And forget about quotes outside DatabaseCollector,
I can't see how we can forget about quotes outside db collector ?
But seems we can't easy and beautiful transfer the function
quote() from
JDBCReader to DatabaseCollector offsprings.
So statement is just disputable.
I'm sorry, but I don't understand your point. "is just disputable" ?
/max
8:17 a.m.
> Any news ?
I have attached screenshots with unit test for
JBIDE-2173:
1. HSQLDB_JUnit_Screens.zip
2. MySQL_JUnit_Screens.zip
3. Oracle_JUnit_Screens.zip
4. Postgre_JUnit_Screens.zip
You can check it all easily. Just unzip and look at pictures.
You can see what in both cases result the same - so patch
JBIDE-2173_20080630.zip should be correct.
For testing both, open JDBCReader -> readDatabaseSchema -> comment/uncomment
// method 1: - old method
// method 2: - new one
Remember what I update some unit tests for better work with dbs Oracle &
MySQL and
update some ambiguous places - like IncrementalSchemaReadingTest and others.
So if you apply the patch without my junit updates and try to run tests for
both cases you can get some other results - but I expect this as unit tests
problems.
JBIDE-2229 - JBIDE-2229_20080529.zip - my junits update patch.
IncrementalSchemaReadingTest is still have no patch to download and apply.
IncrementalSchemaReadingTest has the iterator dependents - it expected first
element "CHILD",
In general case thin is not a true.
> I'm sorry, but I don't understand your point. "is
just disputable" ?
At that moment I did not see all benefits and drawbacks of
one and other
solution - so
I just wanted to discuss both. Now I see what current solution is better -
so
we can terminate this topic (the topic about "incorporate quotes process
into DatabaseCollector").
Vitali
-----Original Message-----
From: Max Rydahl Andersen [mailto:max.andersen@redhat.com]
Sent: Thursday, August 07, 2008 2:05 PM
To: Vitali Yemialyanchyk
Cc: jbosstools-dev(a)lists.jboss.org
Subject: Re: JBIDE-2173 & JBIDE-2162 - performance improvement (for MySQL?)
>> Did you create a patch for this ? Does all unittests passes ?
I assume
alot of logic
>> has to change now that foreignkeys are in "reverse" order ?
Yes I have created patch for this ->
https://jira.jboss.org/jira/browse/JBIDE-2173 -> JBIDE-2173_20080630.zip.
I added some info about process time before and after the patch.
I compare unittests before patch and after - it had the same number of
Runs/Errors/Failures in both cases.
You could check the patch to find possible problems - as I expect it
should
work -
may be it's logic not so smooth - we could discuss it to find better
solution.
I'll compare all unittests once more for better confidence and report you
about all problems.
Any news ?
>> sorry, but I don't see what is obvious in that ?
You could see this changes in JBIDE-2173_20080630.zip ->
org.hibernate.cfg.reveng_patch.txt ->
>>
- Table fkTable = dbs.getTable(fkSchema,
fkCatalog, fkTableName);
+ Table fkTable =
dbs.getTable(quote(fkSchema), quote(fkCatalog), quote(fkTableName));
>>
I find this error in some unittest for quotes (when I tested
JBIDE-2173_20080630 patch with unittestes) ->
This is a way to add table into dbs:
Table table = dbs.addTable(quote(getSchemaForModel(schemaName)),
getCatalogForModel(catalogName), quote(tableName));
You see - we add quotes for schema & table (if it's necessary to have
quotes
added),
but then we try to get table -> dbs.getTable we have fkSchema without
quotes
- and get an error.
In the case without my patch this situation was no reproducible.
>> > I think the better way incorporate quotes process into
DatabaseCollector.
>> Why ?
>> Please explain.
My statement is disputable - " incorporate quotes process into
DatabaseCollector " means
We can use
Table fkTable = dbs.getTable(fkSchema, fkCatalog, fkTableName);
&
Table table = dbs.addTable(getSchemaForModel(schemaName),
getCatalogForModel(catalogName), tableName);
And forget about quotes outside DatabaseCollector,
I can't see how we can forget about quotes outside db collector ?
But seems we can't easy and beautiful transfer the function
quote() from
JDBCReader to DatabaseCollector offsprings.
So statement is just disputable.
I'm sorry, but I don't understand your point. "is just disputable" ?
/max
5738
days inactive
5780
days old
jbosstools-dev@lists.jboss.org
4 comments
2 participants
participants (2)
-
Max Rydahl Andersen -
Vitali Yemialyanchyk