Background: When integrating persistent state, individual insert operations are being
triggered on JDBCCacheLoader. This might be optimized by batching these insert calls,
though gaining significant performance.
Now this is the short story, the long one is as that it's not only inserts taking
place when transferring persistent state. The general algorithm of transferring state for
a node '/a/b/c' is:
a)if ('/a/b/c' does not exists) add nodes /a, /a/b and /a/b/c (ones that !exist)
else //it exists
b) replace ('/a/b/c') existing attributes with the new ones.
in the case of a) there are 2*Fqn.size db interactions (an exists and an insert for each
new node) + an initial query to check whether the node exists
in the case of b) there are 2 query only (initial one for existence and one for update)
Considering that this is done for each transfered node, performance is low.
Another approach would be to:
- read all the pre-existing state in memory (1 query)
- do a merge in memory
- persist all merged state in one batch ( 2 batch operations are actually necessary as
some updates also need to be performed)
At a glance this should be much efficient as it reduces the number of DB interactions to
3.
Issues that might be with the new approach:
- all pre-existing state would be loaded in memory for the merge. High memory consumption;
possible OOM
- if there are too many statements in the batch, there might be a failure at commit time:
I remember having problems on Oracle with 10k stamens in a batch. This might be fixed
though, splinting into 1k (configurable) sized batches
- others?
View the original post :
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4155281#...
Reply to the post :
http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...