[jboss-dev-forums] [Design of JBossCache] - state transfer optimizations - persisitng state with JDBCCac

Background: When integrating persistent state, individual insert operations are being triggered on JDBCCacheLoader. This might be optimized by batching these insert calls, though gaining significant performance. Now this is the short story, the long one is as that it's not only inserts taking place when transferring persistent state. The general algorithm of transferring state for a node '/a/b/c' is: a)if ('/a/b/c' does not exists) add nodes /a, /a/b and /a/b/c (ones that !exist) else //it exists b) replace ('/a/b/c') existing attributes with the new ones. in the case of a) there are 2*Fqn.size db interactions (an exists and an insert for each new node) + an initial query to check whether the node exists in the case of b) there are 2 query only (initial one for existence and one for update) Considering that this is done for each transfered node, performance is low. Another approach would be to: - read all the pre-existing state in memory (1 query) - do a merge in memory - persist all merged state in one batch ( 2 batch operations are actually necessary as some updates also need to be performed) At a glance this should be much efficient as it reduces the number of DB interactions to 3. Issues that might be with the new approach: - all pre-existing state would be loaded in memory for the merge. High memory consumption; possible OOM - if there are too many statements in the batch, there might be a failure at commit time: I remember having problems on Oracle with 10k stamens in a batch. This might be fixed though, splinting into 1k (configurable) sized batches - others? View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4155281#... Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...