[jboss-dev-forums] [Design of JBoss ESB] - Re: JCRMessageStoreImpl store only message body?

[maeste]

"michael.neale at jboss.com" wrote : Hmm... I would think JCR maybe not the right persistence mechanism for large amounts of transactional data - it certainly is suited for heterogenous message payloads, but it is generally more about content then billions of little messages. 
  | 
Ok, and what else do you have in mind? Relational DB? HDFS?
We can evaluate to leave data in these repository, write a DNA connector and view them with federation engine "as if it's in JCR".
For sure I'll make a first (easier) implementation of JCR message store, and I'll focus on sequencer, but other solution for storing and then federate them can be evaluated in next future.
So, what do you think is the best store for ESB message?
My 2c: 
1) JCR could have limitation with large amounts of messages, not with big messages, it would be quite easy to configure and deploy. Moreover with DNA a 100% JBoss solution could be implemented ;)
2) Relational DB. No problem with large number of messages, maybe some problem with very very large messages. It is a well known technology, probably customers have at least one installation in their server farms. Heterogeneity of data may be an issue, even if we consider DNA federation the only (or at least the main) solution to intercat with data it would be a secondary problem (IOW brute storing data and leaving DNA the responsibility to make them able to be queried). 
3) HDFS. A lot of benefits storing data (as far as I understood it, studing it too fewer to feel me confident): distributed, high available, fast(?), no problem for tons of file, no problem with huge files. What are the minus in my opinion? Not so easy to install and maintain (from customer point of view), another API to implement on both side (ESB and DNA federation). May be the best solution at long term, would be not so easy as starting point, it should be on our road map, but IMHO not at short terms.

But one more time my question is: does it make some sense to keep this large amount of data for a long time for all? I think, the best approach, would be to have more than one repository available for different purpose (and I think ESB already have both solution 1 and 2, even if they can be revisited as I'm saying in this 3ad). 
A JCR implementation keeping data for a relative short period of time with governance goals in mind would be an useful kick start.



View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4152219#4152219

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4152219