[jboss-dev-forums] [Design of JBossCache] - Re: Amazon S3 cache loader

[genman]

I got the basic implementation done, though there are still some problems.

I don't think the Amazon S3 API aligns well with the CacheLoader interface. First of all, there are too many "merge" operations, such as put(Map) or put(key, value) which require an additional HTTP request to fetch the old Map. I came up with an option to get rid of merging, so put(Map) always overwrites the old values.

But the biggest problem is mostly to do with "getChildrenNames()" and how nodes are created. S3 allows nodes to be queried in lexicographic order. So, here's an example dump of S3 keys after put("/a/b/c") put("/a/b/d") etc.:

3/a/b/c
3/a/b/d
4/a/b/c/1
4/a/b/c/2
 ...
4/a/b/c/1000

So, to get children of /a/b, I look for nodes prefixed with the string "3/a/b" ... 

The "3" comes from the depth. Without including depth, you'd see leaf nodes /a/b/c/1 ... 1000. This wouldn't be great.

So, the node depth works well, except how can I get  the children of root? I look for nodes prefixed with 1/ .. But there are no such nodes. The solution is to create 1/a , 2/a/b when creating 3/a/b/c . But to ensure creation, it means extra HTTP requests for every put. It's doable but sucks. There's all sorts of possible race conditions as well for concurrent remove/put.

I'm wondering if I should just give up on this.

View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4126888#4126888

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4126888