[infinispan-dev] [ISPN-78] Alternative interface for writing large objects

[Olaf Bergner]

Hi Galder,

-------- Original-Nachricht --------
> Datum: Mon, 4 Apr 2011 11:29:06 +0200
> Von: "Galder Zamarreño" <galder at redhat.com>
> > in size. Anyway, it should be easy to refactor if reusing 
> > PutKeyValueCommand should prove viable.
> 
> The only reason it reads the previous value is to return it as part of
> contract of "V put(K, V)" - but that can be skipped.

Meanwhile, I replaced my custom WriteLargeObjectCommand with a customized PutKeyValueCommand sporting a new putLargeObject flag.

> > Since this is important and might reveal a fundamental misunderstanding 
> > on my part, I need to sort this out before moving on. These are my 
> > assumptions, please point out any errors:
> > 
> > 1. We want to partition a large object into chunks since, by definition,
> > a large object is too big to be stored in a single node in the cluster. 
> > It follows that it is paramount that no two chunks be stored in the same
> > node, correct?
> 
> No. The idea is that the whole object should not end up being stored in a
> single JVM, but nothing should stop you from storing two chunks of the same
> object in the same node.

Ah, this takes away some complexity. Good.

> What we somehow need to avoid is chunks ending up in nodes that do not
> have enough memory to store them, and that could complicate things. 

Definitely. What about replication, for instance? Does INFINISPAN use the replication mechanism suggested by Dynamo, i.e. walking the constant hash ring in clockwise direction until the desired number of replicas is reached (if I recall correctl)? I'm afraid this might fail in our case.

Plus, I fear rehashing would have to be aware of wheter it is dealing with relocating a large object chunk or a "regular" value.


> Hmmmm, maybe what's needed here is a mix of the two. You want metadata
> information to be transactional, so when you start writing and chunking an
> object and you keep updating the metadata object, this is transactionally
> protected, so no one can read the metadata in the mean time, however, the
> actual chunk writing in the cache could be non-transactional to make chunks do
> not pile up in the transaction context.

Sounds reasonable. Something definitely worth thinking about.

Cheers,
Olaf
> 
> > 
> > So many questions ...
> > 
> > Cheers,
> > Olaf
> > 
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> --
> Galder Zamarreño
> Sr. Software Engineer
> Infinispan, JBoss Cache
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 
Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de