Re: [infinispan-dev] Partial state transfer in Infinispan
On 17 Jun 2011, at 13:49, Mircea Markus wrote:
> But yes, there is no reason why we can't replace this with RPC as per
Distribution, however I think we do need a streaming solution - not just for replication
but distribution as well. As such I'd only want to re-implement this bit once, rather
than a temp RPC based solution first. So we need a mechanism to either:
Now this might sound a bit too radical but do we really need REPLICATED mode?
Yes. :-) REPL is ideal for certain common usage patterns: read-heavy, small clusters
(under 10 nodes), small overall data volume (fits in any single JVM's heap). This
gives fast reads since reads are always local, etc.
This is not fully brewed, but if e.g. we set numOwners =
Integer.MAX_INTEGER the cluster is effectively in replicated mode, so can't we just
drop the REPLICATION entirely? This would reduce the code size significantly...
We could in theory achieve this as you suggest with numOwners = MAX_INTEGER, but
internally we'd still be best off implementing this as we have right now. Saves on a
lot of overhead (memory as well as processing) compared to a DIST-like setup with an
unlimited number of data owners.
>
> (1) open a separate TCP socket for the sake of streaming state, or
> (2) reuse the sockets JGroups opens.
> They both have their pros and cons.
>
> (1) is more configuration, firewall setup, and a spiderweb of connections in a large
grid
> (2) would mean multiplexing with JGroups' use of the socket.
Having our own sockets might cause an administration complications. Also borrowing
sockets from jgroups doesn't seems nice...I'm not a fan of either solution really:
I think this should be transport's responsibility and we should enhance jgroups to
offer the streaming service.
Yes, it would be implemented in our Transport abstraction, for sure. So code wouldn't
leak, but all the same there is no hard requirement that JGroups provides this (since it
could be impl'd in JGroupsTransport).
--
Manik Surtani
manik(a)jboss.org
twitter.com/maniksurtani
Lead, Infinispan
http://www.infinispan.org
Attachments:
- attachment.html (text/html — 3.7 KB)