[jbosscache-dev] Re: Integrating steaming transfer in JBC

[Manik Surtani]

(This should be on the jbc dev mail list, btw - or on the design  
forum if it warrants it)

Vladimir/Brian:

Does this relate to the need for something like:

void storeEntireState(OutputStream s);
InputStream loadEntireState();
void storeState(OutputStream s, Fqn subtree);
InputStream loadState(Fqn subtree);

In the CacheLoader interface?

Firstly, is this feasible when it comes to cache loaders - e.g., a  
JDBC cache loader - what sort of stream markers will we have to let  
the cache loader know when it can start chunking up data from the  
stream, parsing it and pushing this state down to storage?  Or will  
it just be an API-level kludge where the CL impl attempts to keep  
reading from the stream until the stream closes, yanks out a byte[]  
and passes that into storeEntireState(byte[] b)?  (May be the case  
for an initial impl, just so we have the interface methods in place  
for 2.0.0)

Cheers,
--
Manik Surtani

Lead, JBoss Cache
JBoss, a division of Red Hat

Email: manik at jboss.org
Telephone: +44 7786 702 706
MSN: manik at surtani.org
Yahoo/AIM/Skype: maniksurtani


On 15 Aug 2006, at 16:57, Vladimir Blagojevic wrote:

> I should have sent a copy to Manik and Bela as well.  See below.
>
> Main point is: we need additional methods in CacheLoader to  
> accommodate
> streaming state transfer.
> How should we proceed?
>
> Vladimir
>
> -----Original Message-----
> From: Vladimir Blagojevic
> Sent: Tuesday, August 15, 2006 11:41 AM
> To: Brian Stansberry
> Subject: RE: Integrating steaming transfer in JBC
>
> Oyyyy. Not only that but it also affects CacheLoader API:
>
> byte[] loadEntireState() throws Exception;
> void storeEntireState(byte[] state) throws Exception;
> byte[] loadState(Fqn subtree) throws Exception;
> void storeState(byte[] state, Fqn subtree) throws Exception;
>
> These should have streaming counterparts if we really want large state
> OOME to go away....
>
>
>
> -----Original Message-----
> From: Brian Stansberry
> Sent: Tuesday, August 15, 2006 11:27 AM
> To: Vladimir Blagojevic
> Subject: RE: Integrating steaming transfer in JBC
>
> Yep, plus there are different types of state that will be handled by
> different subsystems; that will be interesting for streaming state
> transfer.
>
> Also, for buddy replication, the state transfer is a push rather  
> than a
> pull.  Not sure what to do about that :(
>
> Vladimir Blagojevic wrote:
>> Ok I see. So it is not only integration of state transfer but  
>> removing
>
>> rpc mechanism for partial state integration. I am looking at the code
>> and learning how is this done. Then I'll ping you tomorrow/ 
>> Thursday to
>
>> have a discussion with you.
>>
>> -----Original Message-----
>> From: Brian Stansberry
>> Sent: Monday, August 14, 2006 3:02 PM
>> To: Vladimir Blagojevic
>> Subject: RE: Integrating steaming transfer in JBC
>>
>> Just look at the JBCACHE issues and filter for the ones assigned to
>> you :)
>>
>> Vladimir Blagojevic wrote:
>>> Hey,
>>>
>>> What do you mean by looking at the non  streaming state transfer
>>> tasks? Which ones are those?
>>>
>>> Vladimir
>>>
>>>
>>> -----Original Message-----
>>> From: "Brian Stansberry" <brian.stansberry at jboss.com>
>>> Date: Mon, 14 Aug 2006 12:24:35
>>> To:<vladimirb at rogers.blackberry.net>
>>> Subject: RE: Integrating steaming transfer in JBC
>>>
>>> Sure, just let me know.  But, I'd start with looking at the
>>> non-streaming state transfer tasks.  That will get you familiar with
>>> the use cases and existing code.
>>>