[jbosscache-dev] Re: Integrating steaming transfer in JBC

[Manik Surtani]

Well then this is the ideal time to do it, while the interfaces are  
still malleable.

I suspect the quick and dirty impl for most cache loader impls will  
be to wait till the stream closes to capture the byte[] and process  
this rather than a more efficient on-the-fly approach.
--
Manik Surtani

Lead, JBoss Cache
JBoss, a division of Red Hat

Email: manik at jboss.org
Telephone: +44 7786 702 706
MSN: manik at surtani.org
Yahoo/AIM/Skype: maniksurtani


On 15 Aug 2006, at 17:16, Brian Stansberry wrote:

> Yep.  The thread kind of morphed into a design discussion :)
>
> jbosscache-dev-bounces at lists.jboss.org wrote:
>> (This should be on the jbc dev mail list, btw - or on the
>> design forum if it warrants it)
>>
>> Vladimir/Brian:
>>
>> Does this relate to the need for something like:
>>
>> void storeEntireState(OutputStream s);
>> InputStream loadEntireState();
>> void storeState(OutputStream s, Fqn subtree); InputStream
>> loadState(Fqn subtree);
>>
>> In the CacheLoader interface?
>>
>> Firstly, is this feasible when it comes to cache loaders -
>> e.g., a JDBC cache loader - what sort of stream markers will
>> we have to let the cache loader know when it can start
>> chunking up data from the stream, parsing it and pushing this
>> state down to storage?  Or will it just be an API-level
>> kludge where the CL impl attempts to keep reading from the
>> stream until the stream closes, yanks out a byte[] and passes
>> that into storeEntireState(byte[] b)?  (May be the case for
>> an initial impl, just so we have the interface methods in place for
>> 2.0.0)
>>
>> Cheers,
>>
>>> I should have sent a copy to Manik and Bela as well.  See below.
>>>
>>> Main point is: we need additional methods in CacheLoader to
>>> accommodate streaming state transfer.
>>> How should we proceed?
>>>
>>> Vladimir
>>>
>>> -----Original Message-----
>>> From: Vladimir Blagojevic
>>> Sent: Tuesday, August 15, 2006 11:41 AM
>>> To: Brian Stansberry
>>> Subject: RE: Integrating steaming transfer in JBC
>>>
>>> Oyyyy. Not only that but it also affects CacheLoader API:
>>>
>>> byte[] loadEntireState() throws Exception; void
>>> storeEntireState(byte[] state) throws Exception; byte[] loadState 
>>> (Fqn
>>> subtree) throws Exception; void storeState(byte[] state, Fqn
>>> subtree) throws Exception;
>>>
>>> These should have streaming counterparts if we really want large
>>> state OOME to go away....
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Brian Stansberry
>>> Sent: Tuesday, August 15, 2006 11:27 AM
>>> To: Vladimir Blagojevic
>>> Subject: RE: Integrating steaming transfer in JBC
>>>
>>> Yep, plus there are different types of state that will be handled by
>>> different subsystems; that will be interesting for streaming state
>>> transfer.
>>>
>>> Also, for buddy replication, the state transfer is a push rather  
>>> than
>>> a pull.  Not sure what to do about that :(
>>>
>>> Vladimir Blagojevic wrote:
>>>> Ok I see. So it is not only integration of state transfer but
>>>> removing
>>>
>>>> rpc mechanism for partial state integration. I am looking at the
>>>> code and learning how is this done. Then I'll ping you tomorrow/
>>>> Thursday to
>>>
>>>> have a discussion with you.
>>>>
>>>> -----Original Message-----
>>>> From: Brian Stansberry
>>>> Sent: Monday, August 14, 2006 3:02 PM
>>>> To: Vladimir Blagojevic
>>>> Subject: RE: Integrating steaming transfer in JBC
>>>>
>>>> Just look at the JBCACHE issues and filter for the ones assigned to
>>>> you :)
>>>>
>>>> Vladimir Blagojevic wrote:
>>>>> Hey,
>>>>>
>>>>> What do you mean by looking at the non  streaming state transfer
>>>>> tasks? Which ones are those?
>>>>>
>>>>> Vladimir
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: "Brian Stansberry" <brian.stansberry at jboss.com>
>>>>> Date: Mon, 14 Aug 2006 12:24:35
>>>>> To:<vladimirb at rogers.blackberry.net>
>>>>> Subject: RE: Integrating steaming transfer in JBC
>>>>>
>>>>> Sure, just let me know.  But, I'd start with looking at the
>>>>> non-streaming state transfer tasks.  That will get you familiar
>>>>> with the use cases and existing code.
>>>>>
>>
>> _______________________________________________
>> jbosscache-dev mailing list
>> jbosscache-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jbosscache-dev
>
>
>
> Brian Stansberry
> Lead, AS Clustering
> JBoss, a division of Red Hat
> Ph: 510-396-3864
> skype: bstansberry