[infinispan-dev] ISPN-78 and Large Object Support/Streaming API

[Galder Zamarreno]

Done: https://jira.jboss.org/browse/ISPN-449

----- "Manik Surtani" <manik at jboss.org> wrote:

> And another spin on this - we'd also need to think about such
> streaming functionality for HotRod.  :)  
> 
> Galder, care to create a related JIRA for streaming over HotRod?
> 
> On 18 May 2010, at 16:29, Philippe Van Dyck wrote:
> 
> > 
> > 
> > On Tue, May 18, 2010 at 5:11 PM, Manik Surtani <manik at jboss.org>
> wrote:
> > 
> > On 18 May 2010, at 16:02, Philippe Van Dyck wrote:
> > 
> >> It may sound a bit extreme but what about using streams as the base
> interface and converting values to streams asap ?
> > 
> > It's certainly worth thinking about.
> > 
> > How would his affect listeners though, where listeners expect
> objects, not streams?
> > 
> > 
> > Welcome to stream plumbing ;-)
> >  
> > You are not supposed to read or write to the stream... producers and
> consumers do ! (read cache stores & cache users - not you !)
> > 
> > Now, the question is ... are your listeners stream consumers or even
> producers ?
> > 
> > How much consumers do you have ? Each of them will need a copy of
> the stream... that's a lot of plumbing. 
> > 
> > Think about some kind of spider tube, with one input and multiple
> outputs... Nothing complicated really.
> >  
> > The real question with streams, after they are all connected, is to
> find leaks and seriously think about what procedure to apply when the
> system is overheating ... how do you collect all the leaks
> (exceptions) ?
> > 
> > Funny isn't it ?
> > 
> > phil
> > 
> >> The conversion could be seen as the marshaling operation and the
> result as a stream of bytes ?
> >> 
> >> The 'base' interface would be as simple as:
> >> OutputStream<byte[]> get(K)
> >> put(K, InputStream<byte[]>)
> >> 
> >> the Map<K,V> facade simply 'marshalls' values to byte arrays and
> calls the 'base' interface.
> >> 
> >> This way, you get rid of any buffering from the very beginning and
> give streams to cache stores... (You actually replace byte arrays by
> piped(in/out)putstreams in your futures)... 
> >> 
> >> Btw, I can assure you that some cache store maintainers will be
> super happy to receive and produce streams ;-)
> >> 
> >> my 2 cents...
> >> 
> >> phil
> >> 
> >> 
> >> On Tue, May 18, 2010 at 3:59 PM, Manik Surtani <manik at jboss.org>
> wrote:
> >> I have put together a brief design for ISPN-78.  Please take a
> look, it is on the wiki:
> >> 
> >>        https://community.jboss.org/wiki/LargeObjectSupport
> >> 
> >> I have also deferred ISPN-78 to 5.0.0 rather than 4.1.0 as I'd
> rather not hold up 4.1.0 for new features at this stage.
> >> 
> >> Please have a look at the designs and let me know what you think -
> or comment on the wiki page.
> >> 
> >> Cheers
> >> Manik
> >> --
> >> Manik Surtani
> >> manik at jboss.org
> >> Lead, Infinispan
> >> Lead, JBoss Cache
> >> http://www.infinispan.org
> >> http://www.jbosscache.org
> >> 
> >> 
> >> 
> >> 
> >> 
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >> 
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > 
> > 
> > --
> > Manik Surtani
> > manik at jboss.org
> > Lead, Infinispan
> > Lead, JBoss Cache
> > http://www.infinispan.org
> > http://www.jbosscache.org
> > 
> > 
> > 
> > 
> > 
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > 
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> --
> Manik Surtani
> manik at jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
> 
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev