Re: [infinispan-dev] ISPN-78 and Large Object Support/Streaming API
On Tue, May 18, 2010 at 5:11 PM, Manik Surtani <manik(a)jboss.org> wrote:
On 18 May 2010, at 16:02, Philippe Van Dyck wrote:
It may sound a bit extreme but what about using streams as the base
interface and converting values to streams asap ?
It's certainly worth thinking about.
How would his affect listeners though, where listeners expect objects, not
streams?
Welcome to stream plumbing ;-)
You are not supposed to read or write to the stream... producers and
consumers do ! (read cache stores & cache users - not you !)
Now, the question is ... are your listeners stream consumers or even
producers ?
How much consumers do you have ? Each of them will need a copy of the
stream... that's a lot of plumbing.
Think about some kind of spider tube, with one input and multiple outputs...
Nothing complicated really.
The real question with streams, after they are all connected, is to find
leaks and seriously think about what procedure to apply when the system is
overheating ... how do you collect all the leaks (exceptions) ?
Funny isn't it ?
phil
The conversion could be seen as the marshaling operation and the result as a
stream of bytes ?
The 'base' interface would be as simple as:
OutputStream<byte[]> get(K)
put(K, InputStream<byte[]>)
the Map<K,V> facade simply 'marshalls' values to byte arrays and calls the
'base' interface.
This way, you get rid of any buffering from the very beginning and give
streams to cache stores... (You actually replace byte arrays by
piped(in/out)putstreams in your futures)...
Btw, I can assure you that some cache store maintainers will be super happy
to receive and produce streams ;-)
my 2 cents...
phil
On Tue, May 18, 2010 at 3:59 PM, Manik Surtani <manik(a)jboss.org> wrote:
> I have put together a brief design for ISPN-78. Please take a look, it is
> on the wiki:
>
> https://community.jboss.org/wiki/LargeObjectSupport
>
> I have also deferred ISPN-78 to 5.0.0 rather than 4.1.0 as I'd rather not
> hold up 4.1.0 for new features at this stage.
>
> Please have a look at the designs and let me know what you think - or
> comment on the wiki page.
>
> Cheers
> Manik
> --
> Manik Surtani
> manik(a)jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Attachments:
- attachment.html (text/html — 5.9 KB)