I have put together a brief design for ISPN-78. Please take a look, it is on the wiki: https://community.jboss.org/wiki/LargeObjectSupport I have also deferred ISPN-78 to 5.0.0 rather than 4.1.0 as I'd rather not hold up 4.1.0 for new features at this stage. Please have a look at the designs and let me know what you think - or comment on the wiki page. Cheers Manik -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 18 May 2010, at 16:02, Philippe Van Dyck wrote: It may sound a bit extreme but what about using streams as the base interface and converting values to streams asap ? It's certainly worth thinking about. How would his affect listeners though, where listeners expect objects, not streams?

stream of bytes ? The 'base' interface would be as simple as: OutputStream<byte[]> get(K) put(K, InputStream<byte[]>) the Map<K,V> facade simply 'marshalls' values to byte arrays and calls the 'base' interface. This way, you get rid of any buffering from the very beginning and give streams to cache stores... (You actually replace byte arrays by piped(in/out)putstreams in your futures)... Btw, I can assure you that some cache store maintainers will be super happy to receive and produce streams ;-) my 2 cents... phil On Tue, May 18, 2010 at 3:59 PM, Manik Surtani <manik(a)jboss.org&gt; wrote: > I have put together a brief design for ISPN-78. Please take a look, it is > on the wiki: > > https://community.jboss.org/wiki/LargeObjectSupport > > I have also deferred ISPN-78 to 5.0.0 rather than 4.1.0 as I'd rather not > hold up 4.1.0 for new features at this stage. > > Please have a look at the designs and let me know what you think - or > comment on the wiki page. > > Cheers > Manik > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

And another spin on this - we'd also need to think about such streaming functionality for HotRod. :) Galder, care to create a related JIRA for streaming over HotRod? On 18 May 2010, at 16:29, Philippe Van Dyck wrote: > > > On Tue, May 18, 2010 at 5:11 PM, Manik Surtani <manik(a)jboss.org&gt; wrote: > > On 18 May 2010, at 16:02, Philippe Van Dyck wrote: > >> It may sound a bit extreme but what about using streams as the base interface and converting values to streams asap ? > > It's certainly worth thinking about. > > How would his affect listeners though, where listeners expect objects, not streams? > > > Welcome to stream plumbing ;-) > > You are not supposed to read or write to the stream... producers and consumers do ! (read cache stores & cache users - not you !) > > Now, the question is ... are your listeners stream consumers or even producers ? > > How much consumers do you have ? Each of them will need a copy of the stream... that's a lot of plumbing. > > Think about some kind of spider tube, with one input and multiple outputs... Nothing complicated really. > > The real question with streams, after they are all connected, is to find leaks and seriously think about what procedure to apply when the system is overheating ... how do you collect all the leaks (exceptions) ? > > Funny isn't it ? > > phil > >> The conversion could be seen as the marshaling operation and the result as a stream of bytes ? >> >> The 'base' interface would be as simple as: >> OutputStream<byte[]> get(K) >> put(K, InputStream<byte[]>) >> >> the Map<K,V> facade simply 'marshalls' values to byte arrays and calls the 'base' interface. >> >> This way, you get rid of any buffering from the very beginning and give streams to cache stores... (You actually replace byte arrays by piped(in/out)putstreams in your futures)... >> >> Btw, I can assure you that some cache store maintainers will be super happy to receive and produce streams ;-) >> >> my 2 cents... >> >> phil >> >> >> On Tue, May 18, 2010 at 3:59 PM, Manik Surtani <manik(a)jboss.org&gt; wrote: >> I have put together a brief design for ISPN-78. Please take a look, it is on the wiki: >> >> https://community.jboss.org/wiki/LargeObjectSupport >> >> I have also deferred ISPN-78 to 5.0.0 rather than 4.1.0 as I'd rather not hold up 4.1.0 for new features at this stage. >> >> Please have a look at the designs and let me know what you think - or comment on the wiki page. >> >> Cheers >> Manik >> -- >> Manik Surtani >> manik(a)jboss.org >> Lead, Infinispan >> Lead, JBoss Cache >> http://www.infinispan.org >> http://www.jbosscache.org >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

I have put together a brief design for ISPN-78. Please take a look, it is on the wiki: https://community.jboss.org/wiki/LargeObjectSupport I have also deferred ISPN-78 to 5.0.0 rather than 4.1.0 as I'd rather not hold up 4.1.0 for new features at this stage. Please have a look at the designs and let me know what you think - or comment on the wiki page. Cheers Manik -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 20 May 2010, at 16:14, Galder Zamarreno wrote: > The following ties in a bit with my comment yesterday about return values: > > If these two methods would always return null, you'd never be able to find out whether a remove/replace worked on its own. Returning previous value would indeed not work, but you could still return a marker different to null. The problem with markers is typing. Returning a boolean, for example, would make a lot of sense. But returning anything else would in effect be used *as a boolean*. Using any other return type for this would be abuse of the type system.

> > V remove(K key); // except that this will always return a null > V replace(K key, V value); // except that this will always return a null > > So, if remove worked cos the key was present, a marker value would be returned, otherwise if the key was not present, null would be returned. > > Cheers, > > ----- "Manik Surtani" <manik(a)jboss.org&gt; wrote: > >> I have put together a brief design for ISPN-78. Please take a look, >> it is on the wiki: >> >> https://community.jboss.org/wiki/LargeObjectSupport >> >> I have also deferred ISPN-78 to 5.0.0 rather than 4.1.0 as I'd rather >> not hold up 4.1.0 for new features at this stage. >> >> Please have a look at the designs and let me know what you think - or >> comment on the wiki page. >> >> Cheers >> Manik >> -- >> Manik Surtani >> manik(a)jboss.org >> Lead, Infinispan >> Lead, JBoss Cache >> http://www.infinispan.org >> http://www.jbosscache.org >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

I have put together a brief design for ISPN-78. Please take a look, it is on the wiki: https://community.jboss.org/wiki/LargeObjectSupport I have also deferred ISPN-78 to 5.0.0 rather than 4.1.0 as I'd rather not hold up 4.1.0 for new features at this stage. Please have a look at the designs and let me know what you think - or comment on the wiki page.