Hi, I've committed[1] the interfaces for the key-affinity service in ISPN-232. Would you mind taking a look and let me know what you think? Thanks! Mircea [1] http://anonsvn.jboss.org/repos/infinispan/trunk/core/src/main/java/org/in... _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 24 May 2010, at 14:10, Manik Surtani wrote: > Hi there > > * Do you have an example usage? I presume the entry point here is the KeyAffinityServiceFactory? Perhaps some usage examples in the KeyAffinityServiceFactory javadoc would be best, both for now and in future to document the API. added sample usage in KeyAffinityService together with some other aspects. > > * KeyAffinityService.getCollocatedKey() -> KeyAffinityService.getColocatedKey() (typo) The bundled IDEA speller (I'm not able to configure a locale for it) sees "collocated" as the right spelling. Also the "New English American Dictionary" on my mac book only recognises "collocated". Is this some of UK/US difference in spelling?

> * KeyAffinityService usage examples? Again best if this is in Javadoc added > * Perhaps KeyGenerator is a better name for KeyProvider? Since keys are, in fact, generated rather than retrieved from elsewhere/looked up +1 > * KeyProvider/KeyGenerator to be parameterized with <K> as well? +1 > * Methods on interfaces don't need "public". :) [1] live and learn :) > > Cheers > Manik > > [1] http://java.sun.com/docs/books/jls/third_edition/html/interfaces.html#9.4 - "It is permitted, but strongly discouraged as a matter of style, to redundantly specify the public modifier for interface methods." > > On 24 May 2010, at 11:15, Mircea Markus wrote: > >> Hi, >> >> I've committed[1] the interfaces for the key-affinity service in ISPN-232. >> Would you mind taking a look and let me know what you think? >> >> Thanks! >> Mircea >> >> [1] http://anonsvn.jboss.org/repos/infinispan/trunk/core/src/main/java/org/in... >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Hi, I've committed[1] the interfaces for the key-affinity service in ISPN-232. Would you mind taking a look and let me know what you think? Thanks! Mircea [1] http://anonsvn.jboss.org/repos/infinispan/trunk/core/src/main/java/org/in... _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 24 May 2010, at 19:30, Brian Stansberry wrote: > Thanks, Mircea: > > A related use case I need to cover[1] is checking whether an existing > key still hashes to the local node and if not, where it hashes. This may > not belong in KeyAffinityService, but I'd like it in some place easily > accessible to the same caller the creates KeyAffinityService. > > Set<Address> getAddressesForKey(K key) > > Usage: > > Set<Address> addresses = getAddressesForKey(sessionId); > if (!addresses.contains(localAddress)) { > redirectRequest(addresses); > } > > getAddressesForKey() could return a List<Address> but then checking > whether the local address is OK would require a scan. AIUI there > ordering in the set of DIST nodes is not meaningful, so a Set should be > fine. there's already something similar in the cache API: cache.getAdvancedCache().getDistributionManager().locate(Object key)::List<Address>

If you have many sessions(which I think is the case), then transforming the List in a Set is quicker indeed.

Taking the idea one step further, this would be an possibly useful service as well: a notification service, to which you register keys and listeners, and the service would call the listener whenever key's distribution is changed. Wdyt?

> > Re: K getCollocatedKey(K otherKey) please note that for the AS use cases > I know of where colocation is helpful, I can't help but notice your American spelling of "colocation". I know, I know, you won a War :) > the colocated data will be stored > in separate Cache instances, while the KAS is scoped to a cache. The > method should still be helpful though, as long as the caches use the > same underlying CacheManager/Channel. It's up to the caller to check that. > agreed. All the caches associated to a CM share the same transport. They can potentially have different ConsistentHash functions though, that's why the service runs per Cache instance, rather than running per CacheManager instance. > On KeyAffinityServiceFactory, there's the bufferSize param: > > * @param keyBufferSize the number of generated keys per {@link > org.infinispan.remoting.transport.Address}. > > That implies the factory will maintain a buffer of keys for *all* > addresses, not just the local address. The use cases I'm concerned with, > the only address for which I want keys is the local one. Perhaps it > makes sense to offer a separate param for the local address buffer size? > > In a small cluster maintaining some keys in a buffer for irrelevant > addresses is no big deal, but for large clusters it may waste a fair > amount of memory. Good point! what about adding a new parameter to the factory's methods: Collection<Address> - to specify the list of addresses for which keys will be generated.

And another factory method that would transparently pass the local address to the previous method so that the caller won't be concerned with getting the local address.

Cheers, Mircea > > [1] https://jira.jboss.org/browse/JBAS-7853 plus a similar requirement > for EJB3 as discussed on https://community.jboss.org/docs/DOC-15052 > > > On 05/24/2010 05:15 AM, Mircea Markus wrote: >> Hi, >> >> I've committed[1] the interfaces for the key-affinity service in >> ISPN-232. >> Would you mind taking a look and let me know what you think? >> >> Thanks! >> Mircea >> >> [1] >> http://anonsvn.jboss.org/repos/infinispan/trunk/core/src/main/java/org/in... >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Brian Stansberry > Lead, AS Clustering > JBoss by Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 24 May 2010, at 20:55, Brian Stansberry wrote: > On 05/24/2010 01:39 PM, Mircea Markus wrote: >> On 24 May 2010, at 19:30, Brian Stansberry wrote: >> >>> Thanks, Mircea: >>> >>> A related use case I need to cover[1] is checking whether an existing >>> key still hashes to the local node and if not, where it hashes. This may >>> not belong in KeyAffinityService, but I'd like it in some place easily >>> accessible to the same caller the creates KeyAffinityService. >>> >>> Set<Address> getAddressesForKey(K key) >>> >>> Usage: >>> >>> Set<Address> addresses = getAddressesForKey(sessionId); >>> if (!addresses.contains(localAddress)) { >>> redirectRequest(addresses); >>> } >>> >>> getAddressesForKey() could return a List<Address> but then checking >>> whether the local address is OK would require a scan. AIUI there >>> ordering in the set of DIST nodes is not meaningful, so a Set should be >>> fine. >> there's already something similar in the cache API: >> cache.getAdvancedCache().getDistributionManager().locate(Object >> key)::List<Address> > > Haha, I figured there was something, but was lazy. Thanks! > >> If you have many sessions(which I think is the case), then transforming >> the List in a Set is quicker indeed. > > Yes, this could be pretty heavily invoked. Although... > >> Taking the idea one step further, this would be an possibly useful >> service as well: a notification service, to which >> you register keys and listeners, and the service would call the listener >> whenever key's distribution is changed. >> Wdyt? > > That would be helpful. I was thinking this morning about how to avoid > calling getAddressesForKey() all the time (i.e. every request) and was > thinking of using a view change notification as a trigger to mark all > sessions as needing a check. What you propose would be more efficient > for me. Maybe, but what you described is precisely what we'd do, except that it would be the rehashing thread that does this. > But, I'd be concerned about an Infinispan thread that's needed > for doing a lot of critical work during a view change getting tied up > making a ton of notifications. ^^ Yeah that's my concern. If during a rehash we need to stop at every entry that is being moved and issue a notification, that could be costly and really slow down the rehashing process.

> >>> >>> Re: K getCollocatedKey(K otherKey) please note that for the AS use cases >>> I know of where colocation is helpful, >> I can't help but notice your American spelling of "colocation". I know, >> I know, you won a War :) >>> the colocated data will be stored >>> in separate Cache instances, while the KAS is scoped to a cache. The >>> method should still be helpful though, as long as the caches use the >>> same underlying CacheManager/Channel. It's up to the caller to check that. >>> >> agreed. All the caches associated to a CM share the same transport. They >> can potentially have different ConsistentHash functions though, that's >> why the service runs per Cache instance, rather than running per >> CacheManager instance. >>> On KeyAffinityServiceFactory, there's the bufferSize param: >>> >>> * @param keyBufferSize the number of generated keys per {@link >>> org.infinispan.remoting.transport.Address}. >>> >>> That implies the factory will maintain a buffer of keys for *all* >>> addresses, not just the local address. The use cases I'm concerned with, >>> the only address for which I want keys is the local one. Perhaps it >>> makes sense to offer a separate param for the local address buffer size? >>> >>> In a small cluster maintaining some keys in a buffer for irrelevant >>> addresses is no big deal, but for large clusters it may waste a fair >>> amount of memory. >> Good point! what about adding a new parameter to the factory's methods: >> Collection<Address> - to specify the list of addresses for which keys >> will be generated. > > Perhaps take Map<Address addr, Integer keyBufferSize> ? > > TBH, I can't think of a use case where you'd use different size buffers > (other than 0) for different addresses, but a map gives more imaginative > people flexibility. > > Hmm, for any use case where people are interested in particular > addresses other than the local address, they are vulnerable to view > changes introducing unexpected members. So, to be complete perhaps be a > default buffer size for members not specifically listed: > > public static <K,V> KeyAffinityService<K> > newKeyAffinityService(Cache<K,V> cache, ExecutorFactory ex, KeyGenerator > keyGenerator, Map<Address, Integer> keyBufferSizes, int > defaultKeyBufferSize) { > > Kind of complex! :-) > >> And another factory method that would transparently pass the local >> address to the previous method so that the caller won't be concerned >> with getting the local address. >> > > Sure, if you can come up with a signature for that isn't confusing vs. > the other factory methods. Otherwise I don't mind getting the local address. > >> Cheers, >> Mircea >>> >>> [1] https://jira.jboss.org/browse/JBAS-7853 plus a similar requirement >>> for EJB3 as discussed on https://community.jboss.org/docs/DOC-15052 >>> >>> >>> On 05/24/2010 05:15 AM, Mircea Markus wrote: >>>> Hi, >>>> >>>> I've committed[1] the interfaces for the key-affinity service in >>>> ISPN-232. >>>> Would you mind taking a look and let me know what you think? >>>> >>>> Thanks! >>>> Mircea >>>> >>>> [1] >>>> http://anonsvn.jboss.org/repos/infinispan/trunk/core/src/main/java/org/in... >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> -- >>> Brian Stansberry >>> Lead, AS Clustering >>> JBoss by Red Hat >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Brian Stansberry > Lead, AS Clustering > JBoss by Red Hat > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 25 May 2010, at 14:15, Mircea Markus wrote: >>> >>> But, I'd be concerned about an Infinispan thread that's needed >>> for doing a lot of critical work during a view change getting tied up >>> making a ton of notifications. >> >> ^^ Yeah that's my concern. If during a rehash we need to stop at every entry that is being moved and issue a notification, that could be costly and really slow down the rehashing process. > Can't we register an *async* notification listener on ViewChanged ? You'd still have a *lot* of notifications being queued up for the notification executor since you will have 1 event *per entry* that is moved.

Could be thousands, tens of thousands in cases. All other notifications will be severely delayed (depending on your async notifier executor threadpool size)

-- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 25 May 2010, at 15:09, Mircea Markus wrote: > > On 25 May 2010, at 16:26, Manik Surtani wrote: > >> >> On 25 May 2010, at 14:15, Mircea Markus wrote: >> >>>>> >>>>> But, I'd be concerned about an Infinispan thread that's needed >>>>> for doing a lot of critical work during a view change getting tied up >>>>> making a ton of notifications. >>>> >>>> ^^ Yeah that's my concern. If during a rehash we need to stop at every entry that is being moved and issue a notification, that could be costly and really slow down the rehashing process. >>> Can't we register an *async* notification listener on ViewChanged ? >> >> You'd still have a *lot* of notifications being queued up for the notification executor since you will have 1 event *per entry* that is moved. > this would still happen in the same JVM as this information is needed. Just that we offer this as a service, so that users(AS being one of them) won't have to write this code. No, if the AS does it, it will be one notification (view change) + a scan of known session ids (keys). If we do it, it will involve 1 notification *per key* being migrated.

The cost of each notification is the construction and initialization of an event object - which includes cloning the invocation context - and placing it on an executor queue. >> Could be thousands, tens of thousands in cases. >> All other notifications will be severely delayed (depending on your async notifier executor threadpool size) > If you have an big enough thread pool (actually at least 2 threads) this should be no issue - and this should be made clear to the user. Executor queue size? This could start blocking? >> >> >> -- >> Manik Surtani >> manik(a)jboss.org >> Lead, Infinispan >> Lead, JBoss Cache >> http://www.infinispan.org >> http://www.jbosscache.org >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 25 May 2010, at 15:34, Mircea Markus wrote: > > On 25 May 2010, at 17:22, Manik Surtani wrote: > >> >> On 25 May 2010, at 15:09, Mircea Markus wrote: >> >>> >>> On 25 May 2010, at 16:26, Manik Surtani wrote: >>> >>>> >>>> On 25 May 2010, at 14:15, Mircea Markus wrote: >>>> >>>>>>> >>>>>>> But, I'd be concerned about an Infinispan thread that's needed >>>>>>> for doing a lot of critical work during a view change getting tied up >>>>>>> making a ton of notifications. >>>>>> >>>>>> ^^ Yeah that's my concern. If during a rehash we need to stop at every entry that is being moved and issue a notification, that could be costly and really slow down the rehashing process. >>>>> Can't we register an *async* notification listener on ViewChanged ? >>>> >>>> You'd still have a *lot* of notifications being queued up for the notification executor since you will have 1 event *per entry* that is moved. >>> this would still happen in the same JVM as this information is needed. Just that we offer this as a service, so that users(AS being one of them) won't have to write this code. >> >> No, if the AS does it, it will be one notification (view change) + a scan of known session ids (keys). If we do it, it will involve 1 notification *per key* being migrated. > I don't intend to register more than one ViewChange listener per service. Then AS (or other clients) can register as many keys with that service instance (and consequently with that single listener) and, in that listener to iterate over the keys etc. You may just register 1 listener, but the event gets dispatched by the rehasher thread, which does:

1. Loop thru all entries. For each entry in the cache: 2. Determine if key needs to be moved, given the old topology and the new one after considering a new joiner. If the key needs to be moved: 3. Add key to digest of entries to be moved. 4. Transfer state Now if we were to notify listeners of each change, this would happen in the loop, after the condition, around the same time as step 3. Which means a large number of events generated and dispatched (even with just 1 listener). Unless you are suggesting that the listener gets a set of keys (extracted from the digest, after the loop) in a single notification event? > As per my understanding, Brian's concern was that the list of keys can be quite large (all the web sessions) and that would delay the rehashing thread - this is now mitigated by using @ViewChange(async=true). >> The cost of each notification is the construction and initialization of an event object - which includes cloning the invocation context - and placing it on an executor queue. >> >> >>>> Could be thousands, tens of thousands in cases. >>>> All other notifications will be severely delayed (depending on your async notifier executor threadpool size) >>> If you have an big enough thread pool (actually at least 2 threads) this should be no issue - and this should be made clear to the user. >> >> Executor queue size? This could start blocking? >> >>>> >>>> >>>> -- >>>> Manik Surtani >>>> manik(a)jboss.org >>>> Lead, Infinispan >>>> Lead, JBoss Cache >>>> http://www.infinispan.org >>>> http://www.jbosscache.org >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> -- >> Manik Surtani >> manik(a)jboss.org >> Lead, Infinispan >> Lead, JBoss Cache >> http://www.infinispan.org >> http://www.jbosscache.org >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 25 May 2010, at 16:06, Mircea Markus wrote: > > On 25 May 2010, at 17:44, Manik Surtani wrote: > >> >> On 25 May 2010, at 15:34, Mircea Markus wrote: >> >>> >>> On 25 May 2010, at 17:22, Manik Surtani wrote: >>> >>>> >>>> On 25 May 2010, at 15:09, Mircea Markus wrote: >>>> >>>>> >>>>> On 25 May 2010, at 16:26, Manik Surtani wrote: >>>>> >>>>>> >>>>>> On 25 May 2010, at 14:15, Mircea Markus wrote: >>>>>> >>>>>>>>> >>>>>>>>> But, I'd be concerned about an Infinispan thread that's needed >>>>>>>>> for doing a lot of critical work during a view change getting tied up >>>>>>>>> making a ton of notifications. >>>>>>>> >>>>>>>> ^^ Yeah that's my concern. If during a rehash we need to stop at every entry that is being moved and issue a notification, that could be costly and really slow down the rehashing process. >>>>>>> Can't we register an *async* notification listener on ViewChanged ? >>>>>> >>>>>> You'd still have a *lot* of notifications being queued up for the notification executor since you will have 1 event *per entry* that is moved. >>>>> this would still happen in the same JVM as this information is needed. Just that we offer this as a service, so that users(AS being one of them) won't have to write this code. >>>> >>>> No, if the AS does it, it will be one notification (view change) + a scan of known session ids (keys). If we do it, it will involve 1 notification *per key* being migrated. >>> I don't intend to register more than one ViewChange listener per service. Then AS (or other clients) can register as many keys with that service instance (and consequently with that single listener) and, in that listener to iterate over the keys etc. >> >> You may just register 1 listener, but the event gets dispatched by the rehasher thread, which does: > I was thinking about something totally different. > 1. The NotificationService (NS) is started. Internally it registers a single listener: @ViewChange(async=true) > 2. The user has registers to NS the set of keys for which he wants to be notified on topology changes. Whenever a key is registered, the NS calculates and holds current key address based on the CH. > 3. View change happens, NS is notified with a thread from the async pool > 4. It iterates over the set of registered keys and checks weather the prev address has changed. That's a whole alternate notification system to what we have. :)

-1 to supporting 2 such systems. >> >> 1. Loop thru all entries. For each entry in the cache: >> 2. Determine if key needs to be moved, given the old topology and the new one after considering a new joiner. If the key needs to be moved: >> 3. Add key to digest of entries to be moved. >> 4. Transfer state >> >> Now if we were to notify listeners of each change, this would happen in the loop, after the condition, around the same time as step 3. Which means a large number of events generated and dispatched (even with just 1 listener). >> >> Unless you are suggesting that the listener gets a set of keys (extracted from the digest, after the loop) in a single notification event? >> >> >>> As per my understanding, Brian's concern was that the list of keys can be quite large (all the web sessions) and that would delay the rehashing thread - this is now mitigated by using @ViewChange(async=true). >>>> The cost of each notification is the construction and initialization of an event object - which includes cloning the invocation context - and placing it on an executor queue. >>>> >>>> >>>>>> Could be thousands, tens of thousands in cases. >>>>>> All other notifications will be severely delayed (depending on your async notifier executor threadpool size) >>>>> If you have an big enough thread pool (actually at least 2 threads) this should be no issue - and this should be made clear to the user. >>>> >>>> Executor queue size? This could start blocking? >>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Manik Surtani >>>>>> manik(a)jboss.org >>>>>> Lead, Infinispan >>>>>> Lead, JBoss Cache >>>>>> http://www.infinispan.org >>>>>> http://www.jbosscache.org >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> infinispan-dev mailing list >>>>>> infinispan-dev(a)lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>> >>>>> >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev(a)lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> -- >>>> Manik Surtani >>>> manik(a)jboss.org >>>> Lead, Infinispan >>>> Lead, JBoss Cache >>>> http://www.infinispan.org >>>> http://www.jbosscache.org >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> -- >> Manik Surtani >> manik(a)jboss.org >> Lead, Infinispan >> Lead, JBoss Cache >> http://www.infinispan.org >> http://www.jbosscache.org >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev