thoughts on CacheLoader concurrency
by Mircea Markus
Hi,
AbstractCacheLoader.int looks like this:
@Override
public void init(CacheLoaderConfig config, Cache<?, ?> cache, Marshaller m) throws CacheLoaderException {
this.marshaller = m;
if (config == null) throw new IllegalStateException("Null config!!!");
this.cache = cache;
}
This sets the non-volatile fields marshaller and cache. AbstractCacheLoader.int is called by the thread that starts the CacheManager which is different from e.g. expiration thread or other application threads. Shouldn't we safely publish these fields by making them volatile?
Cheers,
Mircea
14 years
Removing dependencies from parent
by Sanne Grinovero
Hello,
currently even to start a little Lucene/Infinispan demo Maven is
adding a huge set of dependencies to classpath,
this is scaring off.
For example the parent pom is forcing presence for commons-logging and
log4j on the compile and test classpath of all Infinispan modules;
while looking in the Infinispan logging adapter
org.infinispan.util.logging.LogFactory all modules should work even if
both logging frameworks are missing, falling back to Java's
java.util.logging.Logger. Of course this is not tested, as the current
build process is adding all loggers to the classpath.
So my proposal is to move out all dependencies in parent pom from
<dependencies> to <dependencyManagement>, submodules should explicitly
add the test dependencies they really needed;
are you ok with me doing this? Alternatively, I'm going to cut the
parent relationship from lucene modules to infinispan parent, but will
need to duplicate the good part of parent settings.
Regards,
Sanne
14 years
REST server war deployment issues
by galder＠redhat.com
Deploying rest server in Tomcat 6.0.26 throws the exception below.
It seems like that it can't find slf4j-api jar. Michael, did you try the war with any particular tomcat versions?
I managed to get around the issue by both removing the exclusion in the RESTEasy depends and adding slf4j-api and slf4j-simple dependencies.
java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at org.jboss.resteasy.plugins.server.servlet.ConfigurationBootstrap.<clinit>(ConfigurationBootstrap.java:27)
at org.jboss.resteasy.plugins.server.servlet.ResteasyBootstrap.contextInitialized(ResteasyBootstrap.java:26)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3972)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4467)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:546)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:905)
at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:740)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:500)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:785)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at org.apache.catalina.core.StandardService.start(StandardService.java:519)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
at org.apache.catalina.startup.Catalina.start(Catalina.java:581)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1516)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1361)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:398)
... 26 more
As a side note, deploying the original war I tried in a recent AS trunk fails with:
12:39:58,448 ERROR [[/infinispan-server-rest]] Exception starting filter Resteasy: java.lang.ClassCastException: org.jboss.resteasy.spi.ResteasyProviderFactory cannot be cast to org.jboss.resteasy.spi.ResteasyProviderFactory
at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:55) [:]
at org.jboss.resteasy.plugins.server.servlet.FilterDispatcher.init(FilterDispatcher.java:39) [:]
at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:462) [:]
at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:3286) [:]
at org.apache.catalina.core.StandardContext.start(StandardContext.java:3887) [:]
AS ships with a different version of RESTEasy library which would explain this issue (latest AS ships 2.0-beta-2 whereas the war includes 1.2.1.GA). However, AFAIR in the past WEB-INF/classes/lib was deployed under a different classloader and hence this shouldn't be a problem.
I'll create a couple of JIRAs for these.
Cheers,
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
14 years
R: Re: ISPN-359 and grouping entries for distribution
by Sanne Grinovero
Is this covering a different use case than having two caches k1 and k2
equally configured?
My current app Scarlet is dynamically creating many different caches using
the configuration names asd templates, I'm happy with that.
Il giorno 19/mag/2010 16:43, "Manik Surtani" <manik(a)jboss.org> ha scritto:
On 13 Apr 2010, at 11:01, Manik Surtani wrote:
>
> On 13 Apr 2010, at 10:50, Sanne Grinovero wrote:
>
>> rightfull concern, I wouldn't personally ha...
Here is an alternative - *do* we want to support scoping? By this, I mean:
cache.withAffinityKey("k1").put("name", "Manik");
cache.withAffinityKey("k1").put("country", "UK");
cache.withAffinityKey("k2").put("name", "Sanne");
cache.withAffinityKey("k2").put("country", "IT");
will be allowed and the two will not overwrite each other, *but* you cannot
retrieve stuff by simply doing:
cache.get("name") anymore. You would have to do:
cache.withAffinityKey("k1").get("name"). Simply doing a cache.get("name")
will return a null.
What do people prefer?
Cheers
Manik
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http:...
14 years
Re: [infinispan-dev] ISPN-359 and grouping entries for distribution
by galder＠jboss.org
See below:
----- "Manik Surtani" <manik(a)jboss.org> wrote:
> On 19 May 2010, at 17:44, galder(a)jboss.org wrote:
>
> >
> > ----- "Manik Surtani" <manik(a)jboss.org> wrote:
> >
> >> So thinking about this some more, it would seem that there is a
> bit
> >> more to this. We can add the functionality easily enough, and add
> the
> >> API to the Cache interface.
> >>
> >> But would people would reasonably expect this kind of
> functionality
> >> over Hot Rod as well? And would we need to think about exposing
> this
> >> in Hot Rod too? Or would this kind of functionality be implemented
> in
> >> the Hot Rod client if needed, so the protocol remains unchanged?
> >
> > Does the affinity mean that all k,v pairs within an affinity group
> would be stored under the same node in distribution mode? Or this a
> wrong assumption about how affinity works? Judging by
> https://jira.jboss.org/browse/ISPN-312, the grouping is more directed
> at flushing, but https://jira.jboss.org/browse/ISPN-359 suggests it
> could be used for distribution.
>
> Primarily for affinity with distribution. I.e., all entries under the
> same affinity key will be colocated on the same server.
>
> > Bearing in mind I need to clarify my understanding of affinity
> groups, baking it into the protocol means that is less work for
> clients in terms of implementations. If you have 3 client impls, you'd
> have to implement it in all 3. If it's baked in protocol, it's down to
> the server implementation.
>
> Right, but it also gives clients flexibility in *how* they implement
> affinity.
Not sure whether flexibility would be a good thing here? AFAIK, affinity on a group would be based on the hash of the group name and then find the spot in the wheel? I would imagine server side affinity would do a similar thing.
I can see however how letting clients how to deal with affinity could speed things up. If an intelligent client can figure out how to make all the data go to the server where it should be stored, it would be faster than handing it over to the server and the server then figuring out how to bundle that into the corresponding node.
>
> >
> >>
> >> On 13 Apr 2010, at 11:01, Manik Surtani wrote:
> >>
> >>>
> >>> On 13 Apr 2010, at 10:50, Sanne Grinovero wrote:
> >>>
> >>>> rightfull concern, I wouldn't personally have expected that but
> >> I'm
> >>>> biased as I follow this thread; it's not hard to imagine some
> >> people
> >>>> falling in this trap.
> >>>
> >>>
> >>> Yes; how do we make sure no one falls into this trap? :) How
> >> about:
> >>>
> >>> cache.affinityKey("Blah").put(k, v)
> >>>
> >>> The problem with "group" or even location/locality/colocation is
> >> that they can all be misconstrued to mean "scope". With something
> >> like "affinity", I suppose it is clearer?
> >>>
> >>> WDYT?
> >>>
> >>> Cheers
> >>> --
> >>> Manik Surtani
> >>> manik(a)jboss.org
> >>> Lead, Infinispan
> >>> Lead, JBoss Cache
> >>> http://www.infinispan.org
> >>> http://www.jbosscache.org
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev(a)lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >> --
> >> Manik Surtani
> >> manik(a)jboss.org
> >> Lead, Infinispan
> >> Lead, JBoss Cache
> >> http://www.infinispan.org
> >> http://www.jbosscache.org
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev(a)lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Manik Surtani
> manik(a)jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
14 years
Re: [infinispan-dev] ISPN-359 and grouping entries for distribution
by galder＠jboss.org
----- "Manik Surtani" <manik(a)jboss.org> wrote:
> So thinking about this some more, it would seem that there is a bit
> more to this. We can add the functionality easily enough, and add the
> API to the Cache interface.
>
> But would people would reasonably expect this kind of functionality
> over Hot Rod as well? And would we need to think about exposing this
> in Hot Rod too? Or would this kind of functionality be implemented in
> the Hot Rod client if needed, so the protocol remains unchanged?
Does the affinity mean that all k,v pairs within an affinity group would be stored under the same node in distribution mode? Or this a wrong assumption about how affinity works? Judging by https://jira.jboss.org/browse/ISPN-312, the grouping is more directed at flushing, but https://jira.jboss.org/browse/ISPN-359 suggests it could be used for distribution.
Bearing in mind I need to clarify my understanding of affinity groups, baking it into the protocol means that is less work for clients in terms of implementations. If you have 3 client impls, you'd have to implement it in all 3. If it's baked in protocol, it's down to the server implementation.
>
> On 13 Apr 2010, at 11:01, Manik Surtani wrote:
>
> >
> > On 13 Apr 2010, at 10:50, Sanne Grinovero wrote:
> >
> >> rightfull concern, I wouldn't personally have expected that but
> I'm
> >> biased as I follow this thread; it's not hard to imagine some
> people
> >> falling in this trap.
> >
> >
> > Yes; how do we make sure no one falls into this trap? :) How
> about:
> >
> > cache.affinityKey("Blah").put(k, v)
> >
> > The problem with "group" or even location/locality/colocation is
> that they can all be misconstrued to mean "scope". With something
> like "affinity", I suppose it is clearer?
> >
> > WDYT?
> >
> > Cheers
> > --
> > Manik Surtani
> > manik(a)jboss.org
> > Lead, Infinispan
> > Lead, JBoss Cache
> > http://www.infinispan.org
> > http://www.jbosscache.org
> >
> >
> >
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Manik Surtani
> manik(a)jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
14 years
Re: [infinispan-dev] ISPN-359 and grouping entries for distribution
by galder＠jboss.org
See below:
----- "Manik Surtani" <manik(a)jboss.org> wrote:
> On 13 Apr 2010, at 11:01, Manik Surtani wrote:
>
> >
> > On 13 Apr 2010, at 10:50, Sanne Grinovero wrote:
> >
> >> rightfull concern, I wouldn't personally have expected that but
> I'm
> >> biased as I follow this thread; it's not hard to imagine some
> people
> >> falling in this trap.
> >
> >
> > Yes; how do we make sure no one falls into this trap? :) How
> about:
> >
> > cache.affinityKey("Blah").put(k, v)
> >
> > The problem with "group" or even location/locality/colocation is
> that they can all be misconstrued to mean "scope". With something
> like "affinity", I suppose it is clearer?
>
> Here is an alternative - *do* we want to support scoping? By this, I
> mean:
>
> cache.withAffinityKey("k1").put("name", "Manik");
> cache.withAffinityKey("k1").put("country", "UK");
>
> cache.withAffinityKey("k2").put("name", "Sanne");
> cache.withAffinityKey("k2").put("country", "IT");
>
> will be allowed and the two will not overwrite each other, *but* you
> cannot retrieve stuff by simply doing:
>
> cache.get("name") anymore. You would have to do:
>
> cache.withAffinityKey("k1").get("name"). Simply doing a
> cache.get("name") will return a null.
>
> What do people prefer?
Not sure I like that. Reminds me of JBC's cache.getNode("k1").get("name"). I think affinity should be limited to location. If u want scoping, I think we should come up with something else, or direct users to use atomic hash maps.
>
> Cheers
> Manik
>
> --
> Manik Surtani
> manik(a)jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
14 years
Remote contains key instead remote get for unsafe return conditional operations?
by galder＠redhat.com
Hi,
Mircea and I have been discussing about https://jira.jboss.org/browse/ISPN-437 on IRC. In principle, using a flag like SKIP_REMOTE_LOOKUP to avoid remote lookups sounds like a good idea but as I said on the JIRA, I sometimes use those returns to find out whether the operation succeeded correctly, independent of the need of Hot Rod to send the previous value to the client.
Now, amongst others, I use returns of putIfAbsent, replace (2 versions), remove to find out whether the operation work correctly or not. However, although I need a return, as Mircea suggested, I don't need the actual previous value unless I need to send it back to the user. I need "something" that allows me to decide whether the operation worked fine.
If:
- we take putIfAbsent as an example,
- client does not require previous value (this could be translated into sending SKIP_REMOTE_LOOKUP)
- we're in a distributed environment and
- key being operated is not local.
Then: I don't need to get the actual value for the key from another node. I need a remote node to tell me if the key exists because that's enough for the client to know whether the key is present or not and see if the put succeeds or not. Something to clarify is what putIfAbsent would actually return if the key had existed, particularly if the key was remote and we don't have the previous value (maybe some kind of flag/dummy value?)
As a FYI, currently putIfAbsent would not be reliable in the scenario above unless transactions were used (I have already another thread to find out why transactions are required). If no transactions are used, pIA call would not retrieve the key from another node.
replace(k, v) would require similar treatment as putIfAbsent. replace(k, old, v) and remove(k, v) would not need such flag/dummy since they return boolean values.
Thoughts?
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
14 years
Re: [infinispan-dev] Hot Rod topology cache update retry timeout
by galder＠jboss.org
----- "Mircea Markus" <mircea.markus(a)jboss.com> wrote:
> Looks good.
> What is causing this unsuccessful add? If it is caused by timeouts due
> to multiple caches operating on the same key an alternative would be
> to only perform the operation on the coordinator and rest of the
> members to have node added listeners ...
Currently, each node when it starts, it's responsible of adding itself to the view and when it stops, it's responsible from removing itself. Apart from this, there's a crashed member listener running only in coordinator that detects whether any member left without updating the topology view. Your suggestion to have the coordinator control it all seems like could work and get around potential timeouts.
I'll create a JIRA to investigate this but won't do it for CR1 since I'm expect this to be a major issue. Metadata size is small and it's not constantly uddated.
> On 18 May 2010, at 19:03, Manik Surtani wrote:
>
> > Fine by me.
> >
> > On 18 May 2010, at 16:48, galder(a)redhat.com wrote:
> >
> >> Hi all,
> >>
> >> For https://jira.jboss.org/browse/ISPN-426 I've created a small
> retry logic for when there's an issue updating the topology
> information on startup. By default, I set the max wait time to 30
> seconds and I was wondering whether this should be made user
> configurable. This would a parameter into startServer.sh but it'd only
> be relevant to the Hot Rod server.
> >>
> >> The primary reason I'm considering not to make it user configurable
> is cos it's related to Hot Rod metadata which user should not really
> be aware of. If after 30 seconds, lightweight metadata hasn't been
> updated, something pretty wrong is going on and I doubt increasing it
> is going to make much difference.
> >>
> >> WDYT?
> >>
> >> Cheers,
> >> --
> >> Galder Zamarreño
> >> Sr. Software Engineer
> >> Infinispan, JBoss Cache
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev(a)lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> > --
> > Manik Surtani
> > manik(a)jboss.org
> > Lead, Infinispan
> > Lead, JBoss Cache
> > http://www.infinispan.org
> > http://www.jbosscache.org
> >
> >
> >
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
14 years
ISPN-359 and grouping entries for distribution
by Manik Surtani
Re: subject (see https://jira.jboss.org/jira/browse/ISPN-359), there are a couple of approaches that could be taken:
1. Don't use key.hashcode() as the seed in determining to which nodes an entry is mapped, but instead on a well-known method or annotated method (e.g., int getGroupID() or a method annotated with @GroupId). The way I see it, this approach has:
+ Will work, no additional overheads of AtomicMaps
- Cost (reflection)
- Intrusive (what if users have no control over the key class, e.g., String keys?)
2. Additional API methods on the cache - cache.put(K, V, G), cache.putAll(Map, G), etc.
+ Non-intrusive
- Overhead of AtomicMaps + additional entries for mappings
+ or - (depending on how you look at it) all keys in the group will be locked together, etc, a side-effect of using AtomicMaps
My pref is for approach #2. In terms of implementation, here is what I have in mind:
* A GroupingInterceptor that intercepts the call early on if the call is a put(K, V, G) or something similar.
* Breaks up the call to a put(K, G) and a getAtomicMap(G).put(K, V). Wrapped in a tx to ensure atomicity.
* get(K), etc intercepted as well, replaced with getAtomicMap(get(K)).get(K)
* remove(K), etc intercepted with getAtomicMap(get(K)).remove(K)
One of the issues with the API approach is that it heavily pollutes the Cache API. It will double the number of put() methods on Cache (currently 18 variants of put, including ones that take in lifespans and maxIdles, async versions that return futures, etc.) Perhaps this could be in an additional sub-interface interface? GroupedCache? Or is this degree of method overloading not too confusing?
Cheers,
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
14 years
