Infinispan S3 Cluster + Store Success ! (+serious performance problems)
by philippe van dyck
Hi Infinispan mailing list,
a couple of days ago, I succeeded in writing an entity store for qi4j (http://www.qi4j.org/) using Infinispan, the S3 store and the S3_PING JGroups clustering configuration.
It works like a charm, discovers new EC2 instances, synchronizes and process transactions perfectly... you did an amazing job.
But I have a serious performance problems.
When I write an update (<1k) to the cache, it takes around 500 ms to be stored on S3.
The best result I achieved was around 10 cache writes per second... it is abysmal (when using httpclient directly I had a min of 100/sec using 20 connections).
When I commit a JTA transaction made of 100 cache writes, it takes around 30 seconds (cpu<5%) and the first write ends on S3 after at least 5 seconds of 'idle' time (SSL negotiation??).
I disabled the store asynchronism and work without JTA transactions, no effect on performance.
I also modified the jClouds configuration, multiplying by 10 all worker threads, connections and the rest... no improvement!
When I (load) test my web app (wicket based+qi4j+...infinispan) the cpu stays idle (<5%) and ... JTA transactions fails (timeouts) and I cannot acquire locks before the 10 seconds timeout.
Is there something fishy in the jclouds configuration ? in the httpnio use of jclouds ? in the version of jclouds (the trunk one with the blob store seems to be so different) ?
Am I missing something ?
Any pointer to any doc/help/experience is welcome ;-)
Philippe
15 years
Configuration doc generation
by Manik Surtani
Just having a look at the latest generated config.html from a snapshot (attached), and comparing it with the equivalent, programmatic approach via the Configuration bean (again, javadoc attached), I see that there is no cross reference (nor any javadoc being generated for most of the config bean elements). Perhaps some mechanism to hyperlink between the two?
Vladimir, I believe you had some thoughts around this?
Cheers
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
15 years
Preparing for Infinispan 4.0.0.CR3
by Manik Surtani
I was hoping to do a GA soon but I think we should cut at least 1 more CR first. There are some critical issues I want to fix for this, namely:
ISPN-277 (Me)
ISPN-249 (Me)
ISPN-280 (Sanne)
ISPN-288 (Navin)
Nice to have:
ISPN-281 (Mic Neale)
ISPN-276 (Sanne)
ISPN-283 (Me)
ISPN-287 (Navin)
How do people feel about finishing their respective JIRAs by close of play tomorrow (Wed) for a Thu morning release?
Cheers
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
15 years
Ability to retrieve a JChannel via multiplexer stack name, where is it gone?
by Galder Zamarreno
Hi,
Re: "* I'm adding the Infinispan 2nd level cache provider to existing
servers that already use JGroups. Should I set Infinispan to use the
same JGroups cluster, or should I use two separate cluster names?" FAQ
Where is the ability to get a JChannel via multiplexerStack gone? I
don't see any reference to such XML configuration property. Is this
supposed to be hiding behind an
org.infinispan.remoting.transport.jgroups.JGroupsChannelLookup
implementation? I can only see a dummy implementation for it currently.
Cheers,
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
15 years
unresolved dependency?
by Michael Neale
I am trying to do an install (clean) and I get the following problem:
[INFO] Unable to find resource
'org.rhq.helpers:rhq-pluginAnnotations:jar:1.4.0-SNAPSHOT' in
repository scala-tools.org (http://scala-tools.org/repo-releases)
..
[ERROR] BUILD ERROR
..
Missing:
----------
1) org.rhq.helpers:rhq-pluginAnnotations:jar:1.4.0-SNAPSHOT
..
Path to dependency:
1) org.infinispan:infinispan-core:jar:4.0.0-SNAPSHOT
2) org.rhq.helpers:rhq-pluginAnnotations:jar:1.4.0-SNAPSHOT
Is this a recent addition - am I unlucky and a repo is down or did
someone forget to add a repo to the parent pom? (or what I am doing
wrong)?
(been getting a tan for a few weeks, so might have missed something !).
--
Michael D Neale
home: www.michaelneale.net
blog: michaelneale.blogspot.com
15 years, 1 month
Adding isRunning() to Lifecycle?
by Sanne Grinovero
Hello,
what would you think of adding an isRunning() method to Lifecycle interface?
public interface Lifecycle {
void start();
void stop();
boolean isRunning(); <-- not existing
}
?
I would use it to throw easy to understand errors, like when trying to
use a cache which was shutdown.
I'm doing it already but code isn't nice and repeated in some places.
Sanne
15 years, 1 month
Entry grouping feature
by Mindaugas Žakšauskas
Hi there,
I am in process of reviewing Infinispan as, among other things, a
replacement of OSCache library we currently use. So far, it looks very
promising project - well done, guys!
Anyway, one of the features we heavily rely on seems to be missing.
The feature is called "grouping", allowing to "bind" multiple
components and possibly flush them even without knowing of their keys.
E.g.:
Cache cache = ....;
cache.put("key1", "value1", "groupX");
cache.put("key2", "value2", "groupX");
cache.flushGroup("groupX"); // this would evict both key1 and key2
as they belong to groupX
For more details, you can have a look at [1].
I've looked at the Infinispan's roadmap/JIRA, but couldn't find a
single trace of this feature mentioned anywhere.
My question is, how hard would be to implement this feature without
having much knowledge of Infinispan internals? I've spoken to Manik
Surtani at Devoxx'09 and he guessed this could be done by using
interceptors. I've had a sneak peek at the interceptors package, but
obviously this is still pretty low level stuff, and apart from
javadocs, documentation resources are rather scarce.
Chances are, I could get some time assigned to implement this feature
myself and contribute the code back to the community, but before doing
anything I'd like to know how feasible is this in general. For
example, what would be the best way to fit these parameters in the
current API? How would this work cluster-wise?
Any ideas are appreciated. Thanks in advance!
Regards,
Mindaugas
[1] http://www.opensymphony.com/oscache/api/com/opensymphony/oscache/base/Cac...
15 years, 1 month
Infinispan and Google app engine
by Vladimir Blagojevic
Hi all,
Manik asked me to investigate what it would take to deploy Infinispan
inside google app engine (GAE). As you might have heard GAE execution
environment is somewhat constricted (can not create threads, write to
files, open sockets, etc). Right from a get go it was clear that we can
not run Infinispan in any mode other than local. However, even if
Infinispan is run in local mode GAE users could benefit from
Infinispan's JSR-107 cache interface that is backed up with cache store
implemented using google datastore api. There probably are use case
scenarios where GAE users interact with both google provided JSR-107
memcache service [1] and datastore [2] separately where in fact
Infinispan's JSR-107 interface that is automatically synced with
datastore would make much more sense.
Manik and I went back and forth and decided to put this sub project on a
back burner at this moment. Even though we could accommodate GAE's
thread restriction it turns out that GAE sandbox restrictions involve
several apis that we use (JAXB, JMX, JNDI among others) and retrofitting
Infinispan into this tight sandbox would be a headache. There is a
strong comunity push to unrestrict these API and it seems pressures from
community would eventually be successful.
Also, I am still awaiting for some feedback from Mircea to see if we can
even implement google datastore cache loader to be transactional. Even
though GAE datastore supports transactions they are only available in
the same entity group [3]. As far as I understand GAE datastore they
implement horizontal partitioning which basically involves putting
different rows into different tables. Think each Person row entry would
have to be in different table unless they are in some owns (say
parent-child) relationship.
Feedback appreciated,
Vladimir
[1] http://code.google.com/appengine/docs/java/memcache/usingjcache.html
[2] http://code.google.com/appengine/docs/java/datastore/
[3] http://www.youtube.com/watch?v=2jW2iSKDipY (between 6:45 and 11:40)
15 years, 1 month
looking for a volunteer to re-integrate with infinispan
by Adrian Cole
Hi, team.
As you may know, we are used as a cachestore for the Infinispan data
grid. That product uses the beta-2 s3 plugin. We have outstanding
work to integrate blobstore, which will allow infinispan to use any
cloud, like rackspace, azure, instead of just s3. I'm pretty swamped
at the moment, but could certainly guide someone, if you can take the
lead on this. With your help, we can make infinispan the first
multi-cloud data grid.
Cheers,
-Adrian
15 years, 1 month
