jclouds 1.0-beta-4 released
by Adrian Cole
Our last beta of jclouds included the new Compute API and multi-cloud
support. Since then, we found found and fixed usage and code concerns
raised during testing from crane and Infinispan.
There have been no updates to the BlobStore API of any significance.
The Compute API has changed and broken compatibility with previous
versions. In particular:
1. The NodeSet object has been dropped for the more consistently
used Map, for tag-based operations.
2. The Compute API no longer attempts to resolve errors when a node
doesn't startup. We've added the option destroyOnError, should you
wish to kill deficient nodes by default. [1]
Here are the new features included in beta-4 of jclouds.
1. A new pooling http engine based on Apache HC Client 4.0.
2. Reboot support in ComputeService.
Known limitations of beta-4 are that virtual directories do not
currently work via VFS.
Thanks for the great feedback and keep it coming!
Cheers,
Adrian
http://anyweight.blogspot.com/2010/02/10-beta-4-released.html
[1] When destroyOnError is set, any nodes who error on executing
runscripts or installing keys will be destroyed. By default, errors
are allowed to occur, and the user must Q/A the state of the node when
using key or runscript features.
14 years, 4 months
eclipse compiler bug prevents Infinispan core compilation
by Sanne Grinovero
Hi all,
the eclipse compiler is not as clever as Intellij's one; since this change
http://fisheye.jboss.org/browse/Infinispan/trunk/core/src/main/java/org/i...
Infinispan core is not building any more in eclipse; could I revert
these changes, or change the <? extends T> types to <T> in method
signatures?
for example:
public static <T> Set<T> immutableSetCopy(Set<? extends T> set) {
could be changed to:
public static <T> Set<T> immutableSetCopy(Set<T> set) {
Alternatives:
1) change the signature
2) revert the change
3) introducing an explicit cast like:
return Collections.singleton(set.iterator().next());
to
return (Set<T>) Collections.singleton(set.iterator().next());
The commit log from Manik sais "IntelliJ Inspection refactoring", and
while he changed several classes I'm only having trouble with this one
class, in three methods.
Of course, this is not important as it doesn't affect the real build
and there is an issue open on the eclipse tracker;
I'm also switching slowly to intelliJ and building from commandline
maven, but could be annoying for others.
Sanne
14 years, 4 months
Re: [infinispan-dev] Initial LIRS implementation
by Bryan Thompson
Vladimir,
I believe that there is a lock problem with the BufferedConcurrentHashMap. This code path does not progress. What is strange is that this is the only thread in the thread dump that is in BufferedConcurrentHashMap, which suggests a lock ordering problem. I don't see the problem offhand, but this happens everytime I run our application with the BufferedConcurrentHashMap.
Bryan
"Thread-3" prio=10 tid=0x00002aafec141000 nid=0x41ba runnable [0x0000000040737000]
java.lang.Thread.State: RUNNABLE
at sun.misc.Unsafe.$$YJP$$compareAndSwapInt(Native Method)
at sun.misc.Unsafe.compareAndSwapInt(Unsafe.java)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.compareAndSetState(AbstractQueuedSynchronizer.java:526)
at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:183)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
at org.infinispan.util.concurrent.BufferedConcurrentHashMap$Segment.remove(BufferedConcurrentHashMap.java:745)
at org.infinispan.util.concurrent.BufferedConcurrentHashMap$Segment$LRU.execute(BufferedConcurrentHashMap.java:832)
at org.infinispan.util.concurrent.BufferedConcurrentHashMap$Segment.put(BufferedConcurrentHashMap.java:659)
at org.infinispan.util.concurrent.BufferedConcurrentHashMap.putIfAbsent(BufferedConcurrentHashMap.java:1421)
at com.bigdata.cache.BCHMGlobalLRU$InnerCacheImpl.putIfAbsent(BCHMGlobalLRU.java:633)
at com.bigdata.cache.BCHMGlobalLRU$InnerCacheImpl.putIfAbsent(BCHMGlobalLRU.java:557)
at com.bigdata.btree.AbstractBTree.readNodeOrLeaf(AbstractBTree.java:3624)
________________________________________
From: Bryan Thompson
Sent: Wednesday, February 03, 2010 5:04 PM
To: Vladimir Blagojevic
Cc: infinispan -Dev List
Subject: RE: [infinispan-dev] Initial LIRS implementation
Vladimir,
Does this match up with what you are seeing? That is a good win for get()/remove() with a loss on put().
Size = 14698
Performance for container ConcurrentHashMap
Average get ops/ms 13931
Average put ops/ms 306
Average remove ops/ms 294
Size = 452
Performance for container BufferedConcurrentHashMap(LRU)
Average get ops/ms 7735
Average put ops/ms 140
Average remove ops/ms 200
Size = 494
Performance for container BufferedConcurrentHashMap(LIRS)
Average get ops/ms 13400
Average put ops/ms 36
Average remove ops/ms 1136
Bryan
> -----Original Message-----
> From: Vladimir Blagojevic [mailto:vblagoje@redhat.com]
> Sent: Wednesday, February 03, 2010 3:36 PM
> To: Bryan Thompson
> Subject: Re: [infinispan-dev] Initial LIRS implementation
>
> Yes,
>
> Have a look at LRU first and leave LIRS for tomorrow :) I'd
> be happiest if you can find a use case for BCHM+LRU in your
> application and test it out that way. Real life scenario!
>
> Cheers
>
> On 2010-02-03, at 3:27 PM, Bryan Thompson wrote:
>
> > Ah. It is inside the same outer class? Bryan
> >
> >
> >> -----Original Message-----
> >> From: Vladimir Blagojevic [mailto:vblagoje@redhat.com]
> >> Sent: Wednesday, February 03, 2010 2:44 PM
> >> To: Bryan Thompson
> >> Subject: Re: [infinispan-dev] Initial LIRS implementation
> >>
> >> Here it is:
> >> http://fisheye.jboss.org/browse/Infinispan/trunk/core/src/main
> >> /java/org/infinispan/util/concurrent/BufferedConcurrentHashMap.java
> >>
> >> Are you familiar with LIRS? If not, don't bother unless you are
> >> willing to dedicate at least a day or two :(
> >>
> >> http://www.ece.eng.wayne.edu/~sjiang/Projects/LIRS/sig02.ppt
> >> http://citeseer.ist.psu.edu/527790.html
> >>
> >> Regards,
> >> Vladimir
> >>
> >>
> >> On 2010-02-03, at 2:31 PM, Bryan Thompson wrote:
> >>
> >>> Can you send me the full class name and I will check it out. Bryan
> >>>
> >>>> -----Original Message-----
> >>>> From: infinispan-dev-bounces(a)lists.jboss.org
> >>>> [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf
> >> Of Vladimir
> >>>> Blagojevic
> >>>> Sent: Wednesday, February 03, 2010 2:09 PM
> >>>> To: infinispan -Dev List
> >>>> Subject: [infinispan-dev] Initial LIRS implementation
> >>>>
> >>>> Hi,
> >>>>
> >>>> I've just committed preliminary attempt to implement LIRS.
> >>>> There does not seem to be serious degradation in terms of
> >> performance
> >>>> when it comes to get/remove commands in comparison with
> >> LRU enabled
> >>>> BufferedConcurrentHashMap.
> >>>> However, put command is about three times slower than in
> >> LRU; put in
> >>>> LIRS is as fast as put command of a single lock
> >> synchronized HashMap.
> >>>>
> >>>> Looking for enthusiasts willing to help out with some code
> >> review :)
> >>>>
> >>>> Regards,
> >>>> Vladimir
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> infinispan-dev mailing list
> >>>> infinispan-dev(a)lists.jboss.org
> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>>
> >>
> >>
>
>
14 years, 4 months
please help test new apache hc 4.0 support in jclouds
by Adrian Cole
Many of you know that the java and nio http engines and jclouds have
their own respective warts:
* the nio one is unreliable, as connection breaks under load are not
self-healing.
* the default java one uses too many connections and memory under load.
Sam Tunnicliffe sent us a patch a while back to add support for apache
hc 4.0 [1]. I've revised this and committed to trunk/1.0-SNAPSHOT.
I've also included this in the enterprise module. Here's how to use
that module:
http://code.google.com/p/jclouds/wiki/jcloudsAPI
Please help kick the tires on this, as it is an important piece of our
core infrastructure. If you see any opportunities for improvement, do
shout out.
Thanks,
-Adrian
[1] http://code.google.com/p/jclouds/issues/detail?id=107
14 years, 4 months
Lock amortization preliminary performance numbers
by Vladimir Blagojevic
Hi all,
As you probably recall I am working on a LIRS eviction algorithm. But before we get there, Manik and I agreed that I push in a direction of implementing ConcurrentHashMap(CHM) variant that will do lock amortization and eviction per segment of CHM. So far I have implemented LRU eviction so we can verify feasibility of this approach rather than eviction algorithm itself. We are hoping that eviction algorithm precision will not suffer if we do eviction per segment and you all are familiar with lock striping benefits of segments in CHM.
So, I've cobbled up a first test to compare eviction and lock amortized enabled CHM (BCHM) with regular CHM and synchronized HashMap. The test is actually based on already existing test [1] used to measure performance of various DataContainers. I've changed it slightly to measure Map directly instead of DataContainer. The test launches in cohort 48 reader, 4 writer and 1 remover threads. All operations are randomized; readers execute map.get(R.nextInt(NUM_KEYS), writers map.put(R.nextInt(NUM_KEYS), "value"), and finally removers execute map.remove(R.nextInt(NUM_KEYS). NUM_KEYS was set to 10K. Each thread does in a loop 1K ops.
Initial capacity for CHM and HashMap were set to 1K, max capacity for eviction and lock amortized enabled CHM was set to 256; therefore BCHM has to do a lot of evictions which is evident in map final size listed below.
Size = 9999
Performance for container ConcurrentHashMap
Average get ops/ms 338
Average put ops/ms 87
Average remove ops/ms 171
Size = 193
Performance for container BufferedConcurrentHashMap
Average get ops/ms 322
Average put ops/ms 32
Average remove ops/ms 74
Size = 8340
Performance for container SynchronizedMap
Average get ops/ms 67
Average put ops/ms 45
Average remove ops/ms 63
If I remove lock amortization for BufferedConcurrentHashMap, that is if we attempt eviction on every get/put/remove, performance of put/remove for BCHM goes to zero basically! As far as I can interpret these results, BCHM get operation performance does not suffer at all in comparison with CHM as it does for single lock HashMap. Predictably, for single lock HashMap each type of operation takes on avg almost the same amount of time. We pay a hit in BCHM put/remove operations in comparison with CHM but the numbers are promising, I think. If you have comments, or even suggestions on how to test these map variants in a different, possibly more insightful approach, speak up!
Cheers,
Vladimir
[1] http://fisheye.jboss.org/browse/~raw,r=1264/Infinispan/trunk/core/src/tes...
14 years, 4 months
Re: [infinispan-dev] Initial LIRS implementation
by Bryan Thompson
Vladimir,
Does this match up with what you are seeing? That is a good win for get()/remove() with a loss on put().
Size = 14698
Performance for container ConcurrentHashMap
Average get ops/ms 13931
Average put ops/ms 306
Average remove ops/ms 294
Size = 452
Performance for container BufferedConcurrentHashMap(LRU)
Average get ops/ms 7735
Average put ops/ms 140
Average remove ops/ms 200
Size = 494
Performance for container BufferedConcurrentHashMap(LIRS)
Average get ops/ms 13400
Average put ops/ms 36
Average remove ops/ms 1136
Bryan
> -----Original Message-----
> From: Vladimir Blagojevic [mailto:vblagoje@redhat.com]
> Sent: Wednesday, February 03, 2010 3:36 PM
> To: Bryan Thompson
> Subject: Re: [infinispan-dev] Initial LIRS implementation
>
> Yes,
>
> Have a look at LRU first and leave LIRS for tomorrow :) I'd
> be happiest if you can find a use case for BCHM+LRU in your
> application and test it out that way. Real life scenario!
>
> Cheers
>
> On 2010-02-03, at 3:27 PM, Bryan Thompson wrote:
>
> > Ah. It is inside the same outer class? Bryan
> >
> >
> >> -----Original Message-----
> >> From: Vladimir Blagojevic [mailto:vblagoje@redhat.com]
> >> Sent: Wednesday, February 03, 2010 2:44 PM
> >> To: Bryan Thompson
> >> Subject: Re: [infinispan-dev] Initial LIRS implementation
> >>
> >> Here it is:
> >> http://fisheye.jboss.org/browse/Infinispan/trunk/core/src/main
> >> /java/org/infinispan/util/concurrent/BufferedConcurrentHashMap.java
> >>
> >> Are you familiar with LIRS? If not, don't bother unless you are
> >> willing to dedicate at least a day or two :(
> >>
> >> http://www.ece.eng.wayne.edu/~sjiang/Projects/LIRS/sig02.ppt
> >> http://citeseer.ist.psu.edu/527790.html
> >>
> >> Regards,
> >> Vladimir
> >>
> >>
> >> On 2010-02-03, at 2:31 PM, Bryan Thompson wrote:
> >>
> >>> Can you send me the full class name and I will check it out. Bryan
> >>>
> >>>> -----Original Message-----
> >>>> From: infinispan-dev-bounces(a)lists.jboss.org
> >>>> [mailto:infinispan-dev-bounces@lists.jboss.org] On Behalf
> >> Of Vladimir
> >>>> Blagojevic
> >>>> Sent: Wednesday, February 03, 2010 2:09 PM
> >>>> To: infinispan -Dev List
> >>>> Subject: [infinispan-dev] Initial LIRS implementation
> >>>>
> >>>> Hi,
> >>>>
> >>>> I've just committed preliminary attempt to implement LIRS.
> >>>> There does not seem to be serious degradation in terms of
> >> performance
> >>>> when it comes to get/remove commands in comparison with
> >> LRU enabled
> >>>> BufferedConcurrentHashMap.
> >>>> However, put command is about three times slower than in
> >> LRU; put in
> >>>> LIRS is as fast as put command of a single lock
> >> synchronized HashMap.
> >>>>
> >>>> Looking for enthusiasts willing to help out with some code
> >> review :)
> >>>>
> >>>> Regards,
> >>>> Vladimir
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> infinispan-dev mailing list
> >>>> infinispan-dev(a)lists.jboss.org
> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>>
> >>
> >>
>
>
14 years, 4 months
Initial LIRS implementation
by Vladimir Blagojevic
Hi,
I've just committed preliminary attempt to implement LIRS. There does not seem to be serious degradation in terms of performance when it comes to get/remove commands in comparison with LRU enabled BufferedConcurrentHashMap. However, put command is about three times slower than in LRU; put in LIRS is as fast as put command of a single lock synchronized HashMap.
Looking for enthusiasts willing to help out with some code review :)
Regards,
Vladimir
14 years, 4 months
JPA Cache Store
by Amin Abbaspour
Hi All,
I know that most of you are already busy fixing the remaining issues
toward first GA release, but I would like to open a small topic here
that may be affective for next major release (4.1.0 or so).
As subject shows, I would like to add a JPA compatible cache store to
Infinispan. I feel that current available cache stores are designed
with this concept that store is just a place for cache persistence and
initialization and do not much consider store as a major logical part
of the system. While to use Infinispan as a write-behind in front of a
previously defined structured RDBMS we need a general purpose
structural cache store.
I myself have conceptual problems with JPA (and in general any sort of
ORM) but yet I believe here is one of the rare cases where ORM is the
best tool around. We can easily use all the progress in the field of
OR mapping without considering much how it works in fact.
As a result less people (like I myself) will need to extends
CacheStores to create their own custom store. (although this will
always remain open for those who like)
You accept that, this store will not only increase our competitiveness
with available commercial products (which already have this feature)
but will also open the door for a feasible 'Infinispan DB' which will
make it also a player in IMDB market.
Thoughts?
Regards,
Amin
14 years, 4 months
