Change AbstractInfinispanTest.nullifyInstanceFields method name to zzzNullifyInstanceFields
by Galder Zamarreno
Hi,
I'm trying to modify SingleCacheManagerTest so that in the same way that
there's a preCreate() callback, there can be a preDestroy() callback
too. This is so that for example, in the
https://jira.jboss.org/jira/browse/ISPN-153 test, I can stop the naming
server created at in preCreate().
The problem though is that by the time either
SingleCacheManagerTest.destroyAfterClass is called,
AbstractInfinispanTest.nullifyInstanceFields has already run and I get NPEs.
However, I see other cases such as
BdbjeCacheStoreFunctionalIntegrationTest where clearTempDir is called
before AbstractInfinispanTest.nullifyInstanceFields.
I just run a test without the preDestroy() in SingleCacheManagerTest so
that there was only nullifyInstanceFields and destroyAfterClass
@AfterClass methods to called and destroyAfterClass was called first.
So, there does appear to be some kind alphabetical order in which testng
calls its @AfterClass methods.
So, I've just tried to change
AbstractInfinispanTest.nullifyInstanceFields method name to
AbstractInfinispanTest.zzzNullifyInstanceFields and now the preDestroy()
gets called first.
Any objections to this change?
The other solution would to have some kind of dependsOnMethods in
AbstractInfinispanTest but that would force specific naming on
subclasses. Changing the name in AbstractInfinispanTest seems like a
much easier option.
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
15 years, 11 months
Resetting Lucene lock at Directory initialization
by Sanne Grinovero
Hello,
Lucene does - in default LockManager implementation - a sort of "lock
cleanup" at index creation: if it detects a lock on the index at
startup, this is cleared.
Łukasz translated the exact same semantic on the Infinispan Directory;
current implementation inserts a "lock marker" at a conventional key,
like Infinispan was a filesystem.
So what is done in this case is to just delete the value from this
key, if any, at startup (for precision: at lockFactory.clearLock()).
But in this situation I would need to "steal" the lock from another
node, if it exists. IMHO this Lucene approach is not considering
concurrent initializations of the FSDirectory.
So my doubts:
1) Is there some API in Infinispan capable to invalidate an existing
Lock on a key, in case another node is still holding it (and will I
have the other node failing?)
2) Does it make sense at all? looks like a bad practice to steal stuff.
I am considering to write this lock using SKIP_CACHE_STORE, in which
case I could assume that if there exists one, there's a good reason to
not delete the lock as other nodes are running and using the index. In
case all nodes are going down, the lock doesn't exists as it wasn't
stored.
So my proposal is to do a no-op on lockFactory.clearLock(), and use
SKIP_CACHE_STORE when the lock is created.
When an IndexWriter re-creates an index (basically making an existing
one empty) it first uses clearLock(), then it tries to acquire one, so
it looks like it should be safe.
WDYT? this concept of SKIP_CACHE_STORE is totally new for me, maybe I
just misunderstood the usage.
Regards,
Sanne
15 years, 11 months
[ISPN-153] Binding Cache/CacheManager to JNDI
by Galder Zamarreno
Hi,
Re: https://jira.jboss.org/jira/browse/ISPN-153
Apart from testing itself, does it make sense to include the ability of
binding to JNDI, using org.jboss.util.naming.NonSerializableFactory to
to bind a non-serializable object into a local JNDI context, within
Infinispan itself?
So, I'm thinking that we could have a <jndi name="xyz"/> configuration
option both at the global level (for Cache Manager) and at the Cache
level. So,
- if <jndi/> was present in the global section, we'd bind CacheManager
to java:CacheManager
- if <jndi name="xyz"/> was present in the global section, we'd bind
CacheManager to java:xyz
- if <jndi/> was present in the cache section, we'd bind Cache to
java:<name-of-the-cache>
- if <jndi name="xyz"/> was present in the cache section, we'd bind
Cache to java:xyz
This should, in theory, ease JNDI binding in AS and other apps servers
without the need of extra code in those environments.
Thoughts? Brian, would you find this useful?
Cheers,
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
15 years, 11 months
storing in memory data in binary format
by Mircea Markus
Hi,
While working on the coherence config converter, I've seen that they
are able to specify how much memory a cache should use at a time (we
also had a thought about this in past but abandoned the idea mainly
due to performance). E.g. :
<backing-map-scheme>
<local-scheme>
<high-units>100m</high-units>
<unit-calculator>BINARY</unit-calculator>
<eviction-policy>LRU</eviction-policy>
</local-scheme>
</backing-map-scheme>
When 100 MB is reached, data will start to be evicted. I know we
support marshaled values, but it's not exactly the same thing.
I've been wondering how do they achieve this, and I've found this http://coherence.oracle.com/display/COH35UG/Storage+and+Backing+Map
.
The way it works (my understanding!) is by keeping the key + values in
serialized form within the map. So, if you have both the key and value
as a byte[], you can easily measure the memory fingerprint of the
cached data.
Now what about keeping data in maps in serialized form?
Pros:
- we would be able to support memory based eviction triggers.
- in DIST mode, when doing a put we won't need to deserialize the data
at the other end. This de-serialization might be redundant, as if
another node asks for this data, we'll have to serialize it back etc.
- the sync puts would be faster, as the data gets only serialized (and
doesn't get deserialized at the other end).
- ???
Cons:
- data would be deserialized for each get request, adding a latency.
Partially compensated by faster puts (see cons) and can be mitigated
by using L1 caches (near caches)
- ???
Well I'm not even sure that this fits with our actual architecture,
just brought this in for brainstorming :)
Cheers,
Mircea
15 years, 11 months
Re: [infinispan-dev] Infinispan target JDK?
by Manik Surtani
The plan was to baseline on JDK 6. Surely the cache-infinispan module
can be for JDK 6 only? What JDK 5 APIs do you specifically rely on,
that don't exist in 6?
On 15 Oct 2009, at 13:03, Hardy Ferentschik wrote:
> Hi Manik,
>
> I was just wondering which target JVM you are using for Infinispan.
> I recall there was a discussion about this on the Core.
> The reason I am asking is that in Hibernate we have a infinispan
> cache provider module which depends obviously on infinispan core.
>
> Now I get the following error when compiling Hibernate:
>
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Building Hibernate Infinispan Integration
> [INFO] task-segment: [clean, install]
> [INFO]
> ------------------------------------------------------------------------
> [INFO] [clean:clean {execution: default-clean}]
> [INFO] Deleting directory /Users/hardy/work/hibernate/core/trunk/
> cache-infinispan/target
> [INFO] snapshot org.infinispan:infinispan-core:4.0.0-SNAPSHOT:
> checking for updates from central
> [INFO] snapshot org.infinispan:infinispan-parent:4.0.0-SNAPSHOT:
> checking for updates from central
> [INFO] [enforcer:enforce {execution: enforce-java}]
> [INFO] [resources:resources {execution: default-resources}]
> [WARNING] Using platform encoding (MacRoman actually) to copy
> filtered resources, i.e. build is platform dependent!
> [INFO] Copying 1 resource
> [INFO] [compiler:compile {execution: default-compile}]
> [INFO] Compiling 18 source files to /Users/hardy/work/hibernate/core/
> trunk/cache-infinispan/target/classes
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Compilation failure
> /Users/hardy/work/hibernate/core/trunk/cache-infinispan/src/main/
> java/org/hibernate/cache/infinispan/InfinispanRegionFactory.java:
> [29,-1] cannot access org.infinispan.Cache
> bad class file: /Users/hardy/tmp/m2/org/infinispan/infinispan-core/
> 4.0.0-SNAPSHOT/infinispan-core-4.0.0-SNAPSHOT.jar(org/infinispan/
> Cache.class)
> class file has wrong version 50.0, should be 49.0
>
>
> I cannot switch to JDK 6, since parts of Hibernate require JDK 5 APIs.
>
> Are you only going to release infinispan with target VM 1.6? Or was
> the deploy a mistake? If so maybe you could push another snapshot.
>
> --Hardy
>
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
15 years, 11 months
Re: [infinispan-dev] Hash calculation in DefaultConsistentHash
by Alex Kluge
> Any chance of sharing this work with the rest of Infinispan community
> at some point?
My apologies, The embedded link in the original message did not
propagate through the mailing list. The doc is at
http://www.vizitsolutions.com/ConsistentHashingCaching.html.
There an an implementation of an AddressHash at the bottom, which should
be very close in functionality to the current DefaultConsistentHash. If
it looks correct it can be used directly, or I can tweak it if need be.
The doc will get more interesting when the section on adding a new server
is finished, as you will need it if you go with the virtual nodes.
Alex
--- On Wed, 9/16/09, Galder Zamarreno <galder.zamarreno(a)redhat.com> wrote:
From: Galder Zamarreno <galder.zamarreno(a)redhat.com>
Subject: Re: [infinispan-dev] Hash calculation in DefaultConsistentHash
To: infinispan-dev(a)lists.jboss.org
Date: Wednesday, September 16, 2009, 2:57 AM
Hey Alex,
On 09/14/2009 10:05 PM, Alex Kluge wrote:
> Hi,
>
> I would even make the case that the abs and the remainder operator are unnecessary. I have an version of a consistent hash which is similar to this, and made a few modifications so that my code delivers the same functionality as the current infinispan DefaultConsistentHash. The consistent hash is much more natural if you directly use the hash values. I had meant to contribute this code a while back, but I was simply too busy getting the full project of which this is a part to a testable state to make the infinispan modifications to the code.
>
> These classes are under the examples section in a Consistent hashing and Caching doc I am working on.
>
> This code also supports virtual nodes, but with the settings in the example do not produce any. The builder is there to facilitate the creation of the hash through an XML parser, while keeping the hash itself immutable. These things can all be tuned or changed for infinispan's specific needs.
Any chance of sharing this work with the rest of Infinispan community at
some point?
Regards,
>
> Alex
>
>
> --- On Mon, 9/14/09, Krzysztof Sobolewski<Krzysztof.Sobolewski(a)atm.com.pl> wrote:
>
> From: Krzysztof Sobolewski<Krzysztof.Sobolewski(a)atm.com.pl>
> Subject: [infinispan-dev] Hash calculation in DefaultConsistentHash
> To: "infinispan-dev(a)lists.jboss.org"<infinispan-dev(a)lists.jboss.org>
> Date: Monday, September 14, 2009, 4:25 AM
>
> I took a look at the implementation of DefaultConsistentHash and I was
> wondering: does the key hash value have to be positive? As in:
>
> int hash = Math.abs(key.hashCode());
>
> I can see that the resulting hash is used to pull a tail map of the hash
> space, but there's no reason the SortedMap key has to be positive. I'm
> actually most concerned about the fact that Math.abs(Integer.MIN_VALUE) ==
> Integer.MIN_VALUE so you have at least one negative key anyway :)
> -KS
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
15 years, 11 months
Test suite memory leaks
by Vladimir Blagojevic
Hi,
I found a cause of memory leaks in our test suite [1]. I corrected easy
fixes but I did not go through individual test classes that need to be
updated (see attachment in JIRA). I thought it was best to leave test
updating to devs that are most familiar with particular test classes.
Best regards,
Vladimir
[1] https://jira.jboss.org/jira/browse/ISPN-220
15 years, 11 months
Re: [infinispan-dev] Test suite memory leaks
by Galder Zamarreno
Hmmm, could you be more precise on what was the issue exaclty? Mircea
has been banging his head getting UDP tests to work on hudson slave but
we couldn't pin point the cause.
So, please share anything you learned with the rest so that we don't
have to go through such pain again :)
On 10/13/2009 05:12 AM, Vladimir Blagojevic wrote:
> UDP build is also up and running, no more problems for members to find
> each other. Hudson did not like binding to 127.0.0.1 for mcast sockets....
>
> On 09-10-12 5:30 AM, Manik Surtani wrote:
>> Great, thanks for looking into this!
>>
>> On 12 Oct 2009, at 06:20, Vladimir Blagojevic wrote:
>>
>>
>>> On 09-10-11 5:53 AM, Manik Surtani wrote:
>>>
>>>> As long as it works and is non-intrusive, I'm happy. :-)
>>>>
>>> Done. There has been a regression in our test suite that I believe is
>>> unrelated to the change I made regarding these leaks. I did not pick
>>> up
>>> commits 936 and 937 and I do not see new failures [1]. So you guys
>>> have
>>> a look there. EhCache and jbc2InfinispanTransformerTest that Mircea
>>> worked on pass if test suite is invoked from core directory but the
>>> fail
>>> if invoked from infinispan directory - as hudson does.
>>>
>>>
>>> [1] http://hudson.jboss.org/hudson/job/Infinispan-trunk-JDK6-tcp/812/
>>>
>>>
>>> http://fisheye.jboss.org/changelog/Infinispan/?cs=937
>>> http://fisheye.jboss.org/changelog/Infinispan/?cs=936
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>> --
>> Manik Surtani
>> manik(a)jboss.org
>> Lead, Infinispan
>> Lead, JBoss Cache
>> http://www.infinispan.org
>> http://www.jbosscache.org
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
15 years, 11 months
