Maven.assembly.useTransitiveDependencies true or false? WAS Re: More demo
by Galder Zamarreno
Hmmmmm, this has a domino effect. Because s3 module depends on
infinispan-core, it now pulls all its dependencies, so we need to
exclude them somehow, cos they're dependencies of the core module and
hence they should reside there rather than have them in each and every
module.
Personally, in spite of the repetition, I prefer explicit dependency
definition rather than letting maven bring all transitive dependencies
and then having to go and do exclusions.
More globally, I think this needs to be put the infinispan-dev list.
During assembly, do we either:
- Leave useTransitiveDependencies as it is, which is false, and make
modules like s3 explicitly define the 3rd party dependencies that
jclouds depends on.
- Or set useTransitiveDependencies to true so that they're all brought
in and exclude direct infinispan-core dependecies from getting to the
lib directories of submodules such as s3.
For the moment, and to make my life easier getting the demo working, I'm
using the latter option.
Thoughts?
On 06/30/2009 05:47 PM, Adrian Cole wrote:
> beware of the assemblies... they are extremely crufty compared to ant ;)
>
> On Tue, Jun 30, 2009 at 5:45 PM, Galder Zamarreno<
> galder.zamarreno(a)redhat.com> wrote:
>
>>
>> On 06/30/2009 05:22 PM, Adrian Cole wrote:
>>
>>> Well.. :) I setup the dependencies of jclouds, so it shouldn't bring
>>> anything crazy in! Besides, a missing dep might be symptomatic of a
>>> larger
>>> issue in the distribution assembly.
>>>
>> Ahh, maybe just found the reason why transitive dependencies are not
>> included:
>>
>> src/main/resources/assemblies/bin.xml:
>> <useTransitiveDependencies>false</useTransitiveDependencies>
>>
>> I'm trying out with that parameter set to true.
>>
>>
>>
>>> Anyway, let me know how it goes.
>>>
>>> on the SNAPSHOT bit, it was irrelevant to the topic. no worries on that.
>>>
>>> Thanks again for sorting this out, Galder.
>>> -a
>>>
>>> On Tue, Jun 30, 2009 at 5:11 PM, Galder Zamarreno<
>>> galder.zamarreno(a)redhat.com> wrote:
>>>
>>>
>>>> On 06/30/2009 05:00 PM, Galder Zamarreno wrote:
>>>>
>>>>
>>>>> On 06/30/2009 04:55 PM, Adrian Cole wrote:
>>>>>
>>>>> many thanks, Galder. It should be a transitive dep from jclouds-s3...
>>>>>> mvn
>>>>>> dependency:tree
>>>>>>
>>>>>> maybe the distribution isn't pulling in deps properly.
>>>>>>
>>>>>> Yeah, that looks to be the issue. At the moment I'm trying to redefine
>>>>> them in the s3 project but maybe there's another way to do this? Google
>>>>> to the rescue...
>>>>>
>>>>> Hmmm, even though it's repetition, I think it might be safer to
>>>> redefine
>>>> the transitive dependencies rather than let it bring them all. You never
>>>> know what Maven might bring with it...
>>>>
>>>>
>>>>
>>>> I will be cutting a new beta within the next couple days, so maybe you
>>>>>> can
>>>>>> switch temporarily to snapshot to ensure it will work?
>>>>>>
>>>>>> Hmmm, why would I need a snapshot of jclouds? I've simply been adding
>>>>> the transitive dependencies manually in s3/pom.xml
>>>>>
>>>>>
>>>>> Thanks,
>>>>>> -Adrian
>>>>>>
>>>>>> On Tue, Jun 30, 2009 at 4:40 PM, Galder Zamarreno<
>>>>>> galder.zamarreno(a)redhat.com> wrote:
>>>>>>
>>>>>> Hi Adrian,
>>>>>>
>>>>>>> I'm trying to make further progress with the demo and I'm
>>>>>>> encountering the
>>>>>>> following issue:
>>>>>>>
>>>>>>> Caused by: java.lang.ClassNotFoundException:
>>>>>>> org.jclouds.logging.config.LoggingModule
>>>>>>>
>>>>>>> The list of jars that is distributed does not include
>>>>>>> jclouds-core-1.0-beta-1.jar and this looks to be cos there's no direct
>>>>>>> dependency on this jar in the pom.xml for the s3 cache store module.
>>>>>>>
>>>>>>> FYI: I'm adding this dependency. I think there's a similar issue with
>>>>>>> google guice...
>>>>>>>
>>>>>>> Anyway, just letting you know of issues as I find them.
>>>>>>>
>>>>>>> See ya,
>>>>>>> --
>>>>>>> Galder Zamarreño
>>>>>>> Sr. Software Engineer
>>>>>>> Infinispan, JBoss Cache
>>>>>>>
>>>>>>>
>>>>>>> --
>>>> Galder Zamarreño
>>>> Sr. Software Engineer
>>>> Infinispan, JBoss Cache
>>>>
>>>>
>> --
>> Galder Zamarreño
>> Sr. Software Engineer
>> Infinispan, JBoss Cache
>>
>
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
14 years, 9 months
[ISPN-116] Async cache store: aggregation of multiple changes on a single key
by Galder Zamarreno
Hi,
Re: https://jira.jboss.org/jira/browse/ISPN-116
I can see two ways of providing such feature:
1. Rather than using a queue, using a data structure similar to the one
used in the data container so that when a newer value for a key already
present in the queue is present, the value can be swapped (map like
lookup required on the key so that O(1) is maintained) while maintaining
queue-like FIFO operations required to empty it.
2. Based on suggestion coming amin59 in
http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4240980#4240980, make
the current queue only store keys and make the draining process query
the cache for the latest value. Because from the time the data was put
to the cache to the time it's stored, the data could have expired or had
been removed all together, such query would need to make sure it doesn't
go to the cache loader again. Also, if the data had been removed, how
would the Async store know that from cache.get()? The queue would need
to record somehow that a key was actually removed.
The benefit of the 1st option is that we can take advantage of fast
drainTo() like operations when draining it, and also the fact that no
contention is added to the cache itself. The downside would be the need
for a more complex data structure.
The benefit of the 2nd option is the keeping a simple queue, less
complex data structure, but requires that each time a key is to be
drained, we have to query the cache. This could slow down the draining
process.
My preference is with 1. but for to use it, I'd need the data container
collection to be made more generic.
Thoughts?
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
14 years, 9 months
property tags in configuration files
by Mircea Markus
Continuing a conversation Vladimir and I had on irc, about how
<property> tags can be be listed in the documentation.
<property> are a way to configure extension points (e.g. CacheStore
implementations, ScheduledExecutorFactory implemetations etc), e.g:
<replicationQueueScheduledExecutor
factory="org.infinispan.executors.DefaultScheduledExecutorFactory">
<property name="threadNamePrefix" value="ReplicationQueueThread"/>
</replicationQueueScheduledExecutor>
"threadNamePrefix" property is specific to
DefaultScheduledExecutorFactory, other implementations of
ScheduledExecutorFactory would not recognize it, and/or use different
properties.
So there's no way to handle "property" tags as the other 'regular'
attributes that map to fixed configuration elements: e.g. transportClass
maps to GlobalConfiguration#transportClass etc.
Even more, in some situations these properties are not being used to
call setters on the target objects, but are being passed all together
through a java.util.Properties: which makes them impossible/harder to
annotate.
On the other hand it would be very nice to have all the properties of
default extension points (e.g. FileCacheStore, JdbcStringBasedCacheStore
etc) having all the documentation exposed through annotations.
A possible way I see to make this working is:
1) create a new annotation to denote an extension point,
@ExtenssionPoint(defaultImplementations={ClassName1, ClassName2,
ClassName3})
e.g.
CacheLoaderConfig {
@ExtenssionPoint(defaultImplementations={FileCacheStore.class,
JdbcStringBasedCacheStore.class}) //this will break module dependency,
guess strings should be used
void setCacheLoaderClassName(String s);
}
2) For all existing extension points the tool will create a different
table containing all its specific properties, obtained by parsing
class's @Property tags
3) Make all the current extension points that use java.util.Property are
replace by setters that can be annotated: this way we would remain
consistent in all code, and IMHO the code would be more readable
wdyt?
Cheers,
Mircea
14 years, 9 months
Re: Distributed hashing - heterogenous nodes
by Alex Kluge
Hi,
I have been working on a use of Jboss cache, which has a lot of overlap with the Infinispan project. I expect to be able to employ significant parts of this work in Infinispan as well. One point of overlap is the use of a consistent hash.
I have looked at the org.infinispan.distribution.DefaultConsistentHash, and this is actually a simple hash, and not a consistent hash. Luckily I have a version of a consistent hash that can almost be dropped in here. There are a number of properties of a consistent hash that make it valuable for a distributed hash table.
- If a server is removed, the number of keys that shift to a different
bin (different server) is minimal.
- The same key is always mapped to the same server.
- If a server is added, the number of keys that shift is minimal.
The current DefaultConsistentHash doesn't deliver on these. I hope you don't mind if I go into some details here.
For example, the hash is sensitive to the order in which servers appear in the initial collection of caches. If one cache is built with a list of servers (S1, S2, S3), and another is built with a list (S3, S2, S1), keys will be mapped to different servers, even though the set of servers is actually the same.
If one server is removed, many, or even all, keys will be shifted. For example one hash with the set of servers (S1, S2, S3) will map many keys to different servers than one with (S2, S3). In a true consistent hash, the keys originally mapped to S2 will remain mapped to S2, and those mapped to S3 will remain mapped to S3. The keys that were mapped to S1 will (depending on the exact implementation) will be divided between S1 and S2.
There are a few differences, specifically, I work with arrays rather than collections – in part for performance, I also support weights for the servers, and the replication count is an instance variable rather than an argument to the locate method. How wedded are you to supplying the replication count as part of the locate method? Other than this, it looks like an adaptation of my implementation to Infinispan would be fairly painless, and I suggest replacing the current implementation with it.
Alex Kluge
14 years, 9 months
https://jira.jboss.org/jira/browse/ISPN-38
by Mircea Markus
Hi,
This is related to https://jira.jboss.org/jira/browse/ISPN-38 (Early
distributed deadlock detection).
When I determine that a deadlock scenario happens, I try to interrupt a
tx originated locally. This translates to an thread.interrupt() call on
a thread that is replicating to other caches. The issue is that this
call is ignored in jgroups's GroupRequest:
private boolean collectResponses(long timeout) {
.....
if(timeout > 0) {
try {
completed.await(timeout, TimeUnit.MILLISECONDS);
//LINE 552
}
catch(Exception e) {
}
}
....
}
So my call to interrupt is ignored (line 552) and the collectResponses
method will only return after timeout elapses - not good for me, as I
want the method to return immediately if thread is interrupted.
Would it be possible to enhance the code to be aware of interrupt calls?
This is in JGroups 2.8.0 Beta2.
Cheers,
Mircea
14 years, 9 months
[ISPN-32] API changes for query module
by Navin Surtani
Heya guys,
I'm just getting going with the query module for infinispan (ISPN-32
in branches) and I just wanted to send around a couple of emails so I
could get an idea on some changes that I was thinking about making.
1 - query module: -
I decided this would be a better name than 'searchable' from JBCS
because I think that this would be a better name, and also JBCS never
really took off imo so I don't really want to stick with the old name.
2 - createQuery(): -
This was JBCS's only extra API method on the cache interface (or
SearchableCache) and I was thinking that maybe this is quite a long
method name - given that the cache has put(), get() etc ... I was
thinking of using query() but given that Lucene and Hibernate Search
both have Query classes that users deal with I'm wondering if there's
too much use of the word query. On the flipside, those are classes and
not API methods so it might work.
I'm not too worried about continuing on with createQuery() but think
it might work.
Another alternative would be search() but I'm not too convinced by that.
3 - SearchableCache: -
Should the name of this class be changed? I don't think so but if
anyone has any ideas ...
Thanks guys,
Navin Surtani
Intern Infinispan
Intern JBoss Cache Searchable
14 years, 9 months
Running GUI from Eclipse
by Galder Zamarreno
Hi guys,
Running the GUI demo from Eclipse throws an NPE:
Exception in thread "main" java.lang.NullPointerException
at org.infinispan.demo.InfinispanDemo.<init>(InfinispanDemo.java:100)
at org.infinispan.demo.InfinispanDemo.main(InfinispanDemo.java:89)
It's clear that IntelliJ does some kind of magic underneath with
InfinispanDemo.form but I haven't been able to figure out yet. If anyone
has seen this b4 and knows how to get such UI running from Eclipse, help
would be appreciated. Maybe something related to the magic done by
https://svn.codehaus.org/mojo/tags/ideauidesigner-maven-plugin-1.0-alpha-2/
?
Alternatively, does anyone know how to enable the right logging to see
why IntelliJ classes are not doing what's needed?
Cheers,
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
14 years, 9 months
Making sure JBC bugs get fixed in ISPN too WAS Re: You pinged me?
by Galder Zamarreno
cc'ing ISPN dev
Takayoshi Kimura wrote:
> Hi Galder,
>
> I wanted to know how we fix ISPN bugs originally found in
> JBCACHE project. I've created separate JIRA issue on ISPN
> project, is this preferred way?
I believe it is. I saw the bug you entered and linked the ISPN one with
the JBC one.
> My concern is some bugs
> could fall thru the cracks because issue reporters might
> open tickets in only JBCACHE and not in ISPN.
That's certainly a valid concern, we need to keep an eye on the JBC bugs
and verify whether they're relevant for ISPN or not.
>
> Takayoshi
>
> At Tue, 23 Jun 2009 11:40:18 +0200,
> Galder Zamarreno <galder.zamarreno(a)redhat.com> wrote:
>> Hey,
>>
>> You pinged me earlier but I was away in French lessons.
>>
>> What's up?
>>
>> Cheers,
>> --
>> Galder Zamarreño
>> Sr. Software Engineer
>> Infinispan, JBoss Cache
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
14 years, 9 months
Document all configuration options available in XML
by Vladimir Blagojevic
Hi,
I finished up https://jira.jboss.org/jira/browse/ISPN-89 today. I
decorated only a few bean configuration classes. I want to get a nod
from Manik first before proceeding further on this front. I also
committed a ConfigDoclet in tools project. ConfigDoclet is supposed to
create configuration documentation in html. However, I could not figure
out how to include ConfigDoclet invocation into maven. ConfigDoclet
requires -classpath flag to include all jars from dependent projects. If
you are maven proficient and have a lot of hair please be my guest :)
For testing purposes I was invoking ConfigDoclet as a simple java class
with main method. We can invoke this doclet as a simple java program. No
need for this doclet nonsense.
Regards,
Vladimir
14 years, 9 months
