[ISPN-134] Scanning MBean annotations for the jopr plugin
by Galder Zamarreno
Hi,
Re: http://www.jboss.org/index.html?module=bb&op=viewtopic&t=161834
I've opened that forum post in the JBoss Build project to figure out the
best way to scan Mbean annotations. This is related to
https://jira.jboss.org/jira/browse/ISPN-134
I can use any of the methods used by Vladimir so far in a
straightforward way cos the requirement here is to execute something
before the jopr plugin is jarred and it needs to have access to the
dependency classpath.
A similar javadoc method as used for Config or JMX documentation could
be used but these methods rely on the doclet being already built in some
kind of artifact. I mentioned a possibility in the forum thread but
another one would be for the jopr doclet that generates the XML to be in
tools. However, tools depends on jopr, why is this so? Apparently this
is due to TestNameVerifier but this has the list of modules hardcoded
and opens File instances to them...
The other possibility would be to have a main class executed but how
would I get hold of the classpath? This would also mean that jopr might,
assuming this was doable, need to depend on all the projects where MBean
annotations could live, cos the most likely way to do this would be via
some kind of mvn dependency classpath access.
The best way IMO would be the jopr doclet method and this to live in
tools project where I think it makes sense for it to live as we don't
wanna ship it but only use it at build time. But for this to work,
tools' dependencies need to be sorted out and tools stop depending on jopr.
Thoughts? I'll have a think overnight.
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
16 years
descriptive/config cache listeners
by Mircea Markus
Hi again,
Coherence has this feature named data trigger: whenever something is
added a trigger is called. Very similar with our listener
functionality. What I liked about it though is the fact that you can
specify it through the configuration, and not necessarily
programatically. This is nice in the sense that you can customize the
listeners/triggers at deployment time, without the need of changing
the code. Also easy to implement..., wdyt? worth the effort?
Cheers,
Mircea
16 years
Continuous Query Caching
by Mircea Markus
Hi,
Again, this is a feature from Coherence[1].
Basic idea is to execute a query against the cache, and hold the
result object. This result object will always have up to date query
result; this means that whenever something is modified in the cache
the result itself is updated. Advantage: if one performs the same
query very often(e.g. several times every millisecond) the response
will be fast and the system will not be overloaded.
E.g.
Filter filter = new AndFilter(new EqualsFilter("getTrader", traderid),
new EqualsFilter("getStatus",
Status.OPEN));
ContinuousQueryCache cacheOpenTrades = new ContinuousQueryCache(cache,
filter);
Iterator iter = cacheOpenTrades.entrySet().iterator(); //*this
entrySet call will be instant!*
FOr a full list of scenario in which this can be used take a look at
[1].
Shall we consider adding something similar?
Cheers,
Mircea
[1] http://download.oracle.com/docs/cd/E14526_01/coh.350/e14509/continuousque...
16 years
Flexible indexing - an idea
by Michael Neale
Hi All.
I have been looking over the Infinispan query module by Navin.
As this is built on Hibernate Search - (and correct me if wrong) the
indexing happens on pojos fields.
This is great for most of the cases, but for my ulterior motive (which
I will reveal in another email) I would like to deal with certain
object types differently. So lets for instance take a media file like
MP3, if I was storing it in the cache -I would know when I go to the
index it that I have an instance of something that has extra data I
would like to index (ie its not really a pojo): at that point I can
extract whatever data out of the "rich" object (meta data, or whatnot)
and stick that in the Work object for HS to do its thing on (say based
on known MIME types, as one instance).
I have tried out something like this, by messing with the
QueryInterceptor (and the tests):
So I would propose some mechanism to register for the QueryInterceptor
a surrogate class for indexing purposes (which while only take effect
when it gets asked) - so when it calls addToIndexes(value, key) -
then if if a surrogate is available it will create it, and pass it to
searchFactory.getWorker().performWork(new Work(surrogate... etc... -
where the surrogate is created based on the value type (as well as its
contents) - and thus searching will return what I want (as opposed to
nothing).
Q1: Does this even make sense? Should I just be pushing a "surrogate"
type object into the cache in the first place (doesn't feel right-
changing what I would store for the purposes of indexing)?
Q2: Is there any way we can query heterogenous caches ie caches like
Cache<String, SomeParent> where there are many children of SomeParent.
(so in a query we would declare we are only interested in specific
instances types? )
Thoughts?
--
Michael D Neale
home: www.michaelneale.net
blog: michaelneale.blogspot.com
16 years
creating a module for Infinispan based Directory
by Sanne Grinovero
Hi Manik,
I'm dwelling with several request-for-changes in email between me and
Łukasz, some changes are really trivial
like style and typos and I could just fix them, but I don't as we
don't have source control.
I'm also developing tests which are probably going to be lost forever
in email-change control :-/
Please, could you create the module to host this sources? this would
greatly benefit development and public interest.
thanks,
Sanne
16 years
code review
by Sanne Grinovero
Hi Łukasz,
I'm still unable to reproduce your same exception, but I started to
develop an additional test
to get more thread interleaving, my original goal was to make your
problem happen more likely
but I got some other errors regarding the lock & transaction management.
Here is some early feedback, hoping to get this somewhere in SVN soon
to make it easier
to help out:
1) make sure all Logger instances are made "static", especially since the logger
factory we are using is quite expensive.
2) Using "synchronized ( cache ) " in InfinispanLock:
there's something wrong with this approach, especially here:
if ( !cache.containsKey( lock ) ) {
cache.put( lock, lock );
acquired = true;
}
this is not atomical: considering at first the "synchronized (cache)"
it means you want to guarantee that nobody else uses it in a similar method,
but this guarantee holds only on the same reference of "cache", especially
only to the current node; other nodes might change the contents of the cache
and synch won't stop that, ending up with two or more nodes owning the Lucene
directory lock.
In the next step the code is asking if the lock is taken, if not you
take it and consider
yourself as owner. actually during cache.put you should check the
return value as some
other node could have acquired it.
Even better, give a look into using Infinispan's atomic API:
cache.putIfAbsent(K,V);
and again make sure you check for the return value.
In any case, I think you'll need to avoid all usages of "synchronized"
as it's not giving
the safety needed for this use case, and exploit the atomic APIs
Infinispan is providing.
cheers,
Sanne
16 years
[HSearch] Query DSL step 1
by Emmanuel Bernard
There are several steps to the query DSL:
1. implement the initial ideas and see what problems we face and how
well that fits
2. add analyzers into the mix to transparently use the right one
3. add parameters that use the conversion bridge (not sure how well
that could fly but an interesting idea
4. build up the stack of operators integrated into the DSL
5. string based QL using this API (not convinced yet but why not).
Navin will start working on 1 and if things go well 2 (we will have a
fantastic tool already we do just that).
Here are my notes based on the initial idea + the feedback received.
A few remarks:
- it asks the analyzer so that we correctly apply the analyzer on
terms
- it has a few query factory methods
- it contains a few orthogonal operations
- I am not quite satisfied with how boolean is handled, any idea?
Design remarks:
- should we use interfaces or plain implementations? I would start
with plain implementations to make things easier
- let's put it in org.hibernate.search.query.dsl for now
Examples
SealedQueryBuilder qb = searchFactory.withEntityAnalyzer(Address.class);
Query luceneQuery = qb.must()
.add(
qb.should()
.add( qb.term("city",
"Atlanta").boostedTo(4).createQuery() )
.add( qb.term("address1",
"Peachtree").fuzzy().createQuery() )
)
.add(
qb
.range
("movingDate").from("200604").to("201201").exclusive().createQuery()
)
.createQuery();
Analyzer choice
queryBuilder.withAnalyzer(Analyzer)
queryBuilder.withEntityAnalyzer(Class<?>)
queryBuilder.basedOnEntityAnalyzer(Class<?>)
.overridesForField(String field, Analyzer)
.overridesForField(String field, Analyzer)
.build() //sucky name
returns a SealedQueryBuilder //sucky name
SealedQueryBuilder contains the factory methods
Factory methods
Hosted onSealedQueryBuilder
//Alternative
.term().on(String field).matches(String text)
.on(String field).matches(String text)
.term(String field, String text) //define a new query
.term(String field, String text) //define a new query
.ignoreAnalyzer() //ignore the analyzer, optional
.fuzzy() //API prevent wildcard calls, optional
.threshold() //optional
.prefixLengh() //optional
.term(String field, String value)
.wildcard() //API prevent fuzzy calls, optional
//range query
.rangeQuery(String field)
.from(String text)
.to(String text)
.exclusive() //optional
.constantScore() //optional, due to constantScoreRangeQuery but
in practice inherited from the common operations
//match all docs
.all()
//phrase query
.phrase(String field)
.ignoreAnalyzer() //ignore the analyzer, optional
.addWord(String text) //at least one
.addWord(String text)
.sentence(String text) //do we need that?
.slop() //optional
//search multiple fields for same value
.searchInMultipleFields()
.onField(String field)
.boostedTo(float) //optional
.ignoreAnalyzer() //optional
.onField(String field)
.forWords(String) //do we need that?
.forWord(String)
Boolean operations
SealedQueryBuilder contains the boolean methods
.must()
.add( qb.from().to() )
.add( ... )
.must().not()
.should()
Works on all queries
.boostedTo()
.constantScore()
.filter(Filter) //filter the current query
.scoreMultipliedByField(field) //FieldScoreQuery +
FunctionQuery?? //Not backed
.createQuery()
Todo
Span*Queries
MultiPhraseQuery - needs to fillup all accepted terms
FieldScoreQuery
ValueSourceQuery
FuzzyLikeThis
MoreLikeThis
16 years
Re: [infinispan-dev] 3 nodes problem
by Manik Surtani
On 24 Sep 2009, at 16:39, Mircea Markus wrote:
>
> On Sep 24, 2009, at 12:42 PM, Galder Zamarreno wrote:
>
>> Hi guys,
>>
>> I'm forwarding here some logs sent to me by Marcin. This is related
>> to the following Infinispan user forum post:
>>
>> http://www.jboss.org/index.html?module=bb&op=viewtopic&t=161077
>>
>> Addition of big maps with DIST seems to be taking too long. Even
>> adding map of 1000 pairs takes a long time:
>>
>> infinispan_osoz.log:
>>
>> 2009-09-18 12:15:03,171 TRACE
>> [org.infinispan.interceptors.InvocationContextInterceptor] (main)
>> Invoked with command PutMapCommand{map={a548=b548...
>> ...
>> 2009-09-18 12:15:59,500 TRACE
>> [org.infinispan.remoting.rpc.RpcManagerImpl] (main) osoz-
>> fd029efaa5-53570 broadcasting call PutMapCommand{map={a548=b548
>> ..
>> 2009-09-18 12:15:59,750 TRACE
>> [org.infinispan.remoting.rpc.RpcManagerImpl] (main) responses=
>> [SuccessfulResponse, SuccessfulResponse]
>>
>> It takes almost 1 minute to send a map of 1000 pairs. This seems
>> (note: having done a deep dive on the log) to be due to individual
>> key remote get operations across the cluster.
>>
>> An operation of 10000 pairs took so long that user abandoned it.
>>
>> Now, if we got a put map of 10000 pairs, instead of doing a
>> remoteGetBeforeWrite for each key. Why don't we figure out the
>> group of keys that do requite a remote get and group them all in
>> single ClusteredGetCommand for all those keys? That would reduce
>> the number of RPCs ops from N, where N is the number of keys
>> requiring a remote get down, to 1.
> I'm not sure this would work for users that need to use the
> previously existing values.
> e.g.
> Object oldValueIDefentlyNeedToKnow = cache.put(k,v);
> If they don't I remember there was a setting that didn't perform the
> remote get, Manik?
Yes you can suppress the remote get with a flag, but lets first think
about why we need a remote get at all for PutMapCommand. We need it
for PutKeyValueCommand since we return the old value. We need it for
RemoveCommand since we return the old value. Etc. But for
PutMapCommand?!? Hmm...
cc'ing Infinispan-dev, BTW.
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
16 years
[hibernate-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new
by Navin Surtani
Incase this email didn't go to the full dev-list. I got it as a
separate thread so forwarding on.
Begin forwarded message:
> From: johng.sst(a)gmail.com
> Date: 25 September 2009 16:00:53 BST
> To: Navin Surtani <nsurtani(a)redhat.com>
> Subject: Re: Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for
> Lucene queries (was: Re: Query module new
>
> All,
>
> I think Hardy's original push back came from the first pass' use of
> the decorator pattern to try to come up with a DSL. That really
> isn't much better than knowing the API. The alternate is to come up
> with a more natural language implementation but that leads to
> parsers, lexers, etc... I'm not saying it's not worth while but it
> may be a lot of work.
>
> John Griffin
>
> On Sep 25, 2009 8:12am, Navin Surtani <nsurtani(a)redhat.com> wrote:
> > Just wanted to get this topic re-started again.
> >
> >
> >
> >
> >
> > Essentially what I think this project/DSL/module/thingy-bob is
> thought
> >
> > to become: -
> >
> >
> >
> > A simple package where a user can build Lucene queries without
> having
> >
> > to know too much about Lucene itself. If I'm headed down the wrong
> >
> > thought path then just thwack me.
> >
> >
> >
> >
> >
> >
> >
> > On 26 Aug 2009, at 21:08, Hardy Ferentschik wrote:
> >
> >
> >
> > > On Wed, 2009-08-26 at 13:39 +0200, Emmanuel Bernard wrote:
> >
> > >> I've been thinking about a DSL to build Lucene queries in the
> last
> >
> > >> day.
> >
> > >> What do you think of this proposal?
> >
> > >
> >
> > > What do you really gain compared to native Lucene queries?
> >
> >
> >
> > What's gained I believe is the fact that people can build complex
> >
> > lucene queries easier. Currently, it's a bit clunky imo so if we
> >
> > provide a cleaner way to build them it can prove beneficial to any
> >
> > lucene user (myself included for querying on Infinispan).
> >
> >
> >
> > Any other thoughts?
> >
> >
> >
> >
> >
> > > If your API achieves exactly the same as what's possible with
> Lucene
> >
> > > it is just a 'useless' wrapper.
> >
> > >
> >
> > > A wrapper around native Lucene queries would make sense if it
> could
> >
> > > somehow use some of the Hibernate Search specific meta data. As an
> >
> > > extreme example one could generate some meta classes a la JPA2.
> This
> >
> > > way
> >
> > > one could ensure that you can get help with which field names are
> >
> > > available.
> >
> > >
> >
> > > --Hardy
> >
> > >
> >
> > > _______________________________________________
> >
> > > infinispan-dev mailing list
> >
> > > infinispan-dev(a)lists.jboss.org
> >
> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> >
> >
> > Navin Surtani
> >
> >
> >
> > Intern Infinispan
> >
> > Intern JBoss Cache Searchable
> >
> >
> >
> > _______________________________________________
> >
> > hibernate-dev mailing list
> >
> > hibernate-dev(a)lists.jboss.org
> >
> > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >
Navin Surtani
Intern Infinispan
Intern JBoss Cache Searchable
16 years
