On 26 août 09, at 22:08, Hardy Ferentschik wrote: Remember, Hibernate Search's mission is to make full-text search as easy to use as possible to increase the overall technology adoption. There are several advantages to the DSL API listed below, but let's compare my example and the Lucene equivalent and see if you can still claim the API to be useless with a straight face. SealedQueryBuilder qb = searchFactory.withEntityAnalyzer(Address.class); Query luceneQuery = qb.must(Occurs.MUST) .add( qb.boolean(Occurs.Should) .add( qb.term("city", "Atlanta").boostedTo(4).createQuery() ) .add( qb.term("address1", "Peachtree").fuzzy().threshold(. 7).createQuery() ) ) .add( qb.from("movingDate", "200604").to("201201").exclusive().createQuery() ) .createQuery(); vs BooleanQuery luceneQuery = new BooleanQuery(); BooleanQuery addressLocationQuery = new BooleanQuery(); Query city = new TermQuery( new Term("city", "Atlanta") ); city.setBoost(4f); addressLocationQuery.add(BooleanClause.Occur.Should, city); Query address1 = new FuzzyQuery( new Term("address1", "Peachtree"), . 7 ); addressLocationQuery.add(BooleanClause.Occur.Should, address1); luceneQuery.add(BooleanClause.Occur.Must, addressLocationQuery); Query range = new RangeQuery( new Term("movingDate", "200604"), new Term("movingDate", "201201", false); luceneQuery.add(BooleanClause.Occur.Must, range); Advantages: - the query is readable and understandable even to new Lucene users. BTW the example is a quite simple one, it does not involve filter, search in multiple fields, query negation etc. - I have normalized some operations that require knowledge of the lucene query hierarchy (eg. ConstantScoreQuery, ConstantScorePrefixQuery, ConstrantScoreRangeQuery or PrefixQuery vs WildcardQuery) - the API shows available options right away using IDE auto- completion, not by looking at the Query hierarchy and its implementations - the API does take the analyzer into account which means that I can take my input and use it without thinking much about the underlying analyzer used at indexing time. In the example, my plain Lucene rewrite of the query will very likely fail because "Atlanta" and "Peachtree" should really be "atlanta" and "peachtree". In the API, we have the analyzer and can take that into account. Likewise for synonyms, phonetic approximation etc. Even worse, trying to search a user query containing several words in different fields is quite difficult in plain Lucene. In the new API it could look like: String search = "harry potter"; SealedQueryBuilder qb = searchFactory.withEntityAnalyzer(Book.class); Query luceneQuery = qb.searchInMultipleFields() .onField("title").boostedTo(4) .onField("title_ngram") .onField("description") .onField("description_ngram").boostedTo(.25) .forWords(search); vs String search = "harry potter"; Analyzer analyzer = searchFactory.getAnalyzer(Book.class); Map<String,Float> boostPerField = new HashMap<String,Float>(2); // boost factors boostPerField.put( "title", (float) 4); boostPerField.put( "title_ngram", (float) 1); boostPerField.put( "description", (float) 1); boostPerField.put( "description_ngram", (float) .25); BooleanQuery luceneQuery = new BooleanQuery(); for ( Map.Entry<String, Float> entry : boostPerField.entrySet() ) { final String fieldName = entry.getKey(); final Float boost = entry.getValue(); List<String> terms = new ArrayList<String>(); try { Reader reader = new StringReader(search); TokenStream stream = analyzer.tokenStream( fieldName, reader); Token token = new Token(); token = stream.next(token); while (token != null) { if (token.termLength() != 0) { String term = new String(token.termBuffer(), 0, token.termLength()); terms.add( term ); } token = stream.next(token); } } catch ( IOException e ) { throw new RuntimeException("IO exception while reading String stream??", e); } for (String term : terms) { TermQuery termQuery = new TermQuery( new Term( fieldName, term ) ); termQuery.setBoost( boost ); luceneQuery.add( termQuery, BooleanClause.Occur.SHOULD ); } } Did I make my case?

Just wanted to get this topic re-started again. Essentially what I think this project/DSL/module/thingy-bob is thought to become: - A simple package where a user can build Lucene queries without having to know too much about Lucene itself. If I'm headed down the wrong thought path then just thwack me. On 26 Aug 2009, at 21:08, Hardy Ferentschik wrote: What's gained I believe is the fact that people can build complex lucene queries easier. Currently, it's a bit clunky imo so if we provide a cleaner way to build them it can prove beneficial to any lucene user (myself included for querying on Infinispan). Any other thoughts? Navin Surtani Intern Infinispan Intern JBoss Cache Searchable

On 27 août 09, at 13:07, Manik Surtani wrote: This is called an internal DSL, ie you use Java as the language, not some external representation. The name is not right yet There are two things: - the query builder that lets you define the analyzer - the query builder that has an analyzer assigned and lets you build query What name is best for each of them. As of today this code is technically pure Lucene but to be honest the idea of passing an analyzer multiplexer (like the one we receive from searchFactory.getAnalyzer<Class<?>)) is not wildly spread in Lucene and potentially cumbersome wo the declarative approach of HSearch. The second problem is that some potential improvements will require inner knowledge of HSearch: - object parameters (and not string params) do require to know the FieldBridge of the property. This is a pure HSearch notion. - "property literal" like JPA is introducing could be added to replace the String-based field approach in some situations. Though I don't think that it would be a perfect fit. - spell checker (the old idea we had) That been said, if the API ends up being pure Lucene and once we stabilize it, we can contribute it back even though I am not necessarily a huge fan of ASL.

if you do that, all of the sudden, a QB can change it's analyzer on the fly making it immutable. Also the overridesForField methods would pollute the API when it's time to create a query. One of the advantages of a fluent API in a strongly typed environment is that we can hide methods that are meaningless in a given context. Let's start it under the Hibernate Search umbrella due to potential synergies and spin it out if needed.

That's what I said as well. Separate maven module. However, one can feel a disturbance in the force when one mentions MAVEN MODULE within the Hibernate Team :) On Thu, 27 Aug 2009 17:18:23 +0200, Manik Surtani <manik(a)jboss.org&gt; wrote: