<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On 26 août 09, at 22:08, Hardy Ferentschik wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>On Wed, 2009-08-26 at 13:39 +0200, Emmanuel Bernard wrote:<br><blockquote type="cite">I've been thinking about a DSL to build Lucene queries in the last<br></blockquote><blockquote type="cite">day.<br></blockquote><blockquote type="cite">What do you think of this proposal?<br></blockquote><br>What do you really gain compared to native Lucene queries?<br>If your API achieves exactly the same as what's possible with Lucene <br>it is just a 'useless' wrapper. <br><br>A wrapper around native Lucene queries would make sense if it could<br>somehow use some of the Hibernate Search specific meta data. As an<br>extreme example one could generate some meta classes a la JPA2. This way<br>one could ensure that you can get help with which field names are<br>available. <br></div></blockquote></div><div><br></div><div>Remember, Hibernate Search's mission is to make full-text search as easy to use as possible to increase the overall technology adoption.</div><div><br></div><div>There are several advantages to the DSL API listed below, but let's compare my example and the Lucene equivalent and see if you can still claim the API to be useless with a straight face.</div><div><br></div><div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">SealedQueryBuilder qb = searchFactory.withEntityAnalyzer(Address.class);</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">Query luceneQuery = </span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">qb.must(Occurs.MUST)</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> .add(</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> qb.boolean(Occurs.Should)</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> .add( qb.term("city", "Atlanta").boostedTo(4).createQuery() )</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> .add( qb.term("address1", "Peachtree").fuzzy().threshold(.7).createQuery() )</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> )</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> .add(</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> qb.from("movingDate", "200604").to("201201").exclusive().createQuery()</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> )</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> .createQuery();</span></font></div><div><br></div><div>vs</div><div><br></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">BooleanQuery luceneQuery = new BooleanQuery();</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">BooleanQuery addressLocationQuery = new BooleanQuery();</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">Query city = new TermQuery( new Term("city", "Atlanta") );</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">city.setBoost(4f);</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">addressLocationQuery.add(BooleanClause.Occur.Should, city);</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">Query address1 = new FuzzyQuery( new Term("address1", "Peachtree"), .7 );</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">addressLocationQuery.add(BooleanClause.Occur.Should, address1);</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">luceneQuery.add(BooleanClause.Occur.Must, addressLocationQuery);</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">Query range = new RangeQuery( new Term("movingDate", "200604"), new Term("movingDate", "201201", false);</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">luceneQuery.add(BooleanClause.Occur.Must, range);</span></font></div><div><br></div><div><br></div><div>Advantages:</div><div> - the query is readable and understandable even to new Lucene users. BTW the example is a quite simple one, it does not involve filter, search in multiple fields, query negation etc.</div><div> - I have normalized some operations that require knowledge of the lucene query hierarchy (eg. ConstantScoreQuery, ConstantScorePrefixQuery, ConstrantScoreRangeQuery or PrefixQuery vs WildcardQuery)</div><div> - the API shows available options right away using IDE auto-completion, not by looking at the Query hierarchy and its implementations</div><div> - the API does take the analyzer into account which means that I can take my input and use it without thinking much about the underlying analyzer used at indexing time. In the example, my plain Lucene rewrite of the query will very likely fail because "Atlanta" and "Peachtree" should really be "atlanta" and "peachtree". In the API, we have the analyzer and can take that into account. Likewise for synonyms, phonetic approximation etc.</div><div><br></div><div>Even worse, trying to search a user query containing several words in different fields is quite difficult in plain Lucene. In the new API it could look like:</div><div><br></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">String search = "harry potter";</span></font></div><div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">SealedQueryBuilder qb = searchFactory.withEntityAnalyzer(Book.class);</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">Query luceneQuery = </span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> qb.searchInMultipleFields()</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> .onField("title").boostedTo(4)</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> .onField("title_ngram")</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> .onField("description")</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> .onField("description_ngram").boostedTo(.25)</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> .forWords(search);</span></font></div><div><br></div><div>vs</div><div><br></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">String search = "harry potter";</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">Analyzer analyzer = searchFactory.getAnalyzer(Book.class);</span></font></div><div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">Map<String,Float> boostPerField = new HashMap<String,Float>(2); // boost factors</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">boostPerField.put( "title", (float) 4);</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">boostPerField.put( "title_ngram", (float) 1);</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">boostPerField.put( "description", (float) 1);</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">boostPerField.put( "description_ngram", (float) .25);</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"><br></span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">BooleanQuery luceneQuery = new BooleanQuery();</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">for ( Map.Entry<String, Float> entry : boostPerField.entrySet() ) {</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">final String fieldName = entry.getKey();</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">final Float boost = entry.getValue();</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"><br></span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">List<String> terms = new ArrayList<String>();</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">try {</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">Reader reader = new StringReader(search);</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">TokenStream stream = analyzer.tokenStream( fieldName, reader);</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">Token token = new Token();</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">token = stream.next(token);</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">while (token != null) {</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">if (token.termLength() != 0) {</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">String term = new String(token.termBuffer(), 0, token.termLength());</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">terms.add( term );</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">}</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">token = stream.next(token);</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">}</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">}</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">catch ( IOException e ) {</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">throw new RuntimeException("IO exception while reading String stream??", e);</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">}</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"><br></span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">for (String term : terms) {</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">TermQuery termQuery = new TermQuery( new Term( fieldName, term ) );</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">termQuery.setBoost( boost );</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">luceneQuery.add( termQuery, BooleanClause.Occur.SHOULD );</span></font></div><div><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">}</span></font></div><div><font class="Apple-style-span" face="'Courier New'" size="3"><span class="Apple-style-span" style="font-size: 12px;">}</span></font></div><div><br></div><div>Did I make my case?</div></div></div></div></body></html>