[hibernate-dev] Elastic search Lucene extension

[Marc Schipperheyn]

On Sun, Mar 6, 2016 at 10:12 AM Sanne Grinovero <sanne at hibernate.org> wrote:

> On 5 March 2016 at 22:04, Marc Schipperheyn <m.schipperheyn at gmail.com>
> wrote:
> > I saw the elastic search blog post and while is not yet sonething on our
> > radar,  it s certainly interesting!
> >
> > I already saw that if you bypass HSearch to go native, the syntax
> changes.
> > What about Lucene extensions? For instance, we extensively use joinutil
> and
> > grouped search. Couldn't live without those.
>
> Hi Marc,
> excellent feedback!
>
> Elasticsearch supports grouping nested joins so it would be able to
> execute such queries, but it needs an explicit schema for that at
> indexing time so we'll need to make sure it indexes things in the
> right way for that.. which is missing in Search today.
>
> I remember you mentioned doing this before, but could you please
> remind us how you are indexing things to use these extensions?
>
Well, talking about joins, there are basically two ways to index this
stuff:
1. implement query joins at index time
2. implement query joins at query time

We use the query time variant. Basically, this works by running a separate
child select agains any kind of index that generates selection criteria for
the parent query. It's a little like a sub query in SQL.

So, why is this relevant for us? Two reasons,
1. Pagination. We can't do this through post processing because we want to
correctly get an x number of results, and then the next x results.
2. Some criteria cannot efficiently be stored inside the document and when
you want to ADD to the query result based on certain criteria in stead of
SUBSTRACT, you can't use filters. And this is solved through the Join query.

With regard to grouping, we also do this at query time. This is more like a
special kind of query result. Where you get a list of items grouped by a
common key. For instance the top 3 likes on a post.

As "cheating" the search-engine will probably not work anymore, so we
> need to implement this feature correctly.
>
> I would appreciate your beta testing on this; I think you'l benefit
> from it is that it means we'll finally have to implement it for Lucene
> (embedded) users too ;)
>
> As always, our tiny team is swamped. So, I'm a little hesitant to commit
to this.
Cheers,
Marc
-- 
Kind regards,

Marc
M.Schipperheyn