12:16 p.m.
Hi all (and especially Gunnar),
I dug a bit in your ElasticSearch work today as I wanted to give a try to
the facet implementation and I was wondering if maybe we should use the
ElasticSearch DSL to build the queries instead of building raw JSON strings.
I experimented here:
https://github.com/gsmet/hibernate-search/commit/18737f720a1200b125adf2bb...
I think the code is a bit clearer now. I haven't changed all the queries in
the tests so I kept the fromJson method in ElasticSearchQueries but I think
we should remove it and rely on the DSL: I'll wait for your opinion as you
might have had a very good reason to not use the DSL.
One of the challenges waiting for us will be to transform the Lucene
Filters into proper Elastic Search queries.
--
Guillaume
12:52 p.m.
Hi Guillaume,
Indeed there is a reason for not using the DSL: I wanted to avoid the
dependency to the ES libraries.
They in turn pull in Lucene in a version which potentially differs
from the Lucene version used by Hibernate Search itself. Also I wanted
to leave the door open for supporting multiple (potentially
incompatibly altered) versions of ES backends, say ES 2 and a future
ES 3.
All in all, these advantages outweigh the slightly simpler usage of
the DSL to me.
One of the challenges waiting for us will be to transform the Lucene
Filters into proper Elastic Search queries
I haven't looked into filters yet, but it's high on my agenda.
Do you think it's more difficult than transforming actual queries?
Btw. the ES PoC allows to pass in plain ES queries, at some point
these should be allowed to contain filter clauses, too (this might
even work already, haven't tried it).
Cheers,
--Gunnar
2016-01-28 18:16 GMT+01:00 Guillaume Smet <guillaume.smet(a)gmail.com>:
Hi all (and especially Gunnar),
I dug a bit in your ElasticSearch work today as I wanted to give a try to
the facet implementation and I was wondering if maybe we should use the
ElasticSearch DSL to build the queries instead of building raw JSON strings.
I experimented here:
https://github.com/gsmet/hibernate-search/commit/18737f720a1200b125adf2bb...
I think the code is a bit clearer now. I haven't changed all the queries in
the tests so I kept the fromJson method in ElasticSearchQueries but I think
we should remove it and rely on the DSL: I'll wait for your opinion as you
might have had a very good reason to not use the DSL.
One of the challenges waiting for us will be to transform the Lucene
Filters into proper Elastic Search queries.
--
Guillaume
_______________________________________________
hibernate-dev mailing list
hibernate-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
2:17 p.m.
Hey Gunnar,
On Thu, Jan 28, 2016 at 6:52 PM, Gunnar Morling <gunnar(a)hibernate.org>
wrote:
They in turn pull in Lucene in a version which potentially differs
from the Lucene version used by Hibernate Search itself. Also I wanted
to leave the door open for supporting multiple (potentially
incompatibly altered) versions of ES backends, say ES 2 and a future
ES 3.
All in all, these advantages outweigh the slightly simpler usage of
the DSL to me.
Yeah, I understand your concerns. Apart from the more complicated usage,
I'm afraid building raw JSON is going to be harder to maintain. We only do
simple things atm and I don't find the JSON manipulation very readable nor
very solid. It's going to be worse when we will add aggregations for
faceting and so on.
As for the dependencies, I could take a look at shading the necessary
classes for the builders and see if we can come up with something clean? I
don't know if it's something acceptable for an Hibernate project.
It doesn't solve the multi versions thingy but if they change the API, it's
going to be quite ugly to generate different JSON queries anyway, don't you
think?
> One of the challenges waiting for us will be to transform the
Lucene
> Filters into proper Elastic Search queries
I haven't looked into filters yet, but it's high on my agenda.
Do you think it's more difficult than transforming actual queries?
Btw. the ES PoC allows to pass in plain ES queries, at some point
these should be allowed to contain filter clauses, too (this might
even work already, haven't tried it).
Yeah I saw you could inject native ES queries and that's also something
that worries me with the raw JSON approach. I find it much more
user-friendly to provide an API to build the queries.
(Using these various discussions to adjust how I work and be more compliant
with the Hibernate way of life :))
--
Guillaume
4:10 p.m.
2016-01-28 20:17 GMT+01:00 Guillaume Smet <guillaume.smet(a)gmail.com>:
Hey Gunnar,
On Thu, Jan 28, 2016 at 6:52 PM, Gunnar Morling <gunnar(a)hibernate.org>
wrote:
>
> They in turn pull in Lucene in a version which potentially differs
> from the Lucene version used by Hibernate Search itself. Also I wanted
> to leave the door open for supporting multiple (potentially
> incompatibly altered) versions of ES backends, say ES 2 and a future
> ES 3.
>
> All in all, these advantages outweigh the slightly simpler usage of
> the DSL to me.
Yeah, I understand your concerns. Apart from the more complicated usage, I'm
afraid building raw JSON is going to be harder to maintain. We only do
simple things atm and I don't find the JSON manipulation very readable nor
very solid. It's going to be worse when we will add aggregations for
faceting and so on.
I don't find too bad what we have atm. :) If becomes too much of a
burden at some point in the future, we still may reconsider.
A general credo we have is to make things as good for the user as
possible also if it means some more trouble for us internally. I think
sparing the user from dependency issues is worth the JSON crafting on
our side.
As for the dependencies, I could take a look at shading the necessary
classes for the builders and see if we can come up with something clean? I
don't know if it's something acceptable for an Hibernate project.
Our experiences with shading are mixed; I think most don't like it.
It might work, but the issue remains that the ES JAR depends on a
specific Lucene version. We may be lucky and the query builder stuff
is nicely isolated from that so that there would be no problem by HS
itself using a different version. Still it'd appear as a potential
source of trouble to me.
The loose coupling via JSON is more practical IMHO. I'd not exclude
the builder stuff for all times, but right now it seems not worth it.
Btw. - I should have mentioned that in my first reply - our vision
regarding queries is to modify and generalize the HSEARCH query
builder DSL so it can emit queries for the backend du jour, be it
plain Lucene, ES or Solr. We just didn't get there yet :)
It doesn't solve the multi versions thingy but if they change the API, it's
going to be quite ugly to generate different JSON queries anyway, don't you
think?
See above, I think some internal ugliness is ok if it helps the user
in a reasonable way.
>
> > One of the challenges waiting for us will be to transform the Lucene
> > Filters into proper Elastic Search queries
>
> I haven't looked into filters yet, but it's high on my agenda.
>
> Do you think it's more difficult than transforming actual queries?
> Btw. the ES PoC allows to pass in plain ES queries, at some point
> these should be allowed to contain filter clauses, too (this might
> even work already, haven't tried it).
Yeah I saw you could inject native ES queries and that's also something that
worries me with the raw JSON approach. I find it much more user-friendly to
provide an API to build the queries.
Yes, fully agreed, hence the idea to enable the HSEARCH query DSL for
that. The native ES query injection is something I built to have
something functional quickly and it also might be helpful for people
giving HSEARCH a quick try.
(Using these various discussions to adjust how I work and be more compliant
with the Hibernate way of life :))
+1 Keep them coming :)
--
Guillaume
3009
days inactive
3009
days old
3 comments
2 participants
participants (2)
-
Guillaume Smet -
Gunnar Morling