[SEARCH] Remove support for (or deprecate) date/calendar resolution?
by Yoann Rodiere
Hi,
That's a catchy subject, now I'm sure I got Guillaume's (angry) attention :)
Sanne wanted us to think about feature removal... Here's a proposal.
The feature I'd like to deprecate/remove: automatic truncation of
date/calendars to a given resolution. It's used like that:
@Field
@CalendarBridge(resolution = Resolution.DAY)
private Calendar dateSent;
I've been working on date-related tests for the past few hours (well,
months, too), and I'm becoming more and more convinced that this resolution
thing is the wrong solution to an admittedly common issue. The other (in my
opinion right) solution being to use range queries.
When we truncate a date to a given resolution, we must use a particular
timezone, obviously. It makes no sense to truncate to the day when we don't
choose a particular timezone.
Then the truncated date will match only if we search for dates whose
truncation is identical in this timezone.
And guess what? In the current implementation, we use... GMT. Always. See
how org.apache.lucene.document.DateTools.round(long, Resolution) works,
getting the calendar used for truncation from TL_CAL, which uses the GMT
timezone.
What it means is that the truncation currently produces rather unexpected
results as soon as applications use a timezone different from GMT.
So it's likely to be considered as broken:
- for multi-timezone applications
- for single-timezone applications using a different timezone than GMT
For instance, let's index 2016-01-01T00:00:00+01:00 and query
2016-01-01T12:00:00+01:00 with a DAY resolution:
- We'll index 2015-12-31 (2016-01-01T00:00:00+01:00 is
2015-12-31T23:00:00+00:00, which once truncated is 2015-12-31+00:00)
- We'll query 2016-01-01 (2016-01-01T12:00:00+01:00 is
2016-01-01T11:00:00+00:00, which once truncated is 2016-01-01+00:00)
That won't match, but from the point of view of a single-timezone user it
should have.
Of course I'm exaggerating a bit, and the issue probably hasn't been
noticed by many, because:
- there is no problem with the MINUTES, SECONDS and MILISECONDS
resolutions, which are not affected by timezones
- there is no problem with the HOUR resolution in most timezones (there
is a problem, however, when the timezone is something like -11:30)
- the problem is less likely to be detected when the user timezone is
close to GMT, because then only date/times close to midnight (i.e. outside
business hours) will be affected.
But my point is: the current solution produces counter-intuitive, sometimes
even useless results.
It was useful when dates were indexed as strings, but now that we (mostly)
index them as integers, this does not seem relevant. Even the DateTools
javadoc hints that DateTools (including "round") are not relevant for
numeric dates:
Another approach is {@link NumericUtils}, which provides
> a sortable binary representation (prefix encoded) of numeric values, which
> date/time are.
> For indexing a {@link Date} or {@link Calendar}, just get the unix
> timestamp as
> <code>long</code> using {@link Date#getTime} or {@link
> Calendar#getTimeInMillis} and
> index this as a numeric value with {@link LongField}
> and use {@link NumericRangeQuery} to query it.
What I'd like to propose is to deprecate or remove date/calendar
resolution, or limit it to STRING encoding, and instead encourage users to
query their dates with range queries whenever they don't want exact matches.
Of course we could also implement a way for users to customize the timezone
to be used when truncating. But range queries have other advantages:
- they fit the multi-timezone use case
- they allow to query the same field with multiple resolutions
At the very least, I'd like us to agree that in the future, we will only
implement automatic truncation with date/time types where the timezone is
either irrelevant or included (for instance ZonedDateTime or LocalDate or
LocalTime, but *not* Instant).
Relevant ticket: https://hibernate.atlassian.net/browse/HSEARCH-2378.
What do you think?
Yoann Rodière <yoann(a)hibernate.org>
Hibernate NoORM Team
9 years
ScrollableResults and 6.0
by Steve Ebersole
For 6.0 I'd like to also make ScrollableResults parameterized wrt the "row
type". E.g.
ScrollableResults<Person> sr = session.createQuery( ..., Person.class
).scroll();
However that changes the signature of its methods returning a "row"
(currently always defined as Object[]).
How would everyone prefer we handle this? Do I just change the signatures
of the existing methods? Or add new methods?
9 years
6.0 - Session#createFilter
by Steve Ebersole
As I have not been hearing hardly any feedback on these 6.0 design
questions I have been trying to start, I'll be doing something different in
this and any additional emails.. I'll state what I propose to do and if
anyone has issue with it they can make a counter proposal. Otherwise I
plan on following through with my proposal.
I plan on removing Session#createFilter. There are numerous reasons why
which I can discuss if anyone is interested in exploring this.
Ultimately I think it makes sense to handle this via Java 8 streams[1]
although I am not sure that needs to happen in 6.0
[1] https://hibernate.atlassian.net/browse/HHH-10962
9 years
"Sister projects" need to be compatible with latest Hibernate ORM
by Sanne Grinovero
Hi all,
as you know we've been late in delivering a version of Hibernate
Search version compatible with Hibernate ORM 5.2, this caused many
questions.
Turns out OGM is also late to this club and this is causing some
headaches too: people simply assume that our latest stuff will work
with each other, and I can't blame them for expecting that.
We need to improve on this; I screwed up scheduling on Search, but
looking forward I hope we can improve on this.
So... let's ramp up the tempo for an OGM 5.2 ?
Thanks,
Sanne
9 years
Re: [hibernate-dev] CI Jenkins upgraded to version 2.38
by Yoann Rodiere
Thanks! Looking forward to using pipelines :)
It seems PR jobs for Hibernate Search do not trigger anymore, though:
* https://github.com/hibernate/hibernate-search/pull/1273 was submitted
yesterday
* there is no build on http://ci.hibernate.org/view/Pull%20Requests/job/
hibernate-search-PR/ since december 23
* and I tried to restart the build explicitly from the PR, with no luck
It's a bit odd, because the OGM PR job seems to work fine... From what I
can see there hasn't been any particular change on the OGM job.
I can't look at the trigger options on GitHub (I lack the access rights),
but maybe the difference between OGM and Search lies there?
Anyway... I'm going for today, maybe I'll see something obvious tomorrow
morning!
Cheers,
Yoann Rodière <yoann(a)hibernate.org>
Hibernate NoORM Team
On 3 January 2017 at 18:22, Yoann Rodiere <yrodiere(a)redhat.com> wrote:
> Thanks! Looking forward to using pipelines :)
>
> It seems PR jobs for Hibernate Search do not trigger anymore, though:
>
> * https://github.com/hibernate/hibernate-search/pull/1273 was submitted
> yesterday
> * there is no build on http://ci.hibernate.org/view/Pull%20Requests/job/
> hibernate-search-PR/ since december 23
> * and I tried to restart the build explicitly from the PR, with no luck
>
> It's a bit odd, because the OGM PR job seems to work fine... From what I
> can see there hasn't been any particular change on the OGM job.
>
> I can't look at the trigger options on GitHub (I lack the access rights),
> but maybe the difference between OGM and Search lies there?
>
> Anyway... I'm going for today, maybe I'll see something obvious tomorrow
> morning!
>
> Cheers,
>
> Yoann Rodière <yrodiere(a)redhat.com>
> Software Engineer
> Red Hat / Hibernate NoORM Team
>
> On 3 January 2017 at 13:31, Davide D'Alto <davide(a)hibernate.org> wrote:
>
>> Thanks for the feedback,
>>
>> header and footer should be fine now.
>>
>> Thanks,
>> Davide
>>
>> On Thu, Dec 29, 2016 at 11:37 PM, Sanne Grinovero <sanne(a)hibernate.org>
>> wrote:
>> > Thanks Davide!
>> > The builds seems to work just fine, the only minor defect I've noticed
>> > is with the stylesheet doing some strange things when scrolling down:
>> > the breadcrumbs will "float" down as well. Definitely not urgent,
>> > great to have all updates!
>> >
>> > Thanks,
>> > Sanne
>> >
>> >
>> > On 28 December 2016 at 01:04, Davide D'Alto <davide(a)hibernate.org>
>> wrote:
>> >> Hi,
>> >> I hope you all had great holidays.
>> >>
>> >> I've upgraded our Jenkin on CI to version 2.38.
>> >>
>> >> As far as i can tell everything seems all right, but if you experience
>> >> some unusual problems,
>> >> please, let me know.
>> >>
>> >>
>> >> Cheers,
>> >> Davide
>> >> _______________________________________________
>> >> hibernate-dev mailing list
>> >> hibernate-dev(a)lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>> _______________________________________________
>> hibernate-dev mailing list
>> hibernate-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>
>
>
9 years
Hibernate OGM 5.1 Beta3 and 5.0.4 released
by Davide D'Alto
Hello everybody, holidays are over and we decided to start back with a
small release.
Hibernate OGM 5.1 Beta3 and a new 5.0 maintainance releases are now available.
In these releases we fixed some issues around sequence generation and
queries on entities using the single table per class inheritance
strategy. An update is highly recommended.
You can find all the details in the blog post [1].
Thanks,
Davide
[1] http://in.relation.to/2017/01/05/hibernate-ogm-5-beta3-and-5/
9 years
[SEARCH] Query-only analyzers with Elasticsearch - new annotation?
by Yoann Rodiere
Hello team,
I'm currently working on HSEARCH-2534, "Query-only analyzer definitions are
never added to the index settings with Elasticsearch".
This issue is about using analyzers only when querying with Elasticsearch.
It is already possible with Lucene, but not in Elasticsearch, because we
assume that any analyzer definition that is not referenced by a @Analyzer
annotation is a Lucene analyzer [1].
To be precise, the exact place where query-only analyzers are used is in
EntityContext.overridesForField [2], and the overrides are leveraged even
with Elasticsearch, for instance in ConnectedMultiFieldsTermQueryBuilder
[3].
I can see two solutions to the issue:
1. Make all analyzer definitions available for all indexing services.
2. Allow users to define, for each entity, which analyzer definitions
will be necessary when querying, even though the definitions are not used
when indexing.
Solution 1 seems quite hard to implement correctly.
First we'd have to have a different namespace for each indexing service,
but I've already implemented that much.
Second, some analyzer definitions are only valid for one indexing service,
and not for the other.
For instance, analyzer definitions using ElasticsearchTokenFilterFactory
are specific to Elasticsearch. And Analyzer definitions using
the WhitespaceTokenizerFactory with the "rule" parameter are only valid
with embedded Lucene. And so on. To sum up, I'm not sure we can do
something smart.
Solution 2 is easier to implement, but requires to add a bit of API: the
way for users to declare that a given analyzer definition is to be
available when querying a given entity. I would add type-level
@QueryAnalyzer(definition = "foo") and @QueryAnalyzers annotation.
I know nobody wants to add new annotations in a minor, but right now that
seems to be the only workable solution.
What do you think?
[1]
https://github.com/hibernate/hibernate-search/blob/1847bd222128395056cdf6...
[2]
https://github.com/hibernate/hibernate-search/blob/1847bd222128395056cdf6...
[3]
https://github.com/hibernate/hibernate-search/blob/1847bd222128395056cdf6...
Yoann Rodière <yoann(a)hibernate.org>
Hibernate NoORM Team
9 years
CI Jenkins upgraded to version 2.38
by Davide D'Alto
Hi,
I hope you all had great holidays.
I've upgraded our Jenkin on CI to version 2.38.
As far as i can tell everything seems all right, but if you experience
some unusual problems,
please, let me know.
Cheers,
Davide
9 years
