hardy: I have some news on HSEARCH-763 as well
[10:34pm] jbossbot: jira [HSEARCH-763] Index numeric properties by using @NumericField by default [Open (Unresolved) Improvement, Major, Hardy Ferentschik] https://hibernate.atlassian.net/browse/HSEARCH-763
[10:34pm] sannegrinovero: yes, that's also why I told him to not bother investigating a better value
[10:34pm] hardy: I investigated a bit and will comment on the issue as well
[10:34pm] sannegrinovero: as we have a relatively easy JIRA open which should be able to get rid of the conf. option and guess the absolute correc value
[10:35pm] sannegrinovero: nice!
[10:35pm] hardy: that sounds like a great feature
[10:35pm] sannegrinovero: I guess that one is going to be complex
[10:35pm] hardy: i don’t think so
[10:35pm] hardy: not on a implementation level
[10:35pm] sannegrinovero: well that's good news Looking forward for your comments then.
[10:35pm] hardy: I acutally meant to ask you why you think it is complex? Maybe I don’t see the elephant in the room
[10:36pm] hardy: there is an issue though with side effects
[10:36pm] hardy: switching to numeric fields will break faceting on numeric fields
[10:36pm] hardy: atm we still only rely on the field cache approach which cannot deal with numericly encoded values
[10:37pm] sannegrinovero: ah
[10:37pm] hardy: there are two alternative approaches which I am plannign to invetigate/implement for ages now, but it just has not happened
[10:37pm] hardy: in both cases we will need more config information from the user
[10:38pm] hardy: we need to know up-front which fields he wants to facet on
[10:38pm] hardy: be it via an explciit @Facet or whatever
[10:38pm] sannegrinovero: ok so the elephant in the room of my mind was "unexpected side effects", interactions with other features,, etc
[10:38pm] sannegrinovero: I didn't expect that one though
[10:38pm] hardy: again, this will/need to happen when we revisit faceting
[10:39pm] hardy: for Search 5 we are pretty much stuck with the field cache approach
[10:39pm] hardy: there is just no time to address faceting
[10:39pm] hardy: that is a topic for a 5.x release alone (almost)
[10:39pm] sannegrinovero: ok
[10:39pm] hardy: btw, I would like to see changes asap, eg 5.1
[10:40pm] hardy: back to numeric fields and Search 5.0
[10:40pm] hardy: if we are going to make numeric fields the default for numbers we break faceting
[10:40pm] sannegrinovero: today we'd get an exception to tell you can't facet on a NumericField right?
[10:40pm] hardy: we would need a way out for the user
[10:41pm] hardy: an exception or wired values - not sure
[10:41pm] hardy: exception one would hope
[10:41pm] sannegrinovero: please add a test for that
[10:41pm] hardy: sure
[10:42pm] hardy: the question becomes what we should do
[10:42pm] sannegrinovero: so the problem is that we don't have a @NonNumericField to allow the user to override.. right?
[10:42pm] hardy: the issue talks about a global flag to switch to old behaviour
[10:42pm] hardy: but that seems hard-core
[10:42pm] hardy: or too much
[10:43pm] hardy: it is more like you want to use numeric fields per default and just for the faceted fields you want to use string based indexing
[10:43pm] hardy: but there is no (easy) way express this
[10:44pm] hardy: the best option is to specify an explciit field bridge
[10:45pm] sannegrinovero: with faceting, AFAIR you support ranges too, to group them in blocks right?
[10:45pm] hardy: either as part of @Field or via @FieldBridge
[10:45pm] hardy: yes
[10:45pm] sannegrinovero: I'm thinking about a @NumericField(facetablefield="name")
[10:45pm] hardy: explicitly specified bridges have precedence over the numeric one (independent of faceting)
[10:46pm] sannegrinovero: (my initial thought was a boolean, but we can't reuse the same field name)
[10:46pm] hardy: hmm
[10:46pm] sannegrinovero: I guess a @Facetable would be nicer
[10:47pm] hardy: i am a bit weary to introduce this explicit facet flag (to @Field @numericField or whatever)
[10:47pm] hardy: right
[10:47pm] sannegrinovero: yes, it doesn't really belong into the @NumericField
[10:47pm] hardy: with the “updated” faceting down the road whatever we do would become obsolete
[10:47pm] sannegrinovero: hardy: are you sure you can't facet on NumericField ?
[10:47pm] hardy: yes
[10:48pm] sannegrinovero: try cheating
[10:48pm] hardy: at least not the way we do faceting atm
[10:48pm] hardy: afaik field caches just don’t work this way
[10:48pm] hardy: but I can have a look
[10:49pm] hardy: as said, the “workaround” is to provide an explicit field bridge
[10:49pm] sannegrinovero: but don't we have field caches for numbers? I thought we did implement those.
[10:49pm] hardy: however, in the case of let’s say ints what you want to use is IntegerBridge
[10:50pm] hardy: which is impl!?
[10:50pm] hardy: on second thought, it is not
[10:50pm] hardy: either way, it is quite fugly
[10:50pm] sannegrinovero: lol you just merged that one
[10:50pm] sannegrinovero: hum org.hibernate.search.bridge.builtin.IntegerBridge is not .impl
[10:51pm] hardy: right, that what I just said
[10:51pm] hardy: so from that part it would work
[10:51pm] hardy: but it puts a bit of a burden to the user in selecting the right field bridge now
[10:51pm] hardy: for the case where he wants to facet on this this field
[10:51pm] sannegrinovero: it's a bit awckard as we don't even validate the user using IntegerBridge on an int, etc..
[10:52pm] hardy: for sure odd
[10:52pm] hardy: so that’s the dilemma
[10:52pm] sannegrinovero: right.
[10:53pm] hardy: you’d almost would like to do the numeric field change to at the same time you refactor/extend faceting
[10:53pm] sannegrinovero: So in terms of big picture, it shouldn't be a drama for users to need to do this as long as we can craft a well-explanatory error message.
[10:53pm] sannegrinovero: I don't expect them to map new facets on numbers every day..
[10:53pm] hardy: or the other extreme is to say, numeric per default, once you want to use faceting on a number you have to use the global switch
[10:54pm] sannegrinovero: I'm not sure if we still want the global switch. I think that came up because we wanted to provide full backwards compatibility / ease of migration.
[10:54pm] sannegrinovero: But that's just impossible for apps moving to 5.0
[10:54pm] hardy: so your pov is to just make it the default
[10:55pm] sannegrinovero: hardy: I guess the main question is how temporary the "oddness" is.
[10:55pm] hardy: overriding is on per field basis using an appropriate bridge type
[10:55pm] sannegrinovero: hardy: my pov is that the global switch is a nice safety thing, but not worth it if it takes you more than 20 minutes.
[10:55pm] hardy: definitey takes more
[10:56pm] sannegrinovero: then don't. You can provide far higher user value on a different task.
[10:56pm] hardy: but you still think we should go ahead and give it a go
[10:56pm] sannegrinovero: So, let's assume that right after 5.0 you start working on integrating the new faceting engine:
[10:57pm] hardy: that would be the plan
[10:57pm] hardy: realistically it would be beginning next year
[10:57pm] sannegrinovero: assuming we have that, would Faceting then "just work" on the Numeric Fields ?
[10:57pm] hardy: once Search 5 is out, I am planning to spend some cycles on some infra strucure things
[10:57pm] hardy: and then Santa is coming soon
[10:57pm] sannegrinovero: I'm asking from a point of view of API progress:
[10:58pm] sannegrinovero: if something doesn't work today - but there is an easy to figure out workaround - and then starts to work much better in 6 months that's nice.
[10:58pm] sannegrinovero: s/nice/not too bad/
[10:58pm] sannegrinovero: But if we then need to change API, then we should probably see that coming now.
[10:59pm] hardy: true. easier to sell though in the case of new features
[10:59pm] hardy: in this case we make something which already exists harder
[10:59pm] sannegrinovero: yes but in this specific moment of time we don't promise API compatibility, so users will migrate, get an error, read it- > fix it.
[10:59pm] hardy: it depends what you understand with change in API
[11:00pm] sannegrinovero: that's what I don't know, and I think we should try do to some reading about now.
[11:00pm] hardy: first of all there will be more annotations (and/or other means of configuration)
[11:00pm] sannegrinovero: Well I guess that even if we go for a new @Faceting / @Facetable annotation, that's just a new feature at that point.
[11:00pm] hardy: then there is a question whether we need to change the current facet DSL to enable some more advanced features
[11:01pm] sannegrinovero: But I guess people who got it working using the workaround, shoudl be able to keep using that even when you replace the faceting engine.
[11:01pm] hardy: so far we only cover the simple use cases and don’t do these fancy things like side ways drill down and what not
[11:01pm] hardy: right adding a new facet annotation is a new feature
[11:01pm] sannegrinovero: DSL -> but that's evolutionary changes I hope
[11:02pm] hardy: I hope so as well
[11:02pm] sannegrinovero: so with the new annotation we can do wathever you want
[11:02pm] sannegrinovero: The question is, if people migrating in 2 weeks and applying @IntegerBridge .. will they be able to facet still when you introduce the new @Faceting ?
[11:02pm] hardy: I am also not convinced that we need all the bells and whistles Lucene’s native faceting supports
[11:02pm] sannegrinovero: not expecting you to answer that
[11:03pm] sannegrinovero: Just trying to nail down the crux.
[11:03pm] hardy: well, they might need to add this new annotation
[11:03pm] ci-bot-hibernate: Project hibernate-search-PR build #515: SUCCESS in 14 min: http://ci.hibernate.org/job/hibernate-search-PR/515/
[11:03pm] jbossbot: Title: hibernate-search-PR #515 [Jenkins]
[11:03pm] sannegrinovero: see, that's where we would be breaking the API.
[11:03pm] hardy: unless we keep the field chache based approach around as well
[11:03pm] hardy: which is probably possible