[Hibernate-JIRA] Created: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[Hibernate-JIRA] (HHH-7737)...

[Hibernate-JIRA] (HHH-7735)...

Emmanuel Bernard (JIRA)

Thursday, 8 April 2010 Thu, 8 Apr '10

8:37 a.m.

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: engine Reporter: Emmanuel Bernard Assignee: Sanne Grinovero I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.... - For more information on JIRA, see: http://www.atlassian.com/software/jira

Show replies by date

Sanne Grinovero (JIRA)

Thursday, 8 April Thu, 8 Apr

10:22 a.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Sanne Grinovero commented on HSEARCH-499: ----------------------------------------- definitely agree, actually in my previous experiments I had this as I needed it, but it's implementation was coupled to my model. I believe we chatted about the possibility to use a DetachedCriteria, but you where not 100% convinced about it. Should try by HQL? oh and the API should be able to pass something like a Map<rootTypeClass,filter> as each root type would have it's own fetching filter. The same filtering would need a way to define efficient purging on the same dataset, or we need to switch to (delete+write)* for each id rather than having purgeAll+(add)*

...

-- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.... - For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

Thursday, 22 April Thu, 22 Apr

3:43 a.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: 3.3.0

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: engine Reporter: Emmanuel Bernard Assignee: Sanne Grinovero Fix For: 3.3.0 I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

Sanne Grinovero (JIRA)

Friday, 21 May Fri, 21 May

12:47 p.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Component/s: (was: engine) massindexer

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero Fix For: 3.3.0 I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

Sanne Grinovero (JIRA)

Wednesday, 3 November Wed, 3 Nov

6:49 p.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero Fix For: 3.3.0 I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

Sanne Grinovero (JIRA)

Monday, 29 November Mon, 29 Nov

11:07 a.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: (was: 3.3.0) 3.4.0

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero Fix For: 3.4.0 I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

Sanne Grinovero (JIRA)

Monday, 27 December Mon, 27 Dec

7 p.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero Fix For: 3.4.0 I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

Sanne Grinovero (JIRA)

Thursday, 27 January Thu, 27 Jan

1:16 p.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Sanne Grinovero commented on HSEARCH-499: ----------------------------------------- use detached Criteria, which are not reausable in different threads, but go via a "CriteriaProvider", or "JPACriteriaProvider" factories

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero Fix For: 3.4.0 I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

Sanne Grinovero (JIRA)

1:18 p.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Sanne Grinovero commented on HSEARCH-499: ----------------------------------------- or ask user for query templates

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero Fix For: 3.4.0 I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

Sanne Grinovero (JIRA)

Monday, 21 February Mon, 21 Feb

2:05 p.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: (was: 3.4.0) 3.4.0.Alpha1

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero Fix For: 3.4.0.Alpha1 I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

Hardy Ferentschik (JIRA)

Monday, 21 March Mon, 21 Mar

9:30 a.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Hardy Ferentschik updated HSEARCH-499: -------------------------------------- Fix Version/s: (was: 3.4.0.Beta1) 3.4.0

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero Fix For: 3.4.0 I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

Emmanuel Bernard (JIRA)

Monday, 18 April Mon, 18 Apr

8:03 a.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Emmanuel Bernard updated HSEARCH-499: ------------------------------------- Fix Version/s: (was: 3.4.0) 3.5

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero Fix For: 3.5 I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

Sanne Grinovero (JIRA)

Monday, 24 October Mon, 24 Oct

10:58 a.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: (was: 4.0.0.Final) 4.1

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero Fix For: 4.1 I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Emmanuel Bernard (JIRA)

Friday, 23 December Fri, 23 Dec

9:40 a.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Emmanuel Bernard updated HSEARCH-499: ------------------------------------- Fix Version/s: (was: 4.1)

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Marc Schipperheyn (JIRA)

Sunday, 22 January Sun, 22 Jan

9:01 p.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

Monday, 23 January Mon, 23 Jan

3:41 a.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Sanne Grinovero commented on HSEARCH-499: ----------------------------------------- Hi Marc, I'd be glad to implement it and had sent some proposals but we couldn't agree on the API this should have. Please join the discussions, and let me know how you think this should work, and how it would look like to use it. I'd appreciate some use cases examples as well.

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Marc Schipperheyn (JIRA)

5:36 a.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Marc Schipperheyn commented on HSEARCH-499: ------------------------------------------- my use case is this. I have an database of classifieds, market place like, they have start and end dates. The classifieds are stored in one table and the validity data in an inner joined other table. Of course, I can query the index based for valid classifieds based on those start and end dates, However, with the MassIndexer, the index will be much, much larger than needed, since the database will have more and more "invalid" classifieds over time. Implementation wise, a detecached criteria query would seem good, but hql would work for me as well, it tends to be a bit more flexible although I cannot see how we would need this flexibility. Criteria seems to be safer because you might want to be able to protect yourself against like "select cat.kittens from Cat cat inner join cat.kittens". So perhaps a criteria query that does not have the full range of flexibility such as projection since you only want to filter what comes back from a "select all from x" type query.

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

6:26 a.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Sanne Grinovero commented on HSEARCH-499: ----------------------------------------- thanks Marc. Technically the MassIndexer needs two queries: a) return all primary keys of a type which needs to be indexed (this is to produce a scrollable result projecting the ID only) b) A query to know in advance how many objects there are. This is mostly to provide accurate progress statistics. So I need the user to provide some kind of filter which I could apply to generate both types, or ask the user to provide both queries. One complexity is that the user thread passing the object is working on a different Session instance than the thread which is going to generate query a), so parameters to the query should not be applied on the query directly but passed to be applied in a second step. Basically it need to provide somthing equivalent to #withFilter(String countingHQLQuery, String idselectionHQLQuery, Map<String,Object> parametersNamesValuesMap) but this isn't looking too good, nor is in any way safe to check for valid parameters, types or syntax. I guess we'll have to go with some compromise or we won't ever make this but better looking suggestions are welcome :)

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Marc Schipperheyn (JIRA)

6:52 a.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499?pag... ] Marc Schipperheyn commented on HSEARCH-499: ------------------------------------------- Right, but why not ask users to supply a DetachedCriteria or create the forClass part in the API and let the user add createAliases and restrictions. Like a kind of intermediary form, such as a MustJunction in HSearch. You can add the rowCount projection and id projection yourself. fts.createIndexer(clazz) .restrictionsForClass( MassIndexer.forClass() .createAlias("validity","validity") .add(Restrictions.le("validity.dteFrom", new Date()) .add(Restrictions.ge("validity.dteTo", new Date()) ) .purgeAllOnStart(true) .optimizeOnFinish(true) .threadsToLoadObjects(3) .batchSizeToLoadObjects(30) .threadsForSubsequentFetching(12) .startAndWait();

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Marc Schipperheyn (JIRA)

Wednesday, 8 February Wed, 8 Feb

4:12 p.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Marc Schipperheyn commented on HSEARCH-499: ------------------------------------------- Perhaps a much simpler approach would be to allow you to feed the primary keys to be indexed to the MassIndexer. It would avoid the complications of working with criteria selections and basically achieve what most people want. I would *love* to see this as part of 4.1

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

4:32 p.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: 4.1.0.Beta2

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

4:34 p.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero commented on HSEARCH-499: ----------------------------------------- great idea! we can't expect you to load them all (could be huge), but we could define an interface for some kind of stream generator. Good timing as well as I'm working on other MassIndexer fixes and improvements.

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Emmanuel Bernard (JIRA)

Thursday, 9 February Thu, 9 Feb

4:53 a.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Emmanuel Bernard commented on HSEARCH-499: ------------------------------------------ Iterator?

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

Monday, 13 February Mon, 13 Feb

12:47 p.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: (was: 4.1.0.Beta2) 4.1.0.CR1

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Emmanuel Bernard (JIRA)

Tuesday, 28 February Tue, 28 Feb

2:26 p.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Emmanuel Bernard commented on HSEARCH-499: ------------------------------------------ So based on SAnne's HSEARCH-499 branch (as of today at least :), here are my comments: h3. General idea The general idea looks good and we should pursue it. h3. MassIndexer API If we open the API to a generic loader + HQL + possibly in the future DetachedCriteria, I would like us to have a more fluid approach to set that up than piling up options in MassIndexer. At least if we make mistakes API wise they would be contained. Roughly sketched that could look like {code} massIndexer .loadId().withHQL() .primaryKeyQuery(...) //should we add fetch size options? .countQuery(...) //optional .addQueryParameter(name, value) ... massIndexer .loadId() .withDetachedCriteria(criteria) //should we add fetch size options? //query parameters? massIndexer .loadId().withCustomStrategy(impl) {code} h3. DetachedCriteria I like the idea of the Criteria or a JP-QL equivalent as we could define the select clause automatically. As Sanne pointed out, he needs a detached criteria or a serializable version to be able to run the query from several threads h3. The HQL approach Seems straightforward enough. count could be made optional. And Sanne had an issue with the MySLQ driver. Would that warrant a more flexible API? h3. The IdentifierLoadingStrategy It looks good but I would allow people to receive ids from an outside source, ie. not the DB via Hibernate ORM. Receiving a StatelessSession is fine as the implementor can ignore it. But ScrollableResults is a heavy interface to implement. And most methods will be useless to us. Maybe returning an interface close to Iterator but with a close() method is the way to go.

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

Wednesday, 29 February Wed, 29 Feb

5 a.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: (was: 4.1.0.CR1) 4.1.0.Final

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

Thursday, 15 March Thu, 15 Mar

7:54 a.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: (was: 4.1.0.CR2) 4.1.0.Final

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

Monday, 26 March Mon, 26 Mar

6:12 p.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: (was: 4.1.0.Final) 4.1.0.CR3

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Marc Schipperheyn (JIRA)

Wednesday, 28 March Wed, 28 Mar

9:04 a.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Marc Schipperheyn commented on HSEARCH-499: ------------------------------------------- This feature should probably also be seen side by side with the new soft delete feature since that filter is not honoured by MassIndexer. Assuming that this is intended, an indexing interceptor should always be matched by a MassIndexer filter. In fact, one might even argue that if the soft delete filter would be honoured by MassIndexer, a lot (but not all) of the use cases of the MassIndexer filter would be covered :-)

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

11:49 a.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: (was: 4.1.0.CR3) 4.1.0.Final

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Laubrino (JIRA)

Thursday, 29 March Thu, 29 Mar

2:40 p.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Laubrino commented on HSEARCH-499: ---------------------------------- I agree with Marc Schipperheyn' comment. An indexing interceptor should be not ignored by a massindexer. There's no reason, why it should behave differently (from my point of view)

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

2:46 p.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero commented on HSEARCH-499: ----------------------------------------- I agree too, but this is unlikely to happen without mayor changes, which requires a lot of performance testing. It's not looking very likely I can make this before 4.1 final. Normally I would say "please do it yourself" but I realize this is a though one. You're welcome to try, even a POC should help, but I need you to test performance on relatively large databases.

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Marc Schipperheyn (JIRA)

Saturday, 31 March Sat, 31 Mar

12:21 p.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Marc Schipperheyn commented on HSEARCH-499: ------------------------------------------- I was thinking about this yesterday and what if in stead of a soft delete filter, you have a soft delete @DocumentFieldBridge at the field level? Each bridge would determine if a document gets removed, updated, deleted, based on the value of the field, just like the filter does now. This way the MassIndexer could more easily access this information. Downside would be a potential (slight?) performance loss with multiple bridges like this in case of multi field evaluations.

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

12:37 p.m.

New subject: [Hibernate-JIRA] Commented: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero commented on HSEARCH-499: ----------------------------------------- Why at field level? What if I have multiple fields, and some request a delete and some other request an update? We could patch the MassIndexer to consider the soft delete logic, my problem is that ideally I would need to translate the filters in SQL predicates so that I can restrict the entities being loaded from the database. Filtering the entities by having the user defined soft delete logic by "looking at the entity state" is easy, but it would be much more efficient by loading only the data we actually need.

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

Tuesday, 3 April Tue, 3 Apr

2:03 p.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: (was: 4.1.0.Final)

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

2:05 p.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: 4.2

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

2:34 p.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Fix Version/s: (was: 4.2)

...

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Sanne Grinovero (JIRA)

Wednesday, 4 April Wed, 4 Apr

5:34 a.m.

New subject: [Hibernate-JIRA] Updated: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

[ https://hibernate.onjira.com/browse/HSEARCH-499?page=com.atlassian.jira.p... ] Sanne Grinovero updated HSEARCH-499: ------------------------------------ Labels: candidate_for_next (was: )

...

Pass a "filter" query to MassIndexing: index changed entity based on timestamp ------------------------------------------------------------------------------ Key: HSEARCH-499 URL: https://hibernate.onjira.com/browse/HSEARCH-499 Project: Hibernate Search Issue Type: New Feature Components: massindexer Reporter: Emmanuel Bernard Assignee: Sanne Grinovero Labels: candidate_for_next I was discussion the MassIndexer API. It would be nice to be able to filter the list of Entity ids to reindex based on a query: For example select u.id from User u where u.lastUpdate >= :yesterday or sue some kind of other flag. This would be a nice useful trick.

-- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

Adrian Meredith (JIRA)

Friday, 19 October Fri, 19 Oct

5:54 a.m.

New subject: [Hibernate-JIRA] (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

Sanne Grinovero (JIRA)

6:38 p.m.

New subject: [Hibernate-JIRA] (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

Adrian Meredith (JIRA)

Tuesday, 30 October Tue, 30 Oct

5:53 a.m.

New subject: [Hibernate-JIRA] (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

Sanne Grinovero (JIRA)

6:05 a.m.

New subject: [Hibernate-JIRA] (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

Adrian Meredith (JIRA)

7:26 a.m.

New subject: [Hibernate-JIRA] (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp

4234

days inactive

5170

days old

hibernate-issues@lists.jboss.org

Manage subscription

42 comments

6 participants

tags (0)

participants (6)

Adrian Meredith (JIRA)
Emmanuel Bernard (JIRA)
Hardy Ferentschik (JIRA)
Laubrino (JIRA)
Marc Schipperheyn (JIRA)
Sanne Grinovero (JIRA)

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

[Hibernate-JIRA] Created: (HSEARCH-499) Pass a "filter" query to MassIndexing: index changed entity based on timestamp