Hi, I've been contemplating the idea of creating a JSR-352-style batch application for re-indexing one or more entity types in Hibernate Search. Functionally, it'd be the same as the current mass indexer, but using JSR 352 would provide some nice benefits: * Operation through standard batch interfaces (e.g. CLI, web console or whatever servers provide) * Standardized monitoring and logging * Standardized error handling, restartability after failures I thought this might be an interesting GSoC idea. It's very isolated and also should not be too complex to do. If the student is quick, one further idea could be to provide some UI functionality for controlling this (I'm not sure what's already available in the WF web console). Any thoughts? Thanks, --Gunnar _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

+1 Sounds like a good idea. On Wed, Feb 24, 2016 at 11:39 AM, Gunnar Morling <gunnar(a)hibernate.org&gt; wrote: > Hi, > > I've been contemplating the idea of creating a JSR-352-style batch > application for re-indexing one or more entity types in Hibernate > Search. > > Functionally, it'd be the same as the current mass indexer, but using > JSR 352 would provide some nice benefits: > > * Operation through standard batch interfaces (e.g. CLI, web console > or whatever servers provide) > * Standardized monitoring and logging > * Standardized error handling, restartability after failures > > I thought this might be an interesting GSoC idea. > > It's very isolated and also should not be too complex to do. If the > student is quick, one further idea could be to provide some UI > functionality for controlling this (I'm not sure what's already > available in the WF web console). > > Any thoughts? > > Thanks, > > --Gunnar > _______________________________________________ > hibernate-dev mailing list > hibernate-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

On 24 Feb 2016, at 11:45, Sanne Grinovero <sanne(a)hibernate.org&gt; wrote: It sounds interesting, but bear with me I'm not familiar with it so I'll throw out some doubts. Will it work without a JEE container? I guess some implementations might be embeddable, but still how simple would that be for the user? Note that we already allow rebuilding the index via JMX commands, that's quite standard too and it's hooked up by other integrators such as Infinispan or the WildFly admin console, and both expose a CLI too. I guess other stacks will hook up their favourite approach easily too. The idea sounds good but I'm not convinced on this to have a compelling benefits/efforts ratio.. especially not if it turns out to be slower than our current implementation (heck it's not as fast as I'd want it yet but it's ok, and I know how to make it much better as long as we control the details) Performance of this component is important. On the same database I've had MassIndexer POCs which would take 6 months to complete, a week to complete, or 3 minutes. The current implementation compared to those tests is the one which takes approximately 4 hours as I could never make the last optimisations generic enough for the general purpose but it would be just a matter of fixing open JIRAs for a couple of very concrete points. My point being that people often need to be able to reindex in a couple of hours - no matter the size. 2 hours is of course very human arbitrary but it seems to be the general acceptable threshold for such an operation: beyond a developer's tool it's also a tool for recovery from critical failures so it's absolutely unacceptable that a system potentially needs weeks to recover from an issue. On 24 February 2016 at 09:56, Vlad Mihalcea <mihalcea.vlad(a)gmail.com&gt; wrote: > +1 > > Sounds like a good idea. > > On Wed, Feb 24, 2016 at 11:39 AM, Gunnar Morling <gunnar(a)hibernate.org&gt; > wrote: > >> Hi, >> >> I've been contemplating the idea of creating a JSR-352-style batch >> application for re-indexing one or more entity types in Hibernate >> Search. >> >> Functionally, it'd be the same as the current mass indexer, but using >> JSR 352 would provide some nice benefits: >> >> * Operation through standard batch interfaces (e.g. CLI, web console >> or whatever servers provide) >> * Standardized monitoring and logging >> * Standardized error handling, restartability after failures >> >> I thought this might be an interesting GSoC idea. >> >> It's very isolated and also should not be too complex to do. If the >> student is quick, one further idea could be to provide some UI >> functionality for controlling this (I'm not sure what's already >> available in the WF web console). >> >> Any thoughts? >> >> Thanks, >> >> --Gunnar >> _______________________________________________ >> hibernate-dev mailing list >> hibernate-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > _______________________________________________ > hibernate-dev mailing list > hibernate-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev