Re: [hibernate-dev] Hibernate Search road map

Hi, I am happy with the "new features" in the roadmap, but still a bit worried about some open issues: HSEARCH-314 is quite annoying and I've run out of ideas to propose; any direction from someone? It would be nice if the solution could be reused also by other listeners (a topic for JPA2 spec?) HSEARCH-178 is very annoying and I see it hitting many people; probably the new HSEARCH-342 is another duplicate. ** About mass indexing ** I've lots of implementations in house, mostly experiments, many working well, but I can't really satisfy all requirements. Could we define exactly which requirements the first public version should be able to cover? In particular: * if it should work on any backend (using the pluggable backend api) or generate a local file index, reusing some backend code but using a direct reference instead of going through the interface and skipping the "Work" value objects. * Composite pk: to have a consistent view of DB even in multithreaded indexing on an online DB the best solution I've found is to have a single-threaded pk producer (load all pk's from db) which hands the pks to multiple object loaders, and at the same time keep track of other enqueued events (to process them in order at the end of indexing). This works ok if the pks are simple value objects, not graphs of composite objects ;-) * The API: my first ideas were about defining the indexing finetuning parameters (loading queries, caching, number of threads, queue sizes...) on an annotation on top of the class. Emmanuel didn't like it, and I don't like it either anymore: still some API should be defined; to be flexible enough I was thinking in something like this: SearchIndexer si = (SearchFactory instance).createIndexer(); // to rebuild all indexes, or sf.createIndexer( Class... types) si.setIndexWritingThreads( 2 ); // optional, default to 1 si.setLoadingThreads( 8 ); // optional, default to 1 si.setCacheMode( CacheMode... ); // optional, default to IGNORE in some acses useful to not ignore. si.setOptimizeAtEnd( false ) // optional, default to true si.setPurgeAllAtStart( false ) // optional, default to true (does purge and optimize) si. --add more options in future as needed -- si.start(); // can call it only once si.waitForFinish( timeout ) // optional Additionally, I would like to add some way that the user could define an optimal loading query, but this gets messy to check, especially in cases I am indexing different types having some inheritance relationship. We could leave this for future, or only add something like a Criteria.setFetchMode scoped per type, which is what I ususally need: si.setFetchMode(Class scopeHintToType, String relationName, FetchMode). The time to implement really depends on the amount of integration desired with the current backends; as to make a trivial local indexer I've lots of code to paste as soon as the API is set. I've some optimizations for the future, but would like to provide at least the basics soon. regards, Sanne 2009/2/20 Emmanuel Bernard <emmanuel(a)hibernate.org&gt;: