[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-5?page=... ] Stephane Epardaud updated HSEARCH-5: ------------------------------------ Attachment: hibernate-search-3.0.1.GA-hashing_patch.diff Patch to add a hashCode of the indexed data to the Lucene document in order to avoid reindexing unchanged documents. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.... - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-5?page=... ] Stephane Epardaud commented on HSEARCH-5: ----------------------------------------- Both patches are needed for me, but the second one isn't if you aren't embedding indexes with no prefix. In order to solve the problem we had I added a hash of the indexed data to the Lucene document so we can compare the new Lucene Document we're about to add with the previously stored Document if any exists. I am not happy with the way I load the old Lucene document from DocumentBuilder but I didn't find any other way. I am not happy with the cost of building the field names used for hashing as it is concatenating a lot, and could be improved with using a reusable char array where we only overwrite the end to change the current field name as we hash along. The reason for this costly and vast usage of absolute field names (as opposed to the _normal_ field names) is that several fields may have the same field name. I am also using indexes in the absolute field names so that different @Field in @Fields can be differentiated. This patch increased the speed of some operations by 10, because it avoids index rebuilding propagation to @ContainedIn entities when the entity hasn't changed as fas as HS is concerned. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.... - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-5?page=... ] Stephane Epardaud commented on HSEARCH-5: ----------------------------------------- I am not sure that HSEARCH-361 fixes the issue for us, since we are using custom bridges, which are supported by my method of using a hash -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.... - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-5?page=... ] Sanne Grinovero commented on HSEARCH-5: --------------------------------------- Hi Stephane, amazing that there has been a patch 2 years for this, I wish we had more time to discuss this earlier. But to eventually merge this some things should still be polished: 1)Patch should be updated to current code base. 2)as mentioned above, we should discuss how safe the "hash" actually is. 3)Having to rely on the index state during a write doesn't seem a good idea anyway: I'm afraid this works only on non-clustered non-async backends. If you use an async backend or the JMS backend, you might be taking a decision during DocumentBuild according to outdated state, we could hardly claim this is still having some transactional semantics. 4)People might not be happy to waste more space in the index, nor extracting a stored field is very efficient in Lucene. So it might be more efficient in some cases, but it should be optional as I think in some cases it would not be faster than a plain update. Main difference with HSEARCH-361 is that this one attempts to skip unneeded index writes, while HSEARCH-361 focuses on avoiding lazyloading of the missing graph needed to rebuild all fields; as consequence likely also prevent unneeded writes. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.... - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-5?page=... ] Stephane Epardaud commented on HSEARCH-5: ----------------------------------------- @Emmanuel: not enough beer certainly ;) @Sanne: Sorry about the late answer :( 1) It is unlikely I'll have time to update my patch to the current code base until there is a reason for my employer to pay me to do this. 2) If two different values hash to the same hash, then your hashing function is bad ;) 3) You make a good point, but I was entirely unable to retrieve enough info from Hibernate to construct the hash of the indexed entities prior to the current change, which is why it is stored at all. Otherwise I would construct both hashes (old/new) while reindexing. 4) Wasted space in the index is not a problem for me, especially compared with the order of magnitude we gained when our entities stopped propagating changes to all the other indexes where they were embedded even though nothing indexed changed. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.... - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://hibernate.onjira.com/browse/HSEARCH-5?page=com.atlassian.jira.plu... ] Sanne Grinovero resolved HSEARCH-5. ----------------------------------- Resolution: Out of Date This is now superseded by the fact we're sniffing dirty-checking capabilities from the ORM itself. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira