[hibernate-issues] [Hibernate-JIRA] Commented: (HSEARCH-513) Experiencing long delays when indexing entities saved in different transactions

[Sanne Grinovero (JIRA)]

    [ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=39275#action_39275 ] 

Sanne Grinovero commented on HSEARCH-513:
-----------------------------------------

Hi Svyatoslav, thank you if you want to contribute on any other issue or suggest improvements that's very welcome, but also the kind of feedback you're providing is appreciated.

About this issue, I've opened HSEARCH-638 as a continuation to this one: your long delays are caused by 1)inefficient use of data structure 2)HSEARCH-638 and 3)HSEARCH-570
With the current change we solve 1 and 3, but I have to postpone 2) to next version.
I can't make such a big change for 3.3 as we need to get this stable and release soon, but I understand your use case is affected by that. Could you tell me if the improvements reached so far are good enough at the moment to keep the speed at acceptable levels, or is further improvement needed in short time?

> Experiencing long delays when indexing entities saved in different transactions 
> --------------------------------------------------------------------------------
>
>                 Key: HSEARCH-513
>                 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-513
>             Project: Hibernate Search
>          Issue Type: Bug
>          Components: engine
>    Affects Versions: 3.2.0.CR1
>         Environment: Hibernate 3.5.1-Final
> Spring 3.0.2
> MySQL Connector 5.1.6
>            Reporter: Florin Haizea
>            Assignee: Sanne Grinovero
>             Fix For: 3.3.0.CR1
>
>         Attachments: AllLibsExceptSpringAndHibernate.zip, build.xml, GedcomManagerTests-2010-11-24-shutdown-1.zip, hibernate-search-jms.zip, hibernate3.jar, HibernateSearchJira.zip, MavenIndexingTest.zip, output.txt, output08052010.rtf, SpringAndHibernateLibs.zip
>
>
> https://forum.hibernate.org/viewtopic.php?f=9&t=1003224
> I have 3 entities Person, Event and ParentOfBirthEvent. I save about 8000 Person objects in batches of 20, each batch is being saved in a different transaction. After a sufficient number of entities have been persisted (the saving and indexing of initial batches takes between150ms and 500ms) the indexing of the entities that are saved in some of the following batches takes a very large amount of time (sometime even minutes for a whole batch).
> The problem is that in order for these delays to appear there has to be a certain relationship between the objects saved in the current batch and objects saved in previous batches. The only way in which I can reproduce this 100% is by importing a file (which was submitted by one of the users of our app), creating entities based on the data in that file and saving the entities in batches of 20. When I reach batch number 273 or so the indexing starts taking a lot of time.
> If I removed the @IndexedEmbedded annotation from the "parent" field in the ParentOfBirthEvent class the problem is solved. Also, if I removed all the hibernate search listeners from the sessionFactory the problem is solved.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira