[hibernate-issues] [Hibernate-JIRA] Updated: (HSEARCH-33) hibernate.search.worker.batch_size to prevent OutOfMemoryException while inserting many objects

[Emmanuel Bernard (JIRA)]

     [ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-33?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Emmanuel Bernard updated HSEARCH-33:
------------------------------------

    Summary: hibernate.search.worker.batch_size to prevent OutOfMemoryException while inserting many objects  (was: batch_size to prevent OutOfMemoryException while inserting many objects)

Done, the batch size apply today to all transactions (through .index or not), After all the OOME applies everywhere. So a wise value is expected

> hibernate.search.worker.batch_size to prevent OutOfMemoryException while inserting many objects
> -----------------------------------------------------------------------------------------------
>
>                 Key: HSEARCH-33
>                 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-33
>             Project: Hibernate Search
>          Issue Type: New Feature
>          Components: engine
>    Affects Versions: 3.0.0.beta1
>            Reporter: Stefan
>
> While inserting many objects in a batch the application runs out of memory because the FullTextIndexEventListener for Lucene collects the updates.
> This is a tough problem. Today the solution is to apply the work in n transactions rather than 1. Maybe the notion of batch_size at some point to force a "flush" when the queue goes up can help. But it sacrifices the transactional semantic. 
> hibernate.cfg.xml
> <event type="post-insert"> 
>   <listener class="org.hibernate.search.event.FullTextIndexEventListener"/> 
> </event> 
> Inserting code something like: 
> Session session = sessionFactory.openSession(); 
> session.setCacheMode(CacheMode.IGNORE); 
> Transaction tx = session.beginTransaction(); 
> for ( int i=0; i<100000; i++ ) { 
>   Item item = new Item(...); 
>   session.save(item); 
>   if ( i % 100 == 0 ) { 
>     session.flush(); 
>     session.clear(); 
>   } 
> } 
> tx.commit(); 
> session.close(); 

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira