[hibernate-issues] [Hibernate-JIRA] Updated: (HSEARCH-33) hibernate.search.worker.batch_size to prevent OutOfMemoryException while inserting many objects
Emmanuel Bernard (JIRA)
noreply at atlassian.com
Tue Jun 5 19:17:04 EDT 2007
[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-33?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Emmanuel Bernard updated HSEARCH-33:
------------------------------------
Summary: hibernate.search.worker.batch_size to prevent OutOfMemoryException while inserting many objects (was: batch_size to prevent OutOfMemoryException while inserting many objects)
Done, the batch size apply today to all transactions (through .index or not), After all the OOME applies everywhere. So a wise value is expected
> hibernate.search.worker.batch_size to prevent OutOfMemoryException while inserting many objects
> -----------------------------------------------------------------------------------------------
>
> Key: HSEARCH-33
> URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-33
> Project: Hibernate Search
> Issue Type: New Feature
> Components: engine
> Affects Versions: 3.0.0.beta1
> Reporter: Stefan
>
> While inserting many objects in a batch the application runs out of memory because the FullTextIndexEventListener for Lucene collects the updates.
> This is a tough problem. Today the solution is to apply the work in n transactions rather than 1. Maybe the notion of batch_size at some point to force a "flush" when the queue goes up can help. But it sacrifices the transactional semantic.
> hibernate.cfg.xml
> <event type="post-insert">
> <listener class="org.hibernate.search.event.FullTextIndexEventListener"/>
> </event>
> Inserting code something like:
> Session session = sessionFactory.openSession();
> session.setCacheMode(CacheMode.IGNORE);
> Transaction tx = session.beginTransaction();
> for ( int i=0; i<100000; i++ ) {
> Item item = new Item(...);
> session.save(item);
> if ( i % 100 == 0 ) {
> session.flush();
> session.clear();
> }
> }
> tx.commit();
> session.close();
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
More information about the hibernate-issues
mailing list