[rules-users] Working memory batch insert performance

[Mark Proctor]

On 20/12/2011 13:09, Zhuo Li wrote:
>
> Hi, folks,
>
> I recently did a benchmark on Drools 5.1.2 and noticed that data 
> insert into a stateful session is very time consuming. It took me 
> about 30 minutes to insert 10,000 data rows on a 512M heapsize JVM. 
> Hence I have to keep inserting data rows when I receive them and keep 
> them in working memory, rather than loading them in a batch at a given 
> time. This is not a friendly way for disaster recovery and I have two 
> questions here to see if anybody has any thoughts:
>
10K rows? is that 10K bean insertions? 30 minutes sounds bad. We know 
people doing far more than that much quicker.
>
> 1.Is there any better way to improve the performance of data insert 
> into a stateful session;
>
There is nothing faster than "insert".
If you don't need inference, you can try turning on "sequential" mode, 
but in general the performance gain is < 5%.
>
> 2.I noticed that there is a method called BatchExecution() for a 
> stateless session. Did not get a chance to test it yet but is this a 
> better way to load data in a batch and then run rules?
>
That is related to scripting an engine, it uses command objects to call 
the inert() method - so definitely not faster.
>
> My requirement is I need to load a batch of data once by end of the 
> day, and then run the rules to filter out matched data against 
> unmatched data. I have a 3-hour processing window to complete this 
> loading and matching process, and the data I need to load is about 1 
> million to 2 millions. My JVM heapsize can be set up to 1024 M.
>
> Best regards
>
> Abe
>
>
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20111220/c1e662b7/attachment.html