Re: [rules-users] Parallel processing of large batches of facts

Interesting suggestions. Couple of questions: Is drools not internally able to multithread a single execution of fireallrules? Does CEP offer performance benefits, or is it just a different way of structuring the problem? Cheers, Tim 2010/10/12 Michael Anstis <michael.anstis(a)gmail.com&gt; > Can the aggregation or timestamp range be used to partion your data? > > e.g. if you're looking for a data pattern where a fact matches X and Y and > Z can the most course constraint, say X, not be used to partion? > > So you may have pre-processing (to partion the data) before hitting other > finer grained rules? > > Would CEP in stream mode provide an opportunity either? > > 2010/10/12 Wolfgang Laun <wolfgang.laun(a)gmail.com&gt; > > If you have to use a stateful session, with new objects being generated >> in RHS code and triggering more rules, then you've had it (since there is no >> way to split the 100k facts). >> >> If you don't create new facts in RHS code, you should investigate >> stateless sessions. It should be more efficient. >> >> Also, think about what you RHS code does. Can this processing be >> delegated to another thread? After a rule has fired, the "fate" of this >> activation is firmly established; rather than executing (timeconsuming i/o?) >> operations inline, queue the collected date to a processor thread and let >> this one crunch it. >> >> (I've only thought about this for a few minutes, so there may be other >> options.) >> >> -W >> >> 2010/10/12 Tim Jones <jones.tim36(a)gmail.com&gt; >> >>> Hello, >>> >>> >>> >>> I’m working on a project that needs a high performance rules system for >>> processing batches of objects. Typically I’ll have a dozen or so rules, the >>> most complex of which will aggregate several objects based on timestamps and >>> specified data patterns. The objects will come in batches of a few 100ks. >>> The system is reset back to the starting point after each batch is >>> processed. >>> >>> >>> >>> My guess at doing this with Drools is that you load up all the rules and >>> enter all the objects as “facts”. You then hit fireallrules and sit back and >>> wait. Doing this, I only get so much performance and I can see that its only >>> using a single thread. Is there a way to process the whole lot in a >>> parallel, or multithreaded way? Unfortunately there's no natural way to >>> partition the objects that would make things easier. >>> >>> >>> >>> Cheers, >>> >>> Tim >>> >>> _______________________________________________ >>> rules-users mailing list >>> rules-users(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/rules-users >>> >>> >> >> _______________________________________________ >> rules-users mailing list >> rules-users(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/rules-users >> >> > > _______________________________________________ > rules-users mailing list > rules-users(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/rules-users > > _______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users