[rules-users] Parallel processing of large batches of facts

[Michael Anstis]

Can the aggregation or timestamp range be used to partion your data?

e.g. if you're looking for a data pattern where a fact matches X and Y and Z
can the most course constraint, say X, not be used to partion?

So you may have pre-processing (to partion the data) before hitting other
finer grained rules?

Would CEP in stream mode provide an opportunity either?

2010/10/12 Wolfgang Laun <wolfgang.laun at gmail.com>

> If you have to use a stateful session, with new objects being generated in
> RHS code and triggering more rules, then you've had it (since there is no
> way to split the 100k facts).
>
> If you don't create new facts in RHS code, you should investigate stateless
> sessions. It should be more efficient.
>
> Also, think about what you RHS code does. Can this processing be delegated
> to another thread? After a rule has fired, the "fate" of this activation is
> firmly established; rather than executing (timeconsuming i/o?) operations
> inline, queue the collected date to a processor thread and let this one
> crunch it.
>
> (I've only thought about this for a few minutes, so there may be other
> options.)
>
> -W
>
> 2010/10/12 Tim Jones <jones.tim36 at gmail.com>
>
>> Hello,
>>
>>
>>
>> I’m working on a project that needs a high performance rules system for
>> processing batches of objects. Typically I’ll have a dozen or so rules, the
>> most complex of which will aggregate several objects based on timestamps and
>> specified data patterns. The objects will come in batches of a few 100ks.
>> The system is reset back to the starting point after each batch is
>> processed.
>>
>>
>>
>> My guess at doing this with Drools is that you load up all the rules and
>> enter all the objects as “facts”. You then hit fireallrules and sit back and
>> wait. Doing this, I only get so much performance and I can see that its only
>> using a single thread. Is there a way to process the whole lot in a
>> parallel, or multithreaded way? Unfortunately there's no natural way to
>> partition the objects that would make things easier.
>>
>>
>>
>> Cheers,
>>
>> Tim
>>
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
>>
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20101012/e8f49d28/attachment.html