10:40 a.m.
Hi all
Looking at "Drools JBoss Rules 5 0 Developer's guide" it addresses the
Fusion component at page 134, stating:
"This problem cannot be solved by the standard Drools rule engine. The
volume of events is huge and it happens asynchronously. If we simply
inserted them into the knowledge session, we would soon run out of
memory. While the Rete algorithm behind Drools doesn't have any
theoretical limitation on number of objects in the session, we could
use the processing power more wisely. Drools Fusion is the right
candidate for this kind of task."
Ok, but my question is if this is enough, because I would guess that
Fusion could be used to derived a more complex/rich event from the
stream of events, and afterwards that complex event would be inserted
in Working Memory (but not the flow of events that have generated the
more complex event), thus reducing RETE algorithm processing. But
since in our case every event will most certainly fire a rule, I am
not sure about Fusion's usage.
This is because our project is more a kind of
Event-Trigger-Rule/Action system, meaning we have a high volume of
incoming events (say more than 5000/minute), but each of the events
will have to trigger a rule (and usually a rule is more or less a
complex action, invoking DB operations and data structures
manipulation).
So, by using Fusion entry-points, would that be different than just
sending the events as "normal" facts to Working Memory, or by dividing
this in entry-points( or so called streams) we would have benefits in
the RETE processing (I am assuming Fusion also uses RETE someway)?
Best regards
Orlando.
PS-Does anyone know where can I find more info on the coupling of
Drools Expert and Fusion?
11:02 a.m.
Hi Orlando,
Yes Fusion runs on top of the rule engine and uses the RETE algorithm.
You can see Fusion as an extension of Drools Expert to support temporal
evaluations.
On Tue, Aug 17, 2010 at 12:40 PM, Orlando Ricardo Da Silva Costa <
orlandoricardo(a)gmail.com> wrote:
Hi all
Looking at "Drools JBoss Rules 5 0 Developer's guide" it addresses the
Fusion component at page 134, stating:
"This problem cannot be solved by the standard Drools rule engine. The
volume of events is huge and it happens asynchronously. If we simply
inserted them into the knowledge session, we would soon run out of
memory. While the Rete algorithm behind Drools doesn't have any
theoretical limitation on number of objects in the session, we could
use the processing power more wisely. Drools Fusion is the right
candidate for this kind of task."
Ok, but my question is if this is enough, because I would guess that
Fusion could be used to derived a more complex/rich event from the
stream of events, and afterwards that complex event would be inserted
in Working Memory (but not the flow of events that have generated the
more complex event), thus reducing RETE algorithm processing. But
since in our case every event will most certainly fire a rule, I am
not sure about Fusion's usage.
This is because our project is more a kind of
Event-Trigger-Rule/Action system, meaning we have a high volume of
incoming events (say more than 5000/minute), but each of the events
will have to trigger a rule (and usually a rule is more or less a
complex action, invoking DB operations and data structures
manipulation).
So, by using Fusion entry-points, would that be different than just
sending the events as "normal" facts to Working Memory, or by dividing
this in entry-points( or so called streams) we would have benefits in
the RETE processing (I am assuming Fusion also uses RETE someway)?
Best regards
Orlando.
PS-Does anyone know where can I find more info on the coupling of
Drools Expert and Fusion?
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
--
- CTO @ http://www.plugtree.com
- MyJourney @ http://salaboy.wordpress.com
- Co-Founder @ http://www.jbug.com.ar
- Salatino "Salaboy" Mauricio -
1 p.m.
Thanks Mauricio, but since in general RETE algorithm does not get along well
with high volumes of information, I was wondering if I could get more
details on:
1. the implementation architecture
2. about Drools Fusion performance, does anyone has already some experience
or links to share?
Br
Orlando.
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/Drools-Fusion-Perform...
Sent from the Drools - User mailing list archive at Nabble.com.
1:28 p.m.
Google for a presentation that shows how fedex monitor a lot of trucks in
near real time with fusion.
I can't find the link, but probably Edson have it.
On Tue, Aug 17, 2010 at 3:00 PM, Orlando Costa <orlandoricardo(a)gmail.com>wrote:
Thanks Mauricio, but since in general RETE algorithm does not get along
well
with high volumes of information, I was wondering if I could get more
details on:
1. the implementation architecture
2. about Drools Fusion performance, does anyone has already some experience
or links to share?
Br
Orlando.
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/Drools-Fusion-Perform...
Sent from the Drools - User mailing list archive at Nabble.com.
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
--
- CTO @ http://www.plugtree.com
- MyJourney @ http://salaboy.wordpress.com
- Co-Founder @ http://www.jbug.com.ar
- Salatino "Salaboy" Mauricio -
1:37 p.m.
Thanks, I already have seen this some days ago:
http://www.redhat.com/f/pdf/jbw/amollenkopf_430_applying_drools.pdf
but it does not say the amount of events coming, and the internals are more
about the whole solution itself than about the internal Drools Expert/Fusion
architecture (though is shows good rules examples with Fusion use of
course).
Br
Orlando.
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/Drools-Fusion-Perform...
Sent from the Drools - User mailing list archive at Nabble.com.
2:55 p.m.
Hi Orlando, Mauricio,
While the goal of the whole Drools platform is to provide a seamless
environment for users to model business behavior using rules, processes and
events, it is a misconception to say that it is all executed on RETE.
Besides implementing several optimizations on top of the classic RETE
algorithm (as designed by Dr. Forgy back in the 70's), Drools has specific
algorithms to which it delegates processing of specific use cases/features
(read some fusion and flow features).
Regarding your question, entry-points are partitions on the RETE network
that allow for parts of it to be evaluated concurrently, besides creating
separate "name spaces" for facts (scopes if you will). So it does help on
reducing the overall load of the matching algorithm. Although, rules are
still fired sequentially as to ensure consistency of the working memory.
Regarding use cases, we have customers that use 24x7 stateful sessions,
keeping over 1M facts in memory constantly, with new facts coming into the
session and facts expiring from the session all the time. The throughput of
such use cases is obviously dependent on the percentage of facts/rules
matchings, so it is hard to foresee what you will achieve on your use case
without knowing in detail your rules.
Although, the chosen architecture for your kbase and sessions will
obviously have a high impact on the performance metrics you will achieve.
Since you mentioned that in your use case every single fact will match and
cause rules to fire, and the actions involve heavy operations like database
access, I would recommend looking into an "agent" architecture, where you
partition your knowledge base into several kbases with related rules. The
kbases will operate on the atomic (or "raw") events and create the composite
events that you can feed into entry points of sessions of subsequent kbases,
effectively increasing the overall system throughput and simplifying the
maintenance of each agent's kbase in particular.
If you are used to CEP nomenclature, you know that the above is a
simplified description of an EPN.
Finally, you asked about the difference between regular facts and
events, and basically all events are facts with special "properties". These
properties allow the engine to do some fancy stuff with them, like the use
of sliding windows or event lifecycle management (i.e., automatically
expiring events). Although, there is nothing wrong in modeling events like
regular facts if that best fits your design.
Hope the above helps,
Edson
2010/8/17 Orlando Costa <orlandoricardo(a)gmail.com>
Thanks, I already have seen this some days ago:
http://www.redhat.com/f/pdf/jbw/amollenkopf_430_applying_drools.pdf
but it does not say the amount of events coming, and the internals are more
about the whole solution itself than about the internal Drools
Expert/Fusion
architecture (though is shows good rules examples with Fusion use of
course).
Br
Orlando.
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/Drools-Fusion-Perform...
Sent from the Drools - User mailing list archive at Nabble.com.
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
--
Edson Tirelli
JBoss Drools Core Development
JBoss by Red Hat @ www.jboss.com
3:37 p.m.
Hi Edson,
This was really the kind of information I was looking for, many thanks! (I
think you should even put this text highlighted on the documentation
available at Drools site, these are very valuable insights).
I am kind of new in ESP/CEP (been reading "Event Processing - Designing IT
Systems for Agile Companies" and many other online info), but when you talk
about EPN (Event-Processing Network), this is a concept that I am not yet
familiar. This paragraph were you say:
"Although, the chosen architecture for your kbase and sessions will
obviously have a high impact on the performance metrics you will achieve.
Since you mentioned that in your use case every single fact will match and
cause rules to fire, and the actions involve heavy operations like database
access, I would recommend looking into an "agent" architecture, where you
partition your knowledge base into several kbases with related rules. The
kbases will operate on the atomic (or "raw") events and create the composite
events that you can feed into entry points of sessions of subsequent kbases,
effectively increasing the overall system throughput and simplifying the
maintenance of each agent's kbase in particular."
looks quite interesting to explore. Besides the book I mentioned (and I am
also going to check this on "Event Processing in Action" by Etzion) do you
recommend any resource (book, article, etc) that elaborates more on the
concepts you have described?
Best regards
Orlando.
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/Drools-Fusion-Perform...
Sent from the Drools - User mailing list archive at Nabble.com.
4:03 p.m.
Hi Orlando,
Dr. David Luckam's website has a lot of content on CEP:
http://www.complexevents.com/
<http://www.complexevents.com/> Also, his book "The Power of Events" is
considered the publication that formalized the concept of CEP. It is a good
book, very helpful in understanding the overall concepts. Just be aware that
being a "visionary" book, on one hand, lots of requirements for CEP products
described there are still wishful thinking that no product on the market
implemented yet and on the other hand, some products offer several features
that were not envisioned by him at the time.
Edson
2010/8/18 Orlando Costa <orlandoricardo(a)gmail.com>
Hi Edson,
This was really the kind of information I was looking for, many thanks! (I
think you should even put this text highlighted on the documentation
available at Drools site, these are very valuable insights).
I am kind of new in ESP/CEP (been reading "Event Processing - Designing IT
Systems for Agile Companies" and many other online info), but when you talk
about EPN (Event-Processing Network), this is a concept that I am not yet
familiar. This paragraph were you say:
"Although, the chosen architecture for your kbase and sessions will
obviously have a high impact on the performance metrics you will achieve.
Since you mentioned that in your use case every single fact will match and
cause rules to fire, and the actions involve heavy operations like database
access, I would recommend looking into an "agent" architecture, where you
partition your knowledge base into several kbases with related rules. The
kbases will operate on the atomic (or "raw") events and create the
composite
events that you can feed into entry points of sessions of subsequent
kbases,
effectively increasing the overall system throughput and simplifying the
maintenance of each agent's kbase in particular."
looks quite interesting to explore. Besides the book I mentioned (and I am
also going to check this on "Event Processing in Action" by Etzion) do you
recommend any resource (book, article, etc) that elaborates more on the
concepts you have described?
Best regards
Orlando.
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/Drools-Fusion-Perform...
Sent from the Drools - User mailing list archive at Nabble.com.
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
--
Edson Tirelli
JBoss Drools Core Development
JBoss by Red Hat @ www.jboss.com
4:21 p.m.
Hi Edson
I was more refering to more sub-set of ESP/CEP in my question, because of
the description you have made about EPN and "agent" architecture that could
be used as a footprint to a possible solution.
Thus, I was wondering if there was more technical resource just on that
topic (and not so much in CEP theory/background in general, as I already
material on that).
Best regards
Orlando.
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/Drools-Fusion-Perform...
Sent from the Drools - User mailing list archive at Nabble.com.
8:56 a.m.
Orlando,
"The Power of Events" has a full chapter on EPN. Also, it is a common
subject on the network.
Edson
2010/8/18 Orlando Costa <orlandoricardo(a)gmail.com>
Hi Edson
I was more refering to more sub-set of ESP/CEP in my question, because of
the description you have made about EPN and "agent" architecture that could
be used as a footprint to a possible solution.
Thus, I was wondering if there was more technical resource just on that
topic (and not so much in CEP theory/background in general, as I already
material on that).
Best regards
Orlando.
--
View this message in context:
http://drools-java-rules-engine.46999.n3.nabble.com/Drools-Fusion-Perform...
Sent from the Drools - User mailing list archive at Nabble.com.
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
--
Edson Tirelli
JBoss Drools Core Development
JBoss by Red Hat @ www.jboss.com
5213
days inactive
5215
days old
9 comments
4 participants
participants (4)
-
Edson Tirelli -
Mauricio Salatino -
Orlando Costa -
Orlando Ricardo Da Silva Costa