Edson Tirelli [
http://community.jboss.org/people/tirelli] replied to the discussion
"How to configure and scale a large stateful CEP service?"
To view the discussion, visit:
http://community.jboss.org/message/554689#554689
--------------------------------------------------------------
That is a broad question! :)
Jeff DeLong wrote:
The question is how to deploy and scale a large stateful CEP service. CEP services could
be required to handle thousands or more events per minute, and hold millions of events in
working memory (along with a huge number of facts). The memory requirements could be
enormous (several gigabytes)
An initial approach to these sorts of requirements is to deploy the CEP service on its
own dedicated SOA-P server, and configure it's VM sizing appropriately. What would be
reasonable here? I.e. what is the largest reasonable single server JVM configuration? 6 GB
RAM, 8 GB RAM, ...?
However, if the volume of events would overwhelm a single server, what are the options
for scalability? Can a working memory be spread across VMs? Could Drools take advantage of
a federated cache such as Infinispan? What other options are there to scale.
Even if a single server is capable of handling the load, what would a highly available
topology look like? Could a second server be deployed with the same configuration, and
configure a service invoker load balancing policy that always went to the first service if
it were available?
Regarding memory, we have customers with up to 32Gb of memory running Drools, although I
am not aware of any actually using over 20Gb. On 64 bits machines there is no real problem
in increasing the heap if the memory is available.
Regarding topology, a WM can not be distributed over multiple JVMs. There is some research
going on on using Infinispan cache, but there is nothing yet on versions up to Drools
5.1.
I think it is important to realize that competing products in general split and execute
each and every query in a separate space with its own "local" memory. An analogy
would be to create a separate knowledge base in drools for each rule/query and then spawn
a separate session for each kbase. This is obviously sub-optimal as we would not benefit
from inter-rule optimizations. So, we will probably have to try tunning large use cases by
creating multiple kbases of "related" rules. This would allow us to easily
distribute each kbase over different JVMs with their respective sessions.
HA is something we need to discuss in separate. The engine does not support HA internally
as of 5.1. It can be implemented as a master/master HA using external tools, and that is
how it is usually done today. Although, alternatives need to be developed for the future.
--------------------------------------------------------------
Reply to this message by going to Community
[
http://community.jboss.org/message/554689#554689]
Start a new discussion in JBoss ESB Development at Community
[
http://community.jboss.org/choose-container!input.jspa?contentType=1&...]