[rules-users] Sessions Sharing KnowledgeBase
Edson Tirelli
ed.tirelli at gmail.com
Tue Dec 22 16:24:13 EST 2009
Cool, thanks for letting us know about the results.
Regarding my suggestion, just to be clear, you would still use one
stateful session for each of your requests. The only difference is that all
sessions would be created from a shared kbase, meaning you would be able to
stop building kbases from packages for each request.
Take a look at the drools flow manual. I believe (without knowing more
details about your project) that all you need is to use flow to coordinate
which rules are allowed to fire and in which sequence for each of your
requests.
Edson
2009/12/22 malkhafaji <moe.alkhafaji at medcpu.com>
>
> Edson,
>
> I just want to report back that when I upgraded to M1, I was able to get
> through 20 or 30 more requests before I got the deadlock again. Then, I
> used
> the link you sent me and downloaded the snapshot branch, and was able to go
> through 875 requests (so far and running) without a deadlock!! It is
> working
> fine, and results are much much faster!!! I shaved over 200 ms per request
> and now I am averaging < 30 ms per request in the engine!
>
> Once I am done with this software release, I want to explore what you
> suggested above. Can you direct me to documentations or some code that will
> show me how I can push multiple objects through the same drools instance
> (stateful) but each one will use a certain set of loaded packages? Those
> objects will come and go at different times (so the engine may be already
> running and it receives a new object.
>
> I am not looking for detailed code or anything, just basic code or a place
> on the documentation, or even an eclipse hello world project that you have.
> Much appreciated. Thank you again for your quick responses, and THE FIX!
>
>
> Edson Tirelli-4 wrote:
> >
> > Well, May 19th I guess you still have Drools 5.0.0. You should at
> least
> > be using 5.0.1. Anyway, latest artifacts are here:
> >
> >
> https://hudson.jboss.org/hudson/job/drools/lastSuccessfulBuild/artifact/trunk/target/
> >
> > "I am sure there are many ways to get it working on a sharing knowledge
> > base,"
> >
> > Yes, and if what you have works for you, go for it. I am just
> > mentioning
> > that the simplest solution would be to load all rules in a single kbase
> > and
> > use ruleflow, by what you described in your 1-3 items.
> >
> > Cheers,
> > Edson
> >
> > 2009/12/22 malkhafaji <moe.alkhafaji at medcpu.com>
> >
> >>
> >> Edson,
> >>
> >> Thank you for replying to my question. I haven't tried the new release
> >> since
> >> May 19th of this year. Are you talking about the binaries (Drools
> >> 5.1.0.M1
> >> Binaries)? Should I go ahead and try that?
> >>
> >> To answer your question about not sharing KnowledgeBase for all
> requests,
> >> we
> >> have a very advanced logic outside of the rules engine. To give you some
> >> flavor, we have only one type of object per request. Each request will
> >> have
> >> its own data in that object. Now, we have wrapper code around the engine
> >> to
> >> do things like:
> >> 1. Last time this engine ran for this object, including what rules
> >> executed,
> >> etc.
> >> 2. Rules within packages loaded by default may load other packages on
> the
> >> fly because of which rules executed, and this only applies to that
> object
> >> that triggered the loading of those packages. For example, by default we
> >> have 1 package that is used for all objects. But object A may trigger a
> >> rule
> >> that loads another package which becomes available to it besides the
> main
> >> one. However, this new package is not yet available to all other
> objects.
> >> So, I did not know how I can load packages but only make them visible to
> >> some of the objects that I push through the session's knowledge base.
> >> 3. Our rules engine is either data driven (and the notification does not
> >> come from the rule, but outside the rule in some common logic), or time
> >> driven (run the engine again after n minutes because that is when our
> >> logic
> >> has determined that one of the conditions would turn true on a
> previously
> >> false condition potentially starting a chain of executions), etc.
> >>
> >> And other things. So, I found it to be much easier for me and my sanity
> >> to
> >> start using a new instance of a session (having its own knowledge base)
> >> per
> >> request. Those requests are very small and take no more than
> milliseconds
> >> on
> >> each object. So, I am using a stateful session that is alive for the
> life
> >> of
> >> the respective object. My objects are finite (you cannot have more than
> >> 1,000 in our memory), and they and their respective instances are very
> >> light
> >> weight and do not need more than a 1 GB of memory (mostly hovers around
> >> 500-600 MB).
> >>
> >> I am sure there are many ways to get it working on a sharing knowledge
> >> base,
> >> I just thought this way I don't need to worry about synchronizations or
> >> any
> >> other complicated logic on top of what we have. So, I made the decision.
> >>
> >> Please confirm that M1 binaries has your latest fix so I can get testing
> >> on
> >> it. Thanks!
> >>
> >>
> >> Edson Tirelli-4 wrote:
> >> >
> >> > Hi,
> >> >
> >> > I fixed a problem related to deadlock when sharing packages among
> >> > multiple kbases a couple weeks ago. Not sure if it is the same, but
> try
> >> > trunk and let me know, plz.
> >> >
> >> > Meanwhile, I am curious why you can't simply create a kbase with
> all
> >> > your
> >> > rules and share the kbase among all your requests, just creating one
> >> > session
> >> > per request?
> >> >
> >> > Reason is that adding packages to a kbase for each request still
> >> adds
> >> > unnecessary overhead to your application, and kbases are designed to
> be
> >> > shared in such cases. If you need to orchestrate multiple execution
> >> paths
> >> > for your rules, you could use rule flow or any other coordination
> >> > mechanism.
> >> > Finally, Rete is known by performing really well with increasing
> number
> >> of
> >> > rules, since the performance is not dependent on the number of
> existing
> >> > rules, but on the number of affected constraints.
> >> >
> >> > Just my .02c
> >> >
> >> > Edson
> >> >
> >> >
> >> > 2009/12/22 malkhafaji <moe.alkhafaji at medcpu.com>
> >> >
> >> >>
> >> >> Hello,
> >> >>
> >> >> I am trying to create multiple sessions, and for each session I will
> >> load
> >> >> one knowledge package representing one drl. Each drl may optionally
> >> load
> >> >> additional knowledge packages (drls). Since all sessions will be
> >> >> accessing
> >> >> the same list of knowledge packages, and since they take a long time
> >> to
> >> >> compile resources, I decided to create a Map with all possible
> >> knowledge
> >> >> packages that all sessions may use. That worked beautifully saving me
> >> >> compilation time (it is done at start up).
> >> >>
> >> >> Now, here is how I am creating the static map:
> >> >>
> >> >> private static ConcurrentMap<String, KnowledgePackage>
> >> knowledgePackages
> >> >> =
> >> >> new
> >> >> ConcurrentHashMap<String, KnowledgePackage>();
> >> >>
> >> >> Here is how I am initializing it:
> >> >>
> >> >> KnowledgeBuilder kbuilder = null;
> >> >>
> >> >> for (MedCPUKnowledgeBase medcpuKnowledgeBase :
> >> >> medcpuKnowledgeBases) {
> >> >> try {
> >> >> kbuilder =
> >> >> KnowledgeBuilderFactory.newKnowledgeBuilder();
> >> >>
> >> >> // This call actually compiles the drl
> >> >> file.
> >> >>
> >> >>
> >> >>
> >>
> kbuilder.add(ResourceFactory.newClassPathResource(medcpuKnowledgeBase.getFileName()),
> >> >> ResourceType.DRL);
> >> >>
> >> >> KnowledgeBuilderErrors errors =
> >> >> kbuilder.getErrors();
> >> >>
> >> >> if (errors.size() > 0) {
> >> >> .....
> >> >> } else {
> >> >>
> >> >> knowledgePackages.put(medcpuKnowledgeBase.getFileName(),
> >> >> [the package
> >> just
> >> >> added above]);
> >> >> }
> >> >> } catch(Exception ex) {
> >> >> ....
> >> >> } else {
> >> >> .....
> >> >> }
> >> >> }
> >> >> }
> >> >>
> >> >>
> >> >> Each session will do this to get the pre-compiled KnowledgePackage:
> >> >>
> >> >> KnowledgePackage knowledgePackage = knowledgePackages.get(kb);
> >> >> ......
> >> >> Collection<KnowledgePackage> packages = new
> >> >> ArrayList<KnowledgePackage>();
> >> >> packages.add(knowledgePackage);
> >> >>
> >> >> // The problem is this guy (after a relatively small number of
> >> requests
> >> >> each
> >> >> opening a new session) just gets stuck!! It does not throw an
> >> exception
> >> >> and
> >> >> it does not return.
> >> >> this.knowledgeBase.addKnowledgePackages(packages);
> >> >>
> >> >> When I did not share the KnowledgePackages like I did above, I was
> >> able
> >> >> to
> >> >> get thousands of requests with each having its own session,
> >> >> knowledgebase,
> >> >> and its own knowledge packages after they compile the respective
> drls.
> >> >> However, because this was slow compiling the same drls over and over
> >> >> again
> >> >> I
> >> >> switched to the design above which after only 20 or 30 requests it
> >> gets
> >> >> stuck on the line above. I still have a knowledgebase per session per
> >> >> request, but now I am sharing the knowledge packages.
> >> >>
> >> >> I cannot find anywhere where it says that KnowledgePackage objects
> are
> >> >> not
> >> >> thread-safe (not safe to be shared between sessions/threads). Any
> idea
> >> >> why
> >> >> I
> >> >> am stuck on the line above? Thanks!
> >> >> --
> >> >> View this message in context:
> >> >>
> http://n3.nabble.com/Sessions-Sharing-KnowledgeBase-tp96894p96894.html
> >> >> Sent from the Drools - User mailing list archive at Nabble.com.
> >> >> _______________________________________________
> >> >> rules-users mailing list
> >> >> rules-users at lists.jboss.org
> >> >> https://lists.jboss.org/mailman/listinfo/rules-users
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Edson Tirelli
> >> > JBoss Drools Core Development
> >> > JBoss by Red Hat @ www.jboss.com
> >> >
> >> > _______________________________________________
> >> > rules-users mailing list
> >> > rules-users at lists.jboss.org
> >> > https://lists.jboss.org/mailman/listinfo/rules-users
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >> http://n3.nabble.com/Sessions-Sharing-KnowledgeBase-tp96894p97452.html
> >> Sent from the Drools - User mailing list archive at Nabble.com.
> >> _______________________________________________
> >> rules-users mailing list
> >> rules-users at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/rules-users
> >>
> >
> >
> >
> > --
> > Edson Tirelli
> > JBoss Drools Core Development
> > JBoss by Red Hat @ www.jboss.com
> >
> > _______________________________________________
> > rules-users mailing list
> > rules-users at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/rules-users
> >
> >
>
> --
> View this message in context:
> http://n3.nabble.com/Sessions-Sharing-KnowledgeBase-tp96894p97506.html
> Sent from the Drools - User mailing list archive at Nabble.com.
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
--
Edson Tirelli
JBoss Drools Core Development
JBoss by Red Hat @ www.jboss.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/rules-users/attachments/20091222/5e87f6dd/attachment.html
More information about the rules-users
mailing list