[rules-users] Sessions Sharing KnowledgeBase
malkhafaji
moe.alkhafaji at medcpu.com
Tue Dec 22 12:51:46 EST 2009
Edson,
I just want to report back that when I upgraded to M1, I was able to get
through 20 or 30 more requests before I got the deadlock again. Then, I used
the link you sent me and downloaded the snapshot branch, and was able to go
through 875 requests (so far and running) without a deadlock!! It is working
fine, and results are much much faster!!! I shaved over 200 ms per request
and now I am averaging < 30 ms per request in the engine!
Once I am done with this software release, I want to explore what you
suggested above. Can you direct me to documentations or some code that will
show me how I can push multiple objects through the same drools instance
(stateful) but each one will use a certain set of loaded packages? Those
objects will come and go at different times (so the engine may be already
running and it receives a new object.
I am not looking for detailed code or anything, just basic code or a place
on the documentation, or even an eclipse hello world project that you have.
Much appreciated. Thank you again for your quick responses, and THE FIX!
Edson Tirelli-4 wrote:
>
> Well, May 19th I guess you still have Drools 5.0.0. You should at least
> be using 5.0.1. Anyway, latest artifacts are here:
>
> https://hudson.jboss.org/hudson/job/drools/lastSuccessfulBuild/artifact/trunk/target/
>
> "I am sure there are many ways to get it working on a sharing knowledge
> base,"
>
> Yes, and if what you have works for you, go for it. I am just
> mentioning
> that the simplest solution would be to load all rules in a single kbase
> and
> use ruleflow, by what you described in your 1-3 items.
>
> Cheers,
> Edson
>
> 2009/12/22 malkhafaji <moe.alkhafaji at medcpu.com>
>
>>
>> Edson,
>>
>> Thank you for replying to my question. I haven't tried the new release
>> since
>> May 19th of this year. Are you talking about the binaries (Drools
>> 5.1.0.M1
>> Binaries)? Should I go ahead and try that?
>>
>> To answer your question about not sharing KnowledgeBase for all requests,
>> we
>> have a very advanced logic outside of the rules engine. To give you some
>> flavor, we have only one type of object per request. Each request will
>> have
>> its own data in that object. Now, we have wrapper code around the engine
>> to
>> do things like:
>> 1. Last time this engine ran for this object, including what rules
>> executed,
>> etc.
>> 2. Rules within packages loaded by default may load other packages on the
>> fly because of which rules executed, and this only applies to that object
>> that triggered the loading of those packages. For example, by default we
>> have 1 package that is used for all objects. But object A may trigger a
>> rule
>> that loads another package which becomes available to it besides the main
>> one. However, this new package is not yet available to all other objects.
>> So, I did not know how I can load packages but only make them visible to
>> some of the objects that I push through the session's knowledge base.
>> 3. Our rules engine is either data driven (and the notification does not
>> come from the rule, but outside the rule in some common logic), or time
>> driven (run the engine again after n minutes because that is when our
>> logic
>> has determined that one of the conditions would turn true on a previously
>> false condition potentially starting a chain of executions), etc.
>>
>> And other things. So, I found it to be much easier for me and my sanity
>> to
>> start using a new instance of a session (having its own knowledge base)
>> per
>> request. Those requests are very small and take no more than milliseconds
>> on
>> each object. So, I am using a stateful session that is alive for the life
>> of
>> the respective object. My objects are finite (you cannot have more than
>> 1,000 in our memory), and they and their respective instances are very
>> light
>> weight and do not need more than a 1 GB of memory (mostly hovers around
>> 500-600 MB).
>>
>> I am sure there are many ways to get it working on a sharing knowledge
>> base,
>> I just thought this way I don't need to worry about synchronizations or
>> any
>> other complicated logic on top of what we have. So, I made the decision.
>>
>> Please confirm that M1 binaries has your latest fix so I can get testing
>> on
>> it. Thanks!
>>
>>
>> Edson Tirelli-4 wrote:
>> >
>> > Hi,
>> >
>> > I fixed a problem related to deadlock when sharing packages among
>> > multiple kbases a couple weeks ago. Not sure if it is the same, but try
>> > trunk and let me know, plz.
>> >
>> > Meanwhile, I am curious why you can't simply create a kbase with all
>> > your
>> > rules and share the kbase among all your requests, just creating one
>> > session
>> > per request?
>> >
>> > Reason is that adding packages to a kbase for each request still
>> adds
>> > unnecessary overhead to your application, and kbases are designed to be
>> > shared in such cases. If you need to orchestrate multiple execution
>> paths
>> > for your rules, you could use rule flow or any other coordination
>> > mechanism.
>> > Finally, Rete is known by performing really well with increasing number
>> of
>> > rules, since the performance is not dependent on the number of existing
>> > rules, but on the number of affected constraints.
>> >
>> > Just my .02c
>> >
>> > Edson
>> >
>> >
>> > 2009/12/22 malkhafaji <moe.alkhafaji at medcpu.com>
>> >
>> >>
>> >> Hello,
>> >>
>> >> I am trying to create multiple sessions, and for each session I will
>> load
>> >> one knowledge package representing one drl. Each drl may optionally
>> load
>> >> additional knowledge packages (drls). Since all sessions will be
>> >> accessing
>> >> the same list of knowledge packages, and since they take a long time
>> to
>> >> compile resources, I decided to create a Map with all possible
>> knowledge
>> >> packages that all sessions may use. That worked beautifully saving me
>> >> compilation time (it is done at start up).
>> >>
>> >> Now, here is how I am creating the static map:
>> >>
>> >> private static ConcurrentMap<String, KnowledgePackage>
>> knowledgePackages
>> >> =
>> >> new
>> >> ConcurrentHashMap<String, KnowledgePackage>();
>> >>
>> >> Here is how I am initializing it:
>> >>
>> >> KnowledgeBuilder kbuilder = null;
>> >>
>> >> for (MedCPUKnowledgeBase medcpuKnowledgeBase :
>> >> medcpuKnowledgeBases) {
>> >> try {
>> >> kbuilder =
>> >> KnowledgeBuilderFactory.newKnowledgeBuilder();
>> >>
>> >> // This call actually compiles the drl
>> >> file.
>> >>
>> >>
>> >>
>> kbuilder.add(ResourceFactory.newClassPathResource(medcpuKnowledgeBase.getFileName()),
>> >> ResourceType.DRL);
>> >>
>> >> KnowledgeBuilderErrors errors =
>> >> kbuilder.getErrors();
>> >>
>> >> if (errors.size() > 0) {
>> >> .....
>> >> } else {
>> >>
>> >> knowledgePackages.put(medcpuKnowledgeBase.getFileName(),
>> >> [the package
>> just
>> >> added above]);
>> >> }
>> >> } catch(Exception ex) {
>> >> ....
>> >> } else {
>> >> .....
>> >> }
>> >> }
>> >> }
>> >>
>> >>
>> >> Each session will do this to get the pre-compiled KnowledgePackage:
>> >>
>> >> KnowledgePackage knowledgePackage = knowledgePackages.get(kb);
>> >> ......
>> >> Collection<KnowledgePackage> packages = new
>> >> ArrayList<KnowledgePackage>();
>> >> packages.add(knowledgePackage);
>> >>
>> >> // The problem is this guy (after a relatively small number of
>> requests
>> >> each
>> >> opening a new session) just gets stuck!! It does not throw an
>> exception
>> >> and
>> >> it does not return.
>> >> this.knowledgeBase.addKnowledgePackages(packages);
>> >>
>> >> When I did not share the KnowledgePackages like I did above, I was
>> able
>> >> to
>> >> get thousands of requests with each having its own session,
>> >> knowledgebase,
>> >> and its own knowledge packages after they compile the respective drls.
>> >> However, because this was slow compiling the same drls over and over
>> >> again
>> >> I
>> >> switched to the design above which after only 20 or 30 requests it
>> gets
>> >> stuck on the line above. I still have a knowledgebase per session per
>> >> request, but now I am sharing the knowledge packages.
>> >>
>> >> I cannot find anywhere where it says that KnowledgePackage objects are
>> >> not
>> >> thread-safe (not safe to be shared between sessions/threads). Any idea
>> >> why
>> >> I
>> >> am stuck on the line above? Thanks!
>> >> --
>> >> View this message in context:
>> >> http://n3.nabble.com/Sessions-Sharing-KnowledgeBase-tp96894p96894.html
>> >> Sent from the Drools - User mailing list archive at Nabble.com.
>> >> _______________________________________________
>> >> rules-users mailing list
>> >> rules-users at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/rules-users
>> >>
>> >
>> >
>> >
>> > --
>> > Edson Tirelli
>> > JBoss Drools Core Development
>> > JBoss by Red Hat @ www.jboss.com
>> >
>> > _______________________________________________
>> > rules-users mailing list
>> > rules-users at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/rules-users
>> >
>> >
>>
>> --
>> View this message in context:
>> http://n3.nabble.com/Sessions-Sharing-KnowledgeBase-tp96894p97452.html
>> Sent from the Drools - User mailing list archive at Nabble.com.
>> _______________________________________________
>> rules-users mailing list
>> rules-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>>
>
>
>
> --
> Edson Tirelli
> JBoss Drools Core Development
> JBoss by Red Hat @ www.jboss.com
>
> _______________________________________________
> rules-users mailing list
> rules-users at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
--
View this message in context: http://n3.nabble.com/Sessions-Sharing-KnowledgeBase-tp96894p97506.html
Sent from the Drools - User mailing list archive at Nabble.com.
More information about the rules-users
mailing list