Re: [rules-users] Sessions Sharing KnowledgeBase

Edson, I just want to report back that when I upgraded to M1, I was able to get through 20 or 30 more requests before I got the deadlock again. Then, I used the link you sent me and downloaded the snapshot branch, and was able to go through 875 requests (so far and running) without a deadlock!! It is working fine, and results are much much faster!!! I shaved over 200 ms per request and now I am averaging < 30 ms per request in the engine! Once I am done with this software release, I want to explore what you suggested above. Can you direct me to documentations or some code that will show me how I can push multiple objects through the same drools instance (stateful) but each one will use a certain set of loaded packages? Those objects will come and go at different times (so the engine may be already running and it receives a new object. I am not looking for detailed code or anything, just basic code or a place on the documentation, or even an eclipse hello world project that you have. Much appreciated. Thank you again for your quick responses, and THE FIX! Edson Tirelli-4 wrote: > > Well, May 19th I guess you still have Drools 5.0.0. You should at least > be using 5.0.1. Anyway, latest artifacts are here: > > https://hudson.jboss.org/hudson/job/drools/lastSuccessfulBuild/artifact/t... > > "I am sure there are many ways to get it working on a sharing knowledge > base," > > Yes, and if what you have works for you, go for it. I am just > mentioning > that the simplest solution would be to load all rules in a single kbase > and > use ruleflow, by what you described in your 1-3 items. > > Cheers, > Edson > > 2009/12/22 malkhafaji <moe.alkhafaji(a)medcpu.com&gt; > >> >> Edson, >> >> Thank you for replying to my question. I haven't tried the new release >> since >> May 19th of this year. Are you talking about the binaries (Drools >> 5.1.0.M1 >> Binaries)? Should I go ahead and try that? >> >> To answer your question about not sharing KnowledgeBase for all requests, >> we >> have a very advanced logic outside of the rules engine. To give you some >> flavor, we have only one type of object per request. Each request will >> have >> its own data in that object. Now, we have wrapper code around the engine >> to >> do things like: >> 1. Last time this engine ran for this object, including what rules >> executed, >> etc. >> 2. Rules within packages loaded by default may load other packages on the >> fly because of which rules executed, and this only applies to that object >> that triggered the loading of those packages. For example, by default we >> have 1 package that is used for all objects. But object A may trigger a >> rule >> that loads another package which becomes available to it besides the main >> one. However, this new package is not yet available to all other objects. >> So, I did not know how I can load packages but only make them visible to >> some of the objects that I push through the session's knowledge base. >> 3. Our rules engine is either data driven (and the notification does not >> come from the rule, but outside the rule in some common logic), or time >> driven (run the engine again after n minutes because that is when our >> logic >> has determined that one of the conditions would turn true on a previously >> false condition potentially starting a chain of executions), etc. >> >> And other things. So, I found it to be much easier for me and my sanity >> to >> start using a new instance of a session (having its own knowledge base) >> per >> request. Those requests are very small and take no more than milliseconds >> on >> each object. So, I am using a stateful session that is alive for the life >> of >> the respective object. My objects are finite (you cannot have more than >> 1,000 in our memory), and they and their respective instances are very >> light >> weight and do not need more than a 1 GB of memory (mostly hovers around >> 500-600 MB). >> >> I am sure there are many ways to get it working on a sharing knowledge >> base, >> I just thought this way I don't need to worry about synchronizations or >> any >> other complicated logic on top of what we have. So, I made the decision. >> >> Please confirm that M1 binaries has your latest fix so I can get testing >> on >> it. Thanks! >> >> >> Edson Tirelli-4 wrote: >> > >> > Hi, >> > >> > I fixed a problem related to deadlock when sharing packages among >> > multiple kbases a couple weeks ago. Not sure if it is the same, but try >> > trunk and let me know, plz. >> > >> > Meanwhile, I am curious why you can't simply create a kbase with all >> > your >> > rules and share the kbase among all your requests, just creating one >> > session >> > per request? >> > >> > Reason is that adding packages to a kbase for each request still >> adds >> > unnecessary overhead to your application, and kbases are designed to be >> > shared in such cases. If you need to orchestrate multiple execution >> paths >> > for your rules, you could use rule flow or any other coordination >> > mechanism. >> > Finally, Rete is known by performing really well with increasing number >> of >> > rules, since the performance is not dependent on the number of existing >> > rules, but on the number of affected constraints. >> > >> > Just my .02c >> > >> > Edson >> > >> > >> > 2009/12/22 malkhafaji <moe.alkhafaji(a)medcpu.com&gt; >> > >> >> >> >> Hello, >> >> >> >> I am trying to create multiple sessions, and for each session I will >> load >> >> one knowledge package representing one drl. Each drl may optionally >> load >> >> additional knowledge packages (drls). Since all sessions will be >> >> accessing >> >> the same list of knowledge packages, and since they take a long time >> to >> >> compile resources, I decided to create a Map with all possible >> knowledge >> >> packages that all sessions may use. That worked beautifully saving me >> >> compilation time (it is done at start up). >> >> >> >> Now, here is how I am creating the static map: >> >> >> >> private static ConcurrentMap<String, KnowledgePackage> >> knowledgePackages >> >> = >> >> new >> >> ConcurrentHashMap<String, KnowledgePackage>(); >> >> >> >> Here is how I am initializing it: >> >> >> >> KnowledgeBuilder kbuilder = null; >> >> >> >> for (MedCPUKnowledgeBase medcpuKnowledgeBase : >> >> medcpuKnowledgeBases) { >> >> try { >> >> kbuilder = >> >> KnowledgeBuilderFactory.newKnowledgeBuilder(); >> >> >> >> // This call actually compiles the drl >> >> file. >> >> >> >> >> >> >> kbuilder.add(ResourceFactory.newClassPathResource(medcpuKnowledgeBase.getFileName()), >> >> ResourceType.DRL); >> >> >> >> KnowledgeBuilderErrors errors = >> >> kbuilder.getErrors(); >> >> >> >> if (errors.size() > 0) { >> >> ..... >> >> } else { >> >> >> >> knowledgePackages.put(medcpuKnowledgeBase.getFileName(), >> >> [the package >> just >> >> added above]); >> >> } >> >> } catch(Exception ex) { >> >> .... >> >> } else { >> >> ..... >> >> } >> >> } >> >> } >> >> >> >> >> >> Each session will do this to get the pre-compiled KnowledgePackage: >> >> >> >> KnowledgePackage knowledgePackage = knowledgePackages.get(kb); >> >> ...... >> >> Collection<KnowledgePackage> packages = new >> >> ArrayList<KnowledgePackage>(); >> >> packages.add(knowledgePackage); >> >> >> >> // The problem is this guy (after a relatively small number of >> requests >> >> each >> >> opening a new session) just gets stuck!! It does not throw an >> exception >> >> and >> >> it does not return. >> >> this.knowledgeBase.addKnowledgePackages(packages); >> >> >> >> When I did not share the KnowledgePackages like I did above, I was >> able >> >> to >> >> get thousands of requests with each having its own session, >> >> knowledgebase, >> >> and its own knowledge packages after they compile the respective drls. >> >> However, because this was slow compiling the same drls over and over >> >> again >> >> I >> >> switched to the design above which after only 20 or 30 requests it >> gets >> >> stuck on the line above. I still have a knowledgebase per session per >> >> request, but now I am sharing the knowledge packages. >> >> >> >> I cannot find anywhere where it says that KnowledgePackage objects are >> >> not >> >> thread-safe (not safe to be shared between sessions/threads). Any idea >> >> why >> >> I >> >> am stuck on the line above? Thanks! >> >> -- >> >> View this message in context: >> >> http://n3.nabble.com/Sessions-Sharing-KnowledgeBase-tp96894p96894.html >> >> Sent from the Drools - User mailing list archive at Nabble.com. >> >> _______________________________________________ >> >> rules-users mailing list >> >> rules-users(a)lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/rules-users >> >> >> > >> > >> > >> > -- >> > Edson Tirelli >> > JBoss Drools Core Development >> > JBoss by Red Hat @ www.jboss.com >> > >> > _______________________________________________ >> > rules-users mailing list >> > rules-users(a)lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/rules-users >> > >> > >> >> -- >> View this message in context: >> http://n3.nabble.com/Sessions-Sharing-KnowledgeBase-tp96894p97452.html >> Sent from the Drools - User mailing list archive at Nabble.com. >> _______________________________________________ >> rules-users mailing list >> rules-users(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/rules-users >> > > > > -- > Edson Tirelli > JBoss Drools Core Development > JBoss by Red Hat @ www.jboss.com > > _______________________________________________ > rules-users mailing list > rules-users(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/rules-users > > -- View this message in context: http://n3.nabble.com/Sessions-Sharing-KnowledgeBase-tp96894p97506.html Sent from the Drools - User mailing list archive at Nabble.com. _______________________________________________ rules-users mailing list rules-users(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users