9:59 p.m.
Hi All I'm new to Drools, I will build a very large Knowledge Base, let's say it
will be larger than 100GB, the files will be stored in a distribute database, so can
Droools supply this large Knowledge Base? Any solution? Thank you very much. CheersRamon
3:11 a.m.
Hi Guys I'm still confused about this question, let's say i may have a very large
rule set, how can Drools support this case, will it load every rule into memory? Any idea?
Thank you guys. CheersRamon
From: ramon_wang(a)hotmail.com
To: rules-users(a)lists.jboss.org
Date: Thu, 8 Dec 2011 11:59:21 +0800
Subject: [rules-users] Can Drools supply large Knowledge Base?
Hi All
I'm new to Drools, I will build a very large Knowledge Base, let's say it will be
larger than 100GB, the files will be stored in a distribute database, so can Droools
supply this large Knowledge Base? Any solution? Thank you very much.
Cheers
Ramon
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
3:24 a.m.
I think you are mixing up terms which doesn't help.
I think you mean you may have 100GB of facts that will be in your distributed database.
Unless you are planning to do some odd automatic rule generation you will have to write
the rules by hand and so will only have a relatively small number of rules in your
application - this will easily fit in memory.
If 100GB is the size of the facts you have then clearly you can't hold them all in
memory of a single machine. Drools is effectively a special in memory database, in order
to perform its processing it needs the relevant data.
You basically have these options which of these will work depends on your use case and in
reality you'll probably need to combine them:
1. Use partitioning, rather than having 1 set of 100 gb of data, have 10 sets of 10
gbs running on ten machines. Many times you may be able to partition by customer ID or
some other natural value which may give you good enough results.
2. Use accumulating facts and only keep the data you need in memory, eg if you need
to know how many orders a customer has made then don't keep all the orders in memory,
just a simple object containing the total.
3. Don't use drools, depending on what you are doing it may be better just to
use normal db queries to extract the data you need. It may be you get the query results,
insert them into a new short lived session, fire the rules and then get the results of the
drools processing.
Thomas
From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org] On
Behalf Of WangRamon
Sent: 12 December 2011 09:11
To: rules-users(a)lists.jboss.org
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
Hi Guys
I'm still confused about this question, let's say i may have a very large rule
set, how can Drools support this case, will it load every rule into memory? Any idea?
Thank you guys.
Cheers
Ramon
________________________________
From: ramon_wang(a)hotmail.com
To: rules-users(a)lists.jboss.org
Date: Thu, 8 Dec 2011 11:59:21 +0800
Subject: [rules-users] Can Drools supply large Knowledge Base?
Hi All
I'm new to Drools, I will build a very large Knowledge Base, let's say it will be
larger than 100GB, the files will be stored in a distribute database, so can Droools
supply this large Knowledge Base? Any solution? Thank you very much.
Cheers
Ramon
_______________________________________________ rules-users mailing list
rules-users@lists.jboss.org<mailto:rules-users@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/rules-users
________________________________
**************************************************************************************
This message is confidential and intended only for the addressee. If you have received
this message in error, please immediately notify the postmaster(a)nds.com and delete it from
your system as well as any copies. The content of e-mails as well as traffic data may be
monitored by NDS for employment and security purposes. To protect the environment please
do not print this e-mail unless necessary.
NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United
Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603
8808 40-00
**************************************************************************************
3:49 a.m.
Thanks Thomas! The solutions you provided are great, can you give more explanation on your
second point "Use accumulating facts and only keep the data you need in memory",
thanks. CheersRamon
From: TSwindells(a)nds.com
To: rules-users(a)lists.jboss.org
Date: Mon, 12 Dec 2011 09:24:14 +0000
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
I think you are mixing up terms which doesn’t help.
I think you mean you may have 100GB of facts that will be in your distributed database.
Unless you are planning to do some odd automatic rule generation you
will have to write the rules by hand and so will only have a relatively small number of
rules in your application � this will easily fit in memory.
If 100GB is the size of the facts you have then clearly you can’t hold them all in memory
of a single machine. Drools is effectively a special in memory database,
in order to perform its processing it needs the relevant data.
You basically have these options which of these will work depends on your use case and in
reality you’ll probably need to combine them:
1.
Use partitioning, rather than having 1 set of 100 gb of data, have 10 sets of 10 gbs
running on ten machines. Many times you
may be able to partition by customer ID or some other natural value which may give you
good enough results.
2.
Use accumulating facts and only keep the data you need in memory, eg if you need to know
how many orders a customer has made
then don’t keep all the orders in memory, just a simple object containing the total.
3.
Don’t use drools, depending on what you are doing it may be better just to use normal db
queries to extract the data you need.
It may be you get the query results, insert them into a new short lived session, fire the
rules and then get the results of the drools processing.
Thomas
From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org]
On Behalf Of WangRamon
Sent: 12 December 2011 09:11
To: rules-users(a)lists.jboss.org
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
Hi Guys
I'm still confused about this question, let's say i may have a very large rule
set, how can Drools support this case, will it load every rule into memory? Any idea?
Thank you guys.
Cheers
Ramon
From: ramon_wang(a)hotmail.com
To: rules-users(a)lists.jboss.org
Date: Thu, 8 Dec 2011 11:59:21 +0800
Subject: [rules-users] Can Drools supply large Knowledge Base?
Hi All
I'm new to Drools, I will build a very large Knowledge Base, let's say it will be
larger than 100GB, the files will be stored in a distribute database, so can Droools
supply this large Knowledge Base? Any solution? Thank you very much.
Cheers
Ramon
_______________________________________________ rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
**************************************************************************************
This message is confidential and intended only for the addressee. If you have received
this message in error, please immediately notify the postmaster(a)nds.com and delete it from
your system as well as any copies. The content of e-mails as well as traffic data
may be monitored by NDS for employment and security purposes. To protect the environment
please do not print this e-mail unless necessary.
NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United
Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603
8808 40-00
**************************************************************************************
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
3:53 a.m.
There's not that much to say - and if you do a search you should find more
information.
Basically only have the information that you need in memory. If you need to make
calculations of previous state, rather than keeping the previous state all loaded up just
have facts which record the state - perhaps an OrderTotal fact which contains the total of
all the orders. When a new order arrives you update the value of that.
It all depends on what you are trying to achieve and what you are doing. Without a
concrete understanding of what sort of problem you are trying to solve and what you are
planning to do we can't give much more guidance beyond that there are various
different solutions and approaches that are appropriate for various different problem
spaces.
Thomas
From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org] On
Behalf Of WangRamon
Sent: 12 December 2011 09:50
To: rules-users(a)lists.jboss.org
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
Thanks Thomas!
The solutions you provided are great, can you give more explanation on your second point
"Use accumulating facts and only keep the data you need in memory", thanks.
Cheers
Ramon
________________________________
From: TSwindells(a)nds.com
To: rules-users(a)lists.jboss.org
Date: Mon, 12 Dec 2011 09:24:14 +0000
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
I think you are mixing up terms which doesn't help.
I think you mean you may have 100GB of facts that will be in your distributed database.
Unless you are planning to do some odd automatic rule generation you will have to write
the rules by hand and so will only have a relatively small number of rules in your
application - this will easily fit in memory.
If 100GB is the size of the facts you have then clearly you can't hold them all in
memory of a single machine. Drools is effectively a special in memory database, in order
to perform its processing it needs the relevant data.
You basically have these options which of these will work depends on your use case and in
reality you'll probably need to combine them:
1. Use partitioning, rather than having 1 set of 100 gb of data, have 10 sets of 10
gbs running on ten machines. Many times you may be able to partition by customer ID or
some other natural value which may give you good enough results.
2. Use accumulating facts and only keep the data you need in memory, eg if you need
to know how many orders a customer has made then don't keep all the orders in memory,
just a simple object containing the total.
3. Don't use drools, depending on what you are doing it may be better just to
use normal db queries to extract the data you need. It may be you get the query results,
insert them into a new short lived session, fire the rules and then get the results of the
drools processing.
Thomas
From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org] On
Behalf Of WangRamon
Sent: 12 December 2011 09:11
To: rules-users(a)lists.jboss.org
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
Hi Guys
I'm still confused about this question, let's say i may have a very large rule
set, how can Drools support this case, will it load every rule into memory? Any idea?
Thank you guys.
Cheers
Ramon
________________________________
From: ramon_wang(a)hotmail.com
To: rules-users(a)lists.jboss.org
Date: Thu, 8 Dec 2011 11:59:21 +0800
Subject: [rules-users] Can Drools supply large Knowledge Base?
Hi All
I'm new to Drools, I will build a very large Knowledge Base, let's say it will be
larger than 100GB, the files will be stored in a distribute database, so can Droools
supply this large Knowledge Base? Any solution? Thank you very much.
Cheers
Ramon
_______________________________________________ rules-users mailing list
rules-users@lists.jboss.org<mailto:rules-users@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/rules-users
________________________________
**************************************************************************************
This message is confidential and intended only for the addressee. If you have received
this message in error, please immediately notify the
postmaster@nds.com<mailto:postmaster@nds.com> and delete it from your system as well
as any copies. The content of e-mails as well as traffic data may be monitored by NDS for
employment and security purposes. To protect the environment please do not print this
e-mail unless necessary.
NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United
Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603
8808 40-00
**************************************************************************************
_______________________________________________ rules-users mailing list
rules-users@lists.jboss.org<mailto:rules-users@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/rules-users
3:33 a.m.
WangRamon wrote:
Hi Guys I'm still confused about this question, let's say i may have a
very large rule set, how can Drools support this case, will it load every
rule into memory? Any idea? Thank you guys. CheersRamon
You are either asking the wrong question or you do not understand what you
are asking.
In your previous post, you mentioned a Knowledge Base of 100GB.
In this post you are talking about a rule set.
These are different things.
A Knowledge Base contains rules *and* facts and can indeed be quite large.
A rule set contains only rules. For a rule set to grow this much, it means
you have seriously misunderstand your problem. I would suggest you go back
to the drawing board.
Real-world example:
Rule set of about 400 rules.
Facts : In the order to 100.000.
Command line execution shows an approximate 400MB RAM consumption, meaning
with the JDK and all (not only Drools memory consumption).
So, unless you want to hear from someone more experienced than me, I'd
suggest you re-phrase your question by stating the exact problem you are
facing or anticipating.
Cheers,
-Stathis
From: ramon_wang(a)hotmail.com
To: rules-users(a)lists.jboss.org
Date: Thu, 8 Dec 2011 11:59:21 +0800
Subject: [rules-users] Can Drools supply large Knowledge Base?
Hi All
I'm new to Drools, I will build a very large Knowledge Base, let's say it
will be larger than 100GB, the files will be stored in a distribute
database, so can Droools supply this large Knowledge Base? Any solution?
Thank you very much.
Cheers
Ramon
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
3:44 a.m.
Hi Stathis You're probably right, maybe it's a wrong question, i'm new to
Drools, so I copy the rule sample from user guide as below to better explain, that's
what i need, but the rule will be automatically generated in the future.. package
com.company.license
rule "Is of valid age"
when
$a : Applicant( age < 18 )
then
$a.setValid( false );
end So in the future the rule may include more and more conditions like below:
rule "Is of valid age2"
when
$a : Applicant( age > 18 and age < 20 )
then
...end
So it may grow to be a very large rule(?), and this is actually my question: how to handle
this big rule? Thanks you guys CheersRamon> Date: Mon, 12 Dec 2011 11:33:10 +0200
From: rouvas(a)mm.di.uoa.gr
To: rules-users(a)lists.jboss.org
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
WangRamon wrote:
>
> Hi Guys I'm still confused about this question, let's say i may have a
> very large rule set, how can Drools support this case, will it load every
> rule into memory? Any idea? Thank you guys. CheersRamon
You are either asking the wrong question or you do not understand what you
are asking.
In your previous post, you mentioned a Knowledge Base of 100GB.
In this post you are talking about a rule set.
These are different things.
A Knowledge Base contains rules *and* facts and can indeed be quite large.
A rule set contains only rules. For a rule set to grow this much, it means
you have seriously misunderstand your problem. I would suggest you go back
to the drawing board.
Real-world example:
Rule set of about 400 rules.
Facts : In the order to 100.000.
Command line execution shows an approximate 400MB RAM consumption, meaning
with the JDK and all (not only Drools memory consumption).
So, unless you want to hear from someone more experienced than me, I'd
suggest you re-phrase your question by stating the exact problem you are
facing or anticipating.
Cheers,
-Stathis
> From: ramon_wang(a)hotmail.com
> To: rules-users(a)lists.jboss.org
> Date: Thu, 8 Dec 2011 11:59:21 +0800
> Subject: [rules-users] Can Drools supply large Knowledge Base?
>
>
>
>
>
>
>
>
>
>
>
> Hi All
>
> I'm new to Drools, I will build a very large Knowledge Base, let's say it
> will be larger than 100GB, the files will be stored in a distribute
> database, so can Droools supply this large Knowledge Base? Any solution?
> Thank you very much.
>
> Cheers
> Ramon
>
>
> _______________________________________________
> rules-users mailing list
> rules-users(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
> _______________________________________________
> rules-users mailing list
> rules-users(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
3:54 a.m.
That's not a big rule, that is a tiny rule.
So I'm still not sure what you are actually worrying about and why you think you may
have an issue.
Thomas
From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org] On
Behalf Of WangRamon
Sent: 12 December 2011 09:44
To: rouvas(a)di.uoa.gr; rules-users(a)lists.jboss.org
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
Hi Stathis
You're probably right, maybe it's a wrong question, i'm new to Drools, so I
copy the rule sample from user guide as below to better explain, that's what i need,
but the rule will be automatically generated in the future..
package com.company.license
rule "Is of valid age"
when
$a : Applicant( age < 18 )
then
$a.setValid( false );
end
So in the future the rule may include more and more conditions like below:
rule "Is of valid age2"
when
$a : Applicant( age > 18 and age < 20 )
then
...
end
So it may grow to be a very large rule(?), and this is actually my question: how to handle
this big rule? Thanks you guys
Cheers
Ramon
Date: Mon, 12 Dec 2011 11:33:10 +0200
From: rouvas@mm.di.uoa.gr<mailto:rouvas@mm.di.uoa.gr>
To: rules-users@lists.jboss.org<mailto:rules-users@lists.jboss.org>
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
WangRamon wrote:
>
> Hi Guys I'm still confused about this question, let's say i may have a
> very large rule set, how can Drools support this case, will it load every
> rule into memory? Any idea? Thank you guys. CheersRamon
You are either asking the wrong question or you do not understand what you
are asking.
In your previous post, you mentioned a Knowledge Base of 100GB.
In this post you are talking about a rule set.
These are different things.
A Knowledge Base contains rules *and* facts and can indeed be quite large.
A rule set contains only rules. For a rule set to grow this much, it means
you have seriously misunderstand your problem. I would suggest you go back
to the drawing board.
Real-world example:
Rule set of about 400 rules.
Facts : In the order to 100.000.
Command line execution shows an approximate 400MB RAM consumption, meaning
with the JDK and all (not only Drools memory consumption).
So, unless you want to hear from someone more experienced than me, I'd
suggest you re-phrase your question by stating the exact problem you are
facing or anticipating.
Cheers,
-Stathis
> From: ramon_wang@hotmail.com<mailto:ramon_wang@hotmail.com>> &g t; To:
rules-users@lists.jboss.org<mailto:rules-users@lists.jboss.org>
> Date: Thu, 8 Dec 2011 11:59:21 +0800
> Subject: [rules-users] Can Drools supply large Knowledge Base?
>
>
>
>
>
>
>
>
>
>
>
> Hi All
>
> I'm new to Drools, I will build a very large Knowledge Base, let's say it
> will be larger than 100GB, the files will be stored in a distribute
> database, so can Droools supply this large Knowledge Base? Any solution?
> Thank you very much.
>
> Cheers
> Ramon
>
>
> _______________________________________________
> rules-users mailing list
> rules-users@lists.jboss.org<mailto:rules-users@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/rules-users
> _______________________________________________
> rules-users mailing list
> rules-users@lists.jboss.org<mailto:rules-users@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/rules-users
>
_______________________________________________
rules-users mailing list
rules-users@lists.jboss.org<mailto:rules-users@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/rules-users
________________________________
**************************************************************************************
This message is confidential and intended only for the addressee. If you have received
this message in error, please immediately notify the postmaster(a)nds.com and delete it from
your system as well as any copies. The content of e-mails as well as traffic data may be
monitored by NDS for employment and security purposes. To protect the environment please
do not print this e-mail unless necessary.
NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United
Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603
8808 40-00
**************************************************************************************
3:58 a.m.
Hi Thomas It's a tiny rule for now, but in the future i will append more conditions
automatically by machines by analyzing some user logs, the size of the rule file may grow
to several GBs and that's what i'm worrying about? ThanksRamon
From: TSwindells(a)nds.com
To: rules-users(a)lists.jboss.org; rouvas(a)di.uoa.gr
Date: Mon, 12 Dec 2011 09:54:01 +0000
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
That’s not a big rule, that is a tiny rule.
So I’m still not sure what you are actually worrying about and why you think you may have
an issue.
Thomas
From: rules-users-bounces(a)lists.jboss.org [mailto:rules-users-bounces@lists.jboss.org]
On Behalf Of WangRamon
Sent: 12 December 2011 09:44
To: rouvas(a)di.uoa.gr; rules-users(a)lists.jboss.org
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
Hi Stathis
You're probably right, maybe it's a wrong question, i'm new to Drools, so I
copy the rule sample from user guide as below to better explain, that's what i need,
but the rule will be automatically generated in the future..
package com.company.license
rule "Is of valid age"
when
$a : Applicant( age < 18 )
then
$a.setValid( false );
end
So in the future the rule may include more and more conditions like below:
rule "Is of valid age2"
when
$a : Applicant( age > 18 and age < 20 )
then
...
end
So it may grow to be a very large rule(?), and this is actually my question: how to handle
this big rule? Thanks you guys
Cheers
Ramon
Date: Mon, 12 Dec 2011 11:33:10 +0200
From: rouvas(a)mm.di.uoa.gr
To: rules-users(a)lists.jboss.org
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
WangRamon wrote:
>
> Hi Guys I'm still confused about this question, let's
say i may have a
> very large rule set, how can Drools support this case, will it
load every
> rule into memory? Any idea? Thank you guys. CheersRamon
You are either asking the wrong question or you do not understand
what you
are asking.
In your previous post, you mentioned a Knowledge Base of 100GB.
In this post you are talking about a rule set.
These are different things.
A Knowledge Base contains rules *and* facts and can indeed be quite
large.
A rule set contains only rules. For a rule set to grow this much, it
means
you have seriously misunderstand your problem. I would suggest you go
back
to the drawing board.
Real-world example:
Rule set of about 400 rules.
Facts : In the order to 100.000.
Command line execution shows an approximate 400MB RAM consumption,
meaning
with the JDK and all (not only Drools memory consumption).
So, unless you want to hear from someone more experienced than me,
I'd
suggest you re-phrase your question by stating the exact problem you
are
facing or anticipating.
Cheers,
-Stathis
> From: ramon_wang(a)hotmail.com> &g t; To:
rules-users(a)lists.jboss.org
> Date: Thu, 8 Dec 2011 11:59:21 +0800
> Subject: [rules-users] Can Drools supply large Knowledge Base?
>
>
>
>
>
>
>
>
>
>
>
> Hi All
>
> I'm new to Drools, I will build a very large Knowledge Base,
let's say it
> will be larger than 100GB, the files will be stored in a
distribute
> database, so can Droools supply this large Knowledge Base? Any
solution?
> Thank you very much.
>
> Cheers
> Ramon
>
>
> _______________________________________________
> rules-users mailing list
> rules-users(a)lists.jboss.org
> _______________________________________________
> rules-users mailing list
> rules-users(a)lists.jboss.org
>
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
**************************************************************************************
This message is confidential and intended only for the addressee. If you have received
this message in error, please immediately notify the postmaster(a)nds.com and delete it from
your system as well as any copies. The content of e-mails as well as traffic data
may be monitored by NDS for employment and security purposes. To protect the environment
please do not print this e-mail unless necessary.
NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United
Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603
8808 40-00
**************************************************************************************
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
4:02 a.m.
Do you think that you are on the right track at all?
A simple, single Java if built this way to have a size of several GBs
is a no-no.
-W
2011/12/12 WangRamon <ramon_wang(a)hotmail.com>:
Hi Thomas
It's a tiny rule for now, but in the future i will append more conditions
automatically by machines by analyzing some user logs, the size of the rule
file may grow to several GBs and that's what i'm worrying about?
Thanks
Ramon
________________________________
From: TSwindells(a)nds.com
To: rules-users(a)lists.jboss.org; rouvas(a)di.uoa.gr
Date: Mon, 12 Dec 2011 09:54:01 +0000
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
That’s not a big rule, that is a tiny rule.
So I’m still not sure what you are actually worrying about and why you think
you may have an issue.
Thomas
From: rules-users-bounces(a)lists.jboss.org
[mailto:rules-users-bounces@lists.jboss.org] On Behalf Of WangRamon
Sent: 12 December 2011 09:44
To: rouvas(a)di.uoa.gr; rules-users(a)lists.jboss.org
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
Hi Stathis
You're probably right, maybe it's a wrong question, i'm new to Drools, so I
copy the rule sample from user guide as below to better explain, that's what
i need, but the rule will be automatically generated in the future..
package com.company.license
rule "Is of valid age"
when
$a : Applicant( age < 18 )
then
$a.setValid( false );
end
So in the future the rule may include more and more conditions like below:
rule "Is of valid age2"
when
$a : Applicant( age > 18 and age < 20 )
then
...
end
So it may grow to be a very large rule(?), and this is actually my question:
how to handle this big rule? Thanks you guys
Cheers
Ramon
> Date: Mon, 12 Dec 2011 11:33:10 +0200
> From: rouvas(a)mm.di.uoa.gr
> To: rules-users(a)lists.jboss.org
> Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
>
> WangRamon wrote:
> >
> > Hi Guys I'm still confused about this question, let's say i may have a
> > very large rule set, how can Drools support this case, will it load
> > every
> > rule into memory? Any idea? Thank you guys. CheersRamon
>
> You are either asking the wrong question or you do not understand what you
> are asking.
>
> In your previous post, you mentioned a Knowledge Base of 100GB.
> In this post you are talking about a rule set.
> These are different things.
> A Knowledge Base contains rules *and* facts and can indeed be quite large.
> A rule set contains only rules. For a rule set to grow this much, it means
> you have seriously misunderstand your problem. I would suggest you go back
> to the drawing board.
>
> Real-world example:
> Rule set of about 400 rules.
> Facts : In the order to 100.000.
> Command line execution shows an approximate 400MB RAM consumption, meaning
> with the JDK and all (not only Drools memory consumption).
>
> So, unless you want to hear from someone more experienced than me, I'd
> suggest you re-phrase your question by stating the exact problem you are
> facing or anticipating.
>
> Cheers,
> -Stathis
>
>
>
> > From: ramon_wang(a)hotmail.com> &g t; To: rules-users(a)lists.jboss.org
> > Date: Thu, 8 Dec 2011 11:59:21 +0800
> > Subject: [rules-users] Can Drools supply large Knowledge Base?
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Hi All
> >
> > I'm new to Drools, I will build a very large Knowledge Base, let's say
> > it
> > will be larger than 100GB, the files will be stored in a distribute
> > database, so can Droools supply this large Knowledge Base? Any solution?
> > Thank you very much.
> >
> > Cheers
> > Ramon
> >
> >
> > _______________________________________________
> > rules-users mailing list
> > rules-users(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/rules-users
> > _______________________________________________
> > rules-users mailing list
> > rules-users(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/rules-users
> >
>
>
> _______________________________________________
> rules-users mailing list
> rules-users(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
________________________________
**************************************************************************************
This message is confidential and intended only for the addressee. If you
have received this message in error, please immediately notify the
postmaster(a)nds.com and delete it from your system as well as any copies. The
content of e-mails as well as traffic data may be monitored by NDS for
employment and security purposes. To protect the environment please do not
print this e-mail unless necessary.
NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18
4EX, United Kingdom. A company registered in England and Wales. Registered
no. 3080780. VAT no. GB 603 8808 40-00
**************************************************************************************
_______________________________________________ rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
4:09 a.m.
Hi Laun I don't know how Drools can handle it or whether it can handle it even after i
persist the rules into some database. But it becomes much more clearer to me that maybe
Drools is not my best choice here, or at least i need to consider the options from Thomas.
Thank you. CheersRamon
Date: Mon, 12 Dec 2011 11:02:28 +0100
From: wolfgang.laun(a)gmail.com
To: rules-users(a)lists.jboss.org
Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
Do you think that you are on the right track at all?
A simple, single Java if built this way to have a size of several GBs
is a no-no.
-W
2011/12/12 WangRamon <ramon_wang(a)hotmail.com>:
> Hi Thomas
>
> It's a tiny rule for now, but in the future i will append more conditions
> automatically by machines by analyzing some user logs, the size of the rule
> file may grow to several GBs and that's what i'm worrying about?
>
> Thanks
> Ramon
>
> ________________________________
> From: TSwindells(a)nds.com
> To: rules-users(a)lists.jboss.org; rouvas(a)di.uoa.gr
> Date: Mon, 12 Dec 2011 09:54:01 +0000
> Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
>
> That’s not a big rule, that is a tiny rule.
>
> So I’m still not sure what you are actually worrying about and why you think
> you may have an issue.
>
>
>
> Thomas
>
>
>
> From: rules-users-bounces(a)lists.jboss.org
> [mailto:rules-users-bounces@lists.jboss.org] On Behalf Of WangRamon
> Sent: 12 December 2011 09:44
> To: rouvas(a)di.uoa.gr; rules-users(a)lists.jboss.org
> Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
>
>
>
> Hi Stathis
>
> You're probably right, maybe it's a wrong question, i'm new to Drools,
so I
> copy the rule sample from user guide as below to better explain, that's what
> i need, but the rule will be automatically generated in the future..
>
>
> package com.company.license
>
>
>
> rule "Is of valid age"
>
> when
>
> $a : Applicant( age < 18 )
>
> then
>
> $a.setValid( false );
>
> end
>
>
> So in the future the rule may include more and more conditions like below:
>
>
>
>
>
>
> rule "Is of valid age2"
>
> when
>
> $a : Applicant( age > 18 and age < 20 )
>
> then
>
>
>
> ...
>
> end
>
>
> So it may grow to be a very large rule(?), and this is actually my question:
> how to handle this big rule? Thanks you guys
>
> Cheers
> Ramon
>
>> Date: Mon, 12 Dec 2011 11:33:10 +0200
>> From: rouvas(a)mm.di.uoa.gr
>> To: rules-users(a)lists.jboss.org
>> Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
>>
>> WangRamon wrote:
>> >
>> > Hi Guys I'm still confused about this question, let's say i may
have a
>> > very large rule set, how can Drools support this case, will it load
>> > every
>> > rule into memory? Any idea? Thank you guys. CheersRamon
>>
>> You are either asking the wrong question or you do not understand what you
>> are asking.
>>
>> In your previous post, you mentioned a Knowledge Base of 100GB.
>> In this post you are talking about a rule set.
>> These are different things.
>> A Knowledge Base contains rules *and* facts and can indeed be quite large.
>> A rule set contains only rules. For a rule set to grow this much, it means
>> you have seriously misunderstand your problem. I would suggest you go back
>> to the drawing board.
>>
>> Real-world example:
>> Rule set of about 400 rules.
>> Facts : In the order to 100.000.
>> Command line execution shows an approximate 400MB RAM consumption, meaning
>> with the JDK and all (not only Drools memory consumption).
>>
>> So, unless you want to hear from someone more experienced than me, I'd
>> suggest you re-phrase your question by stating the exact problem you are
>> facing or anticipating.
>>
>> Cheers,
>> -Stathis
>>
>>
>>
>> > From: ramon_wang(a)hotmail.com> &g t; To: rules-users(a)lists.jboss.org
>> > Date: Thu, 8 Dec 2011 11:59:21 +0800
>> > Subject: [rules-users] Can Drools supply large Knowledge Base?
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > Hi All
>> >
>> > I'm new to Drools, I will build a very large Knowledge Base, let's
say
>> > it
>> > will be larger than 100GB, the files will be stored in a distribute
>> > database, so can Droools supply this large Knowledge Base? Any solution?
>> > Thank you very much.
>> >
>> > Cheers
>> > Ramon
>> >
>> >
>> > _______________________________________________
>> > rules-users mailing list
>> > rules-users(a)lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/rules-users
>> > _______________________________________________
>> > rules-users mailing list
>> > rules-users(a)lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/rules-users
>> >
>>
>>
>> _______________________________________________
>> rules-users mailing list
>> rules-users(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/rules-users
>
>
> ________________________________
>
>
**************************************************************************************
> This message is confidential and intended only for the addressee. If you
> have received this message in error, please immediately notify the
> postmaster(a)nds.com and delete it from your system as well as any copies. The
> content of e-mails as well as traffic data may be monitored by NDS for
> employment and security purposes. To protect the environment please do not
> print this e-mail unless necessary.
>
> NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18
> 4EX, United Kingdom. A company registered in England and Wales. Registered
> no. 3080780. VAT no. GB 603 8808 40-00
>
**************************************************************************************
>
> _______________________________________________ rules-users mailing list
> rules-users(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
> _______________________________________________
> rules-users mailing list
> rules-users(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
>
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
4:12 a.m.
WangRamon wrote:
Hi Stathis You're probably right, maybe it's a wrong question, i'm new to
Drools, so I copy the rule sample from user guide as below to better
explain, that's what i need, but the rule will be automatically generated
in the future.. package com.company.license
rule "Is of valid age"
when
$a : Applicant( age < 18 )
then
$a.setValid( false );
end So in the future the rule may include more and more conditions like
below:
rule "Is of valid age2"
when
$a : Applicant( age > 18 and age < 20 )
then
...end
So it may grow to be a very large rule(?), and this is actually my
Actually you have expressed two rules not one. "Is of valid age" and "Is
of valid age2" are two different rules in Drools-land.
Also saying "very large rule" is problematic. What would you count as
"large"? A lot of rules? A lot of conditions inside each rule? And how big
is that "large"? Tens? Hundreds? Thousands?
I'd suggest that you press "pause" here and ponder on the problem a bit.
You mention that you are new to Drools. I'm guessing that you are also new
to rule based systems, else you wouldn't be having these questions.
I'd suggest you read the first chapters of the Drools Manual and make sure
you understand all the notions that are mentioned there. For example,
rule-based systems, forward-chaining, backward chaining, rules vs facts
and especially the differences between functional and logic programming.
Once you understand these topics, re-read the answers to your post by
Esteban and Wolfang (posted previously). Enlightenment will be bestowed
upon you.
-Stathis
If you're experimenting
question: how to handle this big rule? Thanks you guys
CheersRamon> Date:
Mon, 12 Dec 2011 11:33:10 +0200
> From: rouvas(a)mm.di.uoa.gr
> To: rules-users(a)lists.jboss.org
> Subject: Re: [rules-users] Can Drools supply large Knowledge Base?
>
> WangRamon wrote:
> >
> > Hi Guys I'm still confused about this question, let's say i may have a
> > very large rule set, how can Drools support this case, will it load
> every
> > rule into memory? Any idea? Thank you guys. CheersRamon
>
> You are either asking the wrong question or you do not understand what
> you
> are asking.
>
> In your previous post, you mentioned a Knowledge Base of 100GB.
> In this post you are talking about a rule set.
> These are different things.
> A Knowledge Base contains rules *and* facts and can indeed be quite
> large.
> A rule set contains only rules. For a rule set to grow this much, it
> means
> you have seriously misunderstand your problem. I would suggest you go
> back
> to the drawing board.
>
> Real-world example:
> Rule set of about 400 rules.
> Facts : In the order to 100.000.
> Command line execution shows an approximate 400MB RAM consumption,
> meaning
> with the JDK and all (not only Drools memory consumption).
>
> So, unless you want to hear from someone more experienced than me, I'd
> suggest you re-phrase your question by stating the exact problem you are
> facing or anticipating.
>
> Cheers,
> -Stathis
>
>
>
> > From: ramon_wang(a)hotmail.com
> > To: rules-users(a)lists.jboss.org
> > Date: Thu, 8 Dec 2011 11:59:21 +0800
> > Subject: [rules-users] Can Drools supply large Knowledge Base?
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Hi All
> >
> > I'm new to Drools, I will build a very large Knowledge Base, let's say
> it
> > will be larger than 100GB, the files will be stored in a distribute
> > database, so can Droools supply this large Knowledge Base? Any
> solution?
> > Thank you very much.
> >
> > Cheers
> > Ramon
> >
> >
> > _______________________________________________
> > rules-users mailing list
> > rules-users(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/rules-users
> > _______________________________________________
> > rules-users mailing list
> > rules-users(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/rules-users
> >
>
>
> _______________________________________________
> rules-users mailing list
> rules-users(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/rules-users
3:34 a.m.
Another standard procedure is doing "load fact on demand".
rule "load missing"
when
$r: Request( $id: id )
not Person( id == $id )
then
fetch Person from DB
if found: insert Person
else:
error handling
retract $r
end
rule "process person"
when
$r: Request( $id: id )
$p: Person( id == $id )
then
process Person
retract $r and $p
end
-W
2011/12/12 WangRamon <ramon_wang(a)hotmail.com>:
Hi Guys
I'm still confused about this question, let's say i may have a very large
rule set, how can Drools support this case, will it load every rule into
memory? Any idea? Thank you guys.
Cheers
Ramon
________________________________
From: ramon_wang(a)hotmail.com
To: rules-users(a)lists.jboss.org
Date: Thu, 8 Dec 2011 11:59:21 +0800
Subject: [rules-users] Can Drools supply large Knowledge Base?
Hi All
I'm new to Drools, I will build a very large Knowledge Base, let's say it
will be larger than 100GB, the files will be stored in a distribute
database, so can Droools supply this large Knowledge Base? Any solution?
Thank you very much.
Cheers
Ramon
_______________________________________________ rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
_______________________________________________
rules-users mailing list
rules-users(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
11:25 a.m.
There are a few things going on in these discussions: rule "size" vs number
of facts vs number of rules.
On the latter two, memory thrashes and then crashes when too much is brought
into memory at once. You can use JVM options to increase memory size,
variable space size, etc. The upper limit for the number of rules is based
upon the "size" of the rule. However, if you are planning to have more than
thousands of facts or rules in memory at once, you may have problems in a 32
bit JVM. Using a 64 bit JVM helps.
On rule "size", I have had clients with rules like "if <any of these
symptoms> then you could have cancer". The number of conditions on such a
rule is over 100. DROOLS can handle rules of such complexity, although they
are difficult to debug in any language.
--
View this message in context:
http://drools.46999.n3.nabble.com/rules-users-Can-Drools-supply-large-Kno...
Sent from the Drools: User forum mailing list archive at Nabble.com.
4554
days inactive
4559
days old
13 comments
5 participants
participants (5)
-
David Green -
rouvas＠mm.di.uoa.gr -
Swindells, Thomas -
WangRamon -
Wolfgang Laun