[rules-users] how can I modify a batch of objects
Elran Dvir
elrand at checkpoint.com
Tue Nov 5 05:50:57 EST 2013
Wolfgang, Thanks for the response.
My new implementation, causing the error, is very similar to what you suggest.
This is my new drl:
--------------------------------------
package myimpl.drools.package1;
import java.util.Date
import java.util.HashMap
import java.util.HashSet
import java.util.Collection
import java.util.Set
import java.util.ArrayList
import myimpl.drools.Log
import myimpl.drools.CorrelatedEvent
import myimpl.drools.CandidatesWindow
global myimpl.server.EventsHandler externalEventsHandler;
function String getUniqueId(Log log) {
String uniqueId="";
uniqueId += (log.fieldsMap.get("port") != null ? log.fieldsMap.get("port").toString() : "null");
return uniqueId;
}
function Collection getMarkers(Collection matchedLogs) {
ArrayList<String> markers = new ArrayList<String>();
HashSet<String> idSet = new HashSet<String>();
for (Object matchedLogObj : matchedLogs) {
Log matchedLog = (Log) matchedLogObj;
String id = getUniqueId(matchedLog);
if (!idSet.contains(id)) {
idSet.add(id);
markers.add(matchedLog.fieldsMap.get("marker").toString());
if (markers.size() == 25) break;
}
}
return markers;
}
declare Log
@role( event)
end
declare CorrelatedEvent
@role( event)
@expires( 600s )
end
declare CandidatesWindow
@role( event)
@expires( 60s )
end
rule "Create Port Scan Event - 1"
enabled true
dialect "java"
no-loop
when
$log : Log()
not CorrelatedEvent(getId() == "portScan" , groupByFieldsMap.get("src") == $log.fieldsMap.get("src") , roupByFieldsMap.get("dst") == $log.fieldsMap.get("dst"))
$windows : ArrayList()
from collect(
CandidatesWindow(getRuleId() == "portScan" , groupByFieldsMap.get("src") == $log.fieldsMap.get("src") ,groupByFieldsMap.get("dst") == $log.fieldsMap.get("dst")))
then
String id = getUniqueId($log);
for (Object windowObj : $windows) {
CandidatesWindow window = (CandidatesWindow) windowObj;
modify ( window ) { addLog($log, id) }
}
CandidatesWindow newWindow = new CandidatesWindow("portScan", true);
newWindow.groupByFieldsMap.put("src", $log.fieldsMap.get("src"));
newWindow.groupByFieldsMap.put("dst", $log.fieldsMap.get("dst"));
newWindow.addLog($log, id);
insert(newWindow);
end
rule "Create Port Scan Event - 2"
enabled true
dialect "java"
no-loop
when
$window: CandidatesWindow(getRuleId() == "portScan" , getCount() > 19)
not CorrelatedEvent(getId() == "portScan" , groupByFieldsMap.get("src") == $window.groupByFieldsMap.get("src") , groupByFieldsMap.get("dst") == $window.groupByFieldsMap.get("dst"))
then
Collection markers = getMarkers($window.getLogs());
CorrelatedEvent $ce = new CorrelatedEvent("portScan");
$ce.groupByFieldsMap.put("src", $window.groupByFieldsMap.get("src"));
$ce.groupByFieldsMap.put("dst", $window.groupByFieldsMap.get("dst"));
insert($ce);
HashMap<String,Object> fieldsMap = new HashMap<String,Object>();
Log firstLog = $window.getLogs().iterator().next();
fieldsMap.put("a",firstLog.fieldsMap.get("a"));
fieldsMap.put("b",firstLog.fieldsMap.get("b"));
fieldsMap.put("markers",markers);
retract($window);
externalEventsHandler.handleEvent(fieldsMap);
end
--------------------------------------------------------------------------------
CandidatesWindow is my sliding window. Its expiration determines its length.
The first rule describe this behavior:
For each (fit) log, if no CorrelatedEvent exists, I attach it to existing (not expired) windows, and create a new widow (and attach the log) because each log basically starts a new window.
The second rule creates a new event if the count fits and retracts the activating CandidatesWindow (it doesn't have to be retracted. It will expire)
All other CandidatesWindow expire when time comes.
How can I update/modify an existing CandidatesWindow and activate the second rule?
Thank you very much.
-----Original Message-----
From: rules-users-bounces at lists.jboss.org [mailto:rules-users-bounces at lists.jboss.org] On Behalf Of Wolfgang Laun
Sent: Tuesday, November 05, 2013 8:03 AM
To: Rules Users List
Subject: Re: [rules-users] how can I modify a batch of objects
The memory consumption has to be tackled by reducing the number of half-baked activations.
I understand that you have to monitor certain connections (excluding those that can or have to be filtered out). And an observation window has to keep track of what goes on between one source s1 and one destination d1 within 60 s after the first event.
rule one
when
$log: Log( $src: ..., $dst: ..., $ts: ... )
not Monitor( source == $src, destination == $dst ) then
create Monitor m, register $log in it, m.setStartTime( $ts ); insert m end
rule two
no-loop
when
$m: Monitor( $src:..., $dst:..., $start:... )
$log: Log( ... == $src, ... == $dst, timestamp - $start < 60s )
then
keep track of $log in $m
end
You'll need more rules, one to detect a violation of the limit and another one to discard a Monitor after 60 seconds of inactivity.
Notice that sequences of s1-d1 will not create additional network activity for each member of the sequence - that's the whole point of this exercise.
-W
On 04/11/2013, Elran Dvir <elrand at checkpoint.com> wrote:
> Hi all,
>
> I am trying to identify a port scan event.
>
> The basic fact is connection log. For each combination of src (source
> IP) and dst (destination IP) , detect a port scan event, if over 60
> seconds there were at least 20 connection logs with different service and protocol.
>
> The event will stay closed for 10 minute - no event will be sent
> during this time for this combination of src and dst. The event will
> contain the connection logs' ids (markers).
>
>
>
> I tried to implement it using "accumulate" and "over window:time" but
> it consumes too much memory.
> So I am trying to imitate this functionality using several rules and facts.
>
> My drl contains the following lines (among others):
>
> declare CorrelatedEvent
> @role( event)
> @expires( 600s )
> end
>
> declare CandidatesWindow
> @role( event)
> @expires( 60s )
> end
>
> rule "Create Port Scan Event - 1"
> enabled true
> dialect "java"
> no-loop
> when
> $log : Log()
> not CorrelatedEvent(getId() == "portScan" ,
> groupByFieldsMap.get("src") == $log.fieldsMap.get("src")
> ,groupByFieldsMap.get("dst") == $log.fieldsMap.get("dst"))
> $windows : ArrayList()
> from collect( CandidatesWindow(getRuleId() == "portScan" ,
> groupByFieldsMap.get("src") == $log.fieldsMap.get("src") ,
> groupByFieldsMap.get("dst") == $log.fieldsMap.get("dst"))) then
> String id = $log.fieldsMap.get("port").toString();
> System.out.println(new Date().toString()+" windowSize: " +
> $windows.size());
> for (Object windowObj : $windows) {
> CandidatesWindow window = (CandidatesWindow) windowObj;
> modify ( window ) { addLog($log, id) }
> }
> CandidatesWindow newWindow = new CandidatesWindow("portScan", true);
> newWindow.groupByFieldsMap.put("src", $log.fieldsMap.get("src"));
> newWindow.groupByFieldsMap.put("dst", $log.fieldsMap.get("dst"));
> newWindow.addLog($log, id);
> insert(newWindow);
> end
>
> This imitates sliding time windows.
> when I tested it, I got the following exception:
>
> Exception executing consequence for rule "Create Port Scan Event - 1"
> in
> com.checkpoint.correlation.impl.drools.package1:
> java.util.ConcurrentModificationException
> at
> org.drools.runtime.rule.impl.DefaultConsequenceExceptionHandler.handleException(DefaultConsequenceExceptionHandler.java:39)
> at
> org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1297)
> at
> org.drools.common.DefaultAgenda.fireNextItem(DefaultAgenda.java:1221)
> at
> org.drools.common.DefaultAgenda.fireAllRules(DefaultAgenda.java:1456)
> at
> org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:710)
> at
> org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:674)
> at
> org.drools.impl.StatefulKnowledgeSessionImpl.fireAllRules(StatefulKnowledgeSessionImpl.java:230)
> at
> com.checkpoint.correlation.impl.drools.DroolsCEPEngineV1.insertEvents(DroolsCEPEngineV1.java:173)
> at
> com.checkpoint.correlation.impl.feeder.JsonFileFeeder.init(JsonFileFeeder.java:68)
> at
> com.checkpoint.correlation.server.CorrelationServer.initFeeder(CorrelationServer.java:63)
> at
> com.checkpoint.correlation.server.CorrelationServer.run(CorrelationServer.java:28)
> at
> com.checkpoint.correlation.server.CorrelationServer.runServer(CorrelationServer.java:101)
> at
> com.checkpoint.correlation.server.CorrelationServer.main(CorrelationSe
> rver.java:85) Caused by: java.util.ConcurrentModificationException
> at
> java.util.ArrayList$Itr.checkForComodification(ArrayList.java:819)
> at java.util.ArrayList$Itr.next(ArrayList.java:791)
> at
> com.checkpoint.correlation.impl.drools.package1.Rule_Create_Port_Scan_Event___1_2f94bc67f9064c6e9614982cf9bc8859.defaultConsequence(Rule_Create_Port_Scan_Event___1_2f94bc67f9064c6e9614982cf9bc8859.java:11)
> at
> com.checkpoint.correlation.impl.drools.package1.Rule_Create_Port_Scan_
> Event___1_2f94bc67f9064c6e9614982cf9bc8859DefaultConsequenceInvokerGen
> erated.evaluate(Unknown
> Source)
> at
> com.checkpoint.correlation.impl.drools.package1.Rule_Create_Port_Scan_
> Event___1_2f94bc67f9064c6e9614982cf9bc8859DefaultConsequenceInvoker.ev
> aluate(Unknown
> Source)
> at
> org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1287)
> ... 11 more
>
> It is caused by modify ( window ) in the for loop.
> How can I make it work?
>
> Thanks.
>
_______________________________________________
rules-users mailing list
rules-users at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
Email secured by Check Point
More information about the rules-users
mailing list