[rules-users] how can I modify a batch of objects

[Wolfgang Laun]

The memory consumption has to be tackled by reducing the number of
half-baked activations.

I understand that you have to monitor certain connections (excluding
those that can or have to be filtered out). And an observation window
has to keep track of what goes on between one source s1 and one
destination d1 within 60 s after the first event.

rule one
when
    $log: Log( $src: ..., $dst: ..., $ts: ... )
    not Monitor( source == $src, destination == $dst )
then
   create  Monitor m, register $log in it, m.setStartTime( $ts ); insert m
end

rule two
no-loop
when
    $m: Monitor( $src:..., $dst:..., $start:... )
    $log: Log( ... == $src, ... == $dst, timestamp - $start < 60s   )
then
    keep track of $log in $m
end

You'll need more rules, one to detect a violation of the limit and
another one to discard a Monitor after 60 seconds of inactivity.

Notice that sequences of s1-d1 will not create additional network
activity for each member of the sequence - that's the whole point of
this exercise.

-W






On 04/11/2013, Elran Dvir <elrand at checkpoint.com> wrote:
> Hi all,
>
> I am trying to identify a port scan event.
>
> The basic fact is connection log. For each combination of src (source IP)
> and dst (destination IP) , detect a port scan event, if over 60 seconds
> there were at least 20 connection logs with different service and protocol.
>
> The event will stay closed for 10 minute - no event will be sent during this
> time for this combination of  src and dst. The event will contain the
> connection logs' ids (markers).
>
>
>
> I tried to implement it using "accumulate" and "over window:time" but it
> consumes too much memory.
> So I am trying to imitate this functionality using several rules and facts.
>
> My drl contains the following lines (among others):
>
> declare CorrelatedEvent
>         @role( event)
>         @expires( 600s )
> end
>
> declare CandidatesWindow
>         @role( event)
>         @expires( 60s )
> end
>
> rule "Create Port Scan Event - 1"
> enabled true
> dialect "java"
> no-loop
> when
>      $log : Log()
>       not CorrelatedEvent(getId() == "portScan" ,
> groupByFieldsMap.get("src") == $log.fieldsMap.get("src")
> ,groupByFieldsMap.get("dst") == $log.fieldsMap.get("dst"))
>       $windows : ArrayList()
>            from collect( CandidatesWindow(getRuleId() == "portScan" ,
> groupByFieldsMap.get("src") == $log.fieldsMap.get("src") ,
> groupByFieldsMap.get("dst") == $log.fieldsMap.get("dst")))
> then
>   String id = $log.fieldsMap.get("port").toString();
>   System.out.println(new Date().toString()+" windowSize: " +
> $windows.size());
>   for (Object windowObj : $windows) {
>     CandidatesWindow window = (CandidatesWindow) windowObj;
>     modify ( window ) { addLog($log, id) }
>   }
>   CandidatesWindow newWindow = new CandidatesWindow("portScan", true);
>   newWindow.groupByFieldsMap.put("src", $log.fieldsMap.get("src"));
>   newWindow.groupByFieldsMap.put("dst", $log.fieldsMap.get("dst"));
>   newWindow.addLog($log, id);
>   insert(newWindow);
> end
>
> This imitates sliding time windows.
> when I tested it, I got the following exception:
>
> Exception executing consequence for rule "Create Port Scan Event - 1" in
> com.checkpoint.correlation.impl.drools.package1:
> java.util.ConcurrentModificationException
>                 at
> org.drools.runtime.rule.impl.DefaultConsequenceExceptionHandler.handleException(DefaultConsequenceExceptionHandler.java:39)
>                 at
> org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1297)
>                 at
> org.drools.common.DefaultAgenda.fireNextItem(DefaultAgenda.java:1221)
>                 at
> org.drools.common.DefaultAgenda.fireAllRules(DefaultAgenda.java:1456)
>                 at
> org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:710)
>                 at
> org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:674)
>                 at
> org.drools.impl.StatefulKnowledgeSessionImpl.fireAllRules(StatefulKnowledgeSessionImpl.java:230)
>                 at
> com.checkpoint.correlation.impl.drools.DroolsCEPEngineV1.insertEvents(DroolsCEPEngineV1.java:173)
>                 at
> com.checkpoint.correlation.impl.feeder.JsonFileFeeder.init(JsonFileFeeder.java:68)
>                 at
> com.checkpoint.correlation.server.CorrelationServer.initFeeder(CorrelationServer.java:63)
>                 at
> com.checkpoint.correlation.server.CorrelationServer.run(CorrelationServer.java:28)
>                 at
> com.checkpoint.correlation.server.CorrelationServer.runServer(CorrelationServer.java:101)
>                 at
> com.checkpoint.correlation.server.CorrelationServer.main(CorrelationServer.java:85)
> Caused by: java.util.ConcurrentModificationException
>                 at
> java.util.ArrayList$Itr.checkForComodification(ArrayList.java:819)
>                 at java.util.ArrayList$Itr.next(ArrayList.java:791)
>                 at
> com.checkpoint.correlation.impl.drools.package1.Rule_Create_Port_Scan_Event___1_2f94bc67f9064c6e9614982cf9bc8859.defaultConsequence(Rule_Create_Port_Scan_Event___1_2f94bc67f9064c6e9614982cf9bc8859.java:11)
>                 at
> com.checkpoint.correlation.impl.drools.package1.Rule_Create_Port_Scan_Event___1_2f94bc67f9064c6e9614982cf9bc8859DefaultConsequenceInvokerGenerated.evaluate(Unknown
> Source)
>                 at
> com.checkpoint.correlation.impl.drools.package1.Rule_Create_Port_Scan_Event___1_2f94bc67f9064c6e9614982cf9bc8859DefaultConsequenceInvoker.evaluate(Unknown
> Source)
>                 at
> org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1287)
>                 ... 11 more
>
> It is caused by modify ( window ) in the for loop.
> How can I make it work?
>
> Thanks.
>