Transaction problems in high concurrency JBPM+Drools+Hibernate instalation
by Alberto R. Galdo
Hi guys,
We have an application built around long running JBPM processes with a
large user base and high concurrency ... early on we began to have issues
with transactions in JBPM and nowadays are watching lots of different
realizations of transaction managing problems that lead to dead human tasks
( state: ERROR ), errors running processes and so on ( different
exceptions follows ):
13:22:48,261 WARN [org.drools.persistence.jta.JtaTransactionManager]
(pool-11-thread-1) Unable to commit transaction:
javax.transaction.RollbackException: ARJUNA016053: Could not commit
transaction
Caused by: javax.persistence.PersistenceException: error during managed
flush
Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not
commit transaction.
at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177)
at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117)
at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)
at
org.jboss.tm.usertx.client.ServerVMClientUserTransaction.commit(ServerVMClientUserTransaction.java:167)
at
org.drools.persistence.jta.JtaTransactionManager.commit(JtaTransactionManager.java:179)
[drools-persistence-jpa-6.0.0-SNAPSHOT.jar:6.0.0-SNAPSHOT]
Caused by: javax.persistence.PersistenceException:
org.hibernate.HibernateException: Flush during cascade is dangerous
at
org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1361)
[hibernate-entitymanager-4.0.1.Final.jar:4.0.1.Final]
at
org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1289)
[hibernate-entitymanager-4.0.1.Final.jar:4.0.1.Final]
11:46:23,754 ERROR [stderr] (schedulerVT_Worker-8)
javax.persistence.PersistenceException: org.hibernate.HibernateException:
proxy handle is no longer valid
11:46:23,754 ERROR [stderr] (schedulerVT_Worker-8) at
org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1361)
11:46:23,755 ERROR [stderr] (schedulerVT_Worker-8) at
org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1289)
11:46:23,756 ERROR [stderr] (schedulerVT_Worker-8) at
org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:261)
11:46:23,756 ERROR [stderr] (schedulerVT_Worker-8) at
So, we're running this installation by the manual. Are using JBoss 7 as
application server, it's transaction manager, Hibernate 4.x as entity
manager, our connection pool delivers READ_COMMITED connections ( so does
hibernate ) ...
Is there anything else we should be doing to avoid this transactional
problems? Has anyone ever tested JBPM in such concurrent scenario? Any
experiences?
Alberto R. Galdo
argaldo(a)gmail.com
12 years, 1 month
Drools Decision Table with no rules matches blank rule
by srtatz
I have an xls file with the appropriate header lines to make the xls a Drools
Decision Table that I provide to my users. I validate the content/data they
enter, but do allow it to be "blank", that is, no user data, just the Drools
Decision Table key words, conditions, actions, etc.
When using the stateful knowledge session built using the Drools Decision
Table, I insert an object implementing AgendaEventListener and override the
afterActivationEvent method to record on the object which row was hit. I
found that it is saying it hit the first row in the Drools Decision Table
after the header information. But that corresponds to a blank line, since
the table is blank. It seems to have generated a blank drl rule behind the
scenes with no conditions (so it always succeeds) and no actions. Since I
have logic based on whether or not a rule was successfully selected, this
selection of a non-existent rule breaks my code.
The user won't really be using a blank spreadsheet, but I need to cover the
situation. The validation is pretty set in stone right now so I don't want
to force the user to have some data.
Can anyone suggest an easy way to stop it from matching the blank line?
Thanks,
Steve
--
View this message in context: http://drools.46999.n3.nabble.com/Drools-Decision-Table-with-no-rules-mat...
Sent from the Drools: User forum mailing list archive at Nabble.com.
12 years, 2 months
Guvnor 5.5.Final REST API for getting packages in JSON format throws InternalServerErrorException
by Sean Su
I do not know if anyone else has run into this before or not. This is only
happening with JSON format. The API works for XML.
I am using CXF library.
javax.ws.rs.InternalServerErrorException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.cxf.jaxrs.client.AbstractClient.convertToWebApplicationException(AbstractClient.java:461)
at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:860)
at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:831)
at org.apache.cxf.jaxrs.client.WebClient.invoke(WebClient.java:394)
at org.apache.cxf.jaxrs.client.WebClient.get(WebClient.java:573)
12 years, 2 months
Drools memory consumption
by Elran Dvir
I am sending this message again because maybe the last wasn't sent because of the attached snapshots.
I removed them now.
Thanks.
Hi all,
I am using Drools Fusion. I am getting OutOfMemoryError rather fast. My JVM is running with -Xmx4g flag.
I have rules defined in another (not Drools) language.
Every rule is translated programmatically to a drl file. This is because the user can add and remove rules (in the other language) dynamically.
The default configuration contains 125 rules.
For example, one rule is supposed to identify a port scan event.
The basic fact is connection log. For each combination of src (source IP) and dst (destination IP) , detect a port scan event, if over 60 seconds there were at least 20 connection logs with different service and protocol.
The event will stay closed for 10 minute - no event will be sent during this time for this combination of src and dst. The event the connection logs' ids (markers).
(other rules are very similar in structure, but different in logic, of course)
This is its programmatic drl file:
package com.checkpoint.correlation.impl.drools.package30;
import java.util.Date
import java.util.HashMap
import java.util.Set
import com.checkpoint.correlation.impl.drools.Log
import com.checkpoint.correlation.impl.drools.CorrelatedEvent
global com.checkpoint.correlation.server.EventsHandler externalEventsHandler;
import function com.checkpoint.correlation.impl.utils.UserDefinedFunctions.isInDayHourRange
import function com.checkpoint.correlation.impl.utils.UserDefinedFunctions.isInIpRange
function boolean filter(Log log) {
return (!((log.fieldsMap.get("src")!= null && isInIpRange(log.fieldsMap.get("src").toString(), "10.80.0.0", "10.80.255.255")) || (log.fieldsMap.get("src")!= null && isInIpRange(log.fieldsMap.get("src").toString(), "124.0.0.0", "124.255.255.255")) || (log.fieldsMap.get("src")!= null && isInIpRange(log.fieldsMap.get("src").toString(), "192.168.0.0", "192.168.255.255")) || (log.fieldsMap.get("src")!= null && isInIpRange(log.fieldsMap.get("src").toString(), "195.158.7.0", "195.158.7.255")) || (log.fieldsMap.get("src")!= null && isInIpRange(log.fieldsMap.get("src").toString(), "11.25.0.0", "11.25.255.255")) || (log.fieldsMap.get("src")!= null && isInIpRange(log.fieldsMap.get("src").toString(), "128.157.0.0", "128.157.255.255")) || (log.fieldsMap.get("src")!= null && isInIpRange(log.fieldsMap.get("src").toString(), "213.114.0.0", "213.114.255.255"))));
}
function String markersToString(Set markersSet) {
int i = 0;
String markersString = "";
for (Object marker : markersSet) {
if (i == 25) break;
String markerStr = marker.toString();
if (i > 0) markersString += "\n";
markersString += markerStr;
}
return markersString;
}
function String calcSeverity(Log log) {
return "High";
}
function String getUniqueId(Log log) {
String uniqueId="";
uniqueId += (log.fieldsMap.get("service") != null ? log.fieldsMap.get("service").toString() : "null");
uniqueId += (log.fieldsMap.get("proto") != null ? log.fieldsMap.get("proto").toString() : "null");
return uniqueId;
}
declare Log
@role(event)
end
declare CorrelatedEvent
@role(event)
@expires(600s)
end
rule "Port scan from external network"
enabled true
dialect "java"
no-loop
when
$log : Log(eval(filter($log)))
not CorrelatedEvent(getId() == "{8AC52BA8-1EE8-4f18-9BB4-54492116501C}", groupByFieldsMap.get("src") == $log.fieldsMap.get("src"), groupByFieldsMap.get("dst") == $log.fieldsMap.get("dst"))
accumulate($accumulatedLog : Log(eval(filter($accumulatedLog)), this after[0s,60s] $log, fieldsMap.get("src") == $log.fieldsMap.get("src"), fieldsMap.get("dst") == $log.fieldsMap.get("dst"), $id : getUniqueId(this));
$idSet : collectSet($id);
$idSet.size > 19)
accumulate($accumulatedLog : Log(eval(filter($accumulatedLog)), this after[0s,60s] $log, fieldsMap.get("src") == $log.fieldsMap.get("src"), fieldsMap.get("dst") == $log.fieldsMap.get("dst"), $idSet.contains(getUniqueId(this)), $marker : fieldsMap.get("marker"));
$markerSet : collectSet($marker))
then
CorrelatedEvent $ce = new CorrelatedEvent("{8AC52BA8-1EE8-4f18-9BB4-54492116501C}");
$ce.groupByFieldsMap.put("src", $log.fieldsMap.get("src"));
$ce.groupByFieldsMap.put("dst", $log.fieldsMap.get("dst"));
insert($ce);
HashMap<String,Object> fieldsMap = new HashMap<String,Object>();
fieldsMap.put("cu_rule_id", "{8AC52BA8-1EE8-4f18-9BB4-54492116501C}");
fieldsMap.put("event_name", "Port scan from external network");
fieldsMap.put("cu_rule_severity", calcSeverity($log));
fieldsMap.put("cu_rule_category", "Scans");
fieldsMap.put("cu_log_count", $markerSet.size());
fieldsMap.put("time", new Date());
fieldsMap.put("cu_markers_list", markersToString($markerSet));
fieldsMap.put("src", $log.fieldsMap.get("src"));
fieldsMap.put("src_machine_name", $log.fieldsMap.get("src_machine_name"));
fieldsMap.put("src_user_name", $log.fieldsMap.get("src_user_name"));
fieldsMap.put("dst", $log.fieldsMap.get("dst"));
fieldsMap.put("dst_machine_name", $log.fieldsMap.get("dst_machine_name"));
fieldsMap.put("dst_user_name", $log.fieldsMap.get("dst_user_name"));
fieldsMap.put("service", $log.fieldsMap.get("service"));
fieldsMap.put("proto", $log.fieldsMap.get("proto"));
fieldsMap.put("product", $log.fieldsMap.get("product"));
externalEventsHandler.handleEvent(fieldsMap);
end
I am sending logs in a rate of up to 200 logs/sec. After about 3 minutes, my application starts to be unresponsive.
I monitored the JVM with VisualVM. Two snapshots of VisualVM are attached.
I found out that the class consuming most memory is FromNodeLeftTuple of drools (as can be seen in "instances.png").
1) Is my inserting rate is too high?
2) Is There a way I can make my rules more memory efficient?
Thanks.
Inserting logs:
public void insertEvents(Collection<Map<String, Object>> logs)
{
for (Map<String, Object> map : logs) {
Log log = new Log();
Log.fieldsMap.putAll(map);
session.insert(log);
session.fireAllRules();
}
}
Log class:
public class Log
{
public HashMap<String, Object> fieldsMap = new HashMap<>();
}
CorrelatedEvent class:
public class CorrelatedEvent
{
public Map<String, Object> groupByFieldsMap;
private String id;
public CorrelatedEvent(String id)
{
groupByFieldsMap = new HashMap<>();
this.id = id;
}
public String getId()
{
return id;
}
}
12 years, 2 months
How to use global hashmap in drools
by bbarani
I am trying to use a global hashmap in drools but not sure how to create a
global hashmap.
I have a very simple function that counts the number of values passed hence
I would need a global hash map to increment the count.
The below function initializes the map every time the function is called, I
want to store the values in map until all the data gets processed.
function String getValue(String a, String b){
* Map<String, String> hashValue = new HashMap<String, String>();
* hashValue.put(a,b);
return hashValue.size();
}
--
View this message in context: http://drools.46999.n3.nabble.com/How-to-use-global-hashmap-in-drools-tp4...
Sent from the Drools: User forum mailing list archive at Nabble.com.
12 years, 2 months
KIE Workbench data model import
by rockford
Hello,
I am trying to import a data model from inside the Drools 6 KIE Workbench.
(I have used this functionality in the Guvnor.)
So far I have:
- Uploaded a jar containing a data model to the repository from the "Asset
repository" editor.
Now I am trying to see where to make the data model visible when using the
Guided Rules Editor. So far with no success.
Has anyone used this yet and can shed some light on this?
Ken Helmes
--
View this message in context: http://drools.46999.n3.nabble.com/KIE-Workbench-data-model-import-tp40265...
Sent from the Drools: User forum mailing list archive at Nabble.com.
12 years, 2 months
Drools server and knowledge agent functionality
by Manuel Ortiz
Hello everybody:
I am taking a look at Drools Integration documentation to understand how
Drools server works and how can be used, and have a doubt concerning, let's
say... 'KnowledgeAgent capabilities'. Does Drools server support on the fly
rule updating as when using KnowledgeAgent to create KnowledgeBase objects?
If it does, where can be configured 'KnowledgeAgent related' properties such
as newInstance or resource scanner interval?
Thank you in advance for your time.
Kind regards,
Manuel Ortiz.
12 years, 2 months
