[teiid-issues] [JBoss JIRA] (TEIID-3404) Teiid DatabaseAppender JPA perisist Command log consume high CPU

[Steven Hawkins (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13053959#comment-13053959 ] 

Steven Hawkins commented on TEIID-3404:
---------------------------------------

> From the code [1], DatabaseAppender does a JNDI lookup on every call, then EJB/JPA persist the log entity, probably we "wouldn't do like that".

Presumably the logging configuration is such that this is an asynch appender and that only command/audit log entries are routed to it.  Any other configuration would be highly problematic.  

> One question I first thought is why we need persist log to database?

This is only for the Teiid audit/command log events, not all logs.  I believe this was a customer/product management feature request.

> Do we need consider transaction per log or per request? 

There will be somewhere around 2 + 2*(source queries) command logs per user request.  There will typically only be one audit log entry.

> I also find the default production install use the DEBUG level log, does this reasonable?

The command and audit log events are only logged at the debug and trace level.  So anything higher than debug would see no events.

> can we use H2 in production?

H2 is not required.  The rationale I believe for using JPA was that the logger would be database agnostic.

> I thought to enhance this issue is use JDBC/batch the insertions to replace EJB/JPA, do you think this is a feasible way ?

Queuing/batching can be implemented either way.  Since the asynch appender approach already is maintaining an event queue it would make more sense to batch instead.  As for using straight JDBC vs JPA, I'd let Ramesh weigh in there.

> Teiid DatabaseAppender JPA perisist Command log consume high CPU
> ----------------------------------------------------------------
>
>                 Key: TEIID-3404
>                 URL: https://issues.jboss.org/browse/TEIID-3404
>             Project: Teiid
>          Issue Type: Bug
>          Components: Tools
>    Affects Versions: 8.7.1
>         Environment: * DV 6.1
> * dynamicvdb-datafederation quickstart
>            Reporter: Kylin Soong
>            Assignee: Kylin Soong
>
> I have 4 - 5 concurrent client query to Portfolio VDB, this cause  log  DatabaseAppender JPA perisist Command log thread consume high CPU. In my test, one log threads always swallow high CPU, if the concurrent client query more than 10, it swallow 100% cpu.
> [1] have detailed reproduce steps, including thread dumps.
> [1] https://github.com/kylinsoong/teiid-samples/tree/master/command-logging-highcpu



--
This message was sent by Atlassian JIRA
(v6.3.11#6341)