[teiid-issues] [JBoss JIRA] (TEIID-3404) Teiid DatabaseAppender JPA perisist Command log consume high CPU

[Ramesh Reddy (JIRA)]

    [ https://issues.jboss.org/browse/TEIID-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054083#comment-13054083 ] 

Ramesh Reddy commented on TEIID-3404:
-------------------------------------

I did not wanted to use JDBC as there are enough database specific issues, we need to provide plugins for each source. JPA seemed lot simpler and even can create the tables on the fly. Is batching possible in JPA? if yes, may be we can use it. This mechanism using EJB with JPA, we need to see if this a EJB pooling issue or what is contributing to the high CPU. The thing I do not like about it is there is no further management we offer on it, like archiving, log rotations or viewing or searching, nor I would like to go into that space, that is where below solutions come in..

What would be interesting is move on to new log management systems like 
http://logstash.net/
http://www.fluentd.org/
https://cwiki.apache.org/confluence/display/FLUME/Home (written in java)
http://kafka.apache.org/

I am sure for most of them they may already support the Log4J appenders, we should research into these as separate issue.

> Teiid DatabaseAppender JPA perisist Command log consume high CPU
> ----------------------------------------------------------------
>
>                 Key: TEIID-3404
>                 URL: https://issues.jboss.org/browse/TEIID-3404
>             Project: Teiid
>          Issue Type: Bug
>          Components: Tools
>    Affects Versions: 8.7.1
>         Environment: * DV 6.1
> * dynamicvdb-datafederation quickstart
>            Reporter: Kylin Soong
>            Assignee: Kylin Soong
>
> I have 4 - 5 concurrent client query to Portfolio VDB, this cause  log  DatabaseAppender JPA perisist Command log thread consume high CPU. In my test, one log threads always swallow high CPU, if the concurrent client query more than 10, it swallow 100% cpu.
> [1] have detailed reproduce steps, including thread dumps.
> [1] https://github.com/kylinsoong/teiid-samples/tree/master/command-logging-highcpu



--
This message was sent by Atlassian JIRA
(v6.3.11#6341)