[jboss-jira] [JBoss JIRA] (WFLY-11031) Json encoding of Access Logs

[Frank Schmager (JIRA)]

Frank Schmager created WFLY-11031:
-------------------------------------

             Summary: Json encoding of Access Logs
                 Key: WFLY-11031
                 URL: https://issues.jboss.org/browse/WFLY-11031
             Project: WildFly
          Issue Type: Feature Request
          Components: Web (Undertow)
            Reporter: Frank Schmager
            Assignee: Stuart Douglas
            Priority: Minor


h1. Request
It should be possible to emit access log events json encoded to allow log aggregation systems to ingest pre-parsed events for filtering and analytics.

h1. Background
Application logs can be emitted [json encoded|https://wildscribe.github.io/WildFly/13.0/subsystem/logging/json-formatter/index.html] allowing log aggregation systems like logstash/ELK to ingest and parse the events. This is especially important when working with containers, but anything requiring log aggregation would benefit. Not all log aggregation systems allow after-the-fact parsing of log events (e.g. Kibana) and developers don't have access to the log aggregator to install a parser like logstash/filebeat.

[Access log events|https://wildscribe.github.io/WildFly/13.0/subsystem/undertow/server/host/setting/access-log/index.html], however, can't be json encoded. The whole log event can be, but what's important about access log events can't.

h2. Example result
There are many ways to encode the event into json. Ideally it would be configurable. One way would be to use {{message}}, but instead of it just containing a {{String}} it could contain json.
{code:json}
...
"@timestamp": "2018-09-14T20:59:37.530Z",
...
"message": {
  "x-forwarded-for": "-",
  "remoteHost": "172.20.2.1",
  "remoteUser": "200",
  "dateTime": "[14/Sep/2018:20:59:37 +0000]",
  "requestLine": "GET /auth/resources/4.4.0.final/login/keycloak/img/keycloak-bg.png HTTP/2.0",
  "status": "200",
  "bytesSent": "81862",
  "timeTaken": "10",
  "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36"
}
{code} 

h2. Attempted Workaround
Making the pattern look like json failed due to double quote escaping:
{code:xml}
<access-log pattern='{\"x-forwarded-for\":\"%{i,X-Forwarded-For}\",\"remoteHost\":\"%h\",\"remoteUser\":\"%s\",\"dateTime\":\"%t\",\"requestLine\":\"%r\",\"status\":\"%s\",\"bytesSent\":\"%b\",\"timeTaken\":\"%D\",\"userAgent\":\"%{i,User-Agent}\"}' use-server-log="true" />
{code}
h3. Result
{code}
...
"@timestamp": "2018-09-14T20:59:37.530Z",
...
"message": "{\\\"x-forwarded-for\\\":\\\"-\\\",\\\"remoteHost\\\":\\\"172.20.2.1\\\",\\\"remoteUser\\\":\\\"200\\\",\\\"dateTime\\\":\\\"[14/Sep/2018:20:59:37 +0000]\\\",\\\"requestLine\\\":\\\"GET /auth/resources/4.4.0.final/login/keycloak/img/keycloak-bg.png HTTP/2.0\\\",\\\"status\\\":\\\"200\\\",\\\"bytesSent\\\":\\\"81862\\\",\\\"timeTaken\\\":\\\"10\\\",\\\"userAgent\\\":\\\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36\\\"}",
{code}

h2. Json formatting log configuration
{code:xml}
<subsystem xmlns="urn:jboss:domain:logging:5.0">
            <console-handler name="CONSOLE">
                <level name="DEBUG"/>
                <formatter>
                    <named-formatter name="JSON_FORMATTER"/>
                </formatter>
            </console-handler>
...
            <formatter name="JSON_FORMATTER">
                <json-formatter>
                    <exception-output-type value="formatted"/>
                    <key-overrides timestamp="@timestamp" loggerName="logger_name"/>
                    <meta-data>
                        <property name="@version" value="1"/>
                    </meta-data>
                </json-formatter>
            </formatter>
...
{code}



--
This message was sent by Atlassian JIRA
(v7.5.0#75005)