<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Is it possible to read log files not stored under ${jboss.server.log.dir}? I'm asking because the audit log is per default stored under ${jboss.server.data.dir}<div><br><div apple-content-edited="true">
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">--- <br>Harald Pehl</div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">JBoss by Red Hat<br><div><a href="http://hpehl.info">http://hpehl.info</a></div><div><br></div></div><br class="Apple-interchange-newline"><br class="Apple-interchange-newline">
</div>
<br><div><div>Am 09.10.2013 um 18:10 schrieb James R. Perkins <<a href="mailto:jperkins@redhat.com">jperkins@redhat.com</a>>:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<meta content="text/html; charset=windows-1252" http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000">
In my opinion we shouldn't offer any kind of log parsing options. We
should simple deliver the raw log file and have the end user use
their own parsing. There is a JIRA [1] to allow custom formatters to
be used. This means an end user could use an XML formatter and it
could be easily parsed by them. Since format patterns are so
configurable and we're going to expose the ability to override the
formatter we need to just give them the raw file.<br>
<br>
I suppose saying all that contradicts my reasoning for using the
BufferedReader.readLine() and maybe we should be returning by number
of bytes/chars instead of by line. There seems to be no real good or
solid answer and it would be nice to hear from any users that have
an opinion on how they might use it. From the web console I'm going
to guess either approach, by line or by bytes, will work fine as it
might just construct one large file for download. If using it from a
console just to see the last 20 lines of the log file, a
line-by-line approach to me makes more sense.<br>
<br>
[1]:
<meta http-equiv="content-type" content="text/html;
charset=windows-1252">
<a href="https://issues.jboss.org/browse/WFLY-1188">https://issues.jboss.org/browse/WFLY-1188</a><br>
<br>
<div class="moz-cite-prefix">On 10/09/2013 05:09 AM, Ondrej Zizka
wrote:<br>
</div>
<blockquote cite="mid:5255478F.6060206@redhat.com" type="cite">How
about storing the offsets of the records.
<br>
Then the operation would read this "index" and instead of
format-based parsing, it would simply cut the right part.
<br>
Simple, fast in both write and read, and doesn't consume too much
memory/storage.
<br>
<br>
Ondra
<br>
<br>
<br>
On 8.10.2013 18:57, James R. Perkins wrote:
<br>
<blockquote type="cite">I'm definitely not trying to parse any of
the log message format. I am
<br>
using a BufferedReader.readLine() which I was a little iffy on,
but went
<br>
with "lines" as opposed to reading x bytes. Obviously from a
management
<br>
standpoint reading the number of bytes makes more sense as we're
not
<br>
relying on any kind of normal line terminator being used. From a
user
<br>
standpoint though I think it makes less sense and the number of
lines
<br>
makes more sense.
<br>
<br>
I suppose we could have two operations so the web console could
use the
<br>
one that reads bytes and then have one a simple tail like one
that reads
<br>
by line.
<br>
<br>
On 10/08/2013 09:26 AM, David M. Lloyd wrote:
<br>
<blockquote type="cite">I've said it before and I'll say it
again. Using the format to
<br>
"intelligently" parse the logs looks like a good idea. It is
actually a
<br>
bad idea. A bad idea that looks like a good idea, or maybe
even a "good
<br>
enough" idea. Maybe it looks like an "easy" idea. But it's
bad. Bad,
<br>
and also not good. It will result in bug reports that nobody
will
<br>
bother fixing or implementing because they will be reasonable
yet
<br>
unfeasible to fix.
<br>
<br>
Here are some implementation options that I *won't*
immediately kill:
<br>
<br>
1) Treat the files in the log directory as opaque things that
the server
<br>
does not pretend to have interior knowledge of - in other
words, provide
<br>
operations that work on the file as a whole (or by things
which make
<br>
sense at a file level, like line number or dumb string grep)
which will
<br>
work 100% of the time
<br>
2) Provide in-memory (object) or on-disk (structured binary)
storage of
<br>
log record objects with intelligent search capabilities which
will work
<br>
100% of the time
<br>
<br>
Here are some implementation options that I *will* immediately
kill:
<br>
<br>
1) Try to use the format to figure out record boundaries
<br>
2) Try to use the format to figure out the date of the record
<br>
3) Try to use the format to extract exception information
<br>
4) Any and all other format-based ideas not mentioned here
<br>
<br>
The common theme is - it has to work 100% of the time. No
excuses.
<br>
<br>
On 10/08/2013 10:55 AM, James R. Perkins wrote:
<br>
<blockquote type="cite">On 10/08/2013 03:40 AM, Kabir Khan
wrote:
<br>
<blockquote type="cite">Just a thought regarding your PR
<a class="moz-txt-link-freetext" href="https://github.com/wildfly/wildfly/pull/5156">https://github.com/wildfly/wildfly/pull/5156</a>
<br>
<br>
The thing that stands out to me is that you're able to
read the first 100 lines, e.g.
<br>
:read-log-file(name=server.log, lines=100,
tail=false)
<br>
or the last 100 lines
<br>
:read-log-file(name=server.log, lines=100,
tail=true)
<br>
<br>
Would it also be beneficial to also be allowed to start
somewhere in the middle, e.g
<br>
:read-log-file(name=server.log, lines=100,
start=10235)
<br>
</blockquote>
There is an option skip that will allows something similar.
The
<br>
following would read read the last 100 lines _after_ 50
lines are
<br>
skipped from the bottom. So in a 200 line this would read
lines 50 to
<br>
line 150.
<br>
:read-log-file(name=server.log, lines=100, skip=50,
tail=true)
<br>
<br>
The following would read from line 50 to line 150
<br>
:read-log-file(name=server.log, lines=100, skip=50,
tail=false)
<br>
<br>
I am using a BufferedReader so if an odd line terminator is
used it will
<br>
likely not work so well. I considered allowing for the line
terminator
<br>
to be defined in the operation, but it would require
analyzing bytes for
<br>
the terminator and seemed like more work than it was worth.
I would
<br>
guess the majority of users use the standard /n or /r/n
terminators.
<br>
<blockquote type="cite">or (19:38:01 today)
<br>
:read-log-file(name=server.log, lines=100,
start-time=20131008-10:38:01)
<br>
</blockquote>
The date would be rather difficult. We're not parsing the
log strings
<br>
just reading the raw text and sending it back.
<br>
<blockquote type="cite">I'm not saying it MUST be like that
but it might be worth discussing
<br>
<br>
On 25 Sep 2013, at 16:36, Ondrej Zizka wrote:
<br>
<br>
<blockquote type="cite">Right, by "current" I meant to
read the logging setup and if there's
<br>
just one file appender, take it as default. Or, if that
would be
<br>
inappropriate mixing of abstraction layers, just
"server.log".
<br>
<br>
<br>
On 25.9.2013 12:06, Kabir Khan wrote:
<br>
<blockquote type="cite">Current log could be the
server.log in a standard setup. If someone has done
something more advanced in their logging setup, the
name becomes necessary
<br>
On 25 Sep 2013, at 08:34, Jaikiran Pai wrote:
<br>
<br>
<blockquote type="cite">On Wednesday 25 September 2013
09:25 AM, Ondrej Zizka wrote:
<br>
<blockquote type="cite">1) Could it have a
"read-log-file()" without name= specified, which
would read the "current" log file?
<br>
</blockquote>
Given the way logger categories and
appenders/handlers interact, within a logging
framework, I don't think there's any notion of
"current log file". It's a very valid scenario where
a single logging category can be backed by different
appenders (some of them file appenders) with
different attributes and each such appender writing
out to a different file. So having a name of the log
file you want to view, becomes necessary.
<br>
<br>
-Jaikiran
<br>
<br>
<blockquote type="cite">2) Regarding security -
what, besides logs, do we expect to be in the log
dir? Could the admin block it by setting
write-only rights?
<br>
<br>
Ondra
<br>
<br>
On 25.9.2013 02:40, James R. Perkins wrote:
<br>
<blockquote type="cite">I'm replying to this old
thread to reopen this conversation about reading
log files. I've complete some work [1] on
reading log files via an operation. This is not
exactly like the JIRA suggests where it would
only read the last 10 error messages. All this
change allows is the raw contents of the file to
be read. The idea is this could be used to read
the entire contents of the log file as a whole,
or in chunks.
<br>
<br>
What I've done is added two new operations
list-log-files and read-log-file.
<br>
<br>
The list-log-files simply lists all files in the
jboss.server.log.dir. This may or may not be a
good idea really. I can see some potential
security risks here mainly just seeing files
that may contain sensitive data. One way I've
thought of to get around that is read the
logging subsystem model and only show files from
known types like the file-handlers. The main
issue with that is there is no good way to get
this to work for custom-handlers.
<br>
<br>
The read-log-file simple does what it says and
reads the contents of a log file line by line.
Reading line by line should work for the most
part unless the an non-standard line delimiter
is used. There are 5 options for this option;
<br>
• name (required): the name of the log file
to read
<br>
• encoding: the encoding for the log file
<br>
• lines: the number of lines to read,
defaults to 10
<br>
• skip: the number of lines to skip before
adding the results
<br>
• tail: true to read from the bottom up,
default is true
<br>
The result of this is just a list of lines with
the \n or \r\n stripped. Just to clarify too a
line means a line in the file, not a log record
e.g. stack traces are generally composed of
multiple lines.
<br>
<br>
So this begs the question, will this work for
what we want? What concerns does anyone else
have?
<br>
<br>
I have not yet submitted a PR yet as I wanted to
get some feedback before we bake it in.
<br>
<br>
<br>
[1]:
<a class="moz-txt-link-freetext" href="https://github.com/jamezp/wildfly/compare/WFLY-280-read">https://github.com/jamezp/wildfly/compare/WFLY-280-read</a>
<br>
<br>
<br>
On 08/14/2013 10:03 AM, James R. Perkins wrote:
<br>
<blockquote type="cite">I had posted this to
another list, but this is a more appropriate
place for it. I think there needs to be a
general discussion around this as it's been
mentioned, at least to me, a few times here
and there and I know Heiko raised the issue
some time a go now.
<br>
<br>
The original JIRA, WFLY-280[1], is to display
the last 10 error messages only. To be honest
I wouldn't find that very useful. To me if I'm
looking for logs I want to see all logs, but
that's not always so easy. Like the
syslog-handler which doesn't log to a file so
there is no way to read those messages back.
<br>
<br>
The current plan for the last 10 error
messages is we store messages in a queue that
can be accessed via an operation. This works
fine until the error message you're interested
in is 11 or you want to see warning messages.
<br>
<br>
Another option I had come up with is reading
back the contents of the file, for example the
server.log. This could be problematic too in
that there is no way to filter information
like only see error messages or only see
warning messages. To solve this I have
considered creating a JSON formatter so the
results could be queried, but I don't think it
should be a default which would mean it's not
reliable for the console to assume it's
getting back JSON.
<br>
<br>
I've also thought about, haven't tested this
and it may not work at all, creating a handler
that uses websockets to send messages. I'm not
sure how well this would work and it's
possible it may not even work for bootstrap
logging.
<br>
<br>
With regards to audit logging, we're probably
going to have to do something totally
different from what we'll do in the logging
subsystem since it doesn't use standard
logging.
<br>
<br>
I guess the bottom line is what does the
console want to see? Do you want to see all
raw text log messages? Do you want all
messages but in a format like JSON that you
can query/filter? Do you really want only the
last 10 error messages only? All or none of
these might be possible, but I really need to
understand the needs before I can explore more
in depth what the best option would be.
<br>
<br>
[1]: <a class="moz-txt-link-freetext" href="https://issues.jboss.org/browse/WFLY-280">https://issues.jboss.org/browse/WFLY-280</a>
<br>
--
<br>
James R. Perkins
<br>
Red Hat JBoss Middleware
<br>
<br>
</blockquote>
--
<br>
James R. Perkins
<br>
Red Hat JBoss Middleware
<br>
<br>
<br>
<br>
<br>
_______________________________________________
<br>
wildfly-dev mailing list
<br>
<br>
<a class="moz-txt-link-abbreviated" href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a>
<br>
<a class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/wildfly-dev">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a>
<br>
</blockquote>
_______________________________________________
<br>
wildfly-dev mailing list
<br>
<br>
<a class="moz-txt-link-abbreviated" href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a>
<br>
<a class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/wildfly-dev">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a>
<br>
</blockquote>
_______________________________________________
<br>
wildfly-dev mailing list
<br>
<a class="moz-txt-link-abbreviated" href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a>
<br>
<a class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/wildfly-dev">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a>
<br>
</blockquote>
</blockquote>
_______________________________________________
<br>
wildfly-dev mailing list
<br>
<a class="moz-txt-link-abbreviated" href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a>
<br>
<a class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/wildfly-dev">https://lists.jboss.org/mailman/listinfo/wildfly-dev</a>
<br>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<br>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
James R. Perkins
Red Hat JBoss Middleware</pre>
</div>
_______________________________________________<br>wildfly-dev mailing list<br><a href="mailto:wildfly-dev@lists.jboss.org">wildfly-dev@lists.jboss.org</a><br>https://lists.jboss.org/mailman/listinfo/wildfly-dev<br></blockquote></div><br></div></body></html>