[wildfly-dev] Reading Logs from Web Console

James R. Perkins jperkins at redhat.com
Fri Aug 16 00:45:11 EDT 2013 

Yes, parsing is the biggest issue. There are all kinds of possibilities 
with the format. For example they could use a null terminator for the 
records rather than a new line terminator. That is why I thought 
allowing list files in the jboss.server.log.dir directory was the best 
option.

Essentially there would probably be two operations. One that lists the 
available files and one that reads the contents of the file. The read 
would probably be best in chunks since big files could be an issue. The 
main security issue here would be if someone starts the server with 
-Djboss.server.log.dir=$JBOSS_HOME/standalone/configuration they would 
have access to files like the mgmt-users.properties. This is one of the 
reasons I thought maybe it should be required to have a .log extension.

One idea I just thought of, and it might not even be possible, but it 
maybe have a way to email a compressed version of the log file. I could 
see some benefit with large files, but maybe ssh is better for that.

On 08/15/2013 08:14 PM, Jason Greene wrote:
> I forgot to mention that I chatted with David late last night and he 
> pointed out cases where applications dump multiline data that makes 
> parsing records impossible.
>
> He also brought up a problem with expression based file names, that 
> could make it not possible to reverse rolling logs.
>
> We did come up with several potential solutions but we really need to 
> nail the use cases first.
>
> The download/view case could be done securely and address the rolling 
> expression problem by making the feature more of a file browser that 
> is locked into a set of directories. By default we would allow 
> browsing into the log directories.
>
> On Aug 15, 2013, at 6:45 PM, "James R. Perkins" <jperkins at redhat.com 
> <mailto:jperkins at redhat.com>> wrote:
>
>>
>> On 08/15/2013 03:27 PM, Harald Pehl wrote:
>>> I agree that we should focus on the most important use cases:
>>> - Not able / not like to sshing into the server to look at the logs
>>> - Deployment failed and you want to know why (staying in the console)
>>>
>>> As for the console I would suggest to start with a very limited set 
>>> of features. No filtering, just show the last n log entries, where n 
>>> is specified as part of the  management operation. At first the user 
>>> would see the last n log entries and can navigate further backwards 
>>> in n-steps. For all deeper analysis the user should be able to 
>>> download the full log. When showing the log to the user, I can think 
>>> of some kind of syntax highlighting (see 
>>> http://www.gnu.org/software/src-highlite/error.log.html for an example).
>> The problem here is we don't know the last n entries if we read a 
>> file. We can read back the last n bytes or possibly lines, but that 
>> could contain portions of a stack trace.
>>>
>>> That said there are however use cases where parsing / filtering 
>>> makes sense. The audit log has a well defined format 
>>> (https://docs.jboss.org/author/display/WFLY8/Audit+logging#Auditlogging-JSONFormatter) 
>>> and in the current implementation I'm using a master / detail view 
>>> to display the audit log items.
>> This might be a little easier for audit-logging since we have more 
>> control over what's happening. I have to look at the code, but we 
>> might just want to always write a formatted file in addition to other 
>> types like the syslog.
>>>
>>> .: Harald
>>>
>>> ---
>>> Harald Pehl
>>> JBoss by Red Hat
>>> http://hpehl.info
>>>
>>>
>>>
>>>
>>> Am 15.08.2013 um 03:11 schrieb ssilvert at redhat.com 
>>> <mailto:ssilvert at redhat.com>:
>>>
>>>> On 8/14/2013 5:20 PM, Brian Stansberry wrote:
>>>>> Agreed. IMHO this is the most important driver for this feature.
>>>>>
>>>>> On 8/14/13 3:44 PM, James R. Perkins wrote:
>>>>>> Just to add too in a domain it would be nice to have a central 
>>>>>> spot to
>>>>>> view logs instead of having to ssh into various servers.
>>>> Also realize that in many shops you don't have the option to ssh into
>>>> various servers.  If they are serious about security then access to the
>>>> console is likely to be the only access you have.
>>>>
>>>>>>
>>>>>> On 08/14/2013 12:56 PM, Jason Greene wrote:
>>>>>>> Mainly convenience. You deploy something it fails, you want to 
>>>>>>> look at the log but don't feel like sshing into the server. As 
>>>>>>> to performance the cost would be on request, and not more 
>>>>>>> expensive then looking at the log file via ssh.
>>>>>>>
>>>>>>> On Aug 14, 2013, at 2:24 PM, Scott Marlow <smarlow at redhat.com 
>>>>>>> <mailto:smarlow at redhat.com>> wrote:
>>>>>>>
>>>>>>>> What are the use cases for online reading of the server logs? 
>>>>>>>>  If there
>>>>>>>> are problems occurring on the application server (e.g. perhaps 
>>>>>>>> the cpu
>>>>>>>> is pegged), reading logs online, could make the system even less
>>>>>>>> responsive.
>>>>>>>>
>>>>>>>> If we just want to read the server logs as part of a health 
>>>>>>>> check, not
>>>>>>>> requiring the server console to be working would be better.
>>>>>>>>
>>>>>>>> Should the reading of the logs instead be an external capability?
>>>>>>>> Perhaps using the logs from the JBoss/WildFly Diagnostic 
>>>>>>>> Reporter output
>>>>>>>> (archive) or some other archived copy of logs.
>>>>>>>>
>>>>>>>> Another compromise, add the WildFly Diagnostic Reporter (or at 
>>>>>>>> least the
>>>>>>>> log collection part) to the management console (output archive is
>>>>>>>> downloaded for local viewing).
>>>>>>>>
>>>>>>>> On 08/14/2013 01:03 PM, James R. Perkins wrote:
>>>>>>>>> I had posted this to another list, but this is a more 
>>>>>>>>> appropriate place
>>>>>>>>> for it. I think there needs to be a general discussion around 
>>>>>>>>> this as
>>>>>>>>> it's been mentioned, at least to me, a few times here and 
>>>>>>>>> there and I
>>>>>>>>> know Heiko raised the issue some time a go now.
>>>>>>>>>
>>>>>>>>> The original JIRA, WFLY-280[1], is to display the last 10 
>>>>>>>>> error messages
>>>>>>>>> only. To be honest I wouldn't find that very useful. To me if I'm
>>>>>>>>> looking for logs I want to see all logs, but that's not always 
>>>>>>>>> so easy.
>>>>>>>>> Like the syslog-handler which doesn't log to a file so there 
>>>>>>>>> is no way
>>>>>>>>> to read those messages back.
>>>>>>>>>
>>>>>>>>> The current plan for the last 10 error messages is we store 
>>>>>>>>> messages in
>>>>>>>>> a queue that can be accessed via an operation. This works fine 
>>>>>>>>> until the
>>>>>>>>> error message you're interested in is 11 or you want to see 
>>>>>>>>> warning
>>>>>>>>> messages.
>>>>>>>>>
>>>>>>>>> Another option I had come up with is reading back the contents 
>>>>>>>>> of the
>>>>>>>>> file, for example the server.log. This could be problematic 
>>>>>>>>> too in that
>>>>>>>>> there is no way to filter information like only see error 
>>>>>>>>> messages or
>>>>>>>>> only see warning messages. To solve this I have considered 
>>>>>>>>> creating a
>>>>>>>>> JSON formatter so the results could be queried, but I don't 
>>>>>>>>> think it
>>>>>>>>> should be a default which would mean it's not reliable for the 
>>>>>>>>> console
>>>>>>>>> to assume it's getting back JSON.
>>>>>>>>>
>>>>>>>>> I've also thought about, haven't tested this and it may not 
>>>>>>>>> work at all,
>>>>>>>>> creating a handler that uses websockets to send messages. I'm 
>>>>>>>>> not sure
>>>>>>>>> how well this would work and it's possible it may not even 
>>>>>>>>> work for
>>>>>>>>> bootstrap logging.
>>>>>>>>>
>>>>>>>>> With regards to audit logging, we're probably going to have to do
>>>>>>>>> something totally different from what we'll do in the logging 
>>>>>>>>> subsystem
>>>>>>>>> since it doesn't use standard logging.
>>>>>>>>>
>>>>>>>>> I guess the bottom line is what does the console want to see? 
>>>>>>>>> Do you
>>>>>>>>> want to see all raw text log messages? Do you want all 
>>>>>>>>> messages but in a
>>>>>>>>> format like JSON that you can query/filter? Do you really want 
>>>>>>>>> only the
>>>>>>>>> last 10 error messages only? All or none of these might be 
>>>>>>>>> possible, but
>>>>>>>>> I really need to understand the needs before I can explore 
>>>>>>>>> more in depth
>>>>>>>>> what the best option would be.
>>>>>>>>>
>>>>>>>>> [1]: https://issues.jboss.org/browse/WFLY-280
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> James R. Perkins
>>>>>>>>> Red Hat JBoss Middleware
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> wildfly-dev mailing list
>>>>>>>>> wildfly-dev at lists.jboss.org <mailto:wildfly-dev at lists.jboss.org>
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> wildfly-dev mailing list
>>>>>>>> wildfly-dev at lists.jboss.org <mailto:wildfly-dev at lists.jboss.org>
>>>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>>>> --
>>>>>>> Jason T. Greene
>>>>>>> WildFly Lead / JBoss EAP Platform Architect
>>>>>>> JBoss, a division of Red Hat
>>>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> wildfly-dev mailing list
>>>> wildfly-dev at lists.jboss.org <mailto:wildfly-dev at lists.jboss.org>
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> wildfly-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>> -- 
>> James R. Perkins
>> Red Hat JBoss Middleware
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org <mailto:wildfly-dev at lists.jboss.org>
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev

-- 
James R. Perkins
Red Hat JBoss Middleware

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20130815/0752264b/attachment-0001.html

More information about the wildfly-dev mailing list