Re: [wildfly-dev] Reading Logs from Web Console

I forgot to mention that I chatted with David late last night and he pointed out cases where applications dump multiline data that makes parsing records impossible. He also brought up a problem with expression based file names, that could make it not possible to reverse rolling logs. We did come up with several potential solutions but we really need to nail the use cases first. The download/view case could be done securely and address the rolling expression problem by making the feature more of a file browser that is locked into a set of directories. By default we would allow browsing into the log directories. On Aug 15, 2013, at 6:45 PM, "James R. Perkins" <jperkins(a)redhat.com <mailto:jperkins@redhat.com>> wrote: > > On 08/15/2013 03:27 PM, Harald Pehl wrote: >> I agree that we should focus on the most important use cases: >> - Not able / not like to sshing into the server to look at the logs >> - Deployment failed and you want to know why (staying in the console) >> >> As for the console I would suggest to start with a very limited set >> of features. No filtering, just show the last n log entries, where n >> is specified as part of the management operation. At first the user >> would see the last n log entries and can navigate further backwards >> in n-steps. For all deeper analysis the user should be able to >> download the full log. When showing the log to the user, I can think >> of some kind of syntax highlighting (see >> http://www.gnu.org/software/src-highlite/error.log.html for an example). > The problem here is we don't know the last n entries if we read a > file. We can read back the last n bytes or possibly lines, but that > could contain portions of a stack trace. >> >> That said there are however use cases where parsing / filtering >> makes sense. The audit log has a well defined format >> (https://docs.jboss.org/author/display/WFLY8/Audit+logging#Auditlogging-JS...) >> and in the current implementation I'm using a master / detail view >> to display the audit log items. > This might be a little easier for audit-logging since we have more > control over what's happening. I have to look at the code, but we > might just want to always write a formatted file in addition to other > types like the syslog. >> >> .: Harald >> >> --- >> Harald Pehl >> JBoss by Red Hat >> http://hpehl.info >> >> >> >> >> Am 15.08.2013 um 03:11 schrieb ssilvert(a)redhat.com >> <mailto:ssilvert@redhat.com>: >> >>> On 8/14/2013 5:20 PM, Brian Stansberry wrote: >>>> Agreed. IMHO this is the most important driver for this feature. >>>> >>>> On 8/14/13 3:44 PM, James R. Perkins wrote: >>>>> Just to add too in a domain it would be nice to have a central >>>>> spot to >>>>> view logs instead of having to ssh into various servers. >>> Also realize that in many shops you don't have the option to ssh into >>> various servers. If they are serious about security then access to the >>> console is likely to be the only access you have. >>> >>>>> >>>>> On 08/14/2013 12:56 PM, Jason Greene wrote: >>>>>> Mainly convenience. You deploy something it fails, you want to >>>>>> look at the log but don't feel like sshing into the server. As >>>>>> to performance the cost would be on request, and not more >>>>>> expensive then looking at the log file via ssh. >>>>>> >>>>>> On Aug 14, 2013, at 2:24 PM, Scott Marlow <smarlow(a)redhat.com >>>>>> <mailto:smarlow@redhat.com>> wrote: >>>>>> >>>>>>> What are the use cases for online reading of the server logs? >>>>>>> If there >>>>>>> are problems occurring on the application server (e.g. perhaps >>>>>>> the cpu >>>>>>> is pegged), reading logs online, could make the system even less >>>>>>> responsive. >>>>>>> >>>>>>> If we just want to read the server logs as part of a health >>>>>>> check, not >>>>>>> requiring the server console to be working would be better. >>>>>>> >>>>>>> Should the reading of the logs instead be an external capability? >>>>>>> Perhaps using the logs from the JBoss/WildFly Diagnostic >>>>>>> Reporter output >>>>>>> (archive) or some other archived copy of logs. >>>>>>> >>>>>>> Another compromise, add the WildFly Diagnostic Reporter (or at >>>>>>> least the >>>>>>> log collection part) to the management console (output archive is >>>>>>> downloaded for local viewing). >>>>>>> >>>>>>> On 08/14/2013 01:03 PM, James R. Perkins wrote: >>>>>>>> I had posted this to another list, but this is a more >>>>>>>> appropriate place >>>>>>>> for it. I think there needs to be a general discussion around >>>>>>>> this as >>>>>>>> it's been mentioned, at least to me, a few times here and >>>>>>>> there and I >>>>>>>> know Heiko raised the issue some time a go now. >>>>>>>> >>>>>>>> The original JIRA, WFLY-280[1], is to display the last 10 >>>>>>>> error messages >>>>>>>> only. To be honest I wouldn't find that very useful. To me if I'm >>>>>>>> looking for logs I want to see all logs, but that's not always >>>>>>>> so easy. >>>>>>>> Like the syslog-handler which doesn't log to a file so there >>>>>>>> is no way >>>>>>>> to read those messages back. >>>>>>>> >>>>>>>> The current plan for the last 10 error messages is we store >>>>>>>> messages in >>>>>>>> a queue that can be accessed via an operation. This works fine >>>>>>>> until the >>>>>>>> error message you're interested in is 11 or you want to see >>>>>>>> warning >>>>>>>> messages. >>>>>>>> >>>>>>>> Another option I had come up with is reading back the contents >>>>>>>> of the >>>>>>>> file, for example the server.log. This could be problematic >>>>>>>> too in that >>>>>>>> there is no way to filter information like only see error >>>>>>>> messages or >>>>>>>> only see warning messages. To solve this I have considered >>>>>>>> creating a >>>>>>>> JSON formatter so the results could be queried, but I don't >>>>>>>> think it >>>>>>>> should be a default which would mean it's not reliable for the >>>>>>>> console >>>>>>>> to assume it's getting back JSON. >>>>>>>> >>>>>>>> I've also thought about, haven't tested this and it may not >>>>>>>> work at all, >>>>>>>> creating a handler that uses websockets to send messages. I'm >>>>>>>> not sure >>>>>>>> how well this would work and it's possible it may not even >>>>>>>> work for >>>>>>>> bootstrap logging. >>>>>>>> >>>>>>>> With regards to audit logging, we're probably going to have to do >>>>>>>> something totally different from what we'll do in the logging >>>>>>>> subsystem >>>>>>>> since it doesn't use standard logging. >>>>>>>> >>>>>>>> I guess the bottom line is what does the console want to see? >>>>>>>> Do you >>>>>>>> want to see all raw text log messages? Do you want all >>>>>>>> messages but in a >>>>>>>> format like JSON that you can query/filter? Do you really want >>>>>>>> only the >>>>>>>> last 10 error messages only? All or none of these might be >>>>>>>> possible, but >>>>>>>> I really need to understand the needs before I can explore >>>>>>>> more in depth >>>>>>>> what the best option would be. >>>>>>>> >>>>>>>> [1]: https://issues.jboss.org/browse/WFLY-280 >>>>>>>> >>>>>>>> -- >>>>>>>> James R. Perkins >>>>>>>> Red Hat JBoss Middleware >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> wildfly-dev mailing list >>>>>>>> wildfly-dev(a)lists.jboss.org <mailto:wildfly-dev@lists.jboss.org> >>>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> wildfly-dev mailing list >>>>>>> wildfly-dev(a)lists.jboss.org <mailto:wildfly-dev@lists.jboss.org> >>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev >>>>>> -- >>>>>> Jason T. Greene >>>>>> WildFly Lead / JBoss EAP Platform Architect >>>>>> JBoss, a division of Red Hat >>>>>> >>>> >>> >>> _______________________________________________ >>> wildfly-dev mailing list >>> wildfly-dev(a)lists.jboss.org <mailto:wildfly-dev@lists.jboss.org> >>> https://lists.jboss.org/mailman/listinfo/wildfly-dev >> >> >> >> _______________________________________________ >> wildfly-dev mailing list >> wildfly-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/wildfly-dev > > -- > James R. Perkins > Red Hat JBoss Middleware > _______________________________________________ > wildfly-dev mailing list > wildfly-dev(a)lists.jboss.org <mailto:wildfly-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/wildfly-dev