[wildfly-dev] Gathering diagnostic data

[James Livingston]

Hi,

I do support for JBoss at RH, especially lower level components like the
JVM and core server, and I've come to realise that we don't have
particularly good mechanisms for gathering diagnostic data. Obviously
there is logging, but for a number of problems it isn't enough. There is
also the JDR subsystem, but that has the serious limitation that the
server instance(s) have to be working correctly to use it.


One piece of data that I think can be useful for a common set of
problems are thread dumps. For standalone instances Oracle/OpenJDK's
'jstack' tool is useful, but for domain mode you need to manually go to
each machine and find the correct process ID to use it. Something that
could potentially be useful is to be able to tell Host Controllers to
take a thread dump from the servers it manages (or all of them), using
the same Attach/SA APIs as jstack.

A feature to allow HC-based extensions as mentioned in the thread about
the RHQ agent could be useful for implementing this too. Right now, I
believe it would need to be added to the HC code directly.


Is this something that might be possible or a good idea to do? I have
some ideas about other things that may be useful, like per-thread CPU
usage which RH support currently uses a pile of shell scripts for, but
starting with one thing may be simpler.

-- 
James "Doc" Livingston
JBoss Support Engineering Group