[Hawkular-dev] [hawkular-apm] tracing async operations

[Gary Brown]

Hi John

If I put things in OpenTracing terms, each unit of work we want to measure is a Span, where work performed in the scope of one Span, is created as a child of that Span - so building up a tree structure. So this is how units of work, whether sync or async, get represented.

So the issue is, how we identify the start and end of a Span, when they are not necessarily occurring within the same thread. And secondly, when a child activity is spawned, that is not in the same thread as its parent, how that child Span becomes associated with its parent.

The piece of code you reference is part of the original java agent, and it relies on ThreadLocal as a way to track activity within threads - but to support async behaviour, the agent also enables the trace sessions/context to be propagated across threads, using context relevant ids.

If explicitly instrumenting code with the OpenTracing API, async behaviour can be monitored by simply passing around the Span objects.

Let me know if you need further information in any particular area.

Regards
Gary

----- Original Message -----
> I am reading through some of the Hawkular APM code and have been looking at
> how trace fragments get created and written on the client side. One of the
> classes I am reviewing is FragmentBuilder.java. Its javadocs state that the
> sequence of events within a thread of execution should be in sequence. It
> made me wonder whether it is possible to trace async operations. I found
> https://issues.jboss.org/browse/HWKAPM-77
> <https://issues.jboss.org/browse/HWKAPM-77> which apparently improves
> support for async execution.
> 
> Gary, can you or anyone else familiar how things work, give a brief
> explanation of how things work async flows? Most everything in
> hawkular-metrics is async and reactive. That makes debugging difficult at
> times where you have to rely primarily on logging. I am wondering if
> Hawkular APM would be a good tool for these types of situations.
> 
> - John