Hi Tobias,
I have created a simple version of a node instance based core process engine to explain how jBPM5 works in the jBPM5 Developer guide.
You can find the not finished source code here: https://github.com/Salaboy/Drools_jBPM5-Training-Examples/tree/master/jbpm5/07-jBPM5-NodeInstanceEngineSimpleImpl
You can take a look at the tests to see what is happening. Obviously this is a simplified version of jBPM5 core, but it can help you to understand how it works.
Cheers