Thomas, just tried brand new installation of 5.4.0.Final full installer and was able to run evaluation without any problems.
One thing you could look into is to check your home directory to see if you don't have any jbpm db files there (for example jbpm.h2.db). If so, either remove them manually or run ant clean.demo and then ant install.demo again.
HTH