Solved my issue, was quite obvious. When using the above code in a package, the .setInterfaces command must refer to the fully qualified name. Assuming the two files are in package "test", one must cast
evalClass.setInterfaces(new CtClass[] { pool.makeClass("test.Evaluator") });