Ok, I can reproduce this using my original code by just persisting more entities than the batch size which causes the batch execution to happen on addToBatch rather than performBatch.

As you seem to be aware, this is pretty much explicitly not supported. But you have worked up enough karma that I am trying to help you here

An interesting option is to abort the batch from within the batch anytime there is a problem executing the batch. That is a simple localized change that actually fixes the test failure in this new BatchingBatchFailureTest test. I don't think there are any negative ramifications of that change. Running the test suite to verify.