Hi Radoslav Petrov,
first I noticed you tests are not isolated, you forget to clean up the database after each test, so the order of execution matter, having the 2 tests updateNoTxWithFlush and createNoTransactionWithFlush influencing the results of the others.

If you clean up the db after each test execution you will notice that only the 2 tests where the flush method is called manually will pass.

I also tried your tests with 5.1.11 and 5.0.12 and I have the same behaviour of 5.2.12.

Anyway apart suggesting you to always use transactions, in case you don't want you always need to flush manually in order to be sure to have the data persisted.