It looks like extra parentheses are added to the select columns corresponding to an “collection IS NOT EMPTY” check in JPQL, where the collection is a @OneToMany with mappedBy. Mysql doesn’t like these added parens. The query originally in 5.4.10 came out as

select * from a where not (exists (
select collection.a_id, collection.b_id from collection where a.id = collection.a_id));

But in 5.4.11 it resolves to

select * from a where not (exists (
select (collection.a_id, collection.b_id) from collection where a.id = collection.a_id));