Neither cascade.persist nor cascade.all was working for me. Not sure what is different with my set up.

What is the difference with orphan removal with cascade persist, versus orphan removal without cascade persist? I guess orphan removal would only do deletes versus cascade persist would also deal with inserts and updates?

Ended up doing the delete manually as you suggested with a named hql query, which was not bad actually.