In this case yes. But there is another valid use case which got this problem - lets say you want to delete the 10 oldest (in whatever context) persons and write something like:

delete from person order by creationtime desc limit 10

than you would still need to have an order by clause, even if you don't have that referential integrity problem.