I just realized that another way of implementing this would be to add a singleton work representing refreshes, much as was done with org.hibernate.search.backend.FlushLuceneWork.INSTANCE.
It would definitely work with Elasticsearch; I'll have a look to other index managers.