We should try avoiding initialize NaturalIdXrefDelegate if possible, or at least defer the CHM it uses internally.

Also: why does it use a concurrent map?