This is tricky given how annotation binding is done overall. I have reached out to Emmanuel Bernard for help.
Emmanuel Bernard if you can pick this up great. If not, I at least need guidance on how to best hook this into your annotation binder code.