In option #2 (just @UnwrapValidationValue) you still need to provide for example a JavaFXValueUnwrapper. In this case you would register it as an option to the bootstrap, but you are adding a new public type.
...
#2 or #3 you are exposing the same amount of new types

It depends on who bootstraps the validator factory. If my library is doing this, I would pass the unwrapper to the bootstrap routine as you say, but I wouldn't have to expose the unwrapper type to my users. It would be transparent to them, i.e. it wouldn't have to be public.

Not quite, since the Validator engine cannot know how unwrapping has to occur, the user also needs to tell us how this is happening.

We might have different interpretations of what "user" means here. To me, this is an application developer using Forge in this case, i.e. the one who works with @Size(min=3) UIInput<String> name. Once the unwrapper is registered (which I assume could be done by the library and thus be transparent to such user) , the engine can select the right unwrapper by the type of the validated element, so there is no need for the user to specify which unwrapper should be used.

So my thinking is that whether to unwrap or not is controlled by such user (via @UnwrapValidatedValue) and the how is controlled by the integrating library. Also if the user is setting up the validator and thus needs to know the unwrapper type, I think it's better to have this information in one place (at bootstrap) instead of repeating this information with each @UnwrapValidatedValue annotation. In a way it's similar to CDI interceptors; There you also specify only the interceptor binding annotations, but which interceptor class is invoked for such a binding is transparent at the call site.