I realized I can achieve the reverse of what I want the following way:

public interface Relaxed {};

class A {
	@NotNull(groups={Relaxed.class, Default.class})
	public String a1;

	@NotNull
	public String a2;
}

which means that the constraint on a2 will only be enforced in the default case. This is the reverse of what I want because developers/maintainers need to remember to always put in the groups list and only if they later realize that want these constraints to be part of the special set that can be relaxed do they need remove the groups attribute from the annotation.