In MongoDB we have one subclass per test per config to test AssociationStorage.GLOBAL_COLLECTION / COLLECTION and IN_ENTITY.

With a Rule based approach, we could limit it to one subclass per customized test and add one method per configuration in the subclass. That would be better than today.

Some inputs:

• org.hibernate.ogm.test.utils.PackagingRule
• org.hibernate.search.test.configuration.IndexMetadataCompleteConfiguredTest
• org.hibernate.search.test.configuration.OptimizerStrategyLoadTest https://raw.github.com/hibernate/hibernate-search/master/hibernate-search-engine/src/test/java/org/hibernate/search/test/configuration/OptimizerStrategyLoadTest.java

Sanne:
In case of OGM, it would be nice to have an helper to easily start en EntityManager /SF instance
Something like the org.hibernate.ogm.test.utils.PackagingRule, but not necessarily a Rule.
Actually using a Rule would be aweseome: it cleans up for sure. Nothing stops you to have multiple instances of the same rule, so initializing all SessionFactories you might need at startup - if that suites.
Well like at the link I posted, you still have to define a method for each test. This method then contains "what's different" for each test.
Or you could use JUnit Theories to have it "mix" all different configuration properties.
The default JUnit runner requires at least a different method per test. If that's not enough, a custom Runner would be needed.
And I just made a custom runner to have more tests per method