In Search 5, one of the few cases where we tried to report failures spanning multiple indexes at bootstrap, and did not simply stop after the first, was schema validation. We took care to execute the validation for all indexes, aggregate the errors, and only throw the exception to the user once every index was started.

In Search 6, the indexes are started by mappers, implicitly, when mappers call something like indexManagerBuildingStateHolder.build(). If they fail to start, build() ultimately throws an exception and the mapper is unable to proceed, since it didn't get a reference to the index manager.
As a result, we stop after the very first failing index, and don't validate the others.

To get back to the Search 5 behavior, we should do things differently: have the build() method return a not-yet-started index manager, and start all the index managers later, in the engine, by calling a start() method on the index manager.

This has many advantages:

• we could integrate Elasticsearch schema validation error reporting into the Search 6 "failure collector" (org.hibernate.search.engine.logging.spi.FailureCollector), to display all failures consistently (note that currently the failure collector only accepts exceptions, but we could easily modify it to accept strings representing a failure)
• we could potentially start all the indexes in parallel, by making the start() method asynchronous; see HSEARCH-3084 Open
• we could add settings to postpone the starting of indexes, or start them lazily, as suggested in some other ticket (I can't find it right now, but I think someone from Infinispan opened it). This would allow in particular to start thread pools lazily, which can be very useful in Infinispan.

But on the other hand, that also means we should add some state checking in the index manager methods, to prevent anyone from trying to use a stopped index manager.