Lucene has support to pass in an ExecutorService to the IndexReader in order to do searches in parallel across segments.

Currently when sharding is used, query performance is O(n) where n is the number of IndexManagers involved.

The caveat is that the ExecutorService only is used to certain methods from IndexReader, and also it is not suitable to all cases: it will trade latency for throughput, so ideally this should be configurable.