[hibernate-dev] [Search] Dynamic sharding configuration

[Gunnar Morling]

2013/9/24 Sanne Grinovero <sanne at hibernate.org>

> Correct me if I'm wrong, but trying to synthesize this discussion I
> think that we're fundamentally agreeing that dynamic sharding is a
> "better replacement" for static sharding.


Yes, from what I understand I think that's right.

To me, the question really is what actually is the difference between
"dynamic" and "static" sharding? Isn't both just two variants of the same
concept? In both cases we examine a document (or entity) and determine the
right shard at runtime.

For one implementation of this mechanism we just happen to have an
additional configuration which is the number of shards for hash based
sharding. So in one case we know the number of shards up-front, while we
don't know in other cases. But is this really a conceptual difference which
should reflect in different SPI interfaces?

Which part of the "new" dynamic sharding (i.e. where we don't know the
number of shards up-front) can't be addressed by implementing the
existing IndexShardingStrategy
interface? If the existing getIndexManagerXy() methods are fine and it is
only about additional information to be passed at initialization time, we
could also do the following:

* inject this information into marked constructors or fields at
instantiation time (this would be a new paradigm in Search, though)
* provide an additional contract only for this initialization and support
IndexShardStrategy implementations which also implement this initialization
contract:

    public interface ShardingStrategyInitializable {
        void initialize(WhatEverNeeded initializationContext);
    }

    public class MyDynamicShardStrategy implements IndexShardingStrategy,
ShardingStrategyInitializable { ... }

In both cases a "dynamic" sharding strategy implementation would be
essentially the same as any other strategy, it would only retrieve
additional information during initialization. This seems more consistent
and easier to grok to me than handling "dynamic" sharding in a different
manner.

Such an approach would allow to add support for the feature in 4.4 without
breaking stuff and give us the chance to create a new contract in 5 if
required, e.g. (optionally) based on the entity, with less methods etc.

Btw. I think this case is a good example showing that it is a good idea to
provide abstract base classes for SPI contracts, allowing to add new
methods in later versions without breaking existing implementations (at
least until we can use default methods from Java 8).

--Gunnar