3:48 p.m.
On Jun 10, 2009, at 15:22, Chase Seibert wrote:
>That sounds like an elegant approach but we need a way to make it
easy to declare
>a filter as dump/shard-sensitive only (ie not force the user to
write some Filter
>implementation). With this knowledge, HSearch could ignore the dump
filter for
>the actual Lucene filtering operation.
I actually think it's rather elegant, because it's backwards
compatible. The filter will cause the correct data to return whether
it's executed before or after hitting the indexes. Ie, it returns
the correct results pre-patch.
To avoid the extra over-head, would it be safe to assume that if the
IndexShardingStrategy returned a subset (non-zero, non-all) set of
DirectoryProviders for
getDirectoryProvidersForQuery(FullTextFilter[] filters), that it's
safe to ignore that filter post-search? Effectively, we can delete
that filter from the array in FullTextQueryImpl.
No, that would not be safe.
I'd rather get a special marker (flag or class) to specify that a
filter should be ignored from the list post search.
7:12 p.m.
New subject: HSearch: Using sharding and avoiding query on multiple shards
you're right we need some way to know which filters don't need to be
applied on the "lowlevel" IndexSearcher,
but adding a flag breaks backwards compatibility, and using a type
will get us to use ugly "instanceof"; they're not
bad solutions but what do you think of using a new option of the
@FullTextFilterDef ?
As an annotation it can have a new flag defaulting to the "apply to
index" behaviour; this flag will be read by Search
internals and we can keep track of the options without affecting
public interfaces.
My point is not that writing a "return true" is much work, but that
most filters will not use this feature, as you can define
at most one IndexShardingStrategy per entity.
We could also consider that such a filter implementation is tightly
coupled to the ShardingStrategy itself; it could make
sense to have the filter+shardingStrategy implementations to be merged
in one class, and have a keyword ( "shardsFilter" ?)
to enable THE one sharding-based filter.
This would reduce considerably the amount of code to be written to use
this feature; the filter parameters would be delegated directly to the
shardingStrategy,
which doesn't need to find out which filters it's interested in. There
would be no dump filter implementation at all, it just looks like you
use
a filter from an API point of view but you're actually configuring the
sharding strategy to return what you need.
Sanne
2009/6/10 Emmanuel Bernard <emmanuel(a)hibernate.org>:
On Jun 10, 2009, at 15:22, Chase Seibert wrote:
> >That sounds like an elegant approach but we need a way to make it easy to
> > declare
> >a filter as dump/shard-sensitive only (ie not force the user to write
> > some Filter
> >implementation). With this knowledge, HSearch could ignore the dump
> > filter for
> >the actual Lucene filtering operation.
>
> I actually think it's rather elegant, because it's backwards compatible.
> The filter will cause the correct data to return whether it's executed
> before or after hitting the indexes. Ie, it returns the correct results
> pre-patch.
>
> To avoid the extra over-head, would it be safe to assume that if the
> IndexShardingStrategy returned a subset (non-zero, non-all) set of
> DirectoryProviders for getDirectoryProvidersForQuery(FullTextFilter[]
> filters), that it's safe to ignore that filter post-search? Effectively, we
> can delete that filter from the array in FullTextQueryImpl.
No, that would not be safe.
I'd rather get a special marker (flag or class) to specify that a filter
should be ignored from the list post search.
8:53 p.m.
New subject: HSearch: Using sharding and avoiding query on multiple shards
On Jun 10, 2009, at 19:12, Sanne Grinovero wrote:
you're right we need some way to know which filters don't
need to be
applied on the "lowlevel" IndexSearcher,
but adding a flag breaks backwards compatibility, and using a type
We're breaking backward compatibility by changing the
IndexShardingStrategy contract.
will get us to use ugly "instanceof"; they're not
bad solutions but what do you think of using a new option of the
@FullTextFilterDef ?
Right thats' what I meant by flag. I am on the fence. In the initial
implementation I've committed, I've used a ShardSensitiveOnlyFilter
interface
@FullTextFilterDef(name="customer", impl=ShardSensitiveOnlyFilter.class)
as opposed to
@FullTextFilterDef(name="customer", impl=void.class,
shardSensitiveOnly=true)
or
@FullTextFilterDef(name="customer", shardSensitiveOnly=true)
if we start to default impl (which can lead to other user errors
unfortunately)
Each solution has its set of ugliness, but I am open to suggesting
arguments. Either solution would work for me
We could also consider that such a filter implementation is tightly
coupled to the ShardingStrategy itself; it could make
sense to have the filter+shardingStrategy implementations to be merged
in one class, and have a keyword ( "shardsFilter" ?)
to enable THE one sharding-based filter.
not sure what you mean. We can certainly reduce the amount of
implementation work in
IndexShardingStrategy#getDirectoryProvidersForQuery (using annotations
to declare which filter we are itnerested in etc etc) but I'd rather
know how people use this feature first before adding some convention
over configuration.
This would reduce considerably the amount of code to be written to
use
this feature; the filter parameters would be delegated directly to the
shardingStrategy,
which doesn't need to find out which filters it's interested in. There
would be no dump filter implementation at all, it just looks like you
use
a filter from an API point of view but you're actually configuring the
sharding strategy to return what you need.
Not sure how that eliminates the dump filter. The flag approach does,
not the coupling between Sharding and Filter.
5431
days inactive
5432
days old
2 comments
2 participants
participants (2)
-
Emmanuel Bernard -
Sanne Grinovero