Re: [infinispan-dev] Improving the performance of index writers
On Mon, Oct 20, 2014 at 1:10 PM, Sanne Grinovero <sanne(a)infinispan.org> wrote:
On 20 October 2014 12:59, Emmanuel Bernard
<emmanuel(a)hibernate.org> wrote:
> HSEARCH-1699 looks good. A few comments.
>
> Maybe from a user point of you we want to expose the number of ms the user is ok to
delay a commit due to indexing. Which would mean that you can wait up to that number
before calling it a day and emptying the queue. The big question I have which you elude
too is whether this mechanism should have some kind of back pressure mechanism by also
caping the queue size.
Gustavo is implementing that for the ASYNC backend, but the SYNC
backend will always block the user thread until the commit is done
(and some commit is going to be done ASAP).
About to write a mail to hibernate-dev to discuss the ASYNC backend
property name and exact semantics.
Current ASYNC proposal [1] involves a refresh interval to explicitly
flush the indexes. There's still a queue involved though; the queue is
filled with indexing work to be applied and will block if flooded with
multiple producers.
The difference resides in the consumption side: instead of the flow
{apply, commit, apply, commit} it will do {apply(1..*) - commit -
apply(1..*) - commit}
[1] https://github.com/hibernate/hibernate-search/pull/681
Gustavo