[infinispan-dev] Improving the performance of index writers

[Gustavo Fernandes]

On Mon, Oct 20, 2014 at 1:10 PM, Sanne Grinovero <sanne at infinispan.org> wrote:
> On 20 October 2014 12:59, Emmanuel Bernard <emmanuel at hibernate.org> wrote:
>> HSEARCH-1699 looks good. A few comments.
>>
>> Maybe from a user point of you we want to expose the number of ms the user is ok to delay a commit due to indexing. Which would mean that you can wait up to that number before calling it a day and emptying the queue. The big question I have which you elude too is whether this mechanism should have some kind of back pressure mechanism by also caping the queue size.
>
> Gustavo is implementing that for the ASYNC backend, but the SYNC
> backend will always block the user thread until the commit is done
> (and some commit is going to be done ASAP).
> About to write a mail to hibernate-dev to discuss the ASYNC backend
> property name and exact semantics.
>

Current ASYNC proposal [1] involves a refresh interval to explicitly
flush the indexes. There's still a queue involved though; the queue is
filled with indexing work to be applied and will block if flooded with
multiple producers.
The difference resides in the consumption side: instead of the flow
{apply, commit, apply, commit} it will do {apply(1..*) - commit -
apply(1..*) - commit}


[1] https://github.com/hibernate/hibernate-search/pull/681


Gustavo