[hibernate-commits] [hibernate/hibernate-search] 008e1c: HSEARCH-1317 Optimize tail recursion in NRT IndexR...

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/hibernate/hibernate-search
  Commit: 008e1cba6dbe9c6580fb2d959537b4d3f94f9373
      https://github.com/hibernate/hibernate-search/commit/008e1cba6dbe9c6580fb2d959537b4d3f94f9373
  Author: Sanne Grinovero <sanne at hibernate.org>
  Date:   2013-05-21 (Tue, 21 May 2013)

  Changed paths:
    M engine/src/main/java/org/hibernate/search/backend/impl/lucene/NRTWorkspaceImpl.java

  Log Message:
  -----------
  HSEARCH-1317 Optimize tail recursion in NRT IndexReader reopening


  Commit: fb295ac00911a625d7c0f0ae5384beeafab83cec
      https://github.com/hibernate/hibernate-search/commit/fb295ac00911a625d7c0f0ae5384beeafab83cec
  Author: Sanne Grinovero <sanne at hibernate.org>
  Date:   2013-05-22 (Wed, 22 May 2013)

  Changed paths:
    M engine/src/main/java/org/hibernate/search/backend/impl/lucene/NRTWorkspaceImpl.java

  Log Message:
  -----------
  HSEARCH-1317 Boosting performance: removing contention between writers and readers

Visibility of changes are guaranteed by the volatile flush mentioned in the code comments; now on to why the writes on these different state flags and counters is ok to not be fully atomic.
Initially I considered moving all state in a little valueobject to be replaced atomically, but this:
 - is more complex to replace than a simple CAS operation as there are counter increments involved
 - this level of consistency is not needed

Considering there is commonly [1] just one thread invoking this, this is basically a CAS operation with a value object.

Why is this level of consistency not needed?
* On counters: they grow monotonically and all what the readers care is to not see a value which flags their state as "outdated"
* On in-flight flags: worst case some state (either deletes or writes) is flushed which wasn't strictly required to be flushed yet - but still this is welcome as this state would need to be flushed very soon: it will be a requirement at the next refresh operation, so in practice if we're ready to include those changes as well in the same IO operation now, it's better to do it all in one go as the worst case scenario (the race condition) is beneficial for performance as it allows us to skip some IO.

Note that this is tested by the existing test org.hibernate.search.test.performance.nrt.ReadWriteParallelismTest: it effectively isolates this component to put a lot of contention on this structure, and does validation of correct state being visible.

[1] Only whan a MassIndexer is running we might have parallel invocations of the setupNewReadersRequirements() method.


  Commit: 229e7eb05be5f87a08aba2d1d607f6f05f55bf45
      https://github.com/hibernate/hibernate-search/commit/229e7eb05be5f87a08aba2d1d607f6f05f55bf45
  Author: Sanne Grinovero <sanne at hibernate.org>
  Date:   2013-05-22 (Wed, 22 May 2013)

  Changed paths:
    M engine/src/main/java/org/hibernate/search/backend/impl/lucene/NRTWorkspaceImpl.java

  Log Message:
  -----------
  HSEARCH-1307 Resolve race condition on closing IndexReader still being used for queries


  Commit: 95212c6598aef70f08c4aeed0f3315cbfbc388d3
      https://github.com/hibernate/hibernate-search/commit/95212c6598aef70f08c4aeed0f3315cbfbc388d3
  Author: Hardy Ferentschik <hibernate at ferentschik.de>
  Date:   2013-05-22 (Wed, 22 May 2013)

  Changed paths:
    M engine/src/main/java/org/hibernate/search/backend/impl/lucene/NRTWorkspaceImpl.java

  Log Message:
  -----------
  HSEARCH-1317 Javadoc and comments update/clarification


Compare: https://github.com/hibernate/hibernate-search/compare/f56df3e986ca...95212c6598ae