[hibernate-commits] [hibernate/hibernate-search] 1873fa: HSEARCH-1929 Aggregations on multi-valued fields

Branch: refs/heads/master Home: https://github.com/hibernate/hibernate-search Commit: 1873fa6a19d6b25092f6db54aaeb83eab9ebeffc https://github.com/hibernate/hibernate-search/commit/1873fa6a19d6b25092f6... Author: Waldemar Kłaczyński <wklaczynski(a)sabaservice.pl&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/search/aggregation/impl/AbstractElasticsearchAggregation.java A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/DoubleMultiValues.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/DoubleMultiValuesToSingleValuesSource.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/GeoPointDistanceMultiValuesToSingleValuesSource.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/LongMultiValuesToSingleValuesSource.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/MultiValueMode.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/NumericDoubleValues.java A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/NumericLongValues.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/SortedNumericDoubleDocValues.java A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/DoubleMultiValueRangeFacetCounts.java A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueFacetCounts.java A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueRangeCounter.java A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueRangeFacetCounts.java A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/MultiValueRangeFacetCounts.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/search/aggregation/impl/AggregationRequestContext.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/search/query/impl/LuceneSearchQueryBuilder.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/aggregation/impl/AbstractLuceneBucketAggregation.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/aggregation/impl/LuceneNumericRangeAggregation.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/aggregation/impl/LuceneNumericTermsAggregation.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneDoubleDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneFloatDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneIntegerDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneLongDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneNumericDomain.java M engine/src/main/java/org/hibernate/search/engine/search/aggregation/dsl/RangeAggregationOptionsStep.java M engine/src/main/java/org/hibernate/search/engine/search/aggregation/dsl/TermsAggregationOptionsStep.java M engine/src/main/java/org/hibernate/search/engine/search/aggregation/dsl/impl/RangeAggregationRangeStepImpl.java M engine/src/main/java/org/hibernate/search/engine/search/aggregation/dsl/impl/TermsAggregationOptionsStepImpl.java M engine/src/main/java/org/hibernate/search/engine/search/aggregation/spi/SearchAggregationBuilder.java A engine/src/main/java/org/hibernate/search/engine/search/common/MultiValue.java M util/internal/integrationtest/common/src/main/java/org/hibernate/search/util/impl/integrationtest/common/stub/backend/search/aggregation/impl/StubRangeAggregationBuilder.java M util/internal/integrationtest/common/src/main/java/org/hibernate/search/util/impl/integrationtest/common/stub/backend/search/aggregation/impl/StubTermsAggregationBuilder.java Log Message: ----------- HSEARCH-1929 Aggregations on multi-valued fields Commit: 08f7390ee7d9fb782ff4ccb59b1d13f83c3fbe24 https://github.com/hibernate/hibernate-search/commit/08f7390ee7d9fb782ff4... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/DoubleMultiValuesToSingleValuesSource.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/LongMultiValuesToSingleValuesSource.java Log Message: ----------- HSEARCH-1929 Remove an unnecessary hack (again) This relied on scores to pass the replacement for missing values, but we just don't need that, since missing values are handled correctly in the FieldComparators. See also 95777b95fd10b3bd9037e5f2a8114ea46e8142ab Commit: 91e67697f657779c6b84baeb8422bd5455698a40 https://github.com/hibernate/hibernate-search/commit/91e67697f657779c6b84... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M backend/elasticsearch/src/main/java/org/hibernate/search/backend/elasticsearch/search/aggregation/impl/AbstractElasticsearchAggregation.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/aggregation/impl/AbstractLuceneBucketAggregation.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/aggregation/impl/LuceneNumericRangeAggregation.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/aggregation/impl/LuceneNumericTermsAggregation.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneDoubleDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneFloatDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneIntegerDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneLongDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneNumericDomain.java M engine/src/main/java/org/hibernate/search/engine/search/aggregation/dsl/RangeAggregationOptionsStep.java M engine/src/main/java/org/hibernate/search/engine/search/aggregation/dsl/TermsAggregationOptionsStep.java M engine/src/main/java/org/hibernate/search/engine/search/aggregation/dsl/impl/RangeAggregationRangeStepImpl.java M engine/src/main/java/org/hibernate/search/engine/search/aggregation/dsl/impl/TermsAggregationOptionsStepImpl.java M engine/src/main/java/org/hibernate/search/engine/search/aggregation/spi/SearchAggregationBuilder.java R engine/src/main/java/org/hibernate/search/engine/search/common/MultiValue.java M util/internal/integrationtest/common/src/main/java/org/hibernate/search/util/impl/integrationtest/common/stub/backend/search/aggregation/impl/StubRangeAggregationBuilder.java M util/internal/integrationtest/common/src/main/java/org/hibernate/search/util/impl/integrationtest/common/stub/backend/search/aggregation/impl/StubTermsAggregationBuilder.java Log Message: ----------- HSEARCH-1929 Do not expose the ability to set a "multi-value mode" for aggregations 1. This is not the purpose of this ticket, so this should be done as a second step. 2. We know that we will not be able to implement this for Elasticsearch, so we cannot expose this in common APIs, but should expose it in Lucene-specific APIs instead. Commit: b04fd55a7923c8e2b2a7218f6bee47e0d3be6b84 https://github.com/hibernate/hibernate-search/commit/b04fd55a7923c8e2b2a7... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/DoubleMultiValueRangeFacetCounts.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueFacetCounts.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueRangeCounter.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueRangeFacetCounts.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/MultiValueRangeFacetCounts.java Log Message: ----------- HSEARCH-1929 Add missing credits to javadoc Commit: 31b575ef1523c6af9a039c1aaa81162d1169ffe4 https://github.com/hibernate/hibernate-search/commit/31b575ef1523c6af9a03... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: R backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/DoubleMultiValues.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/DoubleMultiValueRangeFacetCounts.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueFacetCounts.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueRangeFacetCounts.java Log Message: ----------- HSEARCH-1929 Remove some dead code from faceting implementations Commit: 8bed15fd4a617383fc5ca80390d94031607b8c31 https://github.com/hibernate/hibernate-search/commit/8bed15fd4a617383fc5c... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/join/impl/JoinChildrenIdIterator.java Log Message: ----------- HSEARCH-1929 Introduce JoinChildrenIdIterator, for easy implementation of doc values retrieval in nested documents Commit: c9af32cbc35107ddfe1110358388e7683fe298d7 https://github.com/hibernate/hibernate-search/commit/c9af32cbc35107ddfe11... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/DoubleMultiValues.java A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/DoubleMultiValuesSource.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/DoubleMultiValuesToSingleValuesSource.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/GeoPointDistanceMultiValuesToSingleValuesSource.java A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/JoiningDoubleMultiValuesSource.java A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/JoiningLongMultiValuesSource.java A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/LongMultiValues.java A backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/LongMultiValuesSource.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/LongMultiValuesToSingleValuesSource.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/MultiValueMode.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/NumericDoubleValues.java R backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/NumericLongValues.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/DoubleMultiValueRangeFacetCounts.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueFacetCounts.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueRangeFacetCounts.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/aggregation/impl/LuceneNumericTermsAggregation.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneDoubleDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneFloatDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneIntegerDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneLongDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneNumericDomain.java Log Message: ----------- HSEARCH-1929 Clearly separate multi-values accessors from single-values accessors in the Lucene backend 1. To avoid hacks where sub-classes of NumericDocValues can actually provide multiple values, in complete opposition to the purpose of the NumericDocValues interface. 2. To avoid instantiation of one lists per document followed by a sort when iterating over the doc values for aggregations. Aggregations can iterate over millions of documents, so this instantiation and sort can actually become a big deal performance-wise. Commit: 8957786bf788f1aa86c33495f266f124d0b0617f https://github.com/hibernate/hibernate-search/commit/8957786bf788f1aa86c3... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: R backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/DoubleMultiValues.java R backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/DoubleMultiValuesSource.java R backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/JoiningDoubleMultiValuesSource.java R backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/DoubleMultiValueRangeFacetCounts.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/FacetCountsUtils.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneDoubleDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneFloatDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneIntegerDomain.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/types/lowlevel/impl/LuceneLongDomain.java Log Message: ----------- HSEARCH-1929 Work on raw, long values for Lucene float/double aggregations It's less code, and we don't need more at the moment. We can always revert this commit if one day we need more. Commit: 4a0d774c7620a5a21108e880854c0e2b268f4cf0 https://github.com/hibernate/hibernate-search/commit/4a0d774c7620a5a21108... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueFacetCounts.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueRangeCounter.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueRangeFacetCounts.java Log Message: ----------- HSEARCH-1929 Fix Lucene aggregations: count each document only once per term/range This solution is probably not great performance-wise, but at least it will work. It'll have to do until someone has time to find something better. Commit: 35dcb5d5bbf5427837034222ed517143caecd401 https://github.com/hibernate/hibernate-search/commit/35dcb5d5bbf542783703... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M integrationtest/backend/tck/src/main/java/org/hibernate/search/integrationtest/backend/tck/search/aggregation/SingleFieldAggregationBaseIT.java Log Message: ----------- HSEARCH-1927 Mark the test checking aggregations on multi-valued numeric fields Commit: 9da8ef042b72c9daa2544fafff10ae64315fff4e https://github.com/hibernate/hibernate-search/commit/9da8ef042b72c9daa254... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M integrationtest/backend/tck/src/main/java/org/hibernate/search/integrationtest/backend/tck/search/aggregation/SingleFieldAggregationBaseIT.java Log Message: ----------- HSEARCH-1929 Mark the test checking indexing of multi-valued, aggregable numeric fields Commit: 92db06a52a6755d1299b9581329c9e2c102fa72f https://github.com/hibernate/hibernate-search/commit/92db06a52a6755d1299b... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M integrationtest/backend/lucene/src/test/java/org/hibernate/search/integrationtest/backend/lucene/testsupport/util/LuceneTckBackendFeatures.java M integrationtest/backend/tck/src/main/java/org/hibernate/search/integrationtest/backend/tck/search/aggregation/SingleFieldAggregationBaseIT.java M integrationtest/backend/tck/src/main/java/org/hibernate/search/integrationtest/backend/tck/testsupport/util/TckBackendFeatures.java Log Message: ----------- HSEARCH-1929 Re-enable tests for aggregations on multi-valued fields for Lucene Commit: 7a271e11cc35da42c754df4a52a6a639d066ed38 https://github.com/hibernate/hibernate-search/commit/7a271e11cc35da42c754... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/DoubleMultiValuesToSingleValuesSource.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/LongMultiValuesToSingleValuesSource.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/MultiValueMode.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/docvalues/impl/TextMultiValuesToSingleValuesSource.java R backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/join/impl/JoinFirstChildIdIterator.java Log Message: ----------- HSEARCH-1929 Replace JoinFirstChildIdIterator with JoinChildrenIdIterator It's simpler to use. Commit: 903309dc00b4e548a6e0f96e63367358448ac7bd https://github.com/hibernate/hibernate-search/commit/903309dc00b4e548a6e0... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueFacetCounts.java Log Message: ----------- HSEARCH-1929 Sonar: do not assign loop counters within the loop body Commit: e230ee3f7d873fbbb30b8771e982703220ebc042 https://github.com/hibernate/hibernate-search/commit/e230ee3f7d873fbbb30b... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueFacetCounts.java M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/MultiValueRangeFacetCounts.java Log Message: ----------- HSEARCH-1929 Sonar: Boolean literals should not be redundant Commit: 6a8348b6099f2c505eef8191ec5be65c79a81d97 https://github.com/hibernate/hibernate-search/commit/6a8348b6099f2c505eef... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/LongMultiValueRangeCounter.java Log Message: ----------- HSEARCH-1929 Sonar: Sections of code should not be commented out Commit: 9f216b4f8c6f934e935622fe38a8ab64f78d3f65 https://github.com/hibernate/hibernate-search/commit/9f216b4f8c6f934e9356... Author: Yoann Rodière <yoann(a)hibernate.org&gt; Date: 2020-04-15 (Wed, 15 Apr 2020) Changed paths: M backend/lucene/src/main/java/org/hibernate/search/backend/lucene/lowlevel/facet/impl/MultiValueRangeFacetCounts.java Log Message: ----------- HSEARCH-1929 Sonar: "throws" declarations should not be superfluous Compare: https://github.com/hibernate/hibernate-search/compare/03f6bd5d4468...9f21...