[hibernate-issues] [JIRA] (HSEARCH-4700) Configurable behavior on full queues for indexing operations

Yoann Rodière ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... ) *created* an issue Hibernate Search ( https://hibernate.atlassian.net/browse/HSEARCH?atlOrigin=eyJpIjoiM2JmMGY3... ) / Improvement ( https://hibernate.atlassian.net/browse/HSEARCH-4700?atlOrigin=eyJpIjoiM2J... ) HSEARCH-4700 ( https://hibernate.atlassian.net/browse/HSEARCH-4700?atlOrigin=eyJpIjoiM2J... ) Configurable behavior on full queues for indexing operations ( https://hibernate.atlassian.net/browse/HSEARCH-4700?atlOrigin=eyJpIjoiM2J... ) Issue Type: Improvement Assignee: Unassigned Components: backend-elasticsearch, backend-lucene, engine, mapper-pojo-base Created: 09/Sep/2022 08:36 AM Fix Versions: 6.2-backlog Priority: Major Reporter: Yoann Rodière ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... ) See https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/topic/J... ( https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/topic/J... ) Essentially, users of methods such as: * org.hibernate.search.mapper.pojo.work.spi.PojoIndexingPlan#executeAndReport * org.hibernate.search.mapper.pojo.work.spi.PojoIndexer#add * org.hibernate.search.mapper.pojo.work.spi.PojoIndexer#addOrUpdate * … * org.hibernate.search.mapper.pojo.work.spi.PojoIndexer#delete(org.hibernate.search.mapper.pojo.model.spi.PojoRawTypeIdentifier<?>, java.lang.Object, org.hibernate.search.mapper.pojo.route.DocumentRoutesDescriptor, org.hibernate.search.engine.backend.work.execution.DocumentCommitStrategy, org.hibernate.search.engine.backend.work.execution.DocumentRefreshStrategy) * org.hibernate.search.mapper.pojo.work.spi.PojoScopeWorkspace#mergeSegments * … * org.hibernate.search.mapper.pojo.work.spi.PojoScopeWorkspace#refresh … need to be able to specify the behavior they want to enforce back-pressure: * Block the thread: that’s the current behavior, and it’s fine when calling these methods from blocking processes such as the mass indexer or org.hibernate.search.mapper.orm.work.impl.SearchWorkspaceImpl#mergeSegments. But it's a big no-no when calling from non-blocking processes such as Infinispan. * Fail: that’s the preferred behavior when calling these methods from non-blocking processes, e.g. in Infinispan, or in org.hibernate.search.mapper.orm.work.impl.SearchWorkspaceImpl#mergeSegmentsAsync. I think we should add a parameter to those methods to control that behavior, something like a OperationSubmitter. We would propagate that parameter all the way through to the AbstractWorkOrchestrator , which would take it into account as appropriate: submitter.submitToQueue(queue, element) , submitter.submitToExecutor(executor, task) , … We would provide two built-in implementations: BLOCK and REJECTED_EXECUTION_EXCEPTION. We would *not* allow custom implementations (at least for now); allowing things such as offloading to another thread, for example, could be very dangerous as submitting an operation sometimes needs to happen in a certain context (with certain locks held by the current thread). Also, we may need to add more methods to the OperationSubmitter interface and that would be a breaking change. We would *not* add a OperationSubmitter argument to methods that are clearly blocking. *Maybe we should also introduce a configuration property* (also SPI) so that Infinispan can set the default globally. Finally, maybe we could do the same for schema management operations: * org.hibernate.search.mapper.pojo.schema.management.spi.PojoScopeSchemaManager#createIfMissing * … * org.hibernate.search.mapper.pojo.schema.management.spi.PojoScopeSchemaManager#validate ( https://hibernate.atlassian.net/browse/HSEARCH-4700#add-comment?atlOrigin... ) Add Comment ( https://hibernate.atlassian.net/browse/HSEARCH-4700#add-comment?atlOrigin... ) Get Jira notifications on your phone! Download the Jira Cloud app for Android ( https://play.google.com/store/apps/details?id=com.atlassian.android.jira.... ) or iOS ( https://itunes.apple.com/app/apple-store/id1006972087?pt=696495&ct=Em... ) This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100206- sha1:d8a7a40 )