Yoann Rodière ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... ) *created* an issue Hibernate Search ( https://hibernate.atlassian.net/browse/HSEARCH?atlOrigin=eyJpIjoiNGJjMDhk... ) / Improvement ( https://hibernate.atlassian.net/browse/HSEARCH-4736?atlOrigin=eyJpIjoiNGJ... ) HSEARCH-4736 ( https://hibernate.atlassian.net/browse/HSEARCH-4736?atlOrigin=eyJpIjoiNGJ... ) OperationSubmitter to "retry from another thread" ( https://hibernate.atlassian.net/browse/HSEARCH-4736?atlOrigin=eyJpIjoiNGJ... ) Issue Type: Improvement Assignee: Unassigned Components: engine, mapper-pojo-base Created: 31/Oct/2022 01:51 AM Fix Versions: 6.2-backlog Priority: Major Reporter: Yoann Rodière ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... ) https://hibernate.atlassian.net/browse/HSEARCH-4700 ( https://hibernate.atlassian.net/browse/HSEARCH-4700 ) introduced customization for what should happen when submitting an operation to a background processor and the queue of operation is full, giving the choice between: * Blocking until the operation can be added to the queue * Throwing RejectedExecutionException We could introduce a third option, matching what Infinispan does at the moment: offload the submitting of the operation to another another theadpool that is allowed to block. That’s a middle-ground between blocking and failing: we won’t fail, but we won’t block in the current thread either. Might be useful for operations that are not time-critical and not expected to be executed in high volume, such as getting the size of the index. E.g. we’d change the OperationSubmitter API and also add something like this: static OperationSubmitter offload(Consumer< Runnable > executor) { return new OperationSubmitter() { @Override public <T> void submitToQueue(BlockingQueue<T> queue, T element, Function<T, Runnable > blockingRetryProducer) throws InterruptedException { if ( !queue.offer( element ) ) { executor.accept( blockingRetryProducer.apply( element ) ); } } }; } Users (Infinispan) would do something like this: OperationSubmitter offloadSubmitter = OperationSubmitter.offload(myExecutor::submit); While our batching executor would do something like this: operationSubmitter.submitToQueue(workQueue, work, blockingRetry); We’d have to be careful how we implement the blockingRetryProducer , however: it should not just re-submit the element to the queue, but should also re-acquire all relevant locks. In AbstractWorkOrchestrator for example, it would need to re-acquire the lifecycleLock. Basically blockingRetryProducer = work -> orchestrator.submit(work, OperationSubmitter.BLOCKING) as shown above. That means blockingRetryProducer would need to be a parameter of BatchingExecutor#submit , whose value is set in the (calling) orchestrator. ( https://hibernate.atlassian.net/browse/HSEARCH-4736#add-comment?atlOrigin... ) Add Comment ( https://hibernate.atlassian.net/browse/HSEARCH-4736#add-comment?atlOrigin... ) Get Jira notifications on your phone! Download the Jira Cloud app for Android ( https://play.google.com/store/apps/details?id=com.atlassian.android.jira.... ) or iOS ( https://itunes.apple.com/app/apple-store/id1006972087?pt=696495&ct=Em... ) This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100209- sha1:5956172 )