[infinispan-issues] [JBoss JIRA] (ISPN-5452) Query Execution using Hibernate Search slow for large volume data

[Adrian Nistor (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-5452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073823#comment-13073823 ] 

Adrian Nistor commented on ISPN-5452:
-------------------------------------

I've tried measuring and comparing remote vs embedded, see RemoteQueryDslPerfTest in this PR:  https://github.com/infinispan/infinispan/pull/3520

I found remote mode to be 2x slower than embedded, which perfectly fits my expectations. A query that matches exactly one entry takes about 600 microseconds in remote mode and 300 us in embedded mode. But this scenario runs completely in memory, on one machine and the volume of data is very small. It was not meant to measure absolute performance but it does prove the relative difference between remote and embedded is not absurdly big.

So unless we identify why we have such a big difference in your scenario there is not much we can do.

> Query Execution using Hibernate Search slow for large volume data
> -----------------------------------------------------------------
>
>                 Key: ISPN-5452
>                 URL: https://issues.jboss.org/browse/ISPN-5452
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Configuration, Remote Querying
>    Affects Versions: 7.2.1.Final
>         Environment: Linux
>            Reporter: Prashant Thakur
>
> While benchmarking Infinispan we found that Querying is very slow when compared with Hibernate Search in Isolation
> Single node of Infinispan  
> Memory allocated 230GB. No GC seen throughout query operation.
> Total required after full GC was 122GB.
> Setup 240 million records each of avg size 330 bytes .
> System has 16 cores and 40 worker threads were allocated at server side.
> With Single Client thread throughput was 900 req/sec in remote and 3k per sec in embedded more same request with Hibernate Search in Isolation gives throughput of 14000 req/sec.
> For 50 threads of clients the throughput was limited to 15k req/sec while hibernate search gives 80k req/sec for 10 threads.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)