[infinispan-issues] [JBoss JIRA] (ISPN-5452) Query Execution using Hibernate Search slow for large volume data

[Sanne Grinovero (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-5452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13075279#comment-13075279 ] 

Sanne Grinovero commented on ISPN-5452:
---------------------------------------

Hi Prashant,
there are two aspects at play: latency (response time) for a single query, and how many queries you can get done per unit of time (often measures as queries / second).
The response time of a single query is expected to be longer for a remote query (of course! since it goes over your network) but the execution strategy is identical among all three approaches. The amount of queries per second handled by a single server is approximately the same for each of the three as well: Remote Query adds a little overhead toll as it's processing remote calls, but that being only a minor aspect it should only provide a small deviation.

In other words, even though the latency would be higher, I don't expect significant changes in the amount of queries throughput the system can handle. RMI is not as efficient as Hot Rod so that makes your tests even more surprising.

I can't comment on the 50-100 us: that seems a reasonable figure but it highly depends on the index size, the kind of queries, the kind of hardware and general OS tuning. What is important to take away is that wathever figure you can get on remote query, you should be able to better that when using embedded query.

{quote}... then we are sure its an application issue{quote}

Don't forget it might be a problem of tests design. I wouldn't be surprised if some of your clients get in locking problems because of "too good" performance from the server. Similarly you could have some race in data collection on the testing clients. 


> Query Execution using Hibernate Search slow for large volume data
> -----------------------------------------------------------------
>
>                 Key: ISPN-5452
>                 URL: https://issues.jboss.org/browse/ISPN-5452
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Configuration, Remote Querying
>    Affects Versions: 7.2.1.Final
>         Environment: Linux
>            Reporter: Prashant Thakur
>
> While benchmarking Infinispan we found that Querying is very slow when compared with Hibernate Search in Isolation
> Single node of Infinispan  
> Memory allocated 230GB. No GC seen throughout query operation.
> Total required after full GC was 122GB.
> Setup 240 million records each of avg size 330 bytes .
> System has 16 cores and 40 worker threads were allocated at server side.
> With Single Client thread throughput was 900 req/sec in remote and 3k per sec in embedded more same request with Hibernate Search in Isolation gives throughput of 14000 req/sec.
> For 50 threads of clients the throughput was limited to 15k req/sec while hibernate search gives 80k req/sec for 10 threads.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)