[infinispan-issues] [JBoss JIRA] (ISPN-5452) Query Execution using Hibernate Search slow for large volume data

[Prashant Thakur (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-5452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086716#comment-13086716 ] 

Prashant Thakur edited comment on ISPN-5452 at 7/6/15 7:34 AM:
---------------------------------------------------------------

Please find attached the profiling results of two methods i.e. Infinispan and Hibernate Search direct.
One observation we have is Instance of Index Reader is instantiated for each query operation within Infinispan whereas as recommended by Hibernate Search doc it should not be the case.
The difference at present is 1500 req/sec and 4000 req/sec. Need to understand why in Infinispan we want to Instantiate each time Index Reader.

2.	ON Comparision of HS Search and Infinispan code -> The biggest difference is amount of time spent in BuildSearcher and Close Searcher .These 2 options are negligible in hibernate search.Maybe there is a mechanism to reuse Searcher


was (Author: prashant.thakur):
Please find attached the profiling results of two methods i.e. Infinispan and Hibernate Search direct.
One observation we have is Instance of Index Reader is instantiated for each query operation withing Infinispan whereas as recommended by Hibernate Search doc it should not be the case.
The difference at present is 1500 req/sec and 4000 req/sec. Need to understand why in Infinispan we want to Instantiate each time Index Reader.

2.	ON Comparision of HS Search and Infinispan code -> The biggest difference is amount of time spent in BuildSearcher and Close Searcher .These 2 options are negligible in hibernate search.Maybe there is a mechanism to reuse Searcher

> Query Execution using Hibernate Search slow for large volume data
> -----------------------------------------------------------------
>
>                 Key: ISPN-5452
>                 URL: https://issues.jboss.org/browse/ISPN-5452
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Configuration, Remote Querying
>    Affects Versions: 7.2.1.Final
>         Environment: Linux
>            Reporter: Prashant Thakur
>         Attachments: profiling_results.7z
>
>
> While benchmarking Infinispan we found that Querying is very slow when compared with Hibernate Search in Isolation
> Single node of Infinispan  
> Memory allocated 230GB. No GC seen throughout query operation.
> Total required after full GC was 122GB.
> Setup 240 million records each of avg size 330 bytes .
> System has 16 cores and 40 worker threads were allocated at server side.
> With Single Client thread throughput was 900 req/sec in remote and 3k per sec in embedded more same request with Hibernate Search in Isolation gives throughput of 14000 req/sec.
> For 50 threads of clients the throughput was limited to 15k req/sec while hibernate search gives 80k req/sec for 10 threads.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)