[JBoss JIRA] (ISPN-5452) Query Execution using Hibernate Search slow for large volume data
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-5452?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero commented on ISPN-5452:
---------------------------------------
Hi Prashant,
there are two aspects at play: latency (response time) for a single query, and how many queries you can get done per unit of time (often measures as queries / second).
The response time of a single query is expected to be longer for a remote query (of course! since it goes over your network) but the execution strategy is identical among all three approaches. The amount of queries per second handled by a single server is approximately the same for each of the three as well: Remote Query adds a little overhead toll as it's processing remote calls, but that being only a minor aspect it should only provide a small deviation.
In other words, even though the latency would be higher, I don't expect significant changes in the amount of queries throughput the system can handle. RMI is not as efficient as Hot Rod so that makes your tests even more surprising.
I can't comment on the 50-100 us: that seems a reasonable figure but it highly depends on the index size, the kind of queries, the kind of hardware and general OS tuning. What is important to take away is that wathever figure you can get on remote query, you should be able to better that when using embedded query.
{quote}... then we are sure its an application issue{quote}
Don't forget it might be a problem of tests design. I wouldn't be surprised if some of your clients get in locking problems because of "too good" performance from the server. Similarly you could have some race in data collection on the testing clients.
> Query Execution using Hibernate Search slow for large volume data
> -----------------------------------------------------------------
>
> Key: ISPN-5452
> URL: https://issues.jboss.org/browse/ISPN-5452
> Project: Infinispan
> Issue Type: Bug
> Components: Configuration, Remote Querying
> Affects Versions: 7.2.1.Final
> Environment: Linux
> Reporter: Prashant Thakur
>
> While benchmarking Infinispan we found that Querying is very slow when compared with Hibernate Search in Isolation
> Single node of Infinispan
> Memory allocated 230GB. No GC seen throughout query operation.
> Total required after full GC was 122GB.
> Setup 240 million records each of avg size 330 bytes .
> System has 16 cores and 40 worker threads were allocated at server side.
> With Single Client thread throughput was 900 req/sec in remote and 3k per sec in embedded more same request with Hibernate Search in Isolation gives throughput of 14000 req/sec.
> For 50 threads of clients the throughput was limited to 15k req/sec while hibernate search gives 80k req/sec for 10 threads.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5527) Java system properties no longer supported in leveldb expiration path
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5527?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5527:
----------------------------------
Fix Version/s: 8.0.0.Alpha2
7.2.3.Final
8.0.0.Final
> Java system properties no longer supported in leveldb expiration path
> ---------------------------------------------------------------------
>
> Key: ISPN-5527
> URL: https://issues.jboss.org/browse/ISPN-5527
> Project: Infinispan
> Issue Type: Bug
> Components: Configuration
> Affects Versions: 7.2.0.Final
> Reporter: Paul Richardson
> Assignee: Tristan Tarrant
> Fix For: 8.0.0.Alpha2, 7.2.3.Final, 8.0.0.Final
>
>
> In 6.0, it was possible to create a leveldb cache configuration like:
> {code}
> <leveldbStore xmlns="urn:infinispan:config:store:leveldb:6.0"
> location="${user.home}/.komodo/db/data"
> expiredLocation="${user.home}/.komodo/db/expired"
> implementationType="JAVA">
> </leveldbStore>
> {code}
> Both the location and expiredLocation paths could contain java system properties and these would be substituted when the configuration was parsed. In 7.2, the same configuration successfully substitutes the path attribute but NOT the expiration path attribute:
> {code}
> <leveldb-store xmlns="urn:infinispan:config:store:leveldb:7.2"
> path="${user.home}/.komodo/db/data">
> <expiration path="${user.home}/.komodo/db/expired"/>
> <implementation type="JAVA"/>
> </leveldb-store>
> {code}
> Looking at the [LevelDBStoreConfigurationParser72|https://github.com/infinispan/infinispa...] class, the method parseLevelDBCacheStore() (line 55) has a call to:
> {code}
> String value = StringPropertyReplacer.replaceProperties(attributeValue);
> {code}
> while the parseLevelDBCacheStoreExpiry() method does not contain any call. Given the expiry attribute used to be part of the <leveldbStore>, it would seem the replace-properties support has not been carried over into the expiration tag handling.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5527) Java system properties no longer supported in leveldb expiration path
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5527?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5527:
----------------------------------
Status: Open (was: New)
> Java system properties no longer supported in leveldb expiration path
> ---------------------------------------------------------------------
>
> Key: ISPN-5527
> URL: https://issues.jboss.org/browse/ISPN-5527
> Project: Infinispan
> Issue Type: Bug
> Components: Configuration
> Affects Versions: 7.2.0.Final
> Reporter: Paul Richardson
> Fix For: 8.0.0.Alpha2, 7.2.3.Final, 8.0.0.Final
>
>
> In 6.0, it was possible to create a leveldb cache configuration like:
> {code}
> <leveldbStore xmlns="urn:infinispan:config:store:leveldb:6.0"
> location="${user.home}/.komodo/db/data"
> expiredLocation="${user.home}/.komodo/db/expired"
> implementationType="JAVA">
> </leveldbStore>
> {code}
> Both the location and expiredLocation paths could contain java system properties and these would be substituted when the configuration was parsed. In 7.2, the same configuration successfully substitutes the path attribute but NOT the expiration path attribute:
> {code}
> <leveldb-store xmlns="urn:infinispan:config:store:leveldb:7.2"
> path="${user.home}/.komodo/db/data">
> <expiration path="${user.home}/.komodo/db/expired"/>
> <implementation type="JAVA"/>
> </leveldb-store>
> {code}
> Looking at the [LevelDBStoreConfigurationParser72|https://github.com/infinispan/infinispa...] class, the method parseLevelDBCacheStore() (line 55) has a call to:
> {code}
> String value = StringPropertyReplacer.replaceProperties(attributeValue);
> {code}
> while the parseLevelDBCacheStoreExpiry() method does not contain any call. Given the expiry attribute used to be part of the <leveldbStore>, it would seem the replace-properties support has not been carried over into the expiration tag handling.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5527) Java system properties no longer supported in leveldb expiration path
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5527?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant reassigned ISPN-5527:
-------------------------------------
Assignee: Tristan Tarrant
> Java system properties no longer supported in leveldb expiration path
> ---------------------------------------------------------------------
>
> Key: ISPN-5527
> URL: https://issues.jboss.org/browse/ISPN-5527
> Project: Infinispan
> Issue Type: Bug
> Components: Configuration
> Affects Versions: 7.2.0.Final
> Reporter: Paul Richardson
> Assignee: Tristan Tarrant
> Fix For: 8.0.0.Alpha2, 7.2.3.Final, 8.0.0.Final
>
>
> In 6.0, it was possible to create a leveldb cache configuration like:
> {code}
> <leveldbStore xmlns="urn:infinispan:config:store:leveldb:6.0"
> location="${user.home}/.komodo/db/data"
> expiredLocation="${user.home}/.komodo/db/expired"
> implementationType="JAVA">
> </leveldbStore>
> {code}
> Both the location and expiredLocation paths could contain java system properties and these would be substituted when the configuration was parsed. In 7.2, the same configuration successfully substitutes the path attribute but NOT the expiration path attribute:
> {code}
> <leveldb-store xmlns="urn:infinispan:config:store:leveldb:7.2"
> path="${user.home}/.komodo/db/data">
> <expiration path="${user.home}/.komodo/db/expired"/>
> <implementation type="JAVA"/>
> </leveldb-store>
> {code}
> Looking at the [LevelDBStoreConfigurationParser72|https://github.com/infinispan/infinispa...] class, the method parseLevelDBCacheStore() (line 55) has a call to:
> {code}
> String value = StringPropertyReplacer.replaceProperties(attributeValue);
> {code}
> while the parseLevelDBCacheStoreExpiry() method does not contain any call. Given the expiry attribute used to be part of the <leveldbStore>, it would seem the replace-properties support has not been carried over into the expiration tag handling.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5527) Java system properties no longer supported in leveldb expiration path
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-5527?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-5527:
----------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/3526, https://github.com/infinispan/infinispan/pull/3527
> Java system properties no longer supported in leveldb expiration path
> ---------------------------------------------------------------------
>
> Key: ISPN-5527
> URL: https://issues.jboss.org/browse/ISPN-5527
> Project: Infinispan
> Issue Type: Bug
> Components: Configuration
> Affects Versions: 7.2.0.Final
> Reporter: Paul Richardson
> Assignee: Tristan Tarrant
> Fix For: 8.0.0.Alpha2, 7.2.3.Final, 8.0.0.Final
>
>
> In 6.0, it was possible to create a leveldb cache configuration like:
> {code}
> <leveldbStore xmlns="urn:infinispan:config:store:leveldb:6.0"
> location="${user.home}/.komodo/db/data"
> expiredLocation="${user.home}/.komodo/db/expired"
> implementationType="JAVA">
> </leveldbStore>
> {code}
> Both the location and expiredLocation paths could contain java system properties and these would be substituted when the configuration was parsed. In 7.2, the same configuration successfully substitutes the path attribute but NOT the expiration path attribute:
> {code}
> <leveldb-store xmlns="urn:infinispan:config:store:leveldb:7.2"
> path="${user.home}/.komodo/db/data">
> <expiration path="${user.home}/.komodo/db/expired"/>
> <implementation type="JAVA"/>
> </leveldb-store>
> {code}
> Looking at the [LevelDBStoreConfigurationParser72|https://github.com/infinispan/infinispa...] class, the method parseLevelDBCacheStore() (line 55) has a call to:
> {code}
> String value = StringPropertyReplacer.replaceProperties(attributeValue);
> {code}
> while the parseLevelDBCacheStoreExpiry() method does not contain any call. Given the expiry attribute used to be part of the <leveldbStore>, it would seem the replace-properties support has not been carried over into the expiration tag handling.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5524) Race condition in SemaphoreCompletionService.executeFront()
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5524?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-5524:
-----------------------------------------------
Dan Berindei <dberinde(a)redhat.com> changed the Status of [bug 1229181|https://bugzilla.redhat.com/show_bug.cgi?id=1229181] from NEW to ASSIGNED
> Race condition in SemaphoreCompletionService.executeFront()
> -----------------------------------------------------------
>
> Key: ISPN-5524
> URL: https://issues.jboss.org/browse/ISPN-5524
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.2.2.Final, 8.0.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 8.0.0.Alpha2, 7.2.3.Final
>
>
> If two threads call {{executeFront()}} at the same time, it's possible that none of them executes any task, although there are permits available, and one of the threads just added a task in the queue. This scenario causes random initial state transfer timeouts in the testsuite:
> # T1: continueTaskInBackground(): queue = empty, permits = 0
> # T1: backgroundTaskFinished()
> # T1: = semaphore.release(): queue = empty, permits = 1
> # T1: = executeFront()
> # T1: == semaphore.acquire() -> true: queue = empty, permits = 0
> # T2: == queue.poll() -> null
> # T2: submit(task)
> # T2: = executeFront()
> # T2: == semaphore.acquire() -> false
> # T2: return without executing any task
> # T1: return without executing any task
> Failure example: http://ci.infinispan.org/viewLog.html?buildId=27153&tab=buildResultsDiv&b...
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5524) Race condition in SemaphoreCompletionService.executeFront()
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-5524?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration updated ISPN-5524:
------------------------------------------
Bugzilla Update: Perform
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1229181
> Race condition in SemaphoreCompletionService.executeFront()
> -----------------------------------------------------------
>
> Key: ISPN-5524
> URL: https://issues.jboss.org/browse/ISPN-5524
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.2.2.Final, 8.0.0.Alpha1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 8.0.0.Alpha2, 7.2.3.Final
>
>
> If two threads call {{executeFront()}} at the same time, it's possible that none of them executes any task, although there are permits available, and one of the threads just added a task in the queue. This scenario causes random initial state transfer timeouts in the testsuite:
> # T1: continueTaskInBackground(): queue = empty, permits = 0
> # T1: backgroundTaskFinished()
> # T1: = semaphore.release(): queue = empty, permits = 1
> # T1: = executeFront()
> # T1: == semaphore.acquire() -> true: queue = empty, permits = 0
> # T2: == queue.poll() -> null
> # T2: submit(task)
> # T2: = executeFront()
> # T2: == semaphore.acquire() -> false
> # T2: return without executing any task
> # T1: return without executing any task
> Failure example: http://ci.infinispan.org/viewLog.html?buildId=27153&tab=buildResultsDiv&b...
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5527) Java system properties no longer supported in leveldb expiration path
by Horia Chiorean (JIRA)
[ https://issues.jboss.org/browse/ISPN-5527?page=com.atlassian.jira.plugin.... ]
Horia Chiorean updated ISPN-5527:
---------------------------------
Comment: was deleted
(was: Since there is nothing the ModeShape codebase can do, we can keep this open for documentation purposes only. )
> Java system properties no longer supported in leveldb expiration path
> ---------------------------------------------------------------------
>
> Key: ISPN-5527
> URL: https://issues.jboss.org/browse/ISPN-5527
> Project: Infinispan
> Issue Type: Bug
> Components: Configuration
> Affects Versions: 7.2.0.Final
> Reporter: Paul Richardson
>
> In 6.0, it was possible to create a leveldb cache configuration like:
> {code}
> <leveldbStore xmlns="urn:infinispan:config:store:leveldb:6.0"
> location="${user.home}/.komodo/db/data"
> expiredLocation="${user.home}/.komodo/db/expired"
> implementationType="JAVA">
> </leveldbStore>
> {code}
> Both the location and expiredLocation paths could contain java system properties and these would be substituted when the configuration was parsed. In 7.2, the same configuration successfully substitutes the path attribute but NOT the expiration path attribute:
> {code}
> <leveldb-store xmlns="urn:infinispan:config:store:leveldb:7.2"
> path="${user.home}/.komodo/db/data">
> <expiration path="${user.home}/.komodo/db/expired"/>
> <implementation type="JAVA"/>
> </leveldb-store>
> {code}
> Looking at the [LevelDBStoreConfigurationParser72|https://github.com/infinispan/infinispa...] class, the method parseLevelDBCacheStore() (line 55) has a call to:
> {code}
> String value = StringPropertyReplacer.replaceProperties(attributeValue);
> {code}
> while the parseLevelDBCacheStoreExpiry() method does not contain any call. Given the expiry attribute used to be part of the <leveldbStore>, it would seem the replace-properties support has not been carried over into the expiration tag handling.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (ISPN-5527) Java system properties no longer supported in leveldb expiration path
by Horia Chiorean (JIRA)
[ https://issues.jboss.org/browse/ISPN-5527?page=com.atlassian.jira.plugin.... ]
Horia Chiorean commented on ISPN-5527:
--------------------------------------
Since there is nothing the ModeShape codebase can do, we can keep this open for documentation purposes only.
> Java system properties no longer supported in leveldb expiration path
> ---------------------------------------------------------------------
>
> Key: ISPN-5527
> URL: https://issues.jboss.org/browse/ISPN-5527
> Project: Infinispan
> Issue Type: Bug
> Components: Configuration
> Affects Versions: 7.2.0.Final
> Reporter: Paul Richardson
>
> In 6.0, it was possible to create a leveldb cache configuration like:
> {code}
> <leveldbStore xmlns="urn:infinispan:config:store:leveldb:6.0"
> location="${user.home}/.komodo/db/data"
> expiredLocation="${user.home}/.komodo/db/expired"
> implementationType="JAVA">
> </leveldbStore>
> {code}
> Both the location and expiredLocation paths could contain java system properties and these would be substituted when the configuration was parsed. In 7.2, the same configuration successfully substitutes the path attribute but NOT the expiration path attribute:
> {code}
> <leveldb-store xmlns="urn:infinispan:config:store:leveldb:7.2"
> path="${user.home}/.komodo/db/data">
> <expiration path="${user.home}/.komodo/db/expired"/>
> <implementation type="JAVA"/>
> </leveldb-store>
> {code}
> Looking at the [LevelDBStoreConfigurationParser72|https://github.com/infinispan/infinispa...] class, the method parseLevelDBCacheStore() (line 55) has a call to:
> {code}
> String value = StringPropertyReplacer.replaceProperties(attributeValue);
> {code}
> while the parseLevelDBCacheStoreExpiry() method does not contain any call. Given the expiry attribute used to be part of the <leveldbStore>, it would seem the replace-properties support has not been carried over into the expiration tag handling.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months