[JBoss JIRA] (ISPN-11013) Selecting a single field in a query returns wrong "total_count"
by Gustavo Fernandes (Jira)
[ https://issues.jboss.org/browse/ISPN-11013?page=com.atlassian.jira.plugin... ]
Gustavo Fernandes commented on ISPN-11013:
------------------------------------------
Could you see the correct count?
> Selecting a single field in a query returns wrong "total_count"
> ---------------------------------------------------------------
>
> Key: ISPN-11013
> URL: https://issues.jboss.org/browse/ISPN-11013
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Querying
> Affects Versions: 10.0.1.Final
> Reporter: Jens Reimann
> Assignee: Gustavo Fernandes
> Priority: Major
> Attachments: deviceManagement.proto, infinispan.xml
>
>
> Adding a field selector to the query returns the wrong "total_count" in the result:
> {code:java}
> ➜ openshift oc -n enmasse-infra rsh infinispan-0 curl 'http://app:test12@infinispan-0:11222/rest/v2/caches/devices?action=search...'
> {
> "total_results" : 38847,
> "hits" : [ {
> "hit" : {
> "_type" : "io.enmasse.iot.registry.infinispan.data.DeviceInformation",
> "tenantId" : "jbtest.iot/2019-11-29T08:52:39Z",
> "deviceId" : "http-inserter-wh5xn103",
> "version" : "992899e8-c557-4a2c-8bc0-db950a803898",
> "registrationInformation" : "{}",
> "credentials" : [ {
> "authId" : "device-103",
> "type" : "hashed-password",
> "secrets" : [ "{\"hash-function\":\"bcrypt\",\"pwd-hash\":\"$2y$12$JELemetlJuc.6ZgQkCn5X..7UEPQm5iQV23lgno7/2sEKY2i.mPmS\"}" ]
> } ]
> }
> } ]
> }
> {code}
> {code:java}
> ➜ openshift oc -n enmasse-infra rsh infinispan-0 curl 'http://app:test12@infinispan-0:11222/rest/v2/caches/devices?action=search...'
> {
> "total_results" : 1,
> "hits" : [ {
> "hit" : {
> "deviceId" : "http-inserter-wh5xn103"
> }
> } ]
> }
> {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 6 months
[JBoss JIRA] (ISPN-11025) Empty result on query while data is still present
by Jens Reimann (Jira)
[ https://issues.jboss.org/browse/ISPN-11025?page=com.atlassian.jira.plugin... ]
Jens Reimann updated ISPN-11025:
--------------------------------
Attachment: deviceManagement.proto
> Empty result on query while data is still present
> -------------------------------------------------
>
> Key: ISPN-11025
> URL: https://issues.jboss.org/browse/ISPN-11025
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Querying
> Affects Versions: 10.0.1.Final
> Environment: 12 node Infinispan Cluster, OpenShift 4.2
> Reporter: Jens Reimann
> Priority: Major
> Attachments: deviceManagement.proto, infinispan.xml
>
>
> In a loop like this:
> 1) Query 100 next entries
> 2) For each entry: delete entry by key
> The query first returns a full set of 100 entries. Then starts to return non-full entries (while the total number found indicates that there are more than 100 entries still left). At some point it starts returning results with 0 entries, still showing that a the total number of entries is above zero:
> {code}
> 12:33:15.874 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 295419.
> 12:33:16.648 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 68 entries in this iteration. Total remaining: 295351.
> 12:33:17.535 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 295289.
> 12:33:18.206 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 295251.
> 12:33:19.054 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 295189.
> 12:33:19.766 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 295151.
> 12:33:20.593 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 295089.
> 12:33:21.336 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 295051.
> 12:33:22.209 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 294989.
> 12:33:22.458 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 294951.
> 12:33:23.672 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 294851.
> 12:33:24.946 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 294751.
> 12:33:25.065 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 21 entries in this iteration. Total remaining: 294730.
> 12:33:25.114 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.168 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.224 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.274 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.326 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.387 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.442 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.494 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.547 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.606 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.659 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:26.044 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 294630.
> 12:33:26.079 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.121 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.154 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.192 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.236 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.288 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> …
> {code}
> Executing the following code (while the deletion process is still running):
> {code}
> oc -n enmasse-infra rsh $HOST curl "http://app:test12@$HOST:11222/rest/v2/caches/devices?action=search&query=..."
> oc -n enmasse-infra rsh $HOST curl "http://app:test12@$HOST:11222/rest/v2/caches/devices?action=search&query=..."
> oc -n enmasse-infra rsh $HOST curl "http://app:test12@$HOST:11222/rest/v2/caches/devices?action=size"
> {code}
> Shows:
> {code:json}
> {
> "total_results" : 237484,
> "hits" : [ ]
> }
> {
> "total_results" : 237384,
> "hits" : [ ]
> }
> 237184
> {code}
> Stopping the deletion process and re-executing the script above provides the proper information.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 6 months
[JBoss JIRA] (ISPN-11025) Empty result on query while data is still present
by Jens Reimann (Jira)
[ https://issues.jboss.org/browse/ISPN-11025?page=com.atlassian.jira.plugin... ]
Jens Reimann updated ISPN-11025:
--------------------------------
Attachment: infinispan.xml
> Empty result on query while data is still present
> -------------------------------------------------
>
> Key: ISPN-11025
> URL: https://issues.jboss.org/browse/ISPN-11025
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Querying
> Affects Versions: 10.0.1.Final
> Environment: 12 node Infinispan Cluster, OpenShift 4.2
> Reporter: Jens Reimann
> Priority: Major
> Attachments: infinispan.xml
>
>
> In a loop like this:
> 1) Query 100 next entries
> 2) For each entry: delete entry by key
> The query first returns a full set of 100 entries. Then starts to return non-full entries (while the total number found indicates that there are more than 100 entries still left). At some point it starts returning results with 0 entries, still showing that a the total number of entries is above zero:
> {code}
> 12:33:15.874 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 295419.
> 12:33:16.648 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 68 entries in this iteration. Total remaining: 295351.
> 12:33:17.535 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 295289.
> 12:33:18.206 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 295251.
> 12:33:19.054 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 295189.
> 12:33:19.766 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 295151.
> 12:33:20.593 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 295089.
> 12:33:21.336 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 295051.
> 12:33:22.209 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 294989.
> 12:33:22.458 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 294951.
> 12:33:23.672 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 294851.
> 12:33:24.946 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 294751.
> 12:33:25.065 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 21 entries in this iteration. Total remaining: 294730.
> 12:33:25.114 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.168 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.224 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.274 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.326 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.387 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.442 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.494 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.547 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.606 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.659 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:26.044 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 294630.
> 12:33:26.079 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.121 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.154 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.192 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.236 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.288 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> …
> {code}
> Executing the following code (while the deletion process is still running):
> {code}
> oc -n enmasse-infra rsh $HOST curl "http://app:test12@$HOST:11222/rest/v2/caches/devices?action=search&query=..."
> oc -n enmasse-infra rsh $HOST curl "http://app:test12@$HOST:11222/rest/v2/caches/devices?action=search&query=..."
> oc -n enmasse-infra rsh $HOST curl "http://app:test12@$HOST:11222/rest/v2/caches/devices?action=size"
> {code}
> Shows:
> {code:json}
> {
> "total_results" : 237484,
> "hits" : [ ]
> }
> {
> "total_results" : 237384,
> "hits" : [ ]
> }
> 237184
> {code}
> Stopping the deletion process and re-executing the script above provides the proper information.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 6 months
[JBoss JIRA] (ISPN-11013) Selecting a single field in a query returns wrong "total_count"
by Jens Reimann (Jira)
[ https://issues.jboss.org/browse/ISPN-11013?page=com.atlassian.jira.plugin... ]
Jens Reimann commented on ISPN-11013:
-------------------------------------
Ok, so setting Index+Store=YES allows you to "select" the field.
However I am not sure if it is worth indexing this field just for cherry-picking it during the delete process.
> Selecting a single field in a query returns wrong "total_count"
> ---------------------------------------------------------------
>
> Key: ISPN-11013
> URL: https://issues.jboss.org/browse/ISPN-11013
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Querying
> Affects Versions: 10.0.1.Final
> Reporter: Jens Reimann
> Assignee: Gustavo Fernandes
> Priority: Major
> Attachments: deviceManagement.proto, infinispan.xml
>
>
> Adding a field selector to the query returns the wrong "total_count" in the result:
> {code:java}
> ➜ openshift oc -n enmasse-infra rsh infinispan-0 curl 'http://app:test12@infinispan-0:11222/rest/v2/caches/devices?action=search...'
> {
> "total_results" : 38847,
> "hits" : [ {
> "hit" : {
> "_type" : "io.enmasse.iot.registry.infinispan.data.DeviceInformation",
> "tenantId" : "jbtest.iot/2019-11-29T08:52:39Z",
> "deviceId" : "http-inserter-wh5xn103",
> "version" : "992899e8-c557-4a2c-8bc0-db950a803898",
> "registrationInformation" : "{}",
> "credentials" : [ {
> "authId" : "device-103",
> "type" : "hashed-password",
> "secrets" : [ "{\"hash-function\":\"bcrypt\",\"pwd-hash\":\"$2y$12$JELemetlJuc.6ZgQkCn5X..7UEPQm5iQV23lgno7/2sEKY2i.mPmS\"}" ]
> } ]
> }
> } ]
> }
> {code}
> {code:java}
> ➜ openshift oc -n enmasse-infra rsh infinispan-0 curl 'http://app:test12@infinispan-0:11222/rest/v2/caches/devices?action=search...'
> {
> "total_results" : 1,
> "hits" : [ {
> "hit" : {
> "deviceId" : "http-inserter-wh5xn103"
> }
> } ]
> }
> {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 6 months
[JBoss JIRA] (ISPN-11025) Empty result on query while data is still present
by Jens Reimann (Jira)
[ https://issues.jboss.org/browse/ISPN-11025?page=com.atlassian.jira.plugin... ]
Jens Reimann commented on ISPN-11025:
-------------------------------------
To me this looks different than ISPN-11014, as at some point results are returned. Where in ISPN-11014 the server never did return the remaining entries in the query.
> Empty result on query while data is still present
> -------------------------------------------------
>
> Key: ISPN-11025
> URL: https://issues.jboss.org/browse/ISPN-11025
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Querying
> Affects Versions: 10.0.1.Final
> Environment: 12 node Infinispan Cluster, OpenShift 4.2
> Reporter: Jens Reimann
> Priority: Major
>
> In a loop like this:
> 1) Query 100 next entries
> 2) For each entry: delete entry by key
> The query first returns a full set of 100 entries. Then starts to return non-full entries (while the total number found indicates that there are more than 100 entries still left). At some point it starts returning results with 0 entries, still showing that a the total number of entries is above zero:
> {code}
> 12:33:15.874 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 295419.
> 12:33:16.648 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 68 entries in this iteration. Total remaining: 295351.
> 12:33:17.535 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 295289.
> 12:33:18.206 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 295251.
> 12:33:19.054 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 295189.
> 12:33:19.766 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 295151.
> 12:33:20.593 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 295089.
> 12:33:21.336 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 295051.
> 12:33:22.209 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 294989.
> 12:33:22.458 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 294951.
> 12:33:23.672 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 294851.
> 12:33:24.946 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 294751.
> 12:33:25.065 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 21 entries in this iteration. Total remaining: 294730.
> 12:33:25.114 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.168 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.224 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.274 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.326 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.387 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.442 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.494 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.547 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.606 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:25.659 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
> 12:33:26.044 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 294630.
> 12:33:26.079 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.121 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.154 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.192 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.236 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> 12:33:26.288 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
> …
> {code}
> Executing the following code (while the deletion process is still running):
> {code}
> oc -n enmasse-infra rsh $HOST curl "http://app:test12@$HOST:11222/rest/v2/caches/devices?action=search&query=..."
> oc -n enmasse-infra rsh $HOST curl "http://app:test12@$HOST:11222/rest/v2/caches/devices?action=search&query=..."
> oc -n enmasse-infra rsh $HOST curl "http://app:test12@$HOST:11222/rest/v2/caches/devices?action=size"
> {code}
> Shows:
> {code:json}
> {
> "total_results" : 237484,
> "hits" : [ ]
> }
> {
> "total_results" : 237384,
> "hits" : [ ]
> }
> 237184
> {code}
> Stopping the deletion process and re-executing the script above provides the proper information.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 6 months
[JBoss JIRA] (ISPN-11025) Empty result on query while data is still present
by Jens Reimann (Jira)
Jens Reimann created ISPN-11025:
-----------------------------------
Summary: Empty result on query while data is still present
Key: ISPN-11025
URL: https://issues.jboss.org/browse/ISPN-11025
Project: Infinispan
Issue Type: Bug
Components: Remote Querying
Affects Versions: 10.0.1.Final
Environment: 12 node Infinispan Cluster, OpenShift 4.2
Reporter: Jens Reimann
In a loop like this:
1) Query 100 next entries
2) For each entry: delete entry by key
The query first returns a full set of 100 entries. Then starts to return non-full entries (while the total number found indicates that there are more than 100 entries still left). At some point it starts returning results with 0 entries, still showing that a the total number of entries is above zero:
{code}
12:33:15.874 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 295419.
12:33:16.648 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 68 entries in this iteration. Total remaining: 295351.
12:33:17.535 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 295289.
12:33:18.206 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 295251.
12:33:19.054 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 295189.
12:33:19.766 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 295151.
12:33:20.593 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 295089.
12:33:21.336 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 295051.
12:33:22.209 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 62 entries in this iteration. Total remaining: 294989.
12:33:22.458 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 38 entries in this iteration. Total remaining: 294951.
12:33:23.672 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 294851.
12:33:24.946 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 294751.
12:33:25.065 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 21 entries in this iteration. Total remaining: 294730.
12:33:25.114 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
12:33:25.168 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
12:33:25.224 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
12:33:25.274 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
12:33:25.326 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
12:33:25.387 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
12:33:25.442 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
12:33:25.494 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
12:33:25.547 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
12:33:25.606 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
12:33:25.659 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294730.
12:33:26.044 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 100 entries in this iteration. Total remaining: 294630.
12:33:26.079 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
12:33:26.121 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
12:33:26.154 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
12:33:26.192 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
12:33:26.236 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
12:33:26.288 [main] INFO i.e.i.t.c.InfinispanTenantCleaner - Deleted 0 entries in this iteration. Total remaining: 294630.
…
{code}
Executing the following code (while the deletion process is still running):
{code}
oc -n enmasse-infra rsh $HOST curl "http://app:test12@$HOST:11222/rest/v2/caches/devices?action=search&query=..."
oc -n enmasse-infra rsh $HOST curl "http://app:test12@$HOST:11222/rest/v2/caches/devices?action=search&query=..."
oc -n enmasse-infra rsh $HOST curl "http://app:test12@$HOST:11222/rest/v2/caches/devices?action=size"
{code}
Shows:
{code:json}
{
"total_results" : 237484,
"hits" : [ ]
}
{
"total_results" : 237384,
"hits" : [ ]
}
237184
{code}
Stopping the deletion process and re-executing the script above provides the proper information.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 6 months
[JBoss JIRA] (ISPN-11013) Selecting a single field in a query returns wrong "total_count"
by Gustavo Fernandes (Jira)
[ https://issues.jboss.org/browse/ISPN-11013?page=com.atlassian.jira.plugin... ]
Gustavo Fernandes commented on ISPN-11013:
------------------------------------------
* Fields that are used in the select clause should be stored in the index (stored.YES)
* Id fields should not be analysed (Analyze.NO)
* Random text fields that will be searched using full-text queries should be Analyze.YES, Store.NO
> Selecting a single field in a query returns wrong "total_count"
> ---------------------------------------------------------------
>
> Key: ISPN-11013
> URL: https://issues.jboss.org/browse/ISPN-11013
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Querying
> Affects Versions: 10.0.1.Final
> Reporter: Jens Reimann
> Assignee: Gustavo Fernandes
> Priority: Major
> Attachments: deviceManagement.proto, infinispan.xml
>
>
> Adding a field selector to the query returns the wrong "total_count" in the result:
> {code:java}
> ➜ openshift oc -n enmasse-infra rsh infinispan-0 curl 'http://app:test12@infinispan-0:11222/rest/v2/caches/devices?action=search...'
> {
> "total_results" : 38847,
> "hits" : [ {
> "hit" : {
> "_type" : "io.enmasse.iot.registry.infinispan.data.DeviceInformation",
> "tenantId" : "jbtest.iot/2019-11-29T08:52:39Z",
> "deviceId" : "http-inserter-wh5xn103",
> "version" : "992899e8-c557-4a2c-8bc0-db950a803898",
> "registrationInformation" : "{}",
> "credentials" : [ {
> "authId" : "device-103",
> "type" : "hashed-password",
> "secrets" : [ "{\"hash-function\":\"bcrypt\",\"pwd-hash\":\"$2y$12$JELemetlJuc.6ZgQkCn5X..7UEPQm5iQV23lgno7/2sEKY2i.mPmS\"}" ]
> } ]
> }
> } ]
> }
> {code}
> {code:java}
> ➜ openshift oc -n enmasse-infra rsh infinispan-0 curl 'http://app:test12@infinispan-0:11222/rest/v2/caches/devices?action=search...'
> {
> "total_results" : 1,
> "hits" : [ {
> "hit" : {
> "deviceId" : "http-inserter-wh5xn103"
> }
> } ]
> }
> {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 6 months
[JBoss JIRA] (ISPN-11024) Unable to use binary eviction with protobuf
by Jens Reimann (Jira)
Jens Reimann created ISPN-11024:
-----------------------------------
Summary: Unable to use binary eviction with protobuf
Key: ISPN-11024
URL: https://issues.jboss.org/browse/ISPN-11024
Project: Infinispan
Issue Type: Bug
Components: Server
Affects Versions: 10.0.1.Final
Reporter: Jens Reimann
Enabling binary eviction, e.g.:
{code:xml}
<memory>
<binary strategy="REMOVE" size="134217728" eviction="MEMORY"/>
</memory>
{code}
When storing protobuf based types, throws the following exception:
{code}
15:36:29,859 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (async-thread--p2-t18) ISPN000136: Error executing command PrepareCommand on Cache 'devices', writing keys []: java.lang.IllegalArgumentException: Size of Class class org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper cannot be determined using given entry size calculator :class org.infinispan.container.entries.PrimitiveEntrySizeCalculator
{code}
As protobuf produces a binary representation of the data, it is possible to calculate a size for that object.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 6 months
[JBoss JIRA] (ISPN-11013) Selecting a single field in a query returns wrong "total_count"
by Jens Reimann (Jira)
[ https://issues.jboss.org/browse/ISPN-11013?page=com.atlassian.jira.plugin... ]
Jens Reimann commented on ISPN-11013:
-------------------------------------
{quote}You should not analyse those id fields, mark them as
@Field(index=Index.YES, analyze=Analyze.NO, store=Store.YES){quote}
So, which fields should have which value?
> Selecting a single field in a query returns wrong "total_count"
> ---------------------------------------------------------------
>
> Key: ISPN-11013
> URL: https://issues.jboss.org/browse/ISPN-11013
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Querying
> Affects Versions: 10.0.1.Final
> Reporter: Jens Reimann
> Assignee: Gustavo Fernandes
> Priority: Major
> Attachments: deviceManagement.proto, infinispan.xml
>
>
> Adding a field selector to the query returns the wrong "total_count" in the result:
> {code:java}
> ➜ openshift oc -n enmasse-infra rsh infinispan-0 curl 'http://app:test12@infinispan-0:11222/rest/v2/caches/devices?action=search...'
> {
> "total_results" : 38847,
> "hits" : [ {
> "hit" : {
> "_type" : "io.enmasse.iot.registry.infinispan.data.DeviceInformation",
> "tenantId" : "jbtest.iot/2019-11-29T08:52:39Z",
> "deviceId" : "http-inserter-wh5xn103",
> "version" : "992899e8-c557-4a2c-8bc0-db950a803898",
> "registrationInformation" : "{}",
> "credentials" : [ {
> "authId" : "device-103",
> "type" : "hashed-password",
> "secrets" : [ "{\"hash-function\":\"bcrypt\",\"pwd-hash\":\"$2y$12$JELemetlJuc.6ZgQkCn5X..7UEPQm5iQV23lgno7/2sEKY2i.mPmS\"}" ]
> } ]
> }
> } ]
> }
> {code}
> {code:java}
> ➜ openshift oc -n enmasse-infra rsh infinispan-0 curl 'http://app:test12@infinispan-0:11222/rest/v2/caches/devices?action=search...'
> {
> "total_results" : 1,
> "hits" : [ {
> "hit" : {
> "deviceId" : "http-inserter-wh5xn103"
> }
> } ]
> }
> {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 6 months
[JBoss JIRA] (ISPN-11013) Selecting a single field in a query returns wrong "total_count"
by Gustavo Fernandes (Jira)
[ https://issues.jboss.org/browse/ISPN-11013?page=com.atlassian.jira.plugin... ]
Gustavo Fernandes commented on ISPN-11013:
------------------------------------------
You should not analyse those id fields, mark them as
{{(a)Field(index=Index.YES, analyze=Analyze.NO, store=Store.YES)}}
> Selecting a single field in a query returns wrong "total_count"
> ---------------------------------------------------------------
>
> Key: ISPN-11013
> URL: https://issues.jboss.org/browse/ISPN-11013
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Querying
> Affects Versions: 10.0.1.Final
> Reporter: Jens Reimann
> Assignee: Gustavo Fernandes
> Priority: Major
> Attachments: deviceManagement.proto, infinispan.xml
>
>
> Adding a field selector to the query returns the wrong "total_count" in the result:
> {code:java}
> ➜ openshift oc -n enmasse-infra rsh infinispan-0 curl 'http://app:test12@infinispan-0:11222/rest/v2/caches/devices?action=search...'
> {
> "total_results" : 38847,
> "hits" : [ {
> "hit" : {
> "_type" : "io.enmasse.iot.registry.infinispan.data.DeviceInformation",
> "tenantId" : "jbtest.iot/2019-11-29T08:52:39Z",
> "deviceId" : "http-inserter-wh5xn103",
> "version" : "992899e8-c557-4a2c-8bc0-db950a803898",
> "registrationInformation" : "{}",
> "credentials" : [ {
> "authId" : "device-103",
> "type" : "hashed-password",
> "secrets" : [ "{\"hash-function\":\"bcrypt\",\"pwd-hash\":\"$2y$12$JELemetlJuc.6ZgQkCn5X..7UEPQm5iQV23lgno7/2sEKY2i.mPmS\"}" ]
> } ]
> }
> } ]
> }
> {code}
> {code:java}
> ➜ openshift oc -n enmasse-infra rsh infinispan-0 curl 'http://app:test12@infinispan-0:11222/rest/v2/caches/devices?action=search...'
> {
> "total_results" : 1,
> "hits" : [ {
> "hit" : {
> "deviceId" : "http-inserter-wh5xn103"
> }
> } ]
> }
> {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 6 months