[JBoss JIRA] (ISPN-8411) Add support for efficient removeAll
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-8411?page=com.atlassian.jira.plugin.... ]
Radim Vansa commented on ISPN-8411:
-----------------------------------
[~papegaaij] ClearCommand does not participate in transaction; it is broadcast to other nodes and wipes out data without any specific ordering. It does not lock entries. When you do (in single transaction) cache.get(K); cache.clear(); cache.get(K); the second read will still return the original entry (if you use repeatable-read isolation).
The problem is that if the command were invoked transactionally, it would require locking whole cache, blocking all the other operations, which is not acceptable. In fact the {{Caches.removeAll}} in Hibernate does exactly this.
About the configuration - yes, 8.2 documentation was not updated at that time, 9.0 already points to Hibernate docs which is up-to-date, too, and explains the valid options. Hibernate ORM currently has non-tx caches as default (since 5.1 I think), WildFly is rather slow to adopt it.
It is not wrong (in Hibernate 5.x we wanted to support old configurations to allow for some bake-in and fallback), just a tad slower, and you may have issues like the one you experience. For Hibernate 6.0 the transactional caches will be likely discontinued as there are no known benefits in using them and it's a maintenance burden.
> Add support for efficient removeAll
> -----------------------------------
>
> Key: ISPN-8411
> URL: https://issues.jboss.org/browse/ISPN-8411
> Project: Infinispan
> Issue Type: Feature Request
> Components: Hibernate Cache
> Affects Versions: 8.2.8.Final, 9.1.1.Final
> Environment: WildFly 10.1.0, WildFly 11.0.0.CR1, WildFly master, Hibernate 2LC
> Reporter: Emond Papegaaij
> Assignee: Galder Zamarreño
>
> Infinispan currently does not seem to implement an efficient way to clear an entire cache cluster-wide. This forces Hibernate to remove all entries one by one when a cache region needs to be cleared, for example when a buld CriteriaUpdate or CriteriaDelete is used.
> The behavior we are observing is:
> # All nodes in the cluster are queried for the keyset in a region
> # A lock seems to be in place for this region for the duration of the commit
> # The initiating node constructs a message with {{InvalidateCommands}} for all keys
> # This large message (230MB for 200k entries) is sent to all nodes in the cluster
> For large caches this can take very long. We had to increase the remote-timeout to 60 seconds to prevent timeouts. During this time, the entire cluster is locked an busy processing the cache invalidations. As you can understand, this is not a workable solution for us. On some places we can prevent the cache clear by updating the records one by one, but in other places this is not an option.
> The corresponding report at Hibernate can be found here: https://hibernate.atlassian.net/browse/HHH-12036
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 5 months
[JBoss JIRA] (ISPN-8411) Add support for efficient removeAll
by Emond Papegaaij (JIRA)
[ https://issues.jboss.org/browse/ISPN-8411?page=com.atlassian.jira.plugin.... ]
Emond Papegaaij commented on ISPN-8411:
---------------------------------------
[~NadirX] Can you point me to the location of this cluster-wide clear command? The documentation of {{Cache.clear}} states it's local and should not be used: "_Note: This should never be invoked in production unless you can guarantee no other invocations are ran concurrently. If the cache is transactional, it will not interact with the transaction._" However, the implementation seems to execute a ClearCommand and participate in the transaction. Also note that Infinispan's own JCache (JSR 107) implementation uses the same iterate and remove technique: https://github.com/infinispan/infinispan/blob/9.1.1.Final/jcache/remote/s...
[~rvansa] Yes, we are using transaction invalidation caches. We took this configuration from the Infinispan 8.2 documentation and the default configuration provided by WildFly (10.1 and 11.0). From those we concluded that transactional caches are advised. Is this wrong? Prior to using Infinispan, we used EH Cache 2, which was read-write.
> Add support for efficient removeAll
> -----------------------------------
>
> Key: ISPN-8411
> URL: https://issues.jboss.org/browse/ISPN-8411
> Project: Infinispan
> Issue Type: Feature Request
> Components: Hibernate Cache
> Affects Versions: 8.2.8.Final, 9.1.1.Final
> Environment: WildFly 10.1.0, WildFly 11.0.0.CR1, WildFly master, Hibernate 2LC
> Reporter: Emond Papegaaij
> Assignee: Galder Zamarreño
>
> Infinispan currently does not seem to implement an efficient way to clear an entire cache cluster-wide. This forces Hibernate to remove all entries one by one when a cache region needs to be cleared, for example when a buld CriteriaUpdate or CriteriaDelete is used.
> The behavior we are observing is:
> # All nodes in the cluster are queried for the keyset in a region
> # A lock seems to be in place for this region for the duration of the commit
> # The initiating node constructs a message with {{InvalidateCommands}} for all keys
> # This large message (230MB for 200k entries) is sent to all nodes in the cluster
> For large caches this can take very long. We had to increase the remote-timeout to 60 seconds to prevent timeouts. During this time, the entire cluster is locked an busy processing the cache invalidations. As you can understand, this is not a workable solution for us. On some places we can prevent the cache clear by updating the records one by one, but in other places this is not an option.
> The corresponding report at Hibernate can be found here: https://hibernate.atlassian.net/browse/HHH-12036
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 5 months
[JBoss JIRA] (ISPN-8411) Add support for efficient removeAll
by Emond Papegaaij (JIRA)
[ https://issues.jboss.org/browse/ISPN-8411?page=com.atlassian.jira.plugin.... ]
Emond Papegaaij edited comment on ISPN-8411 at 10/17/17 5:31 AM:
-----------------------------------------------------------------
[~NadirX] Can you point me to the location of this cluster-wide clear command? The documentation of {{Cache.clear}} states it's local and should not be used: "_Note: This should never be invoked in production unless you can guarantee no other invocations are ran concurrently. If the cache is transactional, it will not interact with the transaction._" However, the implementation seems to execute a ClearCommand and participate in the transaction. Also note that Infinispan's own JCache (JSR 107) implementation uses the same iterate and remove technique: https://github.com/infinispan/infinispan/blob/9.1.1.Final/jcache/remote/s...
[~rvansa] Yes, we are using transactional invalidation caches. We took this configuration from the Infinispan 8.2 documentation and the default configuration provided by WildFly (10.1 and 11.0). From those we concluded that transactional caches are advised. Is this wrong? Prior to using Infinispan, we used EH Cache 2, which was read-write.
was (Author: papegaaij):
[~NadirX] Can you point me to the location of this cluster-wide clear command? The documentation of {{Cache.clear}} states it's local and should not be used: "_Note: This should never be invoked in production unless you can guarantee no other invocations are ran concurrently. If the cache is transactional, it will not interact with the transaction._" However, the implementation seems to execute a ClearCommand and participate in the transaction. Also note that Infinispan's own JCache (JSR 107) implementation uses the same iterate and remove technique: https://github.com/infinispan/infinispan/blob/9.1.1.Final/jcache/remote/s...
[~rvansa] Yes, we are using transaction invalidation caches. We took this configuration from the Infinispan 8.2 documentation and the default configuration provided by WildFly (10.1 and 11.0). From those we concluded that transactional caches are advised. Is this wrong? Prior to using Infinispan, we used EH Cache 2, which was read-write.
> Add support for efficient removeAll
> -----------------------------------
>
> Key: ISPN-8411
> URL: https://issues.jboss.org/browse/ISPN-8411
> Project: Infinispan
> Issue Type: Feature Request
> Components: Hibernate Cache
> Affects Versions: 8.2.8.Final, 9.1.1.Final
> Environment: WildFly 10.1.0, WildFly 11.0.0.CR1, WildFly master, Hibernate 2LC
> Reporter: Emond Papegaaij
> Assignee: Galder Zamarreño
>
> Infinispan currently does not seem to implement an efficient way to clear an entire cache cluster-wide. This forces Hibernate to remove all entries one by one when a cache region needs to be cleared, for example when a buld CriteriaUpdate or CriteriaDelete is used.
> The behavior we are observing is:
> # All nodes in the cluster are queried for the keyset in a region
> # A lock seems to be in place for this region for the duration of the commit
> # The initiating node constructs a message with {{InvalidateCommands}} for all keys
> # This large message (230MB for 200k entries) is sent to all nodes in the cluster
> For large caches this can take very long. We had to increase the remote-timeout to 60 seconds to prevent timeouts. During this time, the entire cluster is locked an busy processing the cache invalidations. As you can understand, this is not a workable solution for us. On some places we can prevent the cache clear by updating the records one by one, but in other places this is not an option.
> The corresponding report at Hibernate can be found here: https://hibernate.atlassian.net/browse/HHH-12036
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 5 months
[JBoss JIRA] (ISPN-8411) Add support for efficient removeAll
by Radim Vansa (JIRA)
[ https://issues.jboss.org/browse/ISPN-8411?page=com.atlassian.jira.plugin.... ]
Radim Vansa commented on ISPN-8411:
-----------------------------------
I suppose you're using transactional invalidation cache, do you? This behaviour occurs when you issue the cluster-wide clear inside a transaction.
I suggest using non-transactional caches (in WF you set the {{<transaction mode="NONE"/>}}); this is possible with Hibernate 5.x and offers performance benefits as well. Let me know if there are still issues with this setting.
> Add support for efficient removeAll
> -----------------------------------
>
> Key: ISPN-8411
> URL: https://issues.jboss.org/browse/ISPN-8411
> Project: Infinispan
> Issue Type: Feature Request
> Components: Hibernate Cache
> Affects Versions: 8.2.8.Final, 9.1.1.Final
> Environment: WildFly 10.1.0, WildFly 11.0.0.CR1, WildFly master, Hibernate 2LC
> Reporter: Emond Papegaaij
> Assignee: Galder Zamarreño
>
> Infinispan currently does not seem to implement an efficient way to clear an entire cache cluster-wide. This forces Hibernate to remove all entries one by one when a cache region needs to be cleared, for example when a buld CriteriaUpdate or CriteriaDelete is used.
> The behavior we are observing is:
> # All nodes in the cluster are queried for the keyset in a region
> # A lock seems to be in place for this region for the duration of the commit
> # The initiating node constructs a message with {{InvalidateCommands}} for all keys
> # This large message (230MB for 200k entries) is sent to all nodes in the cluster
> For large caches this can take very long. We had to increase the remote-timeout to 60 seconds to prevent timeouts. During this time, the entire cluster is locked an busy processing the cache invalidations. As you can understand, this is not a workable solution for us. On some places we can prevent the cache clear by updating the records one by one, but in other places this is not an option.
> The corresponding report at Hibernate can be found here: https://hibernate.atlassian.net/browse/HHH-12036
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 5 months
[JBoss JIRA] (ISPN-8411) Add support for efficient removeAll
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-8411?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-8411:
----------------------------------
Component/s: Hibernate Cache
(was: Core)
> Add support for efficient removeAll
> -----------------------------------
>
> Key: ISPN-8411
> URL: https://issues.jboss.org/browse/ISPN-8411
> Project: Infinispan
> Issue Type: Feature Request
> Components: Hibernate Cache
> Affects Versions: 8.2.8.Final, 9.1.1.Final
> Environment: WildFly 10.1.0, WildFly 11.0.0.CR1, WildFly master, Hibernate 2LC
> Reporter: Emond Papegaaij
> Assignee: Galder Zamarreño
>
> Infinispan currently does not seem to implement an efficient way to clear an entire cache cluster-wide. This forces Hibernate to remove all entries one by one when a cache region needs to be cleared, for example when a buld CriteriaUpdate or CriteriaDelete is used.
> The behavior we are observing is:
> # All nodes in the cluster are queried for the keyset in a region
> # A lock seems to be in place for this region for the duration of the commit
> # The initiating node constructs a message with {{InvalidateCommands}} for all keys
> # This large message (230MB for 200k entries) is sent to all nodes in the cluster
> For large caches this can take very long. We had to increase the remote-timeout to 60 seconds to prevent timeouts. During this time, the entire cluster is locked an busy processing the cache invalidations. As you can understand, this is not a workable solution for us. On some places we can prevent the cache clear by updating the records one by one, but in other places this is not an option.
> The corresponding report at Hibernate can be found here: https://hibernate.atlassian.net/browse/HHH-12036
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 5 months
[JBoss JIRA] (ISPN-8411) Add support for efficient removeAll
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-8411?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant commented on ISPN-8411:
---------------------------------------
Infinispan does have a cluster-wide efficient clear command. It is hibernate's 2nd-level cache which removes entries using an iterator.
> Add support for efficient removeAll
> -----------------------------------
>
> Key: ISPN-8411
> URL: https://issues.jboss.org/browse/ISPN-8411
> Project: Infinispan
> Issue Type: Feature Request
> Components: Hibernate Cache
> Affects Versions: 8.2.8.Final, 9.1.1.Final
> Environment: WildFly 10.1.0, WildFly 11.0.0.CR1, WildFly master, Hibernate 2LC
> Reporter: Emond Papegaaij
> Assignee: Galder Zamarreño
>
> Infinispan currently does not seem to implement an efficient way to clear an entire cache cluster-wide. This forces Hibernate to remove all entries one by one when a cache region needs to be cleared, for example when a buld CriteriaUpdate or CriteriaDelete is used.
> The behavior we are observing is:
> # All nodes in the cluster are queried for the keyset in a region
> # A lock seems to be in place for this region for the duration of the commit
> # The initiating node constructs a message with {{InvalidateCommands}} for all keys
> # This large message (230MB for 200k entries) is sent to all nodes in the cluster
> For large caches this can take very long. We had to increase the remote-timeout to 60 seconds to prevent timeouts. During this time, the entire cluster is locked an busy processing the cache invalidations. As you can understand, this is not a workable solution for us. On some places we can prevent the cache clear by updating the records one by one, but in other places this is not an option.
> The corresponding report at Hibernate can be found here: https://hibernate.atlassian.net/browse/HHH-12036
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 5 months
[JBoss JIRA] (ISPN-8411) Add support for efficient removeAll
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-8411?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant reassigned ISPN-8411:
-------------------------------------
Assignee: Galder Zamarreño
> Add support for efficient removeAll
> -----------------------------------
>
> Key: ISPN-8411
> URL: https://issues.jboss.org/browse/ISPN-8411
> Project: Infinispan
> Issue Type: Feature Request
> Components: Core
> Affects Versions: 8.2.8.Final, 9.1.1.Final
> Environment: WildFly 10.1.0, WildFly 11.0.0.CR1, WildFly master, Hibernate 2LC
> Reporter: Emond Papegaaij
> Assignee: Galder Zamarreño
>
> Infinispan currently does not seem to implement an efficient way to clear an entire cache cluster-wide. This forces Hibernate to remove all entries one by one when a cache region needs to be cleared, for example when a buld CriteriaUpdate or CriteriaDelete is used.
> The behavior we are observing is:
> # All nodes in the cluster are queried for the keyset in a region
> # A lock seems to be in place for this region for the duration of the commit
> # The initiating node constructs a message with {{InvalidateCommands}} for all keys
> # This large message (230MB for 200k entries) is sent to all nodes in the cluster
> For large caches this can take very long. We had to increase the remote-timeout to 60 seconds to prevent timeouts. During this time, the entire cluster is locked an busy processing the cache invalidations. As you can understand, this is not a workable solution for us. On some places we can prevent the cache clear by updating the records one by one, but in other places this is not an option.
> The corresponding report at Hibernate can be found here: https://hibernate.atlassian.net/browse/HHH-12036
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 5 months
[JBoss JIRA] (ISPN-8411) Add support for efficient removeAll
by Emond Papegaaij (JIRA)
[ https://issues.jboss.org/browse/ISPN-8411?page=com.atlassian.jira.plugin.... ]
Emond Papegaaij updated ISPN-8411:
----------------------------------
Description:
Infinispan currently does not seem to implement an efficient way to clear an entire cache cluster-wide. This forces Hibernate to remove all entries one by one when a cache region needs to be cleared, for example when a buld CriteriaUpdate or CriteriaDelete is used.
The behavior we are observing is:
# All nodes in the cluster are queried for the keyset in a region
# A lock seems to be in place for this region for the duration of the commit
# The initiating node constructs a message with {{InvalidateCommands}} for all keys
# This large message (230MB for 200k entries) is sent to all nodes in the cluster
For large caches this can take very long. We had to increase the remote-timeout to 60 seconds to prevent timeouts. During this time, the entire cluster is locked an busy processing the cache invalidations. As you can understand, this is not a workable solution for us. On some places we can prevent the cache clear by updating the records one by one, but in other places this is not an option.
The corresponding report at Hibernate can be found here: https://hibernate.atlassian.net/browse/HHH-12036
was:
Infinispan currently does not seem to implement an efficient way to clear an entire cache cluster-wide. This forces Hibernate to remove all entries one by one when a cache region needs to be cleared, for example when a buld CriteriaUpdate or CriteriaDelete is used.
The behavior we are observing is:
# All nodes in the cluster are queried for the keyset in a region
# A lock seems to be in place for this region for the duration of the commit
# The initiating node constructs a message with {{InvalidateCommands}} for all keys
# This large message (230MB for 200k entries) is sent to all nodes in the cluster
For large caches this can take very long. We had to increase the remote-timeout to 60 seconds to prevent timeouts. During this time, the entire cluster is locked an busy processing the cache invalidations. As you can understand, this is not a workable solution for us. On some places we can prevent the cache clear by updating the records one by one, but in other places this is not an option.
> Add support for efficient removeAll
> -----------------------------------
>
> Key: ISPN-8411
> URL: https://issues.jboss.org/browse/ISPN-8411
> Project: Infinispan
> Issue Type: Feature Request
> Components: Core
> Affects Versions: 8.2.8.Final, 9.1.1.Final
> Environment: WildFly 10.1.0, WildFly 11.0.0.CR1, WildFly master, Hibernate 2LC
> Reporter: Emond Papegaaij
>
> Infinispan currently does not seem to implement an efficient way to clear an entire cache cluster-wide. This forces Hibernate to remove all entries one by one when a cache region needs to be cleared, for example when a buld CriteriaUpdate or CriteriaDelete is used.
> The behavior we are observing is:
> # All nodes in the cluster are queried for the keyset in a region
> # A lock seems to be in place for this region for the duration of the commit
> # The initiating node constructs a message with {{InvalidateCommands}} for all keys
> # This large message (230MB for 200k entries) is sent to all nodes in the cluster
> For large caches this can take very long. We had to increase the remote-timeout to 60 seconds to prevent timeouts. During this time, the entire cluster is locked an busy processing the cache invalidations. As you can understand, this is not a workable solution for us. On some places we can prevent the cache clear by updating the records one by one, but in other places this is not an option.
> The corresponding report at Hibernate can be found here: https://hibernate.atlassian.net/browse/HHH-12036
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 5 months