May 2016 - infinispan-issues - Jboss List Archives

[JBoss JIRA] (ISPN-6706) Purging cache writers is [mostly] redundant when eviction is disabled and preload is enabled

by Krzysztof Sobolewski (JIRA)

[ https://issues.jboss.org/browse/ISPN-6706?page=com.atlassian.jira.plugin.... ] Krzysztof Sobolewski commented on ISPN-6706: -------------------------------------------- Thankfully I'm not :) So... it looks like these two should satisfy my use case (I hope at least ISPN-6710 is backported to 8.2.x!). Should this JIRA be closed then? > Purging cache writers is [mostly] redundant when eviction is disabled and preload is enabled > -------------------------------------------------------------------------------------------- > > Key: ISPN-6706 > URL: https://issues.jboss.org/browse/ISPN-6706 > Project: Infinispan > Issue Type: Enhancement > Components: Loaders and Stores > Affects Versions: 8.2.2.Final > Reporter: Krzysztof Sobolewski > > This issue arised when I was testing a cluster with about 16 million entries. Our configuration is that all the data is also kept in memory, so eviction is disabled in this cache. But expiration is enabled. During the test I noticed pauses that started small but increased while the test was progressing, reaching more than 20 seconds at one point. After ruling out maintenance tasks in MySQL that could interfere, I discovered that the pause is caused by the expiration thread purging the database for expired entries. This was a huge and unnecessary drag so I hacked Infinispan to skip the purge of persistent state in cases when it's likely to be redundant with purging the transient state. I say "likely" because entries evicted maually via the evict() call poke a huge hole in the underlying assumptions :) Anyway, our cluster no longer regularly pauses for half a minute, so here's something for your consideration. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

7 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-6712) JdbcStringStore doesn't notify entry expiration events

by Ryan Emerson (JIRA)

[ https://issues.jboss.org/browse/ISPN-6712?page=com.atlassian.jira.plugin.... ] Ryan Emerson reassigned ISPN-6712: ---------------------------------- Assignee: Ryan Emerson > JdbcStringStore doesn't notify entry expiration events > ------------------------------------------------------ > > Key: ISPN-6712 > URL: https://issues.jboss.org/browse/ISPN-6712 > Project: Infinispan > Issue Type: Enhancement > Components: Listeners, Loaders and Stores > Affects Versions: 9.0.0.Alpha2 > Reporter: William Burns > Assignee: Ryan Emerson > > The JdbcStringStore does a DELETE directly to the database on a purge. This limits our ability to raise an event due to this expiration occurring. The JdbcBinaryStore does do events and we may want to duplicate that code instead. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

7 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-6712) JdbcStringStore doesn't notify entry expiration events

by William Burns (JIRA)

William Burns created ISPN-6712: ----------------------------------- Summary: JdbcStringStore doesn't notify entry expiration events Key: ISPN-6712 URL: https://issues.jboss.org/browse/ISPN-6712 Project: Infinispan Issue Type: Enhancement Components: Listeners, Loaders and Stores Affects Versions: 9.0.0.Alpha2 Reporter: William Burns The JdbcStringStore does a DELETE directly to the database on a purge. This limits our ability to raise an event due to this expiration occurring. The JdbcBinaryStore does do events and we may want to duplicate that code instead. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

7 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-6706) Purging cache writers is [mostly] redundant when eviction is disabled and preload is enabled

by William Burns (JIRA)

[ https://issues.jboss.org/browse/ISPN-6706?page=com.atlassian.jira.plugin.... ] William Burns commented on ISPN-6706: ------------------------------------- I actually created ISPN-6710 and ISPN-6711. Yes they are doing SELECT or DELETE. Unfortunately I was looking at JdbcBinaryStore which does what I said, but JdbcStringStore is doing a straight DELETE from the DB. This means that JdbcStringStore will not notify listeners of an entry being expired. I don't know if you are relying on that. > Purging cache writers is [mostly] redundant when eviction is disabled and preload is enabled > -------------------------------------------------------------------------------------------- > > Key: ISPN-6706 > URL: https://issues.jboss.org/browse/ISPN-6706 > Project: Infinispan > Issue Type: Enhancement > Components: Loaders and Stores > Affects Versions: 8.2.2.Final > Reporter: Krzysztof Sobolewski > > This issue arised when I was testing a cluster with about 16 million entries. Our configuration is that all the data is also kept in memory, so eviction is disabled in this cache. But expiration is enabled. During the test I noticed pauses that started small but increased while the test was progressing, reaching more than 20 seconds at one point. After ruling out maintenance tasks in MySQL that could interfere, I discovered that the pause is caused by the expiration thread purging the database for expired entries. This was a huge and unnecessary drag so I hacked Infinispan to skip the purge of persistent state in cases when it's likely to be redundant with purging the transient state. I say "likely" because entries evicted maually via the evict() call poke a huge hole in the underlying assumptions :) Anyway, our cluster no longer regularly pauses for half a minute, so here's something for your consideration. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

7 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-6706) Purging cache writers is [mostly] redundant when eviction is disabled and preload is enabled

by Krzysztof Sobolewski (JIRA)

[ https://issues.jboss.org/browse/ISPN-6706?page=com.atlassian.jira.plugin.... ] Krzysztof Sobolewski commented on ISPN-6706: -------------------------------------------- OK, I'm getting confused :) I too was under impression that the query to purge expired rows picks one node at a time by "manually" scanning the entire table, but I looked at the code more closely and the query indeed limits the rows selected to only those with timestamp in the past, and actually it is a DELETE statement. And indeed the CREATE TABLE statement doesn't create any indices beside the PK on the ID column. So it's much clearer now. Should I add another JIRA about the timestamp index? > Purging cache writers is [mostly] redundant when eviction is disabled and preload is enabled > -------------------------------------------------------------------------------------------- > > Key: ISPN-6706 > URL: https://issues.jboss.org/browse/ISPN-6706 > Project: Infinispan > Issue Type: Enhancement > Components: Loaders and Stores > Affects Versions: 8.2.2.Final > Reporter: Krzysztof Sobolewski > > This issue arised when I was testing a cluster with about 16 million entries. Our configuration is that all the data is also kept in memory, so eviction is disabled in this cache. But expiration is enabled. During the test I noticed pauses that started small but increased while the test was progressing, reaching more than 20 seconds at one point. After ruling out maintenance tasks in MySQL that could interfere, I discovered that the pause is caused by the expiration thread purging the database for expired entries. This was a huge and unnecessary drag so I hacked Infinispan to skip the purge of persistent state in cases when it's likely to be redundant with purging the transient state. I say "likely" because entries evicted maually via the evict() call poke a huge hole in the underlying assumptions :) Anyway, our cluster no longer regularly pauses for half a minute, so here's something for your consideration. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

7 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-6711) Shared stores should only be purged from coordinator

by William Burns (JIRA)

William Burns created ISPN-6711: ----------------------------------- Summary: Shared stores should only be purged from coordinator Key: ISPN-6711 URL: https://issues.jboss.org/browse/ISPN-6711 Project: Infinispan Issue Type: Enhancement Reporter: William Burns Assignee: Ryan Emerson Fix For: 9.0.0.Alpha3 A shared store's data is accessible from all nodes. Therefore we should only have to purge from 1 node instead of all. Thus we should only purge shared stores if the node is the coordinator. This will reduce possible load on the stores by an amount equal to the number of nodes in worst case, which could be huge. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

7 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-6522) Cannot use @DateBridge with WildFly modules: java.lang.NoClassDefFoundError: javax/xml/bind/DatatypeConverter

by Adrian Nistor (JIRA)

[ https://issues.jboss.org/browse/ISPN-6522?page=com.atlassian.jira.plugin.... ] Adrian Nistor updated ISPN-6522: -------------------------------- Status: Resolved (was: Pull Request Sent) Resolution: Done Integrated in master and 8.2.x. Thanks [~gustavonalle]! > Cannot use @DateBridge with WildFly modules: java.lang.NoClassDefFoundError: javax/xml/bind/DatatypeConverter > ------------------------------------------------------------------------------------------------------------- > > Key: ISPN-6522 > URL: https://issues.jboss.org/browse/ISPN-6522 > Project: Infinispan > Issue Type: Bug > Components: Embedded Querying, WildFly modules > Affects Versions: 8.2.1.Final, 9.0.0.Alpha1 > Reporter: Gustavo Fernandes > Assignee: Gustavo Fernandes > Priority: Critical > Fix For: 9.0.0.Alpha3, 8.2.3.Final > > > The Hibernate Search engine detects if the elasticsearch backend is on the classpath, > and if so, adds {{o.h.s.backend.elasticsearch.impl.ElasticsearchBridgeProvider}} to the top of the list of the annotation based bridge providers. > When processing an entity with a @DateBridge annotation, it picks the elasticsearch provider to converted from/to {{Date}} objects (since it has priority) which in turn fails with the exception: > {code} > org.infinispan.commons.CacheException: java.lang.NoClassDefFoundError: javax/xml/bind/DatatypeConverter > at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:198) > at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:363) > at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:351) > at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:93) > at org.hibernate.search.backend.elasticsearch.impl.ElasticsearchBridgeProvider$EsDateBridge.convertToString(ElasticsearchBridgeProvider.java:71) > at org.hibernate.search.backend.elasticsearch.impl.ElasticsearchBridgeProvider$EsDateBridge.set(ElasticsearchBridgeProvider.java:54) > at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:110) > at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFieldsForProperties(DocumentBuilderIndexedEntity.java:626) > {code} > > This exception happens only when using the infinispan modules, where the elasticsearch backend is added as a dependency; in embedded mode the aforementioned provider is not loaded since the elasticsearch is not on the classpath. > This current behaviour is not the best for two reasons: > * When using Wildlfy modules, the elasticsearch backend is marked as optional, but still, it's visible in the classpath and gets loaded > * If the user is not using the elasticsearch backend, but the jar is on the classpath, elasticsearch date conversion will take precedence over the built-in bridge providers, and it shouldn't -- This message was sent by Atlassian JIRA (v6.4.11#64026)

7 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-6699) Update to Hibernate Search 5.6.0.Beta1

by Adrian Nistor (JIRA)

[ https://issues.jboss.org/browse/ISPN-6699?page=com.atlassian.jira.plugin.... ] Adrian Nistor updated ISPN-6699: -------------------------------- Status: Resolved (was: Pull Request Sent) Resolution: Done Integrated in master and 8.2.x. Thanks [~gustavonalle]! > Update to Hibernate Search 5.6.0.Beta1 > -------------------------------------- > > Key: ISPN-6699 > URL: https://issues.jboss.org/browse/ISPN-6699 > Project: Infinispan > Issue Type: Component Upgrade > Reporter: Gustavo Fernandes > Assignee: Gustavo Fernandes > Fix For: 9.0.0.Alpha3, 8.2.3.Final > > -- This message was sent by Atlassian JIRA (v6.4.11#64026)

7 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-6710) TableManager when creating a table should also add index for expiration column

by William Burns (JIRA)

William Burns created ISPN-6710: ----------------------------------- Summary: TableManager when creating a table should also add index for expiration column Key: ISPN-6710 URL: https://issues.jboss.org/browse/ISPN-6710 Project: Infinispan Issue Type: Enhancement Components: Loaders and Stores Reporter: William Burns Assignee: Ryan Emerson Fix For: 9.0.0.Alpha3 All stores are required to have purging enabled now. This can cause performance issues with stores like Jdbc*Store since it may cause a full table scan without an index. We should make sure these tables have an index on this field to prevent possible performance issues. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

7 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-6706) Purging cache writers is [mostly] redundant when eviction is disabled and preload is enabled

by William Burns (JIRA)

[ https://issues.jboss.org/browse/ISPN-6706?page=com.atlassian.jira.plugin.... ] William Burns commented on ISPN-6706: ------------------------------------- Unfortunately due to consistency we have to make sure the in memory container and store are updated at the same time, so we can't do a DELETE to remove only the expired entries in one shot. They are removed 1 by 1, but purge is not a time sensitive operation so it should be fine. But having the index shouldn't cause the DB to read the entire table from disk which hopefully is the main cause of killing your DB response times. Also the purge should return much faster in general, limiting its effect on other operations. > Purging cache writers is [mostly] redundant when eviction is disabled and preload is enabled > -------------------------------------------------------------------------------------------- > > Key: ISPN-6706 > URL: https://issues.jboss.org/browse/ISPN-6706 > Project: Infinispan > Issue Type: Enhancement > Components: Loaders and Stores > Affects Versions: 8.2.2.Final > Reporter: Krzysztof Sobolewski > > This issue arised when I was testing a cluster with about 16 million entries. Our configuration is that all the data is also kept in memory, so eviction is disabled in this cache. But expiration is enabled. During the test I noticed pauses that started small but increased while the test was progressing, reaching more than 20 seconds at one point. After ruling out maintenance tasks in MySQL that could interfere, I discovered that the pause is caused by the expiration thread purging the database for expired entries. This was a huge and unnecessary drag so I hacked Infinispan to skip the purge of persistent state in cases when it's likely to be redundant with purging the transient state. I say "likely" because entries evicted maually via the evict() call poke a huge hole in the underlying assumptions :) Anyway, our cluster no longer regularly pauses for half a minute, so here's something for your consideration. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

7 years, 11 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues May 2016