November 2019 - infinispan-issues

[JBoss JIRA] (ISPN-8241) Refactor RocksDB clearThreshold

by Diego Lovison (Jira)

[ https://issues.jboss.org/browse/ISPN-8241?page=com.atlassian.jira.plugin.... ] Diego Lovison edited comment on ISPN-8241 at 11/5/19 3:03 PM: -------------------------------------------------------------- [~william.burns] [~ryanemerson] could we do the following? Create a random sub-path for dataLocation and expirationLocation when starting the server. Once the server is started, call java.nio.file.Files::delete and log what were the files that we were not able to remove. In this case, the sysAdmin can have the system up and running faster and deal with unused files later. was (Author: dlovison): [~william.burns] [~ryanemerson] could we do the following? Create a random sub-path for dataLocation and expirationLocation when starting the server. Once the server is started, call java.nio.file.Files::delete and log what were the files that we were not able to remove. In this case, the sysAdmin call have the system up and running faster and deal with unused files later. > Refactor RocksDB clearThreshold > ------------------------------- > > Key: ISPN-8241 > URL: https://issues.jboss.org/browse/ISPN-8241 > Project: Infinispan > Issue Type: Sub-task > Components: Loaders and Stores > Affects Versions: 9.1.0.Final > Reporter: Ryan Emerson > Assignee: Diego Lovison > Priority: Major > Fix For: 10.1.0.Final > > > Currently the RocksDB store utilises a "clearThreshold" to try to delete entries individually before deleting and re-initiating the database. We should deprecate this threshold and always delete/reinit the database. > Currently when deleting the database, we utilise Util.recursiveFileRemove which does not confirm that the file has actually been deleted. Instead, we should provide a nio based implementation instead, similar to the one stated [here|https://stackoverflow.com/questions/779519/delete-directories-recurs...]. This has the advantage that an IOException is thrown by java.nio.file.Files::delete -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-8241) Refactor RocksDB clearThreshold

by Diego Lovison (Jira)

[ https://issues.jboss.org/browse/ISPN-8241?page=com.atlassian.jira.plugin.... ] Diego Lovison edited comment on ISPN-8241 at 11/5/19 3:00 PM: -------------------------------------------------------------- [~william.burns] [~ryanemerson] could we do the following? Create a random sub-path for dataLocation and expirationLocation when starting the server. Once the server is started, call java.nio.file.Files::delete and log what were the files that we were not able to remove. In this case, the sysAdmin call have the system up and running faster and deal with unused files later. was (Author: dlovison): [~william.burns][~ryanemerson] could we do the following? Create a random sub-path for dataLocation and expirationLocation when starting the server. Once the server is started, call java.nio.file.Files::delete and log what were the files that we were not able to remove. In this case, the sysAdmin call have the system up and running faster and deal with unused files later. > Refactor RocksDB clearThreshold > ------------------------------- > > Key: ISPN-8241 > URL: https://issues.jboss.org/browse/ISPN-8241 > Project: Infinispan > Issue Type: Sub-task > Components: Loaders and Stores > Affects Versions: 9.1.0.Final > Reporter: Ryan Emerson > Assignee: Diego Lovison > Priority: Major > Fix For: 10.1.0.Final > > > Currently the RocksDB store utilises a "clearThreshold" to try to delete entries individually before deleting and re-initiating the database. We should deprecate this threshold and always delete/reinit the database. > Currently when deleting the database, we utilise Util.recursiveFileRemove which does not confirm that the file has actually been deleted. Instead, we should provide a nio based implementation instead, similar to the one stated [here|https://stackoverflow.com/questions/779519/delete-directories-recurs...]. This has the advantage that an IOException is thrown by java.nio.file.Files::delete -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-8241) Refactor RocksDB clearThreshold

by Diego Lovison (Jira)

[ https://issues.jboss.org/browse/ISPN-8241?page=com.atlassian.jira.plugin.... ] Work on ISPN-8241 started by Diego Lovison. ------------------------------------------- > Refactor RocksDB clearThreshold > ------------------------------- > > Key: ISPN-8241 > URL: https://issues.jboss.org/browse/ISPN-8241 > Project: Infinispan > Issue Type: Sub-task > Components: Loaders and Stores > Affects Versions: 9.1.0.Final > Reporter: Ryan Emerson > Assignee: Diego Lovison > Priority: Major > Fix For: 10.1.0.Final > > > Currently the RocksDB store utilises a "clearThreshold" to try to delete entries individually before deleting and re-initiating the database. We should deprecate this threshold and always delete/reinit the database. > Currently when deleting the database, we utilise Util.recursiveFileRemove which does not confirm that the file has actually been deleted. Instead, we should provide a nio based implementation instead, similar to the one stated [here|https://stackoverflow.com/questions/779519/delete-directories-recurs...]. This has the advantage that an IOException is thrown by java.nio.file.Files::delete -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-8241) Refactor RocksDB clearThreshold

by Diego Lovison (Jira)

[ https://issues.jboss.org/browse/ISPN-8241?page=com.atlassian.jira.plugin.... ] Diego Lovison updated ISPN-8241: -------------------------------- Status: Open (was: New) > Refactor RocksDB clearThreshold > ------------------------------- > > Key: ISPN-8241 > URL: https://issues.jboss.org/browse/ISPN-8241 > Project: Infinispan > Issue Type: Sub-task > Components: Loaders and Stores > Affects Versions: 9.1.0.Final > Reporter: Ryan Emerson > Assignee: Ryan Emerson > Priority: Major > Fix For: 10.1.0.Final > > > Currently the RocksDB store utilises a "clearThreshold" to try to delete entries individually before deleting and re-initiating the database. We should deprecate this threshold and always delete/reinit the database. > Currently when deleting the database, we utilise Util.recursiveFileRemove which does not confirm that the file has actually been deleted. Instead, we should provide a nio based implementation instead, similar to the one stated [here|https://stackoverflow.com/questions/779519/delete-directories-recurs...]. This has the advantage that an IOException is thrown by java.nio.file.Files::delete -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-8241) Refactor RocksDB clearThreshold

by Diego Lovison (Jira)

[ https://issues.jboss.org/browse/ISPN-8241?page=com.atlassian.jira.plugin.... ] Diego Lovison commented on ISPN-8241: ------------------------------------- [~william.burns][~ryanemerson] could we do the following? Create a random sub-path for dataLocation and expirationLocation when starting the server. Once the server is started, call java.nio.file.Files::delete and log what were the files that we were not able to remove. In this case, the sysAdmin call have the system up and running faster and deal with unused files later. > Refactor RocksDB clearThreshold > ------------------------------- > > Key: ISPN-8241 > URL: https://issues.jboss.org/browse/ISPN-8241 > Project: Infinispan > Issue Type: Sub-task > Components: Loaders and Stores > Affects Versions: 9.1.0.Final > Reporter: Ryan Emerson > Assignee: Ryan Emerson > Priority: Major > Fix For: 10.1.0.Final > > > Currently the RocksDB store utilises a "clearThreshold" to try to delete entries individually before deleting and re-initiating the database. We should deprecate this threshold and always delete/reinit the database. > Currently when deleting the database, we utilise Util.recursiveFileRemove which does not confirm that the file has actually been deleted. Instead, we should provide a nio based implementation instead, similar to the one stated [here|https://stackoverflow.com/questions/779519/delete-directories-recurs...]. This has the advantage that an IOException is thrown by java.nio.file.Files::delete -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-8199) Limit RocksDB cache size

by Diego Lovison (Jira)

[ https://issues.jboss.org/browse/ISPN-8199?page=com.atlassian.jira.plugin.... ] Work on ISPN-8199 started by Diego Lovison. ------------------------------------------- > Limit RocksDB cache size > ------------------------ > > Key: ISPN-8199 > URL: https://issues.jboss.org/browse/ISPN-8199 > Project: Infinispan > Issue Type: Sub-task > Components: Loaders and Stores > Affects Versions: 9.1.0.Final > Reporter: Ryan Emerson > Assignee: Diego Lovison > Priority: Major > > Look into https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-8199) Limit RocksDB cache size

by Diego Lovison (Jira)

[ https://issues.jboss.org/browse/ISPN-8199?page=com.atlassian.jira.plugin.... ] Diego Lovison updated ISPN-8199: -------------------------------- Status: Open (was: New) > Limit RocksDB cache size > ------------------------ > > Key: ISPN-8199 > URL: https://issues.jboss.org/browse/ISPN-8199 > Project: Infinispan > Issue Type: Sub-task > Components: Loaders and Stores > Affects Versions: 9.1.0.Final > Reporter: Ryan Emerson > Assignee: Ryan Emerson > Priority: Major > > Look into https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-7912) Prevent RocksDBStore writes blocking on full expiry queue

by Diego Lovison (Jira)

[ https://issues.jboss.org/browse/ISPN-7912?page=com.atlassian.jira.plugin.... ] Diego Lovison updated ISPN-7912: -------------------------------- Status: Pull Request Sent (was: Coding In Progress) Git Pull Request: https://github.com/infinispan/infinispan/pull/7543/files > Prevent RocksDBStore writes blocking on full expiry queue > --------------------------------------------------------- > > Key: ISPN-7912 > URL: https://issues.jboss.org/browse/ISPN-7912 > Project: Infinispan > Issue Type: Sub-task > Components: Loaders and Stores > Affects Versions: 9.1.0.Alpha1 > Reporter: Ryan Emerson > Assignee: Diego Lovison > Priority: Major > Fix For: 10.1.0.Final > > > Currently you can only insert 10000 elements into the rocks db store until you will block a thread until the expiration reaper is ran. Instead we should offer elements to the queue and upon failure utilise the persistence executors to run purge. -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-10908) Client timeout Exception should log the connection additional to the key

by Wolf-Dieter Fink (Jira)

Wolf-Dieter Fink created ISPN-10908: --------------------------------------- Summary: Client timeout Exception should log the connection additional to the key Key: ISPN-10908 URL: https://issues.jboss.org/browse/ISPN-10908 Project: Infinispan Issue Type: Enhancement Components: Hot Rod Affects Versions: 10.0.0.Final Environment: HotRod client Reporter: Wolf-Dieter Fink Assignee: Will Burns In case of communication timeout with the server the client should log the connection details as the key will not give enough to analyse the issue. ~~~ ERROR [stderr] (pool-12-thread-3) org.infinispan.client.hotrod.exceptions.TransportException:: java.net.SocketTimeoutException: GetOperation{TestCache, key=[B0x800127656A6B2F6C..[103], flags=0} timed out after 60000 ms ERROR [stderr] (pool-12-thread-3) at org.infinispan.client.hotrod.impl.Util.rewrap(Util.java:54) ERROR [stderr] (pool-12-thread-3) at org.infinispan.client.hotrod.impl.Util.await(Util.java:27) ERROR [stderr] (pool-12-thread-3) at org.infinispan.client.hotrod.impl.RemoteCacheImpl.get(RemoteCacheImpl.java:418) ... ERROR [stderr] (pool-12-thread-3) Caused by: java.net.SocketTimeoutException: GetOperation{TestCache, key=[B0x800127656A6B2F6C..[103], flags=0} timed out after 60000 ms ERROR [stderr] (pool-12-thread-3) at org.infinispan.client.hotrod.impl.operations.HotRodOperation.run(HotRodOperation.java:172) ERROR [stderr] (pool-12-thread-3) at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) ERROR [stderr] (pool-12-thread-3) at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) ERROR [stderr] (pool-12-thread-3) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ERROR [stderr] (pool-12-thread-3) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) ERROR [stderr] (pool-12-thread-3) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:322) ERROR [stderr] (pool-12-thread-3) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) ERROR [stderr] (pool-12-thread-3) ... 3 more ~~~ -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 11 months

1
0
0 / 0

[JBoss JIRA] (ISPN-7912) Prevent RocksDBStore writes blocking on full expiry queue

by Diego Lovison (Jira)

[ https://issues.jboss.org/browse/ISPN-7912?page=com.atlassian.jira.plugin.... ] Diego Lovison updated ISPN-7912: -------------------------------- Status: Open (was: New) > Prevent RocksDBStore writes blocking on full expiry queue > --------------------------------------------------------- > > Key: ISPN-7912 > URL: https://issues.jboss.org/browse/ISPN-7912 > Project: Infinispan > Issue Type: Sub-task > Components: Loaders and Stores > Affects Versions: 9.1.0.Alpha1 > Reporter: Ryan Emerson > Assignee: Ryan Emerson > Priority: Major > Fix For: 10.1.0.Final > > > Currently you can only insert 10000 elements into the rocks db store until you will block a thread until the expiration reaper is ran. Instead we should offer elements to the queue and upon failure utilise the persistence executors to run purge. -- This message was sent by Atlassian Jira (v7.13.8#713008)

4 years, 11 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues November 2019