July 2013 - infinispan-issues - Jboss List Archives

[JBoss JIRA] (ISPN-2156) Benchmark and blog about a fast method of loading data into Infinispan

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2156?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2156: -------------------------------- Fix Version/s: (was: 6.0.0.Final) > Benchmark and blog about a fast method of loading data into Infinispan > ------------------------------------------------------------------------ > > Key: ISPN-2156 > URL: https://issues.jboss.org/browse/ISPN-2156 > Project: Infinispan > Issue Type: Task > Reporter: Mircea Markus > Assignee: Vladimir Blagojevic > Labels: docs > > To summarise: > When using distributed caches, when we need to batch-load a set of data into the cluster inserting batches of keys that map to the same node should significantly increase the performance. > Why? > during the prepare phase each node receives the > complete list of modifications in that transaction and not only the > modification pertaining to it. > E.g. say we have the following key->node mapping: > {code} > k1 -> A > k2 -> B > k3 -> C > {code} > Where k1, k2 and k3 are keys; A, B and C are nodes. > If Tx1 writes (k1,k2,k3) then during the prepare A,B and C will receive > the the same package containing all the modification - namely (k1, > k2,k3). There are several reasons for doing this (apparently) > unoptimized approach: serialize the prepare only once, better handling > of recovery information. > Now if you group transactions/batches base on key distribution the amount of redundant traffic is significantly reduced - and that translates in better performance especially when the datasets > you're inserting is quite high. > This JIRA is basically about benchmarking and blogging about this approach. > A entry in the FAQ would be helpful as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-188) Benchmark Infinispan cache provider

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-188?page=com.atlassian.jira.plugin.s... ] Mircea Markus resolved ISPN-188. -------------------------------- Fix Version/s: (was: 6.0.0.Final) Resolution: Won't Fix Tuning ISPN as a 2nd level cache has been done as part of the SPEC EE benchmarks. > Benchmark Infinispan cache provider > ----------------------------------- > > Key: ISPN-188 > URL: https://issues.jboss.org/browse/ISPN-188 > Project: Infinispan > Issue Type: Sub-task > Reporter: Galder Zamarreño > Assignee: Dan Berindei > Priority: Minor > > We have an EJB3 app that stresses 2nd level cache: http://anonsvn.jboss.org/repos/qa/ejb3demo/ > More work will go in the near future into this app, and so it's worth waiting for the outcome of that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-328) Async store to support frequency based flushing

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-328?page=com.atlassian.jira.plugin.s... ] Mircea Markus updated ISPN-328: ------------------------------- Fix Version/s: (was: 6.0.0.Final) > Async store to support frequency based flushing > ----------------------------------------------- > > Key: ISPN-328 > URL: https://issues.jboss.org/browse/ISPN-328 > Project: Infinispan > Issue Type: Feature Request > Components: Loaders and Stores > Reporter: Galder Zamarreño > Assignee: Galder Zamarreño > Labels: hackathon > > Implement frequency based flushing in the async store. This is also called, Scheduled Write-Behing Strategy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-1439) Async store needs redesigning to support injected executors

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-1439?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-1439: -------------------------------- Fix Version/s: (was: 6.0.0.Final) > Async store needs redesigning to support injected executors > ----------------------------------------------------------- > > Key: ISPN-1439 > URL: https://issues.jboss.org/browse/ISPN-1439 > Project: Infinispan > Issue Type: Enhancement > Components: Configuration, Loaders and Stores > Reporter: Galder Zamarreño > Assignee: Galder Zamarreño > > The CoalescedAsyncStore design is pretty complex and is designed around AsyncStore being able to control the lifecycle of the async store coordinator and processor executors. > A re-design of the async store is needed to enable central management of these executors, in such way that the NamedExecutorFactory can start with the right parameters (take in account that multiple caches could be configured with an async store), and the shutdown procedure can be correctly executed by the cache manager. > The async store's coalesced logic makes this a fairly complex task, particularly since the introduction of Sanne's changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2955) Async marshalling executor retry when queue fills

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2955?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2955: -------------------------------- Fix Version/s: (was: 6.0.0.Alpha1) > Async marshalling executor retry when queue fills > ------------------------------------------------- > > Key: ISPN-2955 > URL: https://issues.jboss.org/browse/ISPN-2955 > Project: Infinispan > Issue Type: Enhancement > Components: Marshalling > Affects Versions: 5.2.5.Final > Reporter: Manik Surtani > Assignee: Manik Surtani > > When using an async transport and async marshalling, an executor is used to process the marshalling task in a separate thread and the caller's thread is allowed to return immediately. > When the executor's queue fills and the queue cannot accept any more tasks, it throws a {{RejectedExecutionException}}, causing a very bad user/developer experience. A more correct approach to this is to catch the {{RejectedExecutionException}}, block and retry the task submission. > The end result is that, in the degenerate case (when the executor queue is full) instead of throwing exceptions, those invocations will perform slightly slower. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2259) Async store is too slow

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2259?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2259: -------------------------------- Fix Version/s: (was: 6.0.0.Final) > Async store is too slow > ----------------------- > > Key: ISPN-2259 > URL: https://issues.jboss.org/browse/ISPN-2259 > Project: Infinispan > Issue Type: Enhancement > Components: Loaders and Stores > Affects Versions: 5.1.6.FINAL > Environment: Windows XP, Java 7 > Reporter: Radim Kolář > Assignee: Galder Zamarreño > Labels: performance > Attachments: profiler.png, testcase.zip > > > Async support for loaders is too slow. I am writing 1000 items into cache backed by jdbm with modification queue size 10000 and it takes 6 seconds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2280) Async cache store shutdown timeout default value should be -1

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2280?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2280: -------------------------------- Fix Version/s: (was: 6.0.0.Final) > Async cache store shutdown timeout default value should be -1 > ------------------------------------------------------------- > > Key: ISPN-2280 > URL: https://issues.jboss.org/browse/ISPN-2280 > Project: Infinispan > Issue Type: Feature Request > Components: Configuration, Loaders and Stores > Reporter: Galder Zamarreño > Assignee: Galder Zamarreño > Priority: Minor > > We should strive for correctness really. > Better to have someone complain about Infinispan not finishing rather than missing data in the cache store. > To avoid confusion, we should wait for a bit of time, i.e. 1 minute, and then if the async store has not finished yet, log a WARN message indicating what's up, but then carry on waiting forever (or whatever the defined user time - minus the 60 seconds) by default. > If the user defined timeout is less than 60 seconds, wait that long. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2280) Async cache store shutdown timeout default value should be -1

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2280?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2280: -------------------------------- Priority: Minor (was: Major) > Async cache store shutdown timeout default value should be -1 > ------------------------------------------------------------- > > Key: ISPN-2280 > URL: https://issues.jboss.org/browse/ISPN-2280 > Project: Infinispan > Issue Type: Feature Request > Components: Configuration, Loaders and Stores > Reporter: Galder Zamarreño > Assignee: Galder Zamarreño > Priority: Minor > Fix For: 6.0.0.Final > > > We should strive for correctness really. > Better to have someone complain about Infinispan not finishing rather than missing data in the cache store. > To avoid confusion, we should wait for a bit of time, i.e. 1 minute, and then if the async store has not finished yet, log a WARN message indicating what's up, but then carry on waiting forever (or whatever the defined user time - minus the 60 seconds) by default. > If the user defined timeout is less than 60 seconds, wait that long. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3102) Analyse the performance of MassIndexingTest

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-3102?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-3102: -------------------------------- Assignee: Adrian Nistor (was: Mircea Markus) > Analyse the performance of MassIndexingTest > ------------------------------------------- > > Key: ISPN-3102 > URL: https://issues.jboss.org/browse/ISPN-3102 > Project: Infinispan > Issue Type: Task > Reporter: Mircea Markus > Assignee: Adrian Nistor > Priority: Blocker > Fix For: 6.0.0.Alpha1 > > > MassIndexingTest runs on CI in 10 mins with TRACE on and 3 mins with TRACE off. According to Sanne this shouldn't take that long: > {quote} > sannegrinovero > 11:50 mmarkus: the org.infinispan.query.distributed.MassIndexingTest test is adding 2000 entries to the cache. True that might be a bit excessive, still if it takes 10 minutes you have a problem in Infinispan core. > mmarkus > 11:52 sannegrinovero: each such write is indexed I take it? > sannegrinovero > 11:53 mmarkus: no: withFlags(Flag.SKIP_INDEXING) > 11:54 mmarkus: but it's a 4 node DIST test using Map/Reduce. > sannegrinovero > 11:54 then the Map/Reduce job does some indexing but the indexing engine didn't change overnight > {quote} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 9 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3102) Analyse the performance of MassIndexingTest

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-3102?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-3102: -------------------------------- Fix Version/s: 6.0.0.Final (was: 6.0.0.Alpha1) > Analyse the performance of MassIndexingTest > ------------------------------------------- > > Key: ISPN-3102 > URL: https://issues.jboss.org/browse/ISPN-3102 > Project: Infinispan > Issue Type: Task > Reporter: Mircea Markus > Assignee: Adrian Nistor > Priority: Blocker > Fix For: 6.0.0.Final > > > MassIndexingTest runs on CI in 10 mins with TRACE on and 3 mins with TRACE off. According to Sanne this shouldn't take that long: > {quote} > sannegrinovero > 11:50 mmarkus: the org.infinispan.query.distributed.MassIndexingTest test is adding 2000 entries to the cache. True that might be a bit excessive, still if it takes 10 minutes you have a problem in Infinispan core. > mmarkus > 11:52 sannegrinovero: each such write is indexed I take it? > sannegrinovero > 11:53 mmarkus: no: withFlags(Flag.SKIP_INDEXING) > 11:54 mmarkus: but it's a 4 node DIST test using Map/Reduce. > sannegrinovero > 11:54 then the Map/Reduce job does some indexing but the indexing engine didn't change overnight > {quote} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 9 months

1
0
0 / 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues July 2013