January 2014 - infinispan-issues - Jboss List Archives

[JBoss JIRA] (ISPN-2156) Benchmark and blog about a fast method of loading data into Infinispan

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2156?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2156: -------------------------------- Component/s: Core > Benchmark and blog about a fast method of loading data into Infinispan > ------------------------------------------------------------------------ > > Key: ISPN-2156 > URL: https://issues.jboss.org/browse/ISPN-2156 > Project: Infinispan > Issue Type: Task > Components: Core > Reporter: Mircea Markus > Assignee: Vladimir Blagojevic > Labels: docs > > To summarise: > When using distributed caches, when we need to batch-load a set of data into the cluster inserting batches of keys that map to the same node should significantly increase the performance. > Why? > during the prepare phase each node receives the > complete list of modifications in that transaction and not only the > modification pertaining to it. > E.g. say we have the following key->node mapping: > {code} > k1 -> A > k2 -> B > k3 -> C > {code} > Where k1, k2 and k3 are keys; A, B and C are nodes. > If Tx1 writes (k1,k2,k3) then during the prepare A,B and C will receive > the the same package containing all the modification - namely (k1, > k2,k3). There are several reasons for doing this (apparently) > unoptimized approach: serialize the prepare only once, better handling > of recovery information. > Now if you group transactions/batches base on key distribution the amount of redundant traffic is significantly reduced - and that translates in better performance especially when the datasets > you're inserting is quite high. > This JIRA is basically about benchmarking and blogging about this approach. > A entry in the FAQ would be helpful as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 2 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2156) Benchmark and blog about a fast method of loading data into Infinispan

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2156?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2156: -------------------------------- Component/s: Documentation-Core > Benchmark and blog about a fast method of loading data into Infinispan > ------------------------------------------------------------------------ > > Key: ISPN-2156 > URL: https://issues.jboss.org/browse/ISPN-2156 > Project: Infinispan > Issue Type: Task > Components: Core, Documentation-Core > Reporter: Mircea Markus > Assignee: Vladimir Blagojevic > Labels: docs > > To summarise: > When using distributed caches, when we need to batch-load a set of data into the cluster inserting batches of keys that map to the same node should significantly increase the performance. > Why? > during the prepare phase each node receives the > complete list of modifications in that transaction and not only the > modification pertaining to it. > E.g. say we have the following key->node mapping: > {code} > k1 -> A > k2 -> B > k3 -> C > {code} > Where k1, k2 and k3 are keys; A, B and C are nodes. > If Tx1 writes (k1,k2,k3) then during the prepare A,B and C will receive > the the same package containing all the modification - namely (k1, > k2,k3). There are several reasons for doing this (apparently) > unoptimized approach: serialize the prepare only once, better handling > of recovery information. > Now if you group transactions/batches base on key distribution the amount of redundant traffic is significantly reduced - and that translates in better performance especially when the datasets > you're inserting is quite high. > This JIRA is basically about benchmarking and blogging about this approach. > A entry in the FAQ would be helpful as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 2 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2946) Apache Hive Implementation

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2946?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2946: -------------------------------- Component/s: Core > Apache Hive Implementation > -------------------------- > > Key: ISPN-2946 > URL: https://issues.jboss.org/browse/ISPN-2946 > Project: Infinispan > Issue Type: Feature Request > Components: Core > Reporter: Shane Johnson > Assignee: Mircea Markus > Priority: Minor > > Create an Apache Hive Implementation on top of ISPN. This would allow users to execute Apache Hive queries on top of ISPN. An example is Shark. It is an Apache Hive implementation on to top of Spark. > http://shark.cs.berkeley.edu/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 2 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2943) Apache Flume NG Cache Store

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2943?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2943: -------------------------------- Priority: Minor (was: Major) > Apache Flume NG Cache Store > --------------------------- > > Key: ISPN-2943 > URL: https://issues.jboss.org/browse/ISPN-2943 > Project: Infinispan > Issue Type: Feature Request > Components: Loaders and Stores > Reporter: Shane Johnson > Assignee: Manik Surtani > Priority: Minor > > Create an Apache Flume NG cache store by creating an ISPN Apache Flume NG source. The ISPN cache store then sends data via events to Apache Flume NG. This might be a nice way to abstract the persistence as Apache Flume NG can then send the event to any number of sinks (e.g. Apache HDFS). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 2 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2946) Apache Hive Implementation

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2946?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2946: -------------------------------- Priority: Minor (was: Major) > Apache Hive Implementation > -------------------------- > > Key: ISPN-2946 > URL: https://issues.jboss.org/browse/ISPN-2946 > Project: Infinispan > Issue Type: Feature Request > Reporter: Shane Johnson > Assignee: Mircea Markus > Priority: Minor > > Create an Apache Hive Implementation on top of ISPN. This would allow users to execute Apache Hive queries on top of ISPN. An example is Shark. It is an Apache Hive implementation on to top of Spark. > http://shark.cs.berkeley.edu/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 2 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2944) Simple Storm Bolt

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2944?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2944: -------------------------------- Labels: hackathon (was: ) > Simple Storm Bolt > ----------------- > > Key: ISPN-2944 > URL: https://issues.jboss.org/browse/ISPN-2944 > Project: Infinispan > Issue Type: Feature Request > Components: Core > Reporter: Shane Johnson > Assignee: Manik Surtani > Labels: hackathon > > Create a simple ISPN Storm bolt. There are currently simply bolts for Apache Cassandra, MongoDB, Apache HBase, and more. > https://github.com/nathanmarz/storm-contrib > Tighter integration with Big Data tools allows for hybrid architectures with ISPN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 2 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2945) Log Structured Hash Table Cache Store

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2945?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2945: -------------------------------- Priority: Minor (was: Major) > Log Structured Hash Table Cache Store > ------------------------------------- > > Key: ISPN-2945 > URL: https://issues.jboss.org/browse/ISPN-2945 > Project: Infinispan > Issue Type: Feature Request > Reporter: Shane Johnson > Assignee: Manik Surtani > Priority: Minor > > Create a cache store based on a log structured hash table a la Bitcask to provide durability for NoSQL. > http://docs.basho.com/riak/1.3.0/tutorials/choosing-a-backend/Bitcask/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 2 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2945) Log Structured Hash Table Cache Store

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2945?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2945: -------------------------------- Component/s: Core > Log Structured Hash Table Cache Store > ------------------------------------- > > Key: ISPN-2945 > URL: https://issues.jboss.org/browse/ISPN-2945 > Project: Infinispan > Issue Type: Feature Request > Components: Core > Reporter: Shane Johnson > Assignee: Manik Surtani > Priority: Minor > > Create a cache store based on a log structured hash table a la Bitcask to provide durability for NoSQL. > http://docs.basho.com/riak/1.3.0/tutorials/choosing-a-backend/Bitcask/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 2 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2944) Simple Storm Bolt

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2944?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2944: -------------------------------- Component/s: Core > Simple Storm Bolt > ----------------- > > Key: ISPN-2944 > URL: https://issues.jboss.org/browse/ISPN-2944 > Project: Infinispan > Issue Type: Feature Request > Components: Core > Reporter: Shane Johnson > Assignee: Manik Surtani > > Create a simple ISPN Storm bolt. There are currently simply bolts for Apache Cassandra, MongoDB, Apache HBase, and more. > https://github.com/nathanmarz/storm-contrib > Tighter integration with Big Data tools allows for hybrid architectures with ISPN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 2 months

1
0
0 / 0

[JBoss JIRA] (ISPN-2945) Log Structured Hash Table Cache Store

by Mircea Markus (JIRA)

[ https://issues.jboss.org/browse/ISPN-2945?page=com.atlassian.jira.plugin.... ] Mircea Markus updated ISPN-2945: -------------------------------- Component/s: Loaders and Stores (was: Core) > Log Structured Hash Table Cache Store > ------------------------------------- > > Key: ISPN-2945 > URL: https://issues.jboss.org/browse/ISPN-2945 > Project: Infinispan > Issue Type: Feature Request > Components: Loaders and Stores > Reporter: Shane Johnson > Assignee: Manik Surtani > Priority: Minor > > Create a cache store based on a log structured hash table a la Bitcask to provide durability for NoSQL. > http://docs.basho.com/riak/1.3.0/tutorials/choosing-a-backend/Bitcask/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 2 months

1
0
0 / 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues January 2014