[Hawkular-dev] Reducing foot print of embedded Cassandra

[John Sanda]

> On Apr 12, 2016, at 3:33 PM, John Sanda <jsanda at redhat.com> wrote:
> 
> Earlier Heiko asked me if there are any changes we can make to reduce the foot print of the embedded Cassandra instance we use for development and testing. There are several configuration changes that we can make which could be helpful. I did a quick check and it looks like the hawkular server is configured with a 512 MB heap. Some recommended values are based on that assumption.
> 
> * disable hinted handoff
> There is no need to run hinted handoff with a single node. Set the hinted_handoff_enabled property in cassandra.yaml to false.
> 
> * disable key and counter caches
> This may decrease read performance but should reduce Cassandra’s overall memory usage. Set the key_cache_size_in_mb and counter_cache_size_mb properties in cassandra.yaml to 0.
> 
> * reduce the concurrent reads/writes
> This will reduce the number of threads that Cassandra uses. Set the concurrent_reads, concurrent_writes, and concurrent_counter_writes properties in cassandra.yaml to 2.
> 
> * reduce file cache size
> This is off-heap memory used for pooling SSTable buffers. For a 512 MB heap it will be 128 MB. Change the file_cache_size_in_mb property to 64.
> 
> * reduce memory used for memtables
> This will increase the number of flushes which should be fine for write-heavy workloads. We probably don’t want to make this too small for read-heavy scenarios. Change the memtable_heap_space_in_mb and memtable_offheap_space_in_mb properties to 64.
> 
> * reduce memory for index summaries
> This defaults to a little over 25 MB. Change the index_summary_capcaity_in_mb property to 13.
> 
> * reduce the native transport threads
> The native transport server will use a max of 128 threads by default. Change the native_transport_max_threads property to 32.
> 
> * reduce thrift server threads
> Change the rpc_min_threads property to 8 and pc_max_threads to 512.
> 
> * tune compaction
> Set concurrent_compactors to 1.
> 
> * disable gossip
> No need to run gossip with a single node. Since Cassandra runs in its own deployment, the easiest way to do this is via JMX using the stopGossiping() method on StorageServiceMBean. I need to do some testing with this to see how it works. It looks like the call has to happen after some initialization has completed.
> 
> * disable compression
> Compression is enabled by default and is configured per table. It reduces disk space but increases CPU utilization.
> 
> ALTER TABLE my_table WITH compression = { ’sstable_compression’: ‘’ };
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev <https://lists.jboss.org/mailman/listinfo/hawkular-dev>

There is something else I want to point out that is not related specifically to the use of embedded Cassandra. Each hawkular component uses its own, separate driver instance. This is a big no, no. We should be using a shared instance per physical cluster. I think Juca may have looked into a resource adapter at one point, but I am not sure the outcome. Might be good to revisit.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160414/cb27176a/attachment.html