Hi,
I managed to delete that data already, so here comes another calculation based on the real
values (not theoretical). These come from Cassandra 2.2.8 and the dataset is Gauges of
idling MiQ instance Cassandra data..
2168597 values in 1528 keys:
LZ4: 54331865 bytes -> 25.05 bytes per value
Deflate: 28491886 bytes -> 13.14 bytes per value
LZ4 + Gorilla: 1574659 bytes -> 0.726 bytes per value
The last one is too good because the dataset includes a lot of gauges that have
"0" as all of their values. In the dataset the Gorilla compression's worst
cases are around 5 bytes per value, but as you can see, the 0 valued measurements
considerably improve the compression ratio. How realistic measurements these idlings are
.. I don't know.
Note: Cassandra 3.8 improves these values, especially with LZ4:
LZ4 -> ~15 bytes per value
Deflate: ~10 bytes per value
In this Openshift example, with just Cassandra upgrade + Deflate we would have improved 25
-> 10.
- Micke
----- Original Message -----
From: "Heiko W.Rupp" <hrupp(a)redhat.com>
To: "Discussions around Hawkular development"
<hawkular-dev(a)lists.jboss.org>
Sent: Thursday, July 28, 2016 4:56:04 PM
Subject: Re: [Hawkular-dev] Metrics storage usage and compression
On 27 Jul 2016, at 11:19, Michael Burman wrote:
When storing to our current data model, the disk space taken by
"data"
table was 74MB. My prototype uses the method of Facebook's Gorilla
paper (same as what for example Prometheus uses), and in this test I
used a one day block size (storing one metric's one day data to one
row inside Cassandra). The end result was 3,1MB of storage space used.
Code can be found from
bitbucket.org/burmanm/compress_proto (Golang).
I know Prometheus advertises estimated 1.3 bytes per timestamped
value, but those
How many bytes to we currently use per timestamped value?
_______________________________________________
hawkular-dev mailing list
hawkular-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hawkular-dev