[Hawkular-commits] [hawkular/hawkular-metrics] 7b5d80: ETL based compression methods in the core for Gaug...

Tuesday, 4 October 2016

  Branch: refs/heads/master
  Home:   https://github.com/hawkular/hawkular-metrics
  Commit: 7b5d806a7b5c441db5702d2a0f157f80bfd7eae9

https://github.com/hawkular/hawkular-metrics/commit/7b5d806a7b5c441db5702...
  Author: Michael Burman <yak(a)iki.fi&gt;
  Date:   2016-10-04 (Tue, 04 Oct 2016)

  Changed paths:
    M
api/metrics-api-jaxrs/src/main/java/org/hawkular/metrics/api/jaxrs/MetricsServiceLifecycle.java
    M
core/datetime-service/src/main/java/org/hawkular/metrics/datetime/DateTimeService.java
    M core/metrics-core-service/pom.xml
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/jobs/CompressData.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/jobs/JobsService.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/jobs/JobsServiceImpl.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/DataAccess.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/DataAccessImpl.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/MetricsService.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/MetricsServiceImpl.java
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/compress/CompressedPointContainer.java
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/compress/CompressorHeader.java
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/compress/TagsDeserializer.java
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/compress/TagsSerializer.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/transformers/BatchStatementTransformer.java
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/transformers/DataPointCompressTransformer.java
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/transformers/DataPointDecompressTransformer.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/transformers/MetricFromDataRowTransformer.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/transformers/MetricFromFullDataRowTransformer.java
    A
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/compress/CompressorHeaderTest.java
    A
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/compress/TagsMapSerializingTest.java
    A
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/jobs/CompressDataJobITest.java
    M
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/jobs/DeleteTenantITest.java
    M
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/service/DataAccessITest.java
    M
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/service/DelegatingDataAccess.java
    M
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/service/metrics/GaugeITest.java
    M core/metrics-model/src/main/java/org/hawkular/metrics/model/AvailabilityType.java
    M core/schema/src/main/java/org/hawkular/metrics/schema/SchemaService.java
    M core/schema/src/main/resources/org/hawkular/schema/cassalog.groovy
    A core/schema/src/main/resources/org/hawkular/schema/updates/schema-0.20.0.groovy
    M
integration-tests/rest-tests-jaxrs/src/test/groovy/org/hawkular/metrics/rest/RESTTest.groovy
    M job-scheduler/src/main/java/org/hawkular/metrics/scheduler/api/Scheduler.java
    M job-scheduler/src/main/java/org/hawkular/metrics/scheduler/impl/SchedulerImpl.java
    M job-scheduler/src/main/java/org/hawkular/metrics/scheduler/impl/TestScheduler.java
    M pom.xml

  Log Message:
  -----------
  ETL based compression methods in the core for Gauge data

Add token range queries for metric partition keys

Reorder DISTINCT queries to match the partition key (required in the tokens for example),
replace bind(Objects...) with bind().setXXX

Implement the CompressData job and the required changes to the MetricsService

Implement jobServices for ETL compress (not activated yet, needs rebase from master)

Add correct timing functions for the CompressDataJob and add the ability to fetch all jobs
from JobScheduler

Add compressor header to every compressed field

Move bindpoint changes to another PR

Fix trigger time calculation to match UTC

Rebased the TestScheduler changes and updated compression library to 1.1.0

Use range delete in addAndCompress, requires removing batch statement as that does not
preserve order

Add data_compressed table where all the compressed rows are stored in

Add ability to disable the compression job and remove data locality stuff for another
PR..

Add support for compressing COUNTER and AVAILABILITY also

Add Tags serialization/deserialization and schema changes for them

Change CompressData to use jobTriggerTime and the tests to force UTC

Enable findDataPoints to use compressedData for COUNTER & AVAILABILITY and fix
DataAccessITest to clean up compressed_table also before test

Improve CompressionHeader functionality

Enable tagsSerialization and test for that

  Commit: 45da8808ecebc478c9ccea4c15b504b1f229d4b6

https://github.com/hawkular/hawkular-metrics/commit/45da8808ecebc478c9cce...
  Author: jsanda <jsanda(a)redhat.com&gt;
  Date:   2016-10-04 (Tue, 04 Oct 2016)

  Changed paths:
    M
api/metrics-api-jaxrs/src/main/java/org/hawkular/metrics/api/jaxrs/MetricsServiceLifecycle.java
    M
core/datetime-service/src/main/java/org/hawkular/metrics/datetime/DateTimeService.java
    M core/metrics-core-service/pom.xml
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/jobs/CompressData.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/jobs/JobsService.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/jobs/JobsServiceImpl.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/DataAccess.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/DataAccessImpl.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/MetricsService.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/MetricsServiceImpl.java
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/compress/CompressedPointContainer.java
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/compress/CompressorHeader.java
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/compress/TagsDeserializer.java
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/compress/TagsSerializer.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/transformers/BatchStatementTransformer.java
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/transformers/DataPointCompressTransformer.java
    A
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/transformers/DataPointDecompressTransformer.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/transformers/MetricFromDataRowTransformer.java
    M
core/metrics-core-service/src/main/java/org/hawkular/metrics/core/service/transformers/MetricFromFullDataRowTransformer.java
    A
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/compress/CompressorHeaderTest.java
    A
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/compress/TagsMapSerializingTest.java
    A
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/jobs/CompressDataJobITest.java
    M
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/jobs/DeleteTenantITest.java
    M
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/service/DataAccessITest.java
    M
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/service/DelegatingDataAccess.java
    M
core/metrics-core-service/src/test/java/org/hawkular/metrics/core/service/metrics/GaugeITest.java
    M core/metrics-model/src/main/java/org/hawkular/metrics/model/AvailabilityType.java
    M core/schema/src/main/java/org/hawkular/metrics/schema/SchemaService.java
    M core/schema/src/main/resources/org/hawkular/schema/cassalog.groovy
    A core/schema/src/main/resources/org/hawkular/schema/updates/schema-0.20.0.groovy
    M
integration-tests/rest-tests-jaxrs/src/test/groovy/org/hawkular/metrics/rest/RESTTest.groovy
    M job-scheduler/src/main/java/org/hawkular/metrics/scheduler/api/Scheduler.java
    M job-scheduler/src/main/java/org/hawkular/metrics/scheduler/impl/SchedulerImpl.java
    M job-scheduler/src/main/java/org/hawkular/metrics/scheduler/impl/TestScheduler.java
    M pom.xml

  Log Message:
  -----------
  Merge pull request #595 from burmanm/etl_compression

ETL compression process based on Gorilla algorithms

Compare: https://github.com/hawkular/hawkular-metrics/compare/e08798032910...45da8...

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

[Hawkular-commits] [hawkular/hawkular-metrics] 7b5d80: ETL based compression methods in the core for Gaug...