[Hawkular-dev] Each hawkular-metrics PR is now checked for performance regression

[Filip Brychta]

Hello,
Please be aware that second check (first is continuous-integration/travis-ci/pr) on each hawkular-metrics PR now finally does something useful.

Following build (http://jenkins.jonqe.lab.eng.bos.redhat.com:9080/job/haw-perf-stability-test/) is launched for each PR which will:
1 - build the PR
2 - build hawkular-dist using snapshot artifact from step 1
3 - deploy hawkular-dist
4 - start perf test which will
  - generate as high load as the hawkular is able to server. Right now from 300 threads where each thread is using unique tenantId.
  - measure average throughput (one request == POST one data point to http://${server.host}:${server.port}/hawkular/metrics/gauges/test/data and wait for response)
  - check % diff with previous build
  - if the % diff is < -2, it will fail the build. Right now it's still not 100% stable because of shared network, this problem should be fixed soon. Expected instability is now +/- 2 % so anything bigger than 2% should be considered as regression
5 - report status back to the PR


So please from now on pay attention to result of that check and ping me if it fails.
Consider this just as first step, there will be more next week since we have more time now (JON 3.3.3 is out).

It would be nice if anybody could create a PR which would test this (just include some tiny sleep into request processing)

Thanks,

Filip