[Hawkular-dev] Each hawkular-metrics PR is now checked for performance regression

[Filip Brychta]

Currently those steps include our internal jenkins (complete test env is in our internal lab) so the document is not very useful for community right now.
Possible solutions:
1 - report results from our internal jenkins as a new comment in the pull request
2 - publish results somewhere publicly and the 'Details' link in the pull request check would take a user there

Filip

----- Original Message -----
> Ahoj Filip, is there a reason to keep the document RH-internal? -- P
> 
> On 2015-08-13 13:53, Filip Brychta wrote:
> > Hello,
> > if the check fails please take a look at "What to do when the job fails"
> > section in https://mojo.redhat.com/docs/DOC-1042661
> >
> > Thanks,
> >
> > Filip
> >
> > ----- Original Message -----
> >> Hello,
> >> Please be aware that second check (first is
> >> continuous-integration/travis-ci/pr) on each hawkular-metrics PR now
> >> finally
> >> does something useful.
> >>
> >> Following build
> >> (http://jenkins.jonqe.lab.eng.bos.redhat.com:9080/job/haw-perf-stability-test/)
> >> is launched for each PR which will:
> >> 1 - build the PR
> >> 2 - build hawkular-dist using snapshot artifact from step 1
> >> 3 - deploy hawkular-dist
> >> 4 - start perf test which will
> >>    - generate as high load as the hawkular is able to server. Right now
> >>    from
> >>    300 threads where each thread is using unique tenantId.
> >>    - measure average throughput (one request == POST one data point to
> >>    http://${server.host}:${server.port}/hawkular/metrics/gauges/test/data
> >>    and
> >>    wait for response)
> >>    - check % diff with previous build
> >>    - if the % diff is < -2, it will fail the build. Right now it's still
> >>    not
> >>    100% stable because of shared network, this problem should be fixed
> >>    soon.
> >>    Expected instability is now +/- 2 % so anything bigger than 2% should
> >>    be
> >>    considered as regression
> >> 5 - report status back to the PR
> >>
> >>
> >> So please from now on pay attention to result of that check and ping me if
> >> it
> >> fails.
> >> Consider this just as first step, there will be more next week since we
> >> have
> >> more time now (JON 3.3.3 is out).
> >>
> >> It would be nice if anybody could create a PR which would test this (just
> >> include some tiny sleep into request processing)
> >>
> >> Thanks,
> >>
> >> Filip
> >> _______________________________________________
> >> hawkular-dev mailing list
> >> hawkular-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >>
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>