[Hawkular-dev] possible bug in inventory server

[John Mazzitelli]

Crossing fingers - I think this fixes the problem: https://github.com/hawkular/hawkular-agent/pull/400

----- Original Message -----
> On Tue, Oct 31, 2017 at 7:35 PM, Lucas Ponce <lponce at redhat.com> wrote:
> 
> >
> >> Who wants to investigate the inventory service to see if I'm right that
> >> there is a bug here? If there is no bug in inventory, the only other
> >> option
> >> is it's a bug in the itest code, but I checked and everything looks good
> >> to
> >> me there. I can't see a problem in the itest code. And the fact this is
> >> intermittent (passes sometimes, fails sometimes) tells me there is
> >> something sinister going on in the inventory server (i.e. if it is a
> >> problem in the test code, you would think it would always fail).
> >>
> >
> > After a investigation, I don't think there is a problem in the Inventory.
> >
> > In a preliminar status, that was the assumption, and yes, the inventory
> > has a flag that potentially could add an entry and theorically could make
> > that an entry was not indexed if there was a query fast enough after the
> > insertion.
> >
> > But after that PR in commons, the problem still exists, so I ran several
> > itests of commons + agent and this race happens in a ratio of 10%, it took
> > time to test, as itest are time consuming.
> >
> > Basically what happens according the logs in the failed test is the
> > following:
> >
> > 1) The testXaDs datasource is added in a previous test.
> > 2) Agent thread [Hawkular-Agent-Full-Discovery-Scan-Local DMR-1] starts a
> > discovery in its scheduling period.
> > 3) Agent thread [OkHttp http://127.0.0.1:8080/...] process a remove of
> > the testXaDs and succesfully send to the inventory the delete command.
> > 4) Agent thread [Hawkular-Agent-Full-Discovery-Scan-Local DMR-1] might
> > still see the testXaDs resource and it is added into the inventory.
> > 5) Test expects there is no testXaDs resource in inventory, but there is
> > one, as there is a race condition between both threads, for some reason,
> > when the datasource is deleted, it is not updated in the info used by the
> > discovery task.
> >
> > This doesn't happen always, as it is a random scenario due some moving
> > parts between the autodiscovery threads, and the architecture of the itest.
> >
> > I was analyzing why when the testXaDs is removed is still visibile by the
> > autodiscovery thread.
> >
> > Tomorrow I am off and I can't work on this until thursday, so feel free if
> > you want to continue from here.
> >
> 
> Forgot to say, that JIRA for this is
> 
> https://issues.jboss.org/browse/HAWKULAR-1313
> 
> 
> 
> >
> > Lucas
> >
> >
>