[jboss-jira] [JBoss JIRA] (JBRULES-3491) Planner: benchmark ranking algorithms must be failure safe (score is null)

[Geoffrey De Smet (JIRA)]

    [ https://issues.jboss.org/browse/JBRULES-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12690697#comment-12690697 ] 

Geoffrey De Smet commented on JBRULES-3491:
-------------------------------------------

We've decided to go another route: if a solverbenchmark has any failing benchmark, it's not ranked.

                
> Planner: benchmark ranking algorithms must be failure safe (score is null)
> --------------------------------------------------------------------------
>
>                 Key: JBRULES-3491
>                 URL: https://issues.jboss.org/browse/JBRULES-3491
>             Project: Drools
>          Issue Type: Bug
>      Security Level: Public(Everyone can see) 
>          Components: drools-planner
>            Reporter: Geoffrey De Smet
>
> Things that are problematic in the ranking code (all classes under org.drools.planner.benchmark.core.ranking):
> - score can be null, totalScore can be null, ... when there are failures
> - org.drools.planner.benchmark.core.ranking.TotalRankSolverBenchmarkRankingWeightFactory: Usage of SolverBenchmark.getScoreList() because some SolverBenchmark might have extra problem inputFiles.
> The question is, should the ranking first compare the failureCount? Yes, I think it should.
> What code should do that? the DefaultPlannerBenchmark, by default, or the rankers itself?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira