Viewing test execution results

Test execution status

The state of an execution can be the following:

  • Passed ../_images/Passed-scenario.PNGPassed
  • Failed ../_images/Failed-scenario.PNGFailed
  • Other ../_images/Other-scenario.PNGFailed
    • Not executed
    • Skipped
    • A step binding is missing
    • A step was marked as Pending

Test result aggregation

The individual test execution results are aggregated using the following method:

  • Scenario
    • Passed: all executions of this scenario have at least one “Passed” or were “Others”
    • Failed: at least one execution of this scenario has “Failed”
    • Others: all executions of this scenario have the state “Others”
  • Scenario outline example row
    • Passed: all executions of this scenario outline example row have at least one “Passed” or were “Others”
    • Failed: at least one execution of this scenario outline example row has “Failed”
    • Others: all executions of this scenario outline example row have the state “Others”
  • Scenario outline
    • Passed: all executions of its example rows have at least one “Passed” or were “Others”
    • Failed: at least one execution of its example rows has “Failed”
    • Others: all executions of this of its example rows have the state “Others”
  • Feature
    • Passed: all executions of the scenarios of this feature have at least one “Passed” or were “Others”
    • Failed: at least one scenario of this feature has “Failed”
    • Others: all executions of the scenarios of this feature have the state “Others”

Test result summary

A summary of the execution results are display for every folder in the feature tree:

../_images/Folder-result-summary.PNGSummary

Viewing multiple execution results

When a scenario or a scenario outline example row has been executed multiple times during a test execution the following indicator is show:

../_images/Multiple-execution.PNGMultiple Execution Indicator

The details of the executions can be seen by hovering over the triangle icon:

../_images/Multiple-execution-tooltip.PNGMultiple Execution Tooltip

Test result details

If you go to the Test Result Details tab, you can see the test execution results in numbers and percentages:

../_images/test-result-details.pngMultiple Execution Tooltip