testharness.js result statuses

The documentation of testharness.js is available in testharness.js API. Section Basic Usage says that every single test has as result PASS, FAIL, TIMEOUT and NOTRUN. These statuses are also listed in Callback API section. According to testharness.js#L1469, the status of a single test will be initialized with NOTRUN and later set to one of the other values. This means if TIMEOUT is disabled (using explicit_timeout function described in Setup section) and the test is not completed (neither PASS or FAIL), the status will remain NOTRUN. This is the interpretation after reading the source code of the testharness.js since there is no explanation in the documentation for the semantic of status NOTRUN.

A test file may contain multiple tests and there is also an overall status for the test file which can be one of the following values: OK, ERROR and TIMEOUT. There is no further explanation for each of these statuses but from reading the code, the ERROR status will be set if an unexpected exception occurred (try catch block), abort() function is called, timeout occurs when test in cleanup phase or test file contains tests with duplicate names. The status will be set to OK at the end if it is not set yet (it was null) which means no TIMEOUT or ERROR occurred.

PS: in test reports with comparison of multiple test sessions, the status "-" (with orange background of the table cell) means that the test was not executed on the corresponding user agent. This can happen if a test runs only if a certain condition is fulfilled or a specific event is fired.