How Results Map to Test Cases
3 min read
This is the part that decides whether automation is trustworthy. A report row has to become a result on the right test case, and it has to stay right when someone renames a test.
Resolution order
Section titled “Resolution order”Matching on the test’s name alone is a trap — renaming a test silently breaks it. Hawzu resolves each report row in this order:
-
An explicit Hawzu code stated by the report. The most robust option, because it survives renaming the test.
-
A code token embedded in the test name or classname, for example
CHK-123 logs in successfully. No tooling required — just put the code in the test title. -
An exact match on the test case title. A convenient fallback; an ambiguous title is never guessed at.
Codes are compared case-insensitively with surrounding whitespace trimmed, and the report’s own spelling is kept on the stored row.
Stating the code explicitly
Section titled “Stating the code explicitly”| Format | How |
|---|---|
| JUnit | <property name="testcase_code" value="CHK-123"/> on the <testcase> (or the suite) |
| TRX | An MSTest trait named testcase_code |
| NUnit | <property name="testcase_code" value="CHK-123"/> |
| Cucumber | A scenario tag: @CHK-123 |
| Allure | A label named testcase_code, or a CHK-123 tag |
One test covering several test cases
Section titled “One test covering several test cases”An end-to-end test often stands in for several manual test cases. Name them all and each one receives the same verdict:
Or state them explicitly — as one delimited value, or as repeated properties:
If the run doesn’t contain one of the named cases, that case is listed individually as unmatched, so a partially covered test is visible rather than quietly leaving cases Not Executed.
Status mapping
Section titled “Status mapping”Results normalise to Hawzu’s statuses:
| Hawzu | Comes from |
|---|---|
Passed | A passing test |
Failed | A failure or error |
Skipped | A skipped/ignored test, or a JUnit <skipped> |
Blocked | Allure broken, NUnit/TRX Inconclusive, TRX Blocked, a Cucumber undefined/pending step — cases where the test never reached a verdict, so the feature was never actually judged |
JUnit has no way to express Blocked. To set it, add <property name="hawzu_status" value="Blocked"/> to the test case.
Needs Rerun is deliberately unreachable from automation — it is a human judgement about a result, not something a report can assert.
When several report rows map to one test case — parametrised or retried suites — the worst status wins. A case that failed any variant is not green.
The failure message, duration, and stack trace are kept as the result comment.
Scope: which test cases a report may touch
Section titled “Scope: which test cases a report may touch”This differs by run type, and the difference is the whole point.
On a manual test run, the test cases someone picked are the scope. A report row naming a case outside the run is recorded as unmatched and changes nothing.
On an automated test run, the report can extend the scope. A row naming a code that exists in the project’s repository is adopted into the run and given its verdict. Adoption is by code only — never by title — so a coincidental title match can never pull an unrelated case into a run.
In both cases, test cases that are in the run and the report never mentions are left Not Executed on purpose. That is the gap between what you meant to automate and what actually ran, and erasing it would make a readiness dashboard lie.
Rows that match nothing
Section titled “Rows that match nothing”Every row is recorded with an outcome, and the two unmatched outcomes are visible on the build:
- Not in this run — the row named a real test case that is not part of this run.
- No matching test case — nothing in the project matched the row.
Unmatched rows never affect status counts, analytics, test case history, or release readiness. They are kept so you can see what your pipeline is running that Hawzu cannot account for — which is exactly what the Automation Health report summarises over time.