#Flaky Tests
The Flaky Tests report highlights tests that flip statuses across runs — bouncing between pass and fail rather than failing consistently. It gives you a stable triage shortlist so you can fix the noisiest tests first and tell genuine bugs apart from flakes.
#Filtering
- Search by title or code: Find a specific test case.
- Min flips: Set a minimum number of pass↔fail transitions a test must have before it appears.
- Classification: Filter to Flaky or Unstable tests.
- Folder: Limit results to a folder.
#Key metrics
The report summarizes the tests with run data in scope:
- Tests with data
- Flaky
- Unstable
- Stable
#Probable breakages and folder instability
- Probable breakages: Tests whose recent signals point to a real break rather than a flake — for example, a failing streak or open defects.
- Folder instability: Each folder’s instability percentage, calculated as (Flaky + Unstable) ÷ tests with data, so you can see which areas misbehave the most.
#Reading the flaky tests table
The main table lists each flaky test with:
- Streak: How many consecutive recent runs ended with the same status (for example, “3× Failed”).
- Last failed: The most recent run that ended Failed or Blocked.
- Flips: The number of pass-to-fail or fail-to-pass transitions across the recent runs in scope. Higher means more bouncing.
- Fail %: Failed runs ÷ total runs in the flakiness window.
- Open defects: Defects linked to the test that are not closed — a value greater than zero often signals a real bug rather than a flake.
- Classification: Whether the test is classified as Flaky or Unstable by the flakiness model.
#Exporting
Download the report as a PDF or as Excel (XLSX). The PDF dialog lets you choose sections — cover page, executive summary, key metrics, probable breakages, folder instability, the flaky tests table, and a legend explaining flips, fail rate, streaks, and the classifications.