Documentation · Observability

Test results

The Tests tab, read from the JUnit XML your build already writes: what we read, the bounds on it, and the optional comment on a pull request.

If your build writes JUnit XML anywhere in the workspace, the job page grows a Tests tab: how many ran, how many failed, and the name and failure message of each one that did. There is nothing to add to your workflow — we read the files the machine is already writing, which is only possible because we run the machine.

What we read is bounded, and the bounds are these:

A job over the case limit keeps the first 5,000 and the tab says so — the totals it shows are still the ones your reporter printed, not the ones we kept. A file we cannot parse is counted and named rather than skipped: a suite that looks clean because we did not understand the reporter is worse than no tab at all.

Failed and errored are kept apart, because JUnit keeps them apart and they mean different things: <failure> is an assertion that did not hold, <error> is the test not completing.

Test names and failure output are your source code. They are stored against the job they came from, and its page shows them for 90 days, as it does the job's log. The privacy policy says what that means for how long we hold them. We do not keep the XML file itself.

Off by default. Turn on Comment test results on pull requests under Settings and a job whose run belongs to a pull request will leave one comment there with the same summary.

One comment per job, edited in place. A re-run replaces what it said rather than adding a second, so a pull request you push to twenty times has the same number of comments at the end as at the start. Each job of a matrix keeps its own, because a shared one would show you only whichever leg finished last.

We only comment on repositories where the App has been granted write access to pull requests. If it has not, nothing happens and nothing is reported as broken — the Tests tab is unaffected.