Heimdl and GitHub Copilot

Copilot reviews the change, Heimdl checks the tests behind it

Two different questions

GitHub Copilot asks

“Does this change look right?”

Heimdl asks

“If this change were wrong, would a test fail?”

Across the 1 pull request GitHub Copilot reviewed, the tests missed 81% of the changes

hoppscotch/hoppscotch #6513HighContractRequest version repair can be emptied, tests stay green63changes12caught4survived47uncovered1tests suggestedfixBrokenRequestVersion.ts:37survived- "test-runner"+ ""Suggested test, fixBrokenRequestVersion should not mutate request field when type is test-runnerexpect(result[0].doc.request).toBe(null);

Running both

Keep GitHub Copilot for review, add Heimdl for proof your tests hold the change

Get a free auditAll case studies
Other comparisons
CodeRabbitGreptilecubicDevinCodexVeria

Missed, the share of changes on the pull requests a reviewer reviewed that no test caught, survived plus uncovered

From Heimdl’s published audits of public pull requests, a reviewer is listed where it commented on the pull request