Heimdl and Devin

Devin reviews the change, Heimdl checks the tests behind it

Two different questions

Devin asks

“Does this change look right?”

Heimdl asks

“If this change were wrong, would a test fail?”

Across the 1 pull request Devin reviewed, the tests missed 77% of the changes

triggerdotdev/trigger.dev #4815LowSession streams fail to start without the session ID179changes42caught27survived110uncovered2tests suggestedapiClient/index.ts:1397uncovered- `${this.baseUrl}/realtime/v1/sessions/${encodeURIComponent(sessionIdOrExternalId)}`+ ``Suggested test, initializes session stream with correct URL containing session IDexpect(url.pathname).toBe(`/realtime/v1/sessions/${encodeURIComponent(sessionId)}/out`);

Running both

Keep Devin for review, add Heimdl for proof your tests hold the change

Get a free auditAll case studies
Other comparisons
CodeRabbitGreptilecubicCodexVeriaGitHub Copilot

Missed, the share of changes on the pull requests a reviewer reviewed that no test caught, survived plus uncovered

From Heimdl’s published audits of public pull requests, a reviewer is listed where it commented on the pull request