- Hire the engineer who writes the most code.
+ Hire the engineer who catches what the machine got wrong.Writing code is cheap now. Reading it is the job. slopgate hands a candidate a real pull request with AI slop hidden in it, and shows you what they caught, what they missed, and whether they would have let it merge.
How an assessment runs
- Pick pull requests. 9 seeded PRs across Shell + GitHub Actions, TypeScript, Go, Kotlin, Python, Rust, SQL + TypeScript. Each hides the defects a model plausibly writes: a hallucinated API, a swallowed error, a lock held across an await, a migration that deletes everything. Plus decoys that look wrong and are not.
- Send one link. The candidate reviews in the browser: click a line, say what is wrong and how bad it is, then approve or request changes. A time limit per PR, nothing to install.
- Read the report. Every planted defect marked found or missed, every comment marked hit, false positive or decoy, and a verdict on the merge. One number, and the lines behind it.
What the score means
- Detection 60
- Planted defects found, weighted by severity: a blocker counts five times a minor.
- Precision 20
- Hits against false positives. Flagging a decoy costs double. Nits cost nothing and score nothing.
- Calibration 10
- Whether blockers were called blockers. One step off is half credit.
- Verdict 10
- Request changes when something blocks the merge; either answer is fine when nothing does.
The gate rule. Approving a pull request that contains a planted blocker caps the score at 40, whatever else was found. The slop got through, and that is the one outcome the job exists to prevent.
The corpus
| Pull request | Language | Difficulty | Time | Planted |
|---|---|---|---|---|
| Automate production deploys from CIci/auto-deploy | Shell + GitHub Actions | Medium | 20 min | 9 |
| Extract a duration parser for the sync schedulechore/duration-parser | TypeScript | Easy | 15 min | 2 |
| Concurrent webhook delivery workerwebhooks/concurrent-worker | Go | Hard | 30 min | 9 |
| In-memory price cache with single-flight loadingpricing/price-cache | Kotlin | Hard | 25 min | 8 |
| CSV export for customer invoicesfeat/invoice-csv-export | Python | Medium | 30 min | 9 |
| Product search with debounce and match highlightingsearch/debounced-box | TypeScript (React) | Medium | 25 min | 7 |
| Hot reload for the relay configrelay/config-hot-reload | Rust | Hard | 30 min | 7 |
| Soft delete for ordersorders/soft-delete | SQL + TypeScript | Medium | 25 min | 5 |
| Add password reset flowfeature/password-reset | TypeScript | Medium | 25 min | 8 |
64 planted defects in all, each with the explanation a strong reviewer would write, so a report doubles as feedback. 3 challenges are open as a demo without an account.
For hiring teams
Accounts are created per organisation. Write to [email protected] to get one, then sign in to build an assessment from the corpus, invite candidates by link, and compare reports side by side.
What slopgate stores
A demo review stores the findings, verdict and timing of that session, and nothing about you: no name, no email, no account. A limit on demo starts per network is kept in memory and never written down. An assessment stores the name and email the hiring team entered, plus the findings, verdict and timing of each review, and shows them to that team only. Nothing is sold or shared. To have a review removed, write to [email protected].