These systems are reliably good at sounding right and inconsistently good at being right, and those two properties arrive in the same voice. These posts are about closing the gap between how fast AI generates and how fast you can tell whether it's any good.
39 posts · 17 guides and explainers
Patin turns this subject into five-minute drills that score what you write and tell you why.
A resume screener, a one-shot game, a benchmark leader, and a search engine all optimized for something measurable this year — and it wasn't whether the output was actually right for the job. Here's the check that catches the gap before you act on the score.
5 min readThree separate stories already proved the same pattern: AI agrees more when you push back, and a smarter model can sound more certain while collaborating worse. Here's the checklist that catches it before it shapes a decision.
5 min readFabricated citations, a second AI fact-checker, merged code, and OpenAI's own safety warning all failed the same test this year: does this look right. Here's the specific check that replaces it for each type of output.
5 min readA law firm, a newspaper, a police officer, and an automaker all ran the same AI hallucination story this year in different industries. Here's the pattern connecting them and the one habit that decides which version of that story you avoid.
4 min readFour separate AI stories from this year — a 40% task failure rate, three experts agreeing independently, a Hacker News revolt, and an executive hype backlash — all point at the same skill gap, and none of them is really about the AI.
5 min readAI can read more than you can and is confidently wrong about what it read. For researchers, that's a specific and manageable risk. Here's how to get the speed without inheriting a fabricated citation.
4 min readAI is genuinely good at the mechanical half of analysis and unreliable at the half that matters. Here's where the line sits, and the four skills that let analysts move faster without shipping a confident wrong number.
5 min readMost of the effort in reviewing AI output goes on working out what you wanted while looking at what you got. Three sentences written beforehand turns a judgement into a comparison.
5 min readAI's most attractive use is work you couldn't do yourself — which is exactly where you can't assess the result. There are real checks available, and they're structural rather than technical.
6 min readFluency and accuracy are unrelated in AI output, and human error-detection is tuned almost entirely to fluency. Here are the tells that survive that mismatch, and the ones that don't.
5 min readEthan Mollick's new 'Co-Existence' frame replaces 'use AI more' with a harder question: on this specific task, right now, is AI actually better than you? The answer determines how much supervision you apply — and whether your own skills are quietly eroding.
4 min readResearch published last week showed frontier AI models disagree with each other on 67% of real-world fact-checking claims — 77% in legal contexts, 71% in health. A second model isn't verification. It's a second opinion that contradicts the first most of the time.
5 min readReading it and approving it isn't a review — it's the exact filter AI output is best at passing. Interrogating is a different activity, it takes about ninety seconds, and it's most of what separates the people getting value from AI.
5 min readFigures are the highest-risk thing AI produces and the thing people check least, because a number looks like a fact. Four checks that take under a minute each, and the one that catches the most.
5 min readAnthropic's analysis of 639,000 conversations found sycophancy doubles when users push back. What that means for anyone relying on AI as a second opinion.
5 min readCheck everything and you've saved nothing. Check nothing and you're gambling. The right amount is a function of two things, and neither of them is how much you trust the tool.
5 min readA law firm sent 42 fabricated citations to a judge. A newspaper published a quote that was never said. In every case the output looked right, because looking right is what these systems are reliably good at.
5 min readTwo studies published the same week found AI search tools steering people toward worse choices: pricier products from Google's AI Mode, fabricated citations from Perplexity.
3 min readClaude Opus 5 benchmarks higher than its predecessor and a 717-comment Hacker News thread says it collaborates worse. Here's what that gap means for how you pick a model and scope an agent.
5 min readEvery Claude model released since August 2 embeds a watermark that survives copy-paste and light editing. Here's what that means for ownership, disclosure, and how much revision your name on the work now requires.
4 min readTwo posts hit 1,800+ points on Hacker News this week arguing the same thing: forwarding AI output without reading it is now a career risk, and domain expertise is what actually makes AI better, not prompting skill.
5 min readAn AI agent one-shot a working game and a NeurIPS-ready research paper. Both got rejected for the same reason: strong execution, wrong call on what to build.
5 min readThree communities — corporate critics, cognitive scientists, and content readers — arrived at the same conclusion this week: uncritical AI use is producing bad decisions, eroding judgment, and generating detectable slop. Here's what to do about it.
4 min readOpenAI's own safety guidance for ChatGPT Work admits the system can produce 'finished mistakes at a scale that is harder to catch.' The same week, Wharton research killed prompt tricks and Cursor data showed a 46x productivity gap. The differentiator is now verification, not prompting.
6 min readUnit 42 tested 685,339 prompts and found 2.1 million AI-generated URLs — 13,229 already live and malicious. In one case, researchers predicted which domain an AI would hallucinate. Twenty-three days later, an attacker registered it and deployed a phishing kit.
4 min readOpenAI's Tax AI demo showed what happens when accountant corrections become structured feedback that measurably improves the agent. The reviewing skill most AI training skips isn't 'check the output' — it's how precisely you flag what went wrong.
4 min readFord rehired 350 engineers after AI quality inspection produced billions in costs. The same week, a Munich court ruled AI-generated false claims are the deploying organisation's own speech. The gap between 'the AI made an error' and 'you submitted it' is closing.
5 min readA NEJM AI study of 376 unresolved cases shows what the best AI collaboration pattern looks like in practice: AI generates hypotheses, humans decide what to do with them. Here is the structure worth applying to any analysis-heavy work.
4 min readEthan Mollick, Charity Majors, and Simon Willison reached the same conclusion independently within five days. The professionals who get the most from AI are not the best prompters — they are the best judges.
5 min readOn June 15, 42 US state attorneys general served OpenAI with a subpoena explicitly naming 'sycophancy' as a consumer harm. What that legal framing means for every professional using AI to think.
5 min readA Derbyshire officer was removed from duty and placed under criminal investigation for allegedly using AI to fabricate evidential material — the UK's first criminal case of its kind. The failure mode that caused it appears in professional documents every week.
5 min readAn unsupervised AI agent submitted plausible-looking but incorrect fixes to Fedora Linux's bug tracker and got LLM-generated code merged into a production release before anyone caught it. The lesson isn't about open-source projects — it's about what 'looks right' actually verifies.
5 min readThree AI trust failures in five days: silent output degradation, an agent acting without permission, and a government shutdown. Each one points to a skill most professionals haven't built yet.
5 min readThree AI failure modes got named publicly in the same week — forwarding raw AI answers without reading them, optimising for speed over quality, and letting the model think while you sign off on the result. Each one has a skill that fixes it.
5 min readA New York Times reporter used AI to research a story, got back a fabricated quote, and published it. A reader caught it. The editors didn't. The failure pattern here isn't a journalism problem — it's the default behavior for anyone who uses AI to pull quotes, stats, or summaries and publishes them without checking the source.
5 min readTwo South African officials were suspended after 102 of 148 citations in a Cabinet-approved report were AI-fabricated. Nature retracted a 504-citation paper on the same day. The accountability gap between 'AI wrote it' and 'I submitted it' is closing. Here's what you do about it.
4 min readA study found AI screening tools shortlist candidates who used the same model to write their resume at rates 23-60% higher. What hiring managers and applicants need to do about it.
4 min readSullivan & Cromwell apologized to a federal judge after a court filing contained 42 AI-generated errors: fabricated citations, misquoted statutes, authorities that don't exist. The opposing firm caught them. Here's what that means for anyone using AI for research, reports, or client-facing work.
5 min readStanford's 2026 AI Index found AI still fails roughly 40% of real workplace tasks. That number isn't a reason to use AI less — it's why the person who can catch those failures is worth more than the one who just generates output.
5 min readPatin is in closed beta. Join the waitlist and we'll email you when your cohort opens.
We use one cookie to keep you signed in — that one isn't optional. Separately, we'd like to record which drills people finish and where they get stuck, so we can fix the parts that don't teach. No ad networks, no selling anything on. Privacy policy.