You Didn't Request This AI Output. You Still Own Checking It.

The advice to "decide what correct looks like before you generate" has a quiet assumption built in: that you're the one doing the generating. Increasingly you're not. A vendor hands you a report already marked "AI-validated." A colleague forwards a deck an agent drafted overnight. A platform shows you a shortlist a model built while you were asleep. By the time you're looking at any of it, the only decision left is whether to sign off — the decision about what "correct" should have meant was made by someone else, or nobody at all.

What happened

On October 6, OpenAI published 722 AI-generated math manuscripts to GitHub in a single batch, produced by an unreleased model working through roughly 4,000 open problems. Nobody commissioned paper #418 specifically. It arrived as part of a pile, formatted like finished research, and most of the 722 had no independent review behind them by the time the batch went public — including, for many, no formal proof that the underlying math held up at all.

That's an extreme version of something increasingly ordinary: AI output that exists before you're in the room. A compliance tool ships a report stamped "validated." A recruiting platform's agent screens applicants overnight and hands you a ranked list at 9am. None of these ask you what you'd count as correct first, because asking you wasn't part of how they were built.

The skill this calls for

Every popular fix for bad AI output — name the quality bar first, write the criteria down, decide before you generate — depends on a "before" that doesn't exist once the work arrives finished. You can't set a standard in advance for a report someone else already ran. What you can still do is reconstruct a check at the point you receive it, using two moves instead of one.

Ask what "validated" actually checked, not whether it says validated. A label is a claim, not a method. Find out which specific things were tested — against what, by what — before you treat the label as the verification.

Sample before you accept the batch, and sample harder as the batch gets bigger. A bigger pile isn't more trustworthy; it's more volume for the same unknown error rate. Pick a handful of specific items, trace each one to something you can check yourself, and let what you find there tell you whether to trust the rest.

Marisol: the energy audit with one offline sensor

Marisol manages facilities at a 120-person manufacturing plant. A vendor's AI-generated energy audit came back recommending $40,000 in equipment upgrades, the report marked complete and verified. She hadn't set the audit's methodology — that was the vendor's software, running against the vendor's own sensor feed, before she ever saw a line of it.

Instead of approving the spend off the summary, she picked the three most expensive recommendations and asked the vendor to show the sensor readings behind each one. Two held up. The third traced back to a sensor that had been offline for two months; the report had filled the gap with an estimated value and never flagged it as an estimate. That single recommendation was a third of the proposed spend.

Priya: the shortlist that ran overnight

Priya coordinates hiring at a 30-person logistics startup. The recruiting platform's agent screens every applicant overnight against the job posting and delivers a ranked shortlist before she's at her desk — she never writes the scoring criteria; the platform's model does.

Before booking interviews, she pulls five candidates the tool ranked low and checks each one against the job's actual requirements herself, not the tool's score. One rejected candidate had the required certification listed under a slightly different name than the tool's keyword match expected — filtered out on formatting, not qualification. She now runs that same five-candidate check every week, on a fixed day, rather than only when a result looks surprising.

The takeaway

Whoever generated the output decided what "good" meant for it. Whoever receives it still has to decide whether that was the right definition — and that job doesn't go away just because you weren't the one who asked for the work in the first place.