Your [Approve] button is not human oversight.
Most AI products now have a human in the loop. In practice that usually means one button and a line of fine print: “AI can make mistakes. Please check.”
There’s a lot riding on that click.
Six questions it never asks:
- Is the task even possible for a person?
- Does the screen show what’s needed?
- When does attention run out?
- How would a failure surface?
- Where does the accountability land?
- What’s quietly getting worse?
I first ran into this building AI tools for radiologists, where plainly visible findings were missed because of attention, not skill. I’ve been cataloguing the ways in which human review fails ever since. The list is past 50.
None of it is the reviewer’s fault. It’s a design that mostly hasn’t been done.
For the next month, I’m talking to people who run teams that review AI output or build products that depend on it. Approve, Accept, Sign, Send, whatever your button says. I want to know how it works for you, and I’ll tell you what I’d measure.
One question to start: what share of AI output do your reviewers approve unchanged?