Many human-in-the-loop screens turn humans into rubber stamps.
Here are 15 questions to ask about yours. Use them on any screen where a person approves, accepts or signs off on AI output.
They build on Rubber Stamps and on the six questions in Your Approve button is not human oversight.
- Does the AI’s first answer become the reviewer’s answer?
- Does the reviewer stop looking once the answer seems fine?
- Do people trust the AI over what they know?
- Is the reviewer staring at one box while the problem sits elsewhere?
- Does the reviewer know what’s happening around the single decision?
- Was the screen designed around what the reviewer needs to know?
- Does the AI sound like it outranks the person?
- Does the reviewer know which mode the system is in?
- Can the reviewer tell how sure the AI is?
- Does the explanation make sense to this reviewer?
- Does it look right just because it’s well formatted?
- Does the screen invite actions that aren’t safe?
- Is it a chore to say no to a bad suggestion?
- Can the reviewer see where the AI got its facts?
- Are there so many warnings that none of them gets read?