AI will not turn non-experts into experts anymore that Wikipedia turns teenagers into scholars.
For experts, AI has potential, but will remain useless until it is recognizes the cognitive factors at play.
Radiology makes a great case study. The initial promise that radiologists would be replaced was naive to the fact that radiologists do more than read (interpret) images and report findings. Yet, handing them algorithms was never going to be enough.
Under-reading, what AI startups promised to solve, is only one type of error. Here are the other types:
- Finding is seen but dismissed (complacency).
- Finding is seen but attributed incorrectly.
- Finding is seen but not recognized due to a gap in knowledge.
- Finding is seen but miscommunicated.
- Finding is missed due to image acquisition error or quality.
- Finding is seen but clinical history is missing or inaccurate.
- Finding is missed because the interpretation stopped prematurely after another finding was seen.
- Finding is seen but there is an over-reliance on a previous report.
- Finding is missed to to inattentional blindness.
- Finding is missed because of bias of how the patient is presenting.
- Finding is missed because of the anchoring bias of earlier information.
Everyone of these error types will interact with a radiologist using AI in new and different ways. For example, AI results can be the source of anchoring bias or a new source of premature closure.
Furthermore, AI will create new types of errors:
- Automation bias (Over-confidently trusting AI results)
- Attentional tunnelling (Focusing on AI results at the expense of other findings).
- Algorithm Aversion (When a radiologist broadly dismisses AI due to past mistakes).
- Incidentalomas (incidental by inconsequential findings).
- Alarm fatigue (false positives reducing the trustworthiness of all positive AI findings).
- Context bias (the radiologist and AI may be operating with contextually different information).
Engineers, developers, lawyers, truck drivers, financial traders, recruiters, insurance brokers, journalists, adjusters, teachers inspectors, etc. all have such lists of their own.
Every field is full of AI blind spots. Do you know which ones you’re solving for?