Alarm fatigue is an ignored killer.
This is part 1 of “101 Ways AI Can Go Wrong” - a series exploring the interaction of AI and human endeavor through the lens of the Crossfactors framework.
View all posts in this series or explore the Crossfactors Framework
Alarm fatigue is the effect by which humans become desensitized to notifications and alerts due to their high frequency. This effect is worsened by alarms that are too sensitive or simply non-actionable.
Why It Matters
Alarm fatigue causes alarms to be ignored while still potentially being a distraction. Humans fail to take appropriate action, even in high stake environments. This problem is widely recognized in healthcare where studies have shown that there are hundreds of alarms per hospital bed per day. To make matters worse, there is no standardisation of tones or sound patterns to indicate the severity of the alarm or even the source.
Alarm fatigue matters outside of healthcare too. Every alert a system or software produces consumes some of the cognitive bandwidth available to a human to do a task, or can serve as a distraction. Alerts that are ignored or even defeated also provide subtle information to the user as to the operation profile of the system.
A Real-World Example
Tesla’s Autopilot system is known for “nagging”, which is a prompt to force the user to interact with the car while it is driving autonomously. The internet is full of complaints from owners unhappy about the frequency of these alerts. It is also full of examples of people who learn how to subconsciously satisfy the alert by interacting with the controls, or even commercial products such as steering wheel weights to physically defeat the alert.
Plainly, this is because most of the alerts are unactionable.
Another important factor is that the nagging changes with software updates, so the system as a whole may change from one drive to the next. It is unclear what role the various implementations have played in crashes to date, but the NHTSA has mandated changes to Tesla’s software.
Key Dimensions
Here are some key things to consider - but it’s far from a complete list.
Frequency of actionable alerts: what percentage of alerts provide zero (or negative) value?
Safety profile of the environment: are alerts escalated according to the risks involved in a way that’s easily understandable by humans?
Human situational awareness: are the users instinctively aware of both what they are ignoring and paying attention to as it relates to the complex environment the alerts relate to?