Don't fall asleep at the wheel

We just read a new study (here) that's fascinating and has implications for how we all roll out AI to our organisations! Here's a quick overview and some possible lessons from it. 

 

 

It examines an important but under-explored issue in human-AI collaboration: how the quality of AI assistance impacts human performance and engagement. As businesses increasingly rely on AI tools to aid employees in their work, it is crucial to understand how humans will actually interact with and respond to those tools. While improving AI accuracy is an obvious goal, the study shows that higher accuracy does not necessarily translate into better-combined human-AI performance. When AI is "too good", humans may exert less effort and mindlessly follow its recommendations. This "falling asleep at the wheel" phenomenon suggests that maximizing AI performance in isolation is not the same as maximizing overall team output when humans are using that AI. The study sheds light on an underexplored trade-off between AI accuracy and human effort provision that businesses must consider when designing effective human-machine collaboration systems. Its insights can thus help organizations optimize the combined output of their human and AI resources.

 

The researchers ran a controlled experiment where they hired 181 professional recruiters to screen job applications. The recruiters were randomly assigned to receive AI recommendations of varying quality: perfect prediction, high-performing AI (85% accurate), or low-performing AI (75% accurate). The researchers measured the recruiters' accuracy, effort exerted, and time spent. They found that recruiters who received lower-quality AI recommendations performed better and exerted more effort than those with higher-quality AI. Recruiters using higher-quality AI were more likely to just follow the AI's recommendations mindlessly. Meanwhile, more experienced recruiters drove much of the benefit from lower-quality AI, as they thought independently and improved upon the recommendations. The experiment provides evidence that higher AI accuracy can actually reduce human-AI performance by decreasing human engagement, an effect called "falling asleep at the wheel." The key is to design AI tools that keep humans awake and involved.

 

• Improving an AI's technical accuracy does not necessarily translate into improved performance when humans are using that AI. Humans may respond to higher-quality AI by delegating decisions mindlessly and exerting less effort. This can reduce overall human-AI performance.
• To maximize human-AI performance, businesses should consider trading off AI accuracy against potential adverse impacts on human effort. Lower-quality AI that requires humans to stay engaged and provide input may lead to better combined performance than higher-quality AI that humans just follow blindly.
• Businesses should design human-AI collaboration systems to keep humans attentive and engaged in tasks where their focus is required. This may mean creating "custom AI" that performs suboptimally on purpose to ensure humans remain involved.
• More experienced employees drive much of the positive impact of lower-quality AI. They are more likely to think independently and improve on AI's recommendations. However, they perform worse when using higher-quality AI that they just follow. Experts may thus be crucial for effective human-AI augmentation.
• Businesses should take a holistic view of human-AI collaboration as an organization design problem. Beyond AI accuracy, features like transparency, reliability and interpretability can impact human trust and collaboration effectiveness.

In summary, focusing solely on improving AI technical performance may not yield the intended benefits in human-AI teams. Businesses need to consider how humans will actually use and integrate that AI, and design collaboration systems accordingly, in order to maximize the combined output of humans and machines.