Uncalibrated Models Can Improve Human-AI Collaboration

In many practical applications of AI, an AI model is used as a decision aid for human users. The AI provides advice that a human (sometimes) incorporates into their decision-making process. The AI advice is often presented with some measure of “confidence” that the human can use to calibrate how much they depend on or trust the advice. In this paper, we demonstrate that presenting AI models as more confident than they actually are, even when the original AI is well-calibrated, can improve human-AI performance (measured as the accuracy and confidence of the human’s final prediction after seeing the AI advice). We first learn a model for how humans incorporate AI advice using data from thousands of human interactions. This enables us to explicitly estimate how to transform the AI’s prediction confidence, making the AI uncalibrated, in order to improve the final human prediction. We empirically validate our results across four different tasks — dealing with images, text and tabular data — involving hundreds of human participants. We further support our findings with simulation analysis. Our findings suggest the importance of and a framework for jointly optimizing the human-AI system as opposed to the standard paradigm of optimizing the AI model alone.

https://arxiv.org/abs/2202.05983

About
Latest Posts

Ryan Watkins

Professor at George Washington University

I am a Professor with Human-Technology Collaboration and Educational Technology programs at George Washington University in Washington DC. I have written 12 books and more than 100 articles, and I co-host of the Parsing Science podcast where scientists tell the stories behind their research. I am also the developer of the WeShareScience.com online platform for sharing research videos, and SciencePods.com where researchers can create free podcasts about their science. My research interests include human interactions with intelligent machines, needs, needs assessments, and instructional design.