Pak, R., Rovira, E., & McLaughlin, A. C. (in press). Polite AI mitigates user susceptibility to AI hallucinations. Ergonomics. https://doi.org/10.1080/00140139.2024.2434604
Abstract
With their increased capability, AI-based chatbots have become increasingly popular tools to help users answer complex queries. However, these chatbots may hallucinate, or generate incorrect but very plausible-sounding information, more frequently than previously thought. Thus, it is crucial to examine strategies to mitigate human susceptibility to hallucinated output. In a between-subjects experiment, participants completed a difficult quiz with assistance from either a polite or neutral-toned AI chatbot, which occasionally provided hallucinated (incorrect) information. Signal detection analysis revealed that participants interacting with polite-AI showed modestly higher sensitivity in detecting hallucinations and a more conservative response bias compared to those interacting with neutral-toned AI. While the observed effect sizes were modest, even small improvements in users’ ability to detect AI hallucinations can have significant consequences, particularly in high-stakes domains or when aggregated across millions of AI interactions.
Practitioner Summary
This study examined how AI chatbot etiquette affects users’ susceptibility to AI hallucinations. Through a controlled experiment, results showed polite-AI led to modestly higher sensitivity in detecting hallucinations and a more conservative response bias. This suggests a potential design strategy that may enhance users’ critical evaluation of AI-generated content.
