Ziang Xiao 1 2, Michelle X. Zhou 2, Q. Vera Liao3, Gloria Mark 4, Changyan Chi2, Wenxi Chen2, Huahai Yang2

1University of Illinois at Urbana-Champaign, 2Juji. Inc., 3IBM Research AI, 4University of California, Irvine

Abstract

The rise of increasingly more powerful chatbots offers a new way to collect information through conversational surveys, where a chatbot asks open-ended questions, interprets a user’s free-text responses, and probes answers when needed. To investigate the effectiveness and limitations of such a chatbot in conducting surveys, we conducted a field study involving about 600 participants. In this study, half of the participants took a typical online survey on Qualtrics and the other half interacted with an AI-powered chatbot to complete a conversational survey. Our detailed analysis of over 5200 free-text responses revealed that the chatbot drove a significantly higher level of participant engagement and elicited significantly better quality responses in terms of relevance, depth, and readability. Based on our results, we discuss design implications for creating AI-powered chatbots to conduct effective surveys and beyond.

Bibtex

    @article{xiao2019tell,
    title={Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys},
    author={Xiao, Ziang and Zhou, Michelle X and Liao, Q Vera and Mark, Gloria and Chi, Changyan and Chen, Wenxi and Yang, Huahai},
    journal={arXiv preprint arXiv:1905.10700},
    year={2019}
    }