Can you tell that I’m confused? An overhearer study for German backchannels by an embodied agent

Meywirth, Isabel Donya and Götze, Jana

In spoken interaction, humans constantly display and interpret each others’ state of understanding. For an embodied agent, displaying its internal state of understanding in an efficient manner can be an important means for making a user-interaction more natural and initiate error recovery as early as possible. We carry out an overhearer study with 62 participants to investigate whether German verbal and non-verbal backchannels by the virtual Furhat embodied agent can be interpreted by an overhearer of a human-robot conversation. We compare three positive, three negative, and one neutral feedback reaction. We find that even though it is difficult to generate certain verbal backchannels, our participants can recognize displays of understanding with an accuracy of up to 0.92. Trying to communicate a lack of understanding is more often misunderstood (accuracy: 0.55), meaning that interaction designers need to carefully craft them in order to be useful for the interaction flow.

In Companion Publication of the 2022 International Conference on Multimodal Interaction , 2022
[PDF]

@inproceedings{Meywirth-2022,
  address = {New York, NY, USA},
  series = {{ICMI} '22 {Companion}},
  title = {Can you tell that {I}’m confused? {An} overhearer study for {German} backchannels by an embodied agent},
  isbn = {978-1-4503-9389-8},
  shorttitle = {Can you tell that {I}’m confused?},
  url = {https://doi.org/10.1145/3536220.3558804},
  doi = {10.1145/3536220.3558804},
  urldate = {2022-12-06},
  booktitle = {Companion {Publication} of the 2022 {International} {Conference} on {Multimodal} {Interaction}},
  publisher = {Association for Computing Machinery},
  author = {Meywirth, Isabel Donya and Götze, Jana},
  month = nov,
  year = {2022},
  keywords = {backchannels, grounding in conversation, human-robot interaction, listener responses, social virtual agents},
  pages = {89--93},
  file = {Meywirth and Götze - 2022 - Can you tell that I’m confused An overhearer stud.pdf:/Users/jana/work/BIBLIOGRAPHY/zotero/storage/IL8KC96P/Meywirth and Götze - 2022 - Can you tell that I’m confused An overhearer stud.pdf:application/pdf},
  topics = {},
  domains = {},
  approach = {},
  project = {}
}