A Multimodal In-Car Dialogue System That Tracks The Driver’s Attention

Kousidis, Spyridon and Kennington, Casey and Baumann, Timo and Buschmeier, Hendrik and Kopp, Stefan and Schlangen, David

When a passenger speaks to a driver, he or she is co-located with the driver, is generally aware of the situation, and can stop speaking to allow the driver to focus on the driving task. In-car dialogue systems ignore these important aspects, making them more distracting than even cell-phone conversations. We developed and tested a “situationally-aware” dialogue system that can interrupt its speech when a situation which requires more attention from the driver is detected, and can resume when driving conditions return to normal. Furthermore, our system allows driver-controlled resumption of interrupted speech via verbal or visual cues (head nods). Over two experiments, we found that the situationally-aware spoken dialogue system improves driving performance and attention to the speech content, while driver-controlled speech resumption does not hinder performance in either of these two tasks.

In Proceedings of the 16th International Conference on Multimodal Interfaces , 2014
[PDF]

@inproceedings{Kousidis-2014-1,
  author = {Kousidis, Spyridon and Kennington, Casey and Baumann, Timo and Buschmeier, Hendrik and Kopp, Stefan and Schlangen, David},
  booktitle = {Proceedings of the 16th International Conference on Multimodal Interfaces},
  location = {Istanbul, Turkey},
  pages = {26--33},
  title = {{A Multimodal In-Car Dialogue System That Tracks The Driver's Attention}},
  doi = {10.1145/2663204.2663244},
  year = {2014},
  topics = {},
  domains = {},
  approach = {},
  project = {}
}