Combining Incremental Language Generation and Incremental Speech Synthesis for Adaptive Information Presentation

Buschmeier, Hendrik and Baumann, Timo and Dosch, Benjamin and Kopp, Stefan and Schlangen, David

Participants in a conversation are normally receptive to their surroundings and their interlocutors, even while they are speaking and can, if necessary, adapt their ongoing utterance. Typical dialogue systems are not receptive and cannot adapt while uttering. We present combinable components for incremental natural language generation and incremental speech synthesis and demonstrate the flexibility they can achieve with an example system that adapts to a listener’s acoustic understanding problems by pausing, repeating and possibly rephrasing problematic parts of an utterance. In an evaluation, this system was rated as significantly more natural than two systems representing the current state of the art that either ignore the interrupting event or just pause; it also has a lower response time. Video of talk available here: http://www.superlectures.com/sigdial2012/lecture.php?lang=en&id=17

In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue , 2012
[PDF]

@inproceedings{Buschmeier-2012,
  author = {Buschmeier, Hendrik and Baumann, Timo and Dosch, Benjamin and Kopp, Stefan and Schlangen, David},
  booktitle = {Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue},
  keyword = {Dialogue Systems, Incremental Speech Synthesis, Incremental Natural Language Generation},
  location = {Seoul, South Korea},
  pages = {295--303},
  title = {{Combining Incremental Language Generation and Incremental Speech Synthesis for Adaptive Information Presentation}},
  year = {2012},
  topics = {},
  domains = {},
  approach = {},
  project = {}
}