Space Efficient Context Encoding for Non-Task-Oriented Dialogue Generation with Graph Attention Transformer

Galetzka, Fabian and Rose, Jewgeni and Schlangen, David and Lehmann, Jens

To improve the coherence and knowledge retrieval capabilities of non-task-oriented dialogue systems, recent Transformer-based models aim to integrate fixed background context. This often comes in the form of knowledge graphs, and the integration is done by creating pseudo utterances through paraphrasing knowledge triples, added into the accumulated dialogue context. However, the context length is fixed in these architectures, which restricts how much background or dialogue context can be kept. In this work, we propose a more concise encoding for background context structured in the form of knowledge graphs, by expressing the graph connections through restrictions on the attention weights. The results of our human evaluation show that this encoding reduces space requirements without negative effects on the precision of reproduction of knowledge and perceived consistency. Further, models trained with our proposed context encoding generate dialogues that are judged to be more comprehensive and interesting.

In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , 2021
[PDF]

@inproceedings{Galetzka-2021,
  title = {Space Efficient Context Encoding for Non-Task-Oriented Dialogue Generation with Graph Attention Transformer},
  author = {Galetzka, Fabian and Rose, Jewgeni and Schlangen, David and Lehmann, Jens},
  booktitle = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  month = aug,
  year = {2021},
  address = {Online},
  publisher = {Association for Computational Linguistics},
  url = {https://aclanthology.org/2021.acl-long.546},
  doi = {10.18653/v1/2021.acl-long.546},
  pages = {7028--7041},
  topics = {},
  domains = {},
  approach = {},
  project = {}
}