# A Discriminative Model for Perceptually-Grounded Incremental Reference Resolution

## Kennington, Casey and Dia, Livia and Schlangen, David

A large part of human communication involves referring to entities in the world, and often these entities are objects that are visually present for the interlocutors. A computer system that aims to resolve such references needs to tackle a complex task: objects and their visual features must be determined, the referring expressions must be recognised, extra-linguistic information such as eye gaze or pointing gestures must be incorporated — and the intended connection between words and world must be reconstructed. In this paper, we introduce a discriminative model of reference resolution that processes incrementally (i.e., word for word), is perceptually-grounded, and improves when interpolated with information from gaze and pointing gestures. We evaluated our model and found that it performed robustly in a realistic reference resolution task, when compared to a generative model.

In Proceedings of the 11th International Conference on Computational Semantics (IWCS) 2015 , 2015
[PDF]
@inproceedings{Kennington-2015,
author = {Kennington, Casey and Dia, Livia and Schlangen, David},
booktitle = {Proceedings of the 11th International Conference on Computational Semantics (IWCS) 2015},
location = {London},
pages = {195--205},
title = {{A Discriminative Model for Perceptually-Grounded Incremental Reference Resolution}},
year = {2015}
}