Revising with a Backward Glance: Regressions and Skips during Reading as Cognitive Signals for Revision Policies in Incremental Processing

Madureira, Brielen and Çelikkol, Pelin and Schlangen, David

In NLP, incremental processors produce output in instalments, based on incoming prefixes of the linguistic input. Some tokens trigger revisions, causing edits to the output hypothesis, but little is known about why models revise when they revise. A policy that detects the time steps where revisions should happen can improve efficiency. Still, retrieving a suitable signal to train a revision policy is an open problem, since it is not naturally available in datasets. In this work, we investigate the appropriateness of regressions and skips in human reading eye-tracking data as signals to inform revision policies in incremental sequence labelling. Using generalised mixed-effects models, we find that the probability of regressions and skips by humans can potentially serve as useful predictors for revisions in BiLSTMs and Transformer models, with consistent results for various languages.

In Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL) , 2023
[PDF]

@inproceedings{Madureira-2023-4,
  title = {Revising with a Backward Glance: Regressions and Skips during Reading as Cognitive Signals for Revision Policies in Incremental Processing},
  author = {Madureira, Brielen and {\c{C}}elikkol, Pelin and Schlangen, David},
  editor = {Jiang, Jing and Reitter, David and Deng, Shumin},
  booktitle = {Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL)},
  month = dec,
  year = {2023},
  address = {Singapore},
  publisher = {Association for Computational Linguistics},
  url = {https://aclanthology.org/2023.conll-1.22},
  doi = {10.18653/v1/2023.conll-1.22},
  pages = {335--351}
}