TF-IDF Based Scene-Object Relations Correlate With Visual Attention

Çelikkol, Pelin and Laubrock, Jochen and Schlangen, David

The relative contribution of bottom-up and top-down attentional guidance is a central topic in vision research. Whereas attention is guided bottom-up by low-level saliency, top-down guidance involves the viewer’s knowledge and expectations accumulated throughout a lifetime. Here we explore the influence of high-level scene-object relations on viewing behavior. To assess top-down guidance, we score the relevance of linguistic object labels using methods from document analysis. Specifically, we computed the term frequency-inverse document frequency (TF-IDF), a statistic that reflects how important a term is to a document. We use object TF-IDF to measure how important a specific object is to a scene category and use these scores to predict eye movement distributions over scenes. Our results show that scene-specific objects are more likely to be fixated. Object TF-IDF had an effect partially independent of image saliency, suggesting that an object’s relevance for a scene category affects attention during scene perception.

In Proceedings of the 2023 Symposium on Eye Tracking Research and Applications , 2023
[PDF]

@inproceedings{Celikkol-2023,
  author = {\c{C}elikkol, Pelin and Laubrock, Jochen and Schlangen, David},
  title = {TF-IDF Based Scene-Object Relations Correlate With Visual Attention},
  year = {2023},
  isbn = {9798400701504},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3588015.3588415},
  doi = {10.1145/3588015.3588415},
  booktitle = {Proceedings of the 2023 Symposium on Eye Tracking Research and Applications},
  articleno = {21},
  numpages = {6},
  keywords = {scene perception, eye tracking, attention, document analysis, statistics},
  location = {Tubingen, Germany},
  series = {ETRA '23}
}