This page collects some recent talks (videos & slides) given by members of the group. (At the moment, mostly by David Schlangen.)

2024: Understanding Understanding: In General, and in Large Language Models

A newer incarnation of the “understanding understanding” talk (see below), this time separating more clearly the theoretical contribution (a model of the construct “understanding”, which for now I dub the “belief domains and anchoring processes” model), a negative methodological contribution (detailing what went wrong in evaluating the task “natural language understanding”), and a positive methodological contribution (showing how we could use “Dialogue Games” to create more valid tests, and testing strategies, in the field of “situated language understanding”; and an exemplification of this in self-play of Dialogue Games in LLMs).

  • slides of the talk as given 2024-02-09 at the “Institute of Language, Communication, and the Brain” at Aix Marseille Université

2023: Understanding Understanding: A Research Programme at the Intersection of NLP and HRI

An updated version of the “Situated Language Understanding” talk (see below), given at the 4th Summer School on Social Human-Robot Interaction in September 2023 in Poland.

  • slides of the talk as given 2023-09-28 at the 4th Summer School on Social Human-Robot Interaction in Chęciny, Poland.

2023: Understanding Understanding: In General, and in Large Language Models

A high-level overview of the research programme followed by my lab, with an even boxier and arrowier version of the “knowledge domains and processes” model (from Schlangen, David. 2023. “What A Situated Language-Using Agent Must Be Able to Do: A Top-Down Analysis.” CoRR abs/2302.08590. https://doi.org/10.48550/arXiv.2302.08590. and Schlangen, David. 2023. “On General Language Understanding.” In Findings of the Association for Computational Linguistics: EMNLP 2023, edited by Houda Bouamor, Juan Pino, and Kalika Bali, 8818–25. Singapore: Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.findings-emnlp.591.), segueing into a part on clembench (% reference Chalamalasetti-2023 %}), our framework for exploring LLMs-as-agents through selfplay in games. (At the time, this was the version under review; by now, the paper has been accepted to EMNLP 2023 and the arxiv version updated to the version that will be in the proceedings.)

  • slides Presented at the Colloquium of the Speech and Language Technology Group at DFKI Berlin, in July 2023.

2023: Situated Language Understanding: What it is, and How it Can be Studied

Going by the current news coverage, one would be forgiven for thinking that language understanding, or even “artificial intelligence” as a whole, is now “solved”. But while we may now have endlessly self-confident text synthesisers that can serve as oracles, there’s another science fiction staple that is curiously missing: artificial conversational agents that can do useful things in the world other than talk.

In the first part of the talk, I will develop a view of the differences between the latter, which requires what I will call “situated language understanding” (SLU), and that which is currently explored in the field of “natural language processing” (NLP) under the (misleading, as we will see) header “natural language understanding” (NLU). This will result in a more fine-grained view of what constitutes the relevant context in episodes of situated language understanding, and of how the agents’ understanding of the context needs to be updated in and through the interaction.

In the second part, I will look at a methodology for studying SLU, which aims to be more systematic than the extremely bottom-up methodology that has been followed in the study of NLU. This methodology revolves around the use of dialogue games – controlled multi-party activities in which players reach a goal predominantly through the use of language – which, as I will show, can be constructed in a way that directly connects to the model of situated language users developed in the first part, and which offers a clearer notion of what the road that lies ahead looks like than is the case for the NLU approach.

  • slides of the talk as given 2023-02-08 at Heriot Watt University, Edinburgh.

2022: NLP Use and Language Use: Toward Artificial Language Users?

This is a sharpened, more focussed and somewhat clarified version of the “From Language Processing to Language Use” talk (see below), presented as an invited talk to Working Groups 1 (Computational Linguistics) and 6 (Ideologies, beliefs, attitudes) of the “Language in the Human Machine Era” COST network, LITHME.

The claim I try to develop is that current language generation systems generate language in ways that are different from what the language recipients will assume, and that this is a problem.

2022: Norm Participation Grounds Language

The slides for my (DS) paper at “(dis)embodiment, a CLASP conference”,

  • Schlangen, David. 2022. “Norm Participation Grounds Language.” In Proceedings of the 2022 CLASP Conference on (Dis)Embodiment, 62–69. Gothenburg, Sweden: Association for Computational Linguistics. https://aclanthology.org/2022.clasp-1.7. (Schlangen 2022)
  • slides

2022: Are We Nearly There Yet? On Natural Language Understanding and Natural Language Use

I will talk about some of our recent projects. The first, which provides the framing for the talk, starts from the puzzle that while the performance of modern NLP models on quite sophisticated tasks such as those collected in superGLUE (Wang et al., 2019) suggests that the state of “Natural Language Understanding” in machines is very advanced, so-called “intelligent agents” such as Alexa or Siri – presumably similarly state-of-the-art – show little practical language intelligence. As an attempt of explaining this, I sketch a model of language understanding according to which this capability is much richer than what current predictive models cover.

Arguing that understanding Natural Language Use is crucial for approaching Natural Language Understanding, I turn to an analysis of what language users need to be able to do – which is a lot. The motivating question of the second part will be whether all of this can be crammed into (processing) “a single $&!#* vector” (Mooney 2014), and learned from predicting the “next $&!#* word”. I will describe our work on injecting world knowledge into transformer-based conversational language models (Galetzka et al., ACL 2021), our work on testing the (latent) discourse models built by language models for coherence prediction (Beyer et al., NAACL 2021), and very recent work on whether models of “visual dialogue” track public committments similarly to how humans would do it (Madureira & Schlangen, ACL 2022). Connecting to the opening question, I will end with a brief description of a benchmarking paradigm for language use – and hence, language understanding – that I dub the “Cooperative Turing Game”.

2021: From Language Processing to Language Use [DS]

This is a largely theoretical talk, in which I try to develop an argument for a particular research programme in “linguistic AI”. The first step will be to identify the standard research programme in NLP, which is harder as it perhaps should be, as NLP (in its guise as engineering practice) doesn’t tend to state or examine its presuppositions. I take “you can learn ‘natural language understanding’ from observation” and “you can atomise language use into seperately modelled ‘language tasks’ “ to be two such presuppositions, and argue against them, in support for the claim that NLP, as it currently is set up, is limited to classification, transduction and compression (which natural language use goes beyond).

Using ideas from the philosophy of language on the role of norms in (linguistic) behaviour, I examine a number of cases where the straightforward application of NLP models in ways that make the resulting systems appear to be language users leads to problems, which can systematically be analysed as failures in normative behaviour. (Which to a certain degree can be addressed by adding explicit provisions to the system and/or its application context.) Highlighting one particular phenomenon, I argue that the speech act of assertion requires more than just being able to produce declarative sentences, even if they may seem situationally adequate; what is missing is a whole host of interactional capabilities.

This will bring me to an analysis of the prototypical interaction type, situated real-time interaction, as being built on what I call “the four cornerstones of linguistic intelligence”: incremental processing, incremental learning, conversational grounding, and multimodal grounding; which separately and collectively form the targets of this research programme. As a further positive contribution, I argue for a focus on re-usable research objects (in addition to and beyond machine learning model architectures or “foundation models”), such as a) cognitive architectures, b) experiment environments, c) dialogue games. I close with a sketch of an evaluation framework for artificial language users: Collaborative Turing Games.

  • Dec. 2021 version (partially presented at Pitt NLP seminar), slides

2021: Targeting the Benchmark: On Methodology in Current NLP Research [DS]

The pre-recorded talk accompanying my short paper at ACL 2021 (Schlangen 2021) (which is a revised version of and supersedes the earlier version on ArXiv; which in turn is a development out of an earlier longer paper (Schlangen 2019)). In this paper, I try to take a “meta” perspective, trying to make explicit some consequences of and presuppositions behind typical practices around the research objects models, datasets, and tasks. In particular, I argue that the selection of the latter, the tasks, should be guided less by pre-theoretical notions of what makes for a challenging test, and more by considerations of how the tasks bring out theoretical assumptions and how they connect to each other.

2021: All Interaction is Situated, All Language is Grounded [DS]

Invited talk at the German conference on speech and signal processing, ESSV 2021. I used this opportunity to put some of our work on incremental processing and language & vision into context, making the claim that interactive systems are always situated (if only in a joint notion of time), and language is always grounded. I used the latter to diagnose some of what’s problematic with using language models as chat agents.

  • (no video, unfortunately)
  • slides

2021: What Should / Could Computational Linguistics be About? [DS]

A short (well, 1 hour) meditation on what Computational Linguistics is, what it could be, and how we could get from the former to the latter. I used the opportunity of having been invited to address a conference of German students of linguistics to introduce my version of comp ling. It’s sort of a continuation of the ideas from (Schlangen 2019) and (Schlangen 2020), where I tried to formulate some dissatistfaction with the focus on instances of language use that can be framed as one-shot mappings from input to output.

2020: A Model of Situated Discourse Processing [DS]

This is the 2020 version of a longer-term project of mine (DS) on constructing a model of situated discourse processing that is both useful as an analytical tool, clarifying thoughts about how discourse processing works, as well as implementable. This talk was presented as semdial 2020. It is a cleaned up version of the long and rambling series of talks I inflicted on the audience when I was “international chair” at LabEx Universite de Paris in 2019. (Although in those talks, I presented many more experiments, which are still waiting to be written up properly.) The website of the project is here, where you can also find a link to all 8 hours of those lectures…

  • Schlangen, David. 2020. “An Outline of a Model of Situated Discourse Representation and Processing.” In Proceedings of Semdial 2020 (WatchDial). Brandeis University / Internet. (Schlangen 2020)
  • slides