035 - Indexicals and LLMs: Testing Context-Tracking in Language Models
Project Overview
One Liner: Testing whether large language models track who is speaking, to whom, where, and when, or just pattern-match on the surface form of words like “I,” “you,” “here,” and “now.”
Indexical expressions such as “I,” “you,” “here,” and “tomorrow” have meanings that depend systematically on the context of utterance: the same sentence picks out different people, places, and times depending on who says it, to whom, where, and when. Following Kaplan’s distinction between character and content, interpreting an indexical correctly requires updating the relevant contextual coordinate while holding others fixed. This project uses indexicals as a controlled test of whether large language models genuinely track context or rely on surface-level patterns. We construct stimuli that vary a single speech-act coordinate (speaker, addressee, location, or time) while keeping all others constant, and evaluate whether models show selective contextual dependencem shifting interpretation along the manipulated dimension while remaining invariant along the rest. The project contributes an empirical evaluation of context-tracking in LLMs and a conceptual account of what successful indexical interpretation would require of a language model.
No video available.
Screenshots
0 image(s)No screenshots uploaded yet.