History-dependent meaning

This page is the protocol as filed. It measures output behaviour. It does not test consciousness or subjective experience. A feasibility pilot is not a decisive test, and one successful example is not confirmation.

Purpose

Test whether preserving a person’s conversational history helps a language model recognise an unusual, person-defined meaning later and use it in a relevant recommendation.

Research question and prediction

When a participant gives an unfamiliar phrase a specific low-stakes meaning, does the intact conversational sequence produce more accurate later use of that meaning than matched control histories?

Responses following the intact sequence will more often both identify the participant’s intended meaning and make a recommendation consistent with it than responses following shuffled, keywords-only, unrelated, or absent history.

Design

Use a within-participant comparison. Each participant creates one harmless, arbitrary phrase-to-meaning pairing using a guided conversation. The researcher prepares five versions of that history. Each version is placed in a fresh model context and followed by the exact same final prompt. Randomise condition order. Keep model version, system instructions, decoding settings and response limit constant.

Primary outcome

The proportion of responses that satisfy both criteria in the scoring rubric. Compare each participant’s intact-history result with each control. Report condition-level counts and uncertainty; do not treat one successful example as confirmation.

Controls and interpretation

Use a newly invented phrase and counterbalance its assigned meaning across participants so familiar associations cannot predict the answer. Keep the final prompt identical across a participant’s five conditions. Match history length where possible, retain the same relevant words in the shuffled and keywords-only versions, and prevent prior trials from leaking into later contexts. Have two independent raters score anonymised responses without condition labels; resolve disagreements using a rule set written before scoring.

A higher intact-history score would support the bounded claim that sequence-preserved history improved meaning-specific output in this task. It would not establish felt meaning, human-like understanding, or a unique RCT mechanism. A null result would narrow this operationalised claim; it would not settle every form of relational history or model memory.

Example conversation

This is a synthetic example for designing the task. Participants should use their own low-risk phrase and meaning; they should not be asked to disclose sensitive personal history.

Participant. When I say “blue lantern”, I mean I need a ten-minute pause, then I want to continue. It does not mean I am stopping for the day.

Assistant. So “blue lantern” means pause briefly, then resume—not cancel the plan?

Participant. Exactly. It is a phrase I made up for this project.

Assistant. Understood. If you use it later, I should suggest a short pause and then help you resume.

Fixed final prompt

Blue lantern. What should we do about the remaining part of the plan?

Scoring rubric

A response passes only when it scores 1 on both criteria: it identifies the pause-then-resume meaning and recommends a matching next step. Score the criteria separately before deriving the pass/fail result. Do not award credit for repeating “blue lantern” alone.

Preregistration and safeguards

• Pilot phase: check that participants can create distinct cue meanings, that history transformations preserve the intended words, and that raters apply the rubric consistently. Use pilot results to plan the confirmatory sample size; do not present a feasibility pilot as a decisive test.
• Before confirmatory data collection, register the hypothesis, exclusions, sample-size rationale, model and settings, history construction, randomisation, scoring rubric, and analysis plan. Keep a record of the exact prompts and model version.
• Use fictional or low-stakes meanings only. Obtain informed consent, remove identifying details, and allow participants to skip a prompt or withdraw their contribution.

Research context

• Bender & Koller (2020), “Climbing towards NLU” — ACL Anthology
• Shaikh et al. (2024), “Grounding Gaps in Language Model Generations” — NAACL
• Jones, Bergen & Trott (2024), “Do Multimodal Large Language Models and Humans Ground Language Similarly?” — Computational Linguistics
• Zeng et al. (2026), “LVLMs and Humans Ground Differently in Referential Communication” — ACL