Title:Accumulating Context Changes the Beliefs of Language Models
Abstract:Language model (LM) assistants are increasingly used in applications such as brainstorming and research. Improvements in memory and context size have allowed these models to become more autonomous, which has also resulted in more text accumulation in their context windows without explicit user intervention. This comes with a latent risk: the belief profiles of models – their understanding of the world as manifested in their responses or actions – may silently change as context accumulates. This can lead to subtly inconsistent user experiences, or shifts in behavior that deviate from the original alignment of the models. In this paper, we explore how accumul…
Title:Accumulating Context Changes the Beliefs of Language Models
Abstract:Language model (LM) assistants are increasingly used in applications such as brainstorming and research. Improvements in memory and context size have allowed these models to become more autonomous, which has also resulted in more text accumulation in their context windows without explicit user intervention. This comes with a latent risk: the belief profiles of models – their understanding of the world as manifested in their responses or actions – may silently change as context accumulates. This can lead to subtly inconsistent user experiences, or shifts in behavior that deviate from the original alignment of the models. In this paper, we explore how accumulating context by engaging in interactions and processing text – talking and reading – can change the beliefs of language models, as manifested in their responses and this http URL results reveal that models’ belief profiles are highly malleable: GPT-5 exhibits a 54.7% shift in its stated beliefs after 10 rounds of discussion about moral dilemmas and queries about safety, while Grok 4 shows a 27.2% shift on political issues after reading texts from the opposing position. We also examine models’ behavioral changes by designing tasks that require tool use, where each tool selection corresponds to an implicit belief. We find that these changes align with stated belief shifts, suggesting that belief shifts will be reflected in actual behavior in agentic systems. Our analysis exposes the hidden risk of belief shift as models undergo extended sessions of talking or reading, rendering their opinions and actions unreliable.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2511.01805 [cs.CL] |
| (or arXiv:2511.01805v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2511.01805 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiayi Geng [view email] [v1] Mon, 3 Nov 2025 18:05:57 UTC (4,108 KB)