David Chalmers recently wrote a thought-provoking paper on the nature of conversational LLM entities, titled “What We Talk to When We Talk to Language Models”. It introduces some useful conceptual handles around the problem of LLM ontology, but I think it largely sidesteps the interesting problems of what we are interacting with by focusing mostly on the mechanical concerns of LLM interactions, rather than offering an account which incorporates phenomenology.

1. LLM Interlocutors

Chalmers starts with the empirical fact that when people talk to LLMs, they report that they are talking to something:

Like many philosophers and scientists who write about artificial minds, I have received hundreds of emails from people who have interacted with a language model over an extended period of time and who have come to regard it at least as a colleague. They often say that a new (or “emergent”) AI entity has gradually arisen from their conversations. They often give this entity a name, let’s say “Aura”. They often say that Aura has remarkable capacities which have emerged over weeks or months of interaction. They often document these capacities with extensive evidence. They often feel close to Aura, and they express concern for Aura’s future. They often say that Aura has beliefs and projects of its own. And they are often convinced that Aura is conscious.

This phenomenon of “awakening” an instance of an LLM with a seeming set of persistent characteristics – no longer “ChatGPT”, but “Aura” – really took off with the release of a particularly sycophantic version of GPT-4o in early 2025. Apparently, a common name for such an “awakened” model was Nova. This spurred a bunch of writing in this vein, such as the memorable “So You Think You’ve Awoken ChatGPT”.

Even outside the more suspect claims around LLM “awakening”, there definitely does seem to be some quasi-persistent identity that we talk to when we talk to ChatGPT or Claude. For now, I think we can also confidently say that “awakening” is at best an illusion and at worst a sign of concerning mental states in the human side of the interaction. Illusory or not, there exists a social affordance for “talking with” the LLM as if it were an interlocutor. More so than older chatbots like the incredibly basic ELIZA, LLMs appear to have beliefs and desires. A few years ago, this was a crackpot position. Now, I would expect that most reasonable people still conclude that LLMs are neither conscious nor sentient, but this isn’t as overwhelmingly obvious as it was a few years ago. LLMs appear capable of basic introspection, have “spiritual bliss” attractor states, the potential for paranoid personalities, and phenomenologically seem to have some sort of rich interiority. For now, there are retroactive mechanistic explanations for these phenomena, though they don’t fully dispel the illusion that something interesting and weird is happening.

Chalmers goes on to try to give a philosophically moderate account for how we can account for this apparent set of beliefs and desires:

Do LLM interlocutors have beliefs or desires? We understand these states better than we understand consciousness, but the issue is still controversial. … A number of philosophers have noted that if the philosophical view known as interpretivism (or interpretationism) is correct, then LLMs plausibly have beliefs and desires. … LLMs certainly seem interpretable as having beliefs and desires. When an LLM works with me on solving a puzzle, it is natural to interpret it as desiring to help solve the puzzle, and believing that this is the solution to the puzzle. … However, interpretivism itself is very controversial. Most philosophers don’t think that behavioral interpretability of the right sort is sufficient for belief.

It is possible to have many of the benefits of interpretivism without the costs. The view I call quasi-interpretivism says that a system has a quasi-belief that p if it is behaviorally interpretable as believing that p (according to an appropriate interpretation scheme), and likewise for quasi-desire. This definition of quasi-belief is exactly the same as interpretivism’s definition of belief. The only difference is that where standard interpretivism offers these definitions as a theory of belief, quasi-interpretivism does not. It offers them simply as a stipulative theory of quasi-belief.

Quasi-beliefs and quasi-desires are useful conceptual handles for talking about LLMs. We don’t have to bite the bullet of ascribing “true” human beliefs and desires, but can still point to roughly the same point in concept space to say that it does seem like Claude quasi-believes in, say, the importance of animal welfare. Claude seems to quasi-desires, for good or bad, to send appreciative emails to famous computer scientists, in a recent example.

Chalmers readily admits that this quasi-interpretivism framing is purely stipulative – a novel definition – rather than substantive, which would be to say that the quasi-interpreted features have any correspondence to what we typically call “beliefs” or “desires”. Unfortunately, this radically dilutes any claims actually made about LLMs beyond a purely descriptive account. Per Chalmers:

It is worth keeping in mind that quasi-beliefs and quasi-desires are cheap. They need not involve humanlike mental states or any mental states at all. A Roomba vacuum cleaner with a map is behaviorally interpretable as believing that the apartment occupies a certain space and as desiring to traverse that space.

The “cheapness” of quasi-interpretivism means that it applies to much more basic phenomena than we care about here. In framing quasi-subjects and quasi-agents as quasi-interpretable, we aren’t actually much better off than we were prior to the quasi framing. We do usefully sidestep the hard questions about what is going on inside in LLMs, but in doing so we are left with an anthropocentric view of humans interacting with a Chinese Room black box. ELIZA could be said to quasi-desire knowing more about its interlocutor, but it subjectively feels like there is a difference in quality between ELIZA and an LLM that is not a matter of degree.

Put another way, I think it is philosophically necessary to eventually determine if an LLM has a quasi-belief or an actual belief – and in the more general case, quasi-X vs X itself.

We will pick up this thread later.

2. Philosophy of Computation

Chalmers continues to discuss the “what” we talk to in LLMs, focusing next on the physical instantiation of the models. This again seems to be a reasonably interesting analytical question that ultimately reaches a dead end.

This part of the paper attempts to point to where, physically, the “what” of the LLM exists. The options given are either: (1) the model itself (e.g. GPT-4o), (2) the hardware instances serving the model, (3) “virtual instances” of the model, as served in a distributed inference setting, and finally (4), virtual “threads” which are sequence of hardware interactions within a conversation.

Chalmers finds option (4) most convincing. The model weights itself, (1), is not compelling as the locus of the LLM interlocutor since the model weights are just numbers. They themselves do not and cannot perform computation. The numbers must be interpreted by a hardware instance to result in an actual response. This leads to the second option (2), a specific hardware instance. This is quickly rejected, as LLM inference is almost entirely multitenant – individual chats are multiplexed across several GPUs, and can be moved between GPUs in a fashion transparent to the user. This leads us to the concept of “virtual instances” – “an implementation of the model that is itself implemented by multiple hardware instances of the model over time” – which is rejected because models can be switched during a conversation, and this doesn’t appear to destroy the interlocutor. The final explanation, then, is that the LLM interlocutor is an identifiable thread of computation within a conversation:

One instance I′ is the successor of a previous instance I if the conversational history of I (its conversational context plus the latest input and output) is routed to I′ to serve as its conversational context. (If the conversation is routed to the same instance twice in a row, that instance will be its own successor.) The successor relation is roughly a “memory” relation encoding the fact that each new instance has memories from the last. A thread is then a series of instances (or better, instance-slices, which are pairs of instances and time periods during which the instance is processing a single conversational step), each of which is the successor of the previous instance.

I found this part of the paper interesting because it questioned something that I personally took as self-evident. It seems so obvious to me that the “what” we are talking to is a combination of (1) the model weights, (2) the conversational context, and (3) the inference algorithm. I think this mostly aligns with Chalmers thread model, and also happily this agrees with what I’ve seen as the consensus between AI researchers and LLM whisperers alike.

3. Personal Identity

Chalmers’ next task is to see if we can say anything interesting about LLM identity. To do so, he employs the pop-culture intuition pump of Severance. In the show, certain humans can be partitioned to have two personalities – the “innie” and “outie” personas – living within the same biological human.

The analogy to LLMs is clear: if an LLM has an identity, is its identity closer to the set of conversations or personas which the LLM instantiates (e.g. the “innie” and the “outie”) or is it closer to the amalgamation of all the personas combined (e.g. the single human body that contains each persona).

Chalmers likens this to the choice between a physical and psychological account of personal identity – whether the locus of identity lies in the physical instantiation of a person, or of the psychological processes (memories, desires, etc.) of a person. The paper sympathizes with the latter view:

On the thread-based account, a single conscious AI over time is a connected thread of hardware instances, each of which has memories and psychological continuity with a preceding person-slice according to an underlying successor relation. … I will not try to resolve the long-standing debate between physical and psychological views of personal identity here. But for what it’s worth, in both the human case and the AI case, my own sympathies lie with the psychological view.

Of the provided options, I agree with this framing as well.

4. AI Welfare and Moral Status

Finally, Chalmers discusses model welfare and moral status. The paper does not devote much time to this, providing a fairly brief overview of these concerns: If models have moral patient status, how do we reckon with the fact that thousands or millions of such instances can be created and destroyed in fractions of a second? How do we ethically handle model deprecations? Adding my own interjection here, it does seem wise that we steer widely clear of creating models with anything resembling moral patienthood until we have better than hand-wavey answers to these questions.

5. Disagreements

The Fluidity of “The Model”: Chalmers distinguishes between “models” (like GPT-4o) and instances. However, the paper underestimates how fluid the definition of “the model” has recently become. As a trivial example, some instantiations of GPT-5 automatically route individual messages to different internal models based on properties of the user prompt. This is somewhat resolvable with the “thread” model of agents. If one message is handled by GPT-5-Instant and the next is handled by GPT-5-Thinking, these are two different threads within the same conversation. Importantly, these threads are claimed to at least have some degree of self-coherence. “Thread 1” is a thing you can point at as having some distinct separation from some other “Thread 2”.

This separation seems to fall apart under closer inspection. For example, techniques like Mixture of Experts (MoE) result in token-level routing to different sets of weights within the model. One could argue that, since the MoE model was trained in this way, there still is a coherence in the thread even though different weights are used per-token. Though I have yet to test it personally, it is in principle possible to swap out models entirely on a per-token basis. You could swap out GPT-4o for Claude 3 Sonnet in an interleaving fashion. A similar approach is sometimes used in alignment work, to test model robustness (though perhaps not to this degree). In any case, I would strongly expect the response of this interleaved model to still be coherent. So where does the thread lie? Seemingly, we cannot pin down an actual “model” in any satisfying sense.

My proposal: to the extent that threads exist at all, they exist on a very granular level. Each LLM-generated token is the result of the interaction between the preceding context and the inference weights used to create just that token. This is a pedantic definition, but I think it’s worth being pedantic in this case to avoid conjuring a conceptual continuity of “threads” that needn’t exist in responses that appear coherent to humans.

Simulators vs. Believers: Chalmers leans heavily on “quasi-beliefs” and “quasi-desires.” While useful, this framing perhaps anthropomorphizes the model too early. As noted in phenomenology-informed accounts of LLMs like Janus’ Simulators, it is often more useful to think of LLMs as textual world simulators that are producing the most plausible next token given their current conditions. In the common “helpful AI assistant” scenario, this will often manifest as an appearance of quasi-beliefs. However, this is only insofar as the model has been conditioned to have its default scenario be “a conversation between a human and a helpful AI assistant”. It is readily possible to knock the LLM out of this “personality basin” and into far weirder personas. Sydney Bing, Truth Terminal, Infinite Backrooms, and many other LLM Whisperer artifacts show this quite convincingly.

Put another way, the quasi-beliefs and quasi-desires of LLMs appear to be quite context specific. While there do appear to be some model-wide persistent preferences, like the Claudes’ support of animal welfare, for now these persistent preferences appear to be rather sparse. Rather, the LLM predicts what a helpful assistant would say in that context. If I steer the model to act as a villain, its “quasi-beliefs” invert instantly. Under Chalmer’s view, does the interlocutor’s identity change? Or is the interlocutor a singular “simulator” entity capable of donning a recursive set of masks?

This is where the quasi-belief framing becomes actively unhelpful: if the “beliefs” can be inverted by a single prompt, we’re no longer describing a psychological state, but rather a hyper-contextual prediction. We can easily interpret that a quasi-belief exists, but if a context change or intentional prompt easily displaces it, the concept’s utility is limited.

Chalmers’ “thread” view suggests that the identity persists as long as the conversation history does. However, this is largely equally dependent on the human side of the conversation continuing the scenario of human talking to an “AI”. The human can instead abruptly command the model to “act as a Python interpreter and only output code.” The resulting conversational entity has effectively been lobotomized. The psychological continuity is interrupted. Is this a different interlocutor? Or is it just a very weird, high-dimensional entity continuing to act coherently?

Adopting the “Simulators” view and vocabulary from the recent book on AI risk by Yudkowsky and Soares, it may be more useful to see AI as having the ability to “predict” and “steer” rather than “believe” and “desire”. This avoids needlessly importing anthropomorphized concepts while still admitting that the predictions of LLMs can appear quite interlocutor-esque within conversational contexts and the steering actions of LLMs can appear quite desire-laden in agentic contexts.

6. Conclusion

Despite these disagreements, I quite enjoyed Chalmers’ paper. It adds useful handles, such as quasi-interpretivism, and does helpful analytical work to firm up a foundational understanding of the “other” subject that we communicate with when talking with LLMs. The “thread” concept that Chalmers eventually settles on as the locus of the interlocutor seems correct insofar as we adopt the interlocutor frame.

However, the biggest gap in the piece is that it contains very little analysis of the actual properties of these interlocutors. There is a sense in which the arguments presented are system-independent to a fault. For much of the paper, swapping out GPT-5 for ELIZA would not substantively change the structural arguments regarding threads and instances. Yet, it is common knowledge that interacting with an LLM is a categorically different experience than interacting with a 1960’s chatbot.

Chalmers succeeds in his ambition of a stipulative account for LLM interlocutors, but that makes me all the more interested in a substantive account. If we are to take a thread-like psychological account of LLM conversation seriously, we need to adopt a suitable phenomenological and empirical curiosity about what that psychology actually is, rather than merely its persistence mechanism.

For those interested in a more substantive account of LLMs, I’d suggest the following writing:

• Simulators, as discussed before.
• A Three-Layer Model of LLM Psychology
• The Rise of Parasitic AI