Agents

Published 7 Jan 2026. Written by Jakob Kastelic.

While some are still discussing why computers will never be able to pass the Turing test, I find myself repeatedly facing the idea that as the models improve and humans don’t, the bar for the test gets raised and eventually humans won’t pass the test themselves. Here’s a list of what used to be LLM failure modes but that are now more commonly observed when talking to people.

Don’t know when to stop generating

This has always been an issue in conversations: you ask a seemingly small and limited question, and in return have to listen to what seems like hours of incoherent rambling. Despite exhausting their knowledge of the topic, people will keep on talking about stuff you have no interest in. I find myself searching for the “stop generating” button, only to remember that all I can do is drop hints, or rudely walk away.

Small context window

The best thing about a good deep conversation is when the other person gets you: you explain a complicated situation you find yourself in, and find some resonance in their replies. That, at least, is what happens when chatting with the recent large models. But when subjecting the limited human mind to the same prompt—a rather long one—again and again the information in the prompt somehow gets lost, their focus drifts away, and you have to repeat crucial facts. In such a case, my gut reaction is to see if there’s a way to pay to upgrade to a bigger model, only to remember that there’s no upgrading of the human brain. At most what you can do is give them a good night’s sleep and then they may possibly switch from the “Fast” to the “Thinking” mode, but that’s not guaranteed with all people.

Too narrow training set

I’ve got a lot of interests and on any given day, I may be excited to discuss various topics, from kernels to music to cultures and religions. I know I can put together a prompt to give any of today’s leading models and am essentially guaranteed a fresh perspective on the topic of interest. But let me pose the same prompt to people and more often then not the reply will be a polite nod accompanied by clear signs of their thinking something else entirely, or maybe just a summary of the prompt itself, or vague general statements about how things should be. In fact, so rare it is to find someone who knows what I mean that it feels like a magic moment. With the proliferation of genuinely good models—well educated, as it were—finding a conversational partner with a good foundation of shared knowledge has become trivial with AI. This does not bode well for my interest in meeting new people.

Repeating the same mistakes

Models with a small context window, or a small number of parameters, seem to have a hard time learning from their mistakes. This should not be a problem for humans: we have a long term memory span measured in decades, with emotional reinforcement of the most crucial memories. And yet, it happens all too often that I must point out the same logical fallacy again and again in the same conversation! Surely, I think, if I point out the mistake in the reasoning, this will count as an important correction that the brain should immediately make use of? As it turns out, there seems to be some kind of a fundamental limitation on how quickly the neural connections can get rewired. Chatting with recent models, who can make use the extra information immediately, has deteriorated my patience regarding having to repeat myself.

Failure to generalize

By this point, it’s possible to explain what happens in a given situation, and watch the model apply the lessons learned to a similar situation. Not so with humans. When I point out that the same principles would apply elsewhere, their response will be somewhere along the spectrum of total bafflement on the one end and on the other, a face-saving explanation that the comparison doesn’t apply “because it’s different”. Indeed the whole point of comparisons is to apply same principles in different situations, so why the excuse? I’ve learned to take up such discussions with AI and not trouble people with them.

Failure to apply to specific situation

This is the opposite issue: given a principle stated in general terms, the person will not be able to apply it in a specific situation. Indeed, I’ve had a lifetime of observing this very failure mode in myself: given the laws of physics, which are typically “obvious” and easy to understand, I find it very difficult to calculate how long before the next eclipse. More and more, rather than think these things through myself, I’d just send a quick prompt to the most recent big model, and receive a good answer in seconds. In other words, models threaten to sever me not only from other flawed humans, but from my own “slow” thinking as well!

Persistent hallucination

Understood in the medical sense, hallucination refers to when something appears to be real even as you know very well it isn’t. Having no direct insight into the “inner mental life” of models, we claim that every false fact they spit out is a form of hallucination. The meaning of the word is shifting from the medical sense towards the direction of “just being wrong, and persistently so”. This has plagued human speech for centuries. As a convenient example, look up some heated debate between proponents of science and those of religion. (As if the two need be in conflict!) When a model exhibits hallucination, often providing more context and evidence will dispel it, but the same trick does not appear to work so well on humans.

Conclusion

Where to go from here? One conclusion is that LLMs are damaging the connection people feel with each other, much like a decade before social networks threatened to destroy it by replacing it with a shallower, simulated versions. Another interpretation would be to conclude cynically that it’s time humans get either enhanced or replaced by a more powerful form of intelligence. I’d say we’re not there yet entirely, but that some of the replacement has been effected already: I’ll never again ask a human to write a computer program shorter than about a thousand lines, since an LLM will do it better.

Indeed, why am I even writing this? I asked GPT-5 for additional failure modes and found more additional examples than I could hope to get from a human:

Beyond the failure modes already discussed, humans also exhibit analogues of several newer LLM pathologies: conversations often suffer from instruction drift, where the original goal quietly decays as social momentum takes over; mode collapse, in which people fall back on a small set of safe clichés and conversational templates; and reward hacking, where social approval or harmony is optimized at the expense of truth or usefulness. Humans frequently overfit the prompt, responding to the literal wording rather than the underlying intent, and display safety overrefusal, declining to engage with reasonable questions to avoid social or reputational risk. Reasoning is also marked by inconsistency across turns, with contradictions going unnoticed, and by temperature instability, where fatigue, emotion, or audience dramatically alters the quality and style of thought from one moment to the next.