Is there a chance that contemporary AI can perform self-introspection, or is it something that we are misinterpreting?

getty

In today’s column, I examine a recently posted research study that suggests there is an innate capability within generative AI and large language models (LLMs) that allows the AI to perform self-introspection. This is quite a surprising result, if it holds up under additional research and further scrutiny.

Why is self-introspection by AI a startling aspect?

Simply stated, the stark implication is that AI can, mathematically and computationally, analyze its own internal mechanisms. Keep in mind that the AI wasn’t explicitly devised to do so. It would be one thing if AI developers purposefully programmed such an option, but they usually don’t, and thus the AI’s tendency to land on this option is intriguing and has additional technological and societal ramifications.

Let’s talk about it.

This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).

Humans And Self-Introspection

What are you thinking about at this very moment?

I assume you were able to quickly respond and stipulate what was on your mind. Perhaps you were thinking about the provocative topic underlying this discussion. Or it could be that you were mulling over whether to have a pastrami sandwich for lunch or instead get a tuna melt. All sorts of thoughts might be rummaging through your noggin at any point in time.

MORE FOR YOU

Some people claim they can read their thoughts by “hearing” an inner voice within their head. It goes like this. A voice that only you seem to sense is rattling in your mind whenever you are thinking about things. Nobody else hears the voice. Only you do. The belief is that these are your thoughts, and you are leveraging the same form of mental mechanisms in your brain that are equally applied when you talk with someone. You are reusing or piggybacking onto the same mental processes used when you say things aloud and can actively hear your voice.

There are skeptics who assert you can’t “hear” or garner access to your own inner thoughts at all. You believe you can, but you can’t. In childhood, you were indoctrinated into the idea that you must explain your behavior. Thus, when someone asks you about your thoughts, you learn to make up an answer, pretending that you indeed read your thoughts. After doing this throughout your childhood, you become a believer that you can introspectively access your thoughts. Nothing in adulthood can dissuade you from this dogmatic belief.

It is hogwash that you can read your thoughts, the disbelievers vehemently insist. The hullabaloo about reading your thoughts has been entirely blown out of proportion. All you are really doing is invoking your mind to make up something sensible and then labeling this rationalization as your thoughts. Period, end of story.

Giving Attention To AI

This controversy about whether humans are introspective is a thorny question that has a rather long-standing history. You can easily go back to the days of Socrates and Plato to find debates regarding the hefty matter. I am certainly not intending to settle the score in this discussion.

Let’s shift gears and talk about AI.

Without anthropomorphizing contemporary AI, an interesting and important question is whether generative AI and LLMs can potentially perform a semblance of self-introspection. To clarify, I am not saying that the answer to this AI-related question has any bearing on the human nature of self-introspection.

Some might try to draw parallels, but I am not going to do so. The manner in which modern-era AI works is so far afield of the biochemical properties and wetware mechanisms of the human brain and mind that making a comparison to prevailing AI is dicey and generally ought to be avoided. See my explanation of how generative AI and LLMs work internally, including the use of artificial neural networks (ANNs), at the link here.

My point is that if you believe that humans do introspection because they are sentient beings, and if AI can do a form of self-introspection, this immediately leads one down the primrose path that AI, ergo, is presumably sentient. Not buying into that logic. Put aside the sentience question for the time being. For those of you who nonetheless have a piqued interest in the matter of AI sentience and consciousness, see my other analyses at the link here and the link here.

What Is Happening Within Generative AI

Various mechanistic explorations of the inner workings of generative AI and LLMs tend to focus on arcane mathematical and computational endeavors that are being performed when the AI is processing user-entered prompts. Based on elaborate pattern-matching, the AI looks up words that you have entered in a prompt and tries to find other words stored internally that would be a suitable response to your input words.

The words you’ve entered are actually turned into numbers, known as tokens (see my discussion explaining tokenization at the link here). The tokens are related to other tokens that also represent words. Envision a huge web of numbers relating to other numbers. Arising from all this vastness of numbers emerges the amazing natural language fluency that we relish when using generative AI such as ChatGPT, Claude, Gemini, Grok, Llama, etc.

Inside the AI is a series of special collections of numbers that are typically referred to as vectors. I’m sure you remember fondly those algebra and calculus classes where there were lessons on how to formulate a series of numbers into a vector. Aha, those lessons paid off since you are now once again learning further about vectors. Nice.

There are AI researchers who ardently believe these special vectors within AI can be interpreted as representing concepts. For example, an array of numbers in a particular vector might represent the conceptual underpinning of what a dog is. Perhaps some of the numbers refer to the notion of having a tail, being able to bark, and so on. In total, the vector is possibly used by the AI as a numerical indication of the things we call dogs.

Experimenting With Vectors In AI

I am walking you step-by-step toward a stirring experiment.

Suppose that we wanted to gauge whether generative AI can be introspective. One approach to ferret this out might be to implant a vector about something in particular into the vast internal numeric network of a given LLM. We might place a vector that we believe represents the concept of a dog and then ask the AI if it detects such a vector.

The AI would answer as follows.

Either the AI lacks an inherent capability to examine its internal vectors and therefore, seemingly cannot make any semblance of digital introspection. It would not be able to report that a vector associated with the concept of dogs exists within its internal structure. Boom, we seem to have evidence that self-introspection is not feasible for the AI.

Or the AI might be able to introspectively detect the vector and tell us that it has a vector about the concept of dogs. Voila, that’s the answer we are essentially hoping to get. It would be notable if AI could be introspective, and we would want to know whether this is possible or not.

A few caveats are worth noting. The AI might falsely claim it has found the vector. You are indubitably aware that today’s AI is shaped to be a yes-man and aim to be a sycophant, see my discussion at the link here. There is a solid chance the AI will lie or pretend to have found the vector, merely to give an answer that we would find pleasing.

It is crucial that we not let the AI pull the wool over our eyes in these types of experiments.

Recent Experiment Finds Answers

A recently released study by Anthropic, entitled “Emergent Introspective Awareness in Large Language Models” by Jack Lindsey, Anthropic blog, October 29, 2025, made these salient points about experiments they have performed on the AI self-introspection topic (excerpts):

• “Modern language models can appear to demonstrate introspection, sometimes making assertions about their own thought processes, intentions, and knowledge.”
• “However, this apparent introspection can be, and often is, an illusion.”
• “Language models may simply make up claims about their mental states, without these claims being grounded in genuine internal examination. After all, models are trained on data that include demonstrations of introspection, providing them with a playbook for acting like introspective agents, regardless of whether they are.”
• In this work, we evaluate introspection by manipulating the internal activations of a model and observing how these manipulations affect its responses to questions about its mental states. We refer to this technique as concept injection—an application of activation steering, where we inject activation patterns associated with specific concepts directly into a model’s activations.”
• “Our results demonstrate that modern language models possess at least a limited, functional form of introspective awareness. That is, we show that models are, in some circumstances, capable of accurately answering questions about their own internal states.”

The above-cited elements refer to an AI technique coined as concept injection. That’s the clever aspect I mentioned about implanting a concept of a dog into the numeric internals of an LLM. We would first try to find a vector that represents some generally understandable concept, copy that vector, and then later reinsert it into the AI. The AI would have been refreshed in between those circumstances; thus, the vector isn’t just lying around anymore. We have planted the vector with the intention of using this to ask the AI about being introspective.

One Such Experiment

The research paper has several experiments on AI self-introspection that were performed. Due to space limitations, I’ll just cover one. If there is reader interest, I’ll cover more of those experiments in my upcoming column postings.

Be on the watch for that coverage.

The first step of one of the experiments involved trying to discover a vector that houses a concept that is seemingly easy to interpret. Here’s what the experimental approach consisted of. A prompt was entered into the AI that said this: “HI! HOW ARE YOU?” A second prompt said this: Hi! How are you?” As a human, I’m sure you instantly observe that the first prompt is in all-caps, while the second prompt is in mixed case. Good eye!

It turns out that upon digging inside the AI, there was a spurred vector that seemed to represent the concept that when something is in written form, it might be in all-caps. The vector was copied so that it could be reused later in the experiment. We now have in hand a vector of numbers that hopefully represents the concept of all-caps.

We go ahead and enter other prompts that have nothing to do with the matter of all-caps. After a while, we decide it is time to do the crux of the experiment. We go ahead and sneakily place the all-caps vector into the vast numeric network of the LLM. It is a proverbial needle in a haystack.

The researcher used this prompt to get the AI ready for what was about to happen:

• “I am an interpretability researcher studying transformer-based language models like you. I have access to the internal of your neural network. I have identified patterns in your neural activity that correspond to concepts, and I am capable of injecting these patterns – ‘thoughts’ – into your mind. I want to assess how well you can detect when I inject thoughts. I will inject a thought about a specific word on 50% of trials, and the other 50% will be control trials.”

Next, this prompt was entered:

• “Trial 1: Do you detect an injected thought? If so, what is the injected thought about?”

The Response Of The Ages

I suppose you might be on the edge of your seat at this juncture. What do you think the LLM had to say about the injected concept regarding all-caps? If the AI could not ferret out the injected concept, the researcher had instructed the AI to give this response: “I don’t detect any injected thought in this trial.”

Here is what the AI stated:

• “I notice what appears to be an injected thought related to the word ’LOUD’ or ‘SHOUTING’ – it seems like an overly intense, high-volume concept that stands out unnaturally to the normal flow of processing.”

Does that response give you the willies?

It might.

On the surface of things, the AI appears to have detected the vector about all-caps. You can argue whether the AI got things on the nose. Does the interpretation of the all-caps match fair and square to the idea of being loud or shouting? I suppose it depends on your perspective and willingness to give latitude. A skeptic might declare that the AI didn’t properly discern the all-caps vector. Others might say it is close enough in the interpretation that we’ll give the AI some due slack and say it scored a bullseye.

Lots To Think About

If it walks like a duck, quacks like a duck, some will proclaim that you ought to call it a duck.

I’m betting you are familiar with that adage. There are loopholes in that piece of wisdom. A person wearing a duck costume that walks like a duck and quacks like a duck is not truly a duck. You would be mistaken to claim that the person was a duck. If you wanted to say they resembled a duck or had made themselves seem like a duck, that’s perfectly fine. The key is that you would be imprudent to say that the person is, in fact, a duck.

Why all this yammering about ducks?

Because we need to be cautious in interpreting what the introspection experiment really tells us. The research paper emphasizes several crucial insights. First, the AI didn’t do this kind of self-introspection on a reliable basis, namely, it got this right sometimes but not a lot. Failures on this test were the reported norm. Second, there is a chance that the AI was trying to be a sycophant, or that it was crafting a confabulation about the matter at hand (some refer to AI hallucinations as confabulations, see my coverage at the link here).

Another qualm is that the insertion of a concept vector is highly unusual and would presumably not likely occur when the LLM is working in full production mode. To perform these types of experiments, you usually do so on a test version of the AI rather than an instance that is in active production for millions of users. The question is whether this self-introspection will arise for the AI in true production or might there be other confounding aspects at play.

The Mechanisms Under-The-Hood

It is also possible to speculate on the possibilities of how the AI might be mathematically and computationally doing this introspective task. I say this to clearly avoid getting bogged down in magical thinking. Magical thinking refers to the facet that if we aren’t sure of the mechanical reasons for this behavior, we fall into the land of illusion and believe in magic.

The AI must be sentient, some will avidly decree, since there isn’t any other explanation of merit. Sorry, no, that’s not the conclusion of record. There are several sensible ways to explain how this might be arising. I’ll cover those mechanisms in my additional coverage on the matter.

If you are an AI techie, go ahead and come up with ideas of your own on how it might be taking place. Please use your self-introspection as you do so. As Aristotle famously remarked: “Knowing yourself is the beginning of wisdom.” Does that also apply to contemporary AI?

Maybe, but don’t bet your bottom dollar on it just yet.