Credits

Sara Imari Walker is an astrobiologist and theoretical physicist at Arizona State University and the Santa Fe Institute. Walker develops theories and experiments toward characterizing the origin of life. In her book, “Life as No One Knows It: The Physics of Life’s Emergence” (Riverhead Books, 2024), she argues that studying the origin of life requires radical new thinking.

A persistent hubris infects every age of our species’ scientific and technological development. It usually takes the form of individuals or institutions who are confident that — after thousands of years of human cultural evolution and billions of years of biological evolution — we have finally gotten to the bottom of reality. We are finally at the precipice to explain everything.

The newest incarnation is found in discourse around artificial intelligence. Here, at least, it is acknowledged that humans, with our limited memory and information processing capacity, will never really know everything. Still, this newfound and humbler stance is supplemented with the assumption that we are the single superior biological species who can build the technologies that will.

AlphaFold, an AI system developed by Google DeepMind, represents one of AI’s most celebrated achievements in science. Trained on more than 150,000 experimentally determined protein structures, AlphaFold 3 can now predict the structure of more than 200 million proteins as well as that of other biomolecules. Such scale was previously unimaginable. Earlier mathematical models could predict some features of protein structure, but nothing approaching this magnitude. The optimism is palpable: If AI can solve protein folding at this scale, what else might it accomplish?

Some proclaim AI will solve all disease, make scientists obsolete or even that artificial superintelligences will solve all of science. Yet many consider the protein folding problem unsolved. AlphaFold predicts 3D structures, but it does not explain the underlying physics, folding pathways or dynamic conformational ensembles. It works well for proteins made from the 20 or so amino acids found in terrestrial biology. To study proteins from the hundreds of amino acids in meteoritic materials, or to design novel therapeutic proteins, this model needs additional input. The limitation is not the algorithm or its scaling: The necessary data does not exist.

This tension reveals something profound about what science is, and how science defies precise definition. If we view science purely as the scientific method — observation, hypothesis, testing, analysis — then automation seems inevitable. AI algorithms demonstrably perform many, if not all, of these steps, and are getting better at them when guided by scientists.

But as philosopher Paul Feyerabend argued in “Against Method,” the very idea of a universal scientific method is misconceived. Most scientists invoke the scientific method only when writing for peer review, using it as standardization that allows reproducibility. Historically, scientific methods arise after discoveries are made, not before.

The question is not whether AI can execute steps in a method, but whether science generates knowledge in a way that is fundamentally something more.

If scale was all we needed, current AI would provide a mundane solution for science: We could do more because we have larger scale models. However, optimism around AI is not just about automation and scaling, it is also about theory of mind. Large language models (LLMs) like ChatGPT, Gemini and Claude have reshaped how many see intelligence, because interactions with these algorithms, by virtue of their design, give the appearance of a mind.

Yet as neuroscientist Anil Seth keenly observed, AlphaFold relies on the same underlying Transformer architecture as LLMs, and no one confuses AlphaFold with being a mind. Are we supposed to interpret that such an algorithm, instantiated on silicon chips, will comprehend the world in exactly the way we do, and communicate via our language with us so effectively as to describe the world as we understand it? Or should we instead believe it is maybe easier than we thought, after billions of years of the evolution of intelligence, to encode our own predictive and dynamic representational maps within such short spatial and temporal physical scales?

Consider how your own mind constructs your unique representation of reality. Each of us holds within our skulls a volume that generates an entire inner world. We cannot say this with the same certainty about any other entity, alive or not. Your sensory organs convert physical stimuli into electrical signals. In vision, photoreceptors respond to light and send signals along your optic nerve. Your brain processes this in specialized regions, detecting edges, motion and color contrasts in separate areas, then binds these fragmented perceptions into a unified object of awareness — what is called a percept — which forms your conscious experience of the world.

This is the binding problem: how distributed neural activity creates singular, coherent consciousness. Unlike “the hard problem of consciousness,” an open question behind our intrinsic experience, we do have some scientific insights into how binding could be accomplished: Synchronized neural activity and attention mechanisms coordinate information across brain regions to construct your unique mental model of the world. This model is literally the totality of your conscious understanding of what is real.

“The question is not whether AI can execute steps in a method, but whether science generates knowledge in a way that is fundamentally something more.”

Each of us is an inhabitant of one such mental model. What it is like to be inside a physical representation of the world, as we all are within our conscious experience, is nontrivial to explain scientifically (and some argue may not be possible).

Scientific societies face an analogous binding problem. Just as individual minds collect sense data to model the world, societies do the same through what Claire Isabel Webb, director of the Berggruen Institute’s Future Humans program, has called “technologies of perception”: Telescopes reveal cosmic depths, radiometric dating uncovers deep time, microscope expose subatomic structure, and now AI uncovers patterns in massive data.

Danish astronomer Tycho Brahe’s precise astronomical measurements, enabled by mechanical clocks and sophisticated angle-measuring devices, provided sense data that German astronomer Johannes Kepler transformed into mathematical models of elliptical orbits. A society collecting observations across space and time, exemplified across the work of Copernicus, Brahe, Kepler, Galileo and others, came to be bound into a single scientific consensus representation of reality — a societal percept — in the form of a theory that describes what it means to move and to gravitate.

But there is a fundamental difference. Your subjective experience, what philosophers call qualia, is irreducibly private. In a very real sense, it may be the most private information of all that our universe creates, because it is uniquely and intimately tied to the features of your physical existence that cannot be replicated in anything else.

When you see the color red, a specific experience emerges from your neural architecture responding to wavelengths between 620 and 750 nanometers. I can point to something red, and you can acknowledge you are also seeing red, but we cannot transfer the actual experience of redness from your consciousness to mine. We cannot know if we share the same inner experience. All we can share are descriptions.

This is where science radically differs from experience: It is fundamentally intersubjective. If something exists only in one mind and cannot be shared, it cannot become scientific knowledge. Science requires verifying each other’s observations, building on a lineage of past discoveries and developing intergenerational consensus about reality. Scientific models must therefore be expressible in symbols, mathematics and language, because they must be copyable and interpretable between minds.

Science is definitionally unstable because it is not an objective feature of reality; instead, it is more accurately understood as an evolving cultural system, bred of consensus representation and adaptive to the new knowledge we generate.

When Sir Isaac Newton defined F = ma, he was not sharing his inner experience of force or acceleration. He created a symbolic representation of relationships between three core abstractions — force, mass, acceleration — each developed through metrological standardization. The formula became pervasive cultural knowledge because any mind or machine can interpret and apply it, regardless of how each experiences these concepts internally.

This reveals the most fundamental challenge of scientific knowledge: Our primary interface for sharing scientific ideas is symbolic representation. What we communicate are models of the world, not the world itself. Philosopher of science Nancy Cartwright argues scientific theories are simulacra; that is, they are useful fictions in mathematical and conceptual form that help us organize, predict and manipulate phenomena. Theories are cultural technologies.

When we use the ideal gas law (PV = nRT), we model gases as non-interacting points. This is* not* to be interpreted as a claim that real gases are literally points with no volume that never interact, it is merely a simplification that works well enough in many cases. These simplified models matter because they are comprehensible and shareable between minds, and they are copyable between our calculating machines.

The requirement that scientific knowledge must be shareable forces us to create simulacra at every descriptive level. Science’s intersubjective nature places strict physical constraints on what theories can be. Our scientific models must be expressible symbolically and interpretable between human minds. They are therefore necessarily abstractions that never capture reality’s full structure. They can never fully capture reality, because no human mind has sufficient information processing and memory to encode the entire external world. Even societies have limits.

AI will also have limits.

These limits are not solely in terms of available compute power, made acute in the need for more data processing infrastructure to support the AI economy. More fundamentally, the current optimistic, and sometimes hubristic, dialogue around AI and artificial general intelligence (AGI) suggests these algorithms will be “more than human” in their ability to understand and explain the world, breaking what some perceive as limits on intelligence imposed by human biology.

“Our scientific models can never fully capture reality, because no human mind has sufficient information processing and memory to encode the entire external world.”

But this cannot be true by virtue of the very foundations of the theory of computation, and the lineages of human abstraction from which these technologies directly descend. As physicist David Deutsch writes, if the universe is indeed explicable, humans are already “universal explainers” because we are capable of understanding anything any computational system can: In terms of computational repertoire, both computers and brains are equivalently universal.

Other foundational theorems in computer science, like the no free lunch theorems by physicists David Wolpert and William Macready, indicate that when performance is averaged over all possible problems, no optimization algorithm (machine learning algorithms included) is universally better than any other. Stated another way, making an algorithm such that it performs exceptionally well for one class of problems will lead to trade-offs where it is poorer than average at others.

The physical world does not contain all possible problems, but the structure of the ones it does contain changes with biological and technological evolution. Just as no individual can comprehend everything all humans know, or will know, there can be no algorithm (AGI or otherwise) that is indefinitely better than* all* others.

More fundamentally, the possibility of universal computation arises due to a fundamental limitation; universal computers can only describe computable things, but never the uncomputable ones — a limitation intrinsic to any computer we build. This limitation does not apply to individual human minds, only what we share via language, and this is key to how we generate new social knowledge.

Scientific revolutions occur when our shared representational maps break down; that is, when existing concepts prove inadequate to cover phenomena we newly encounter or old ones we wish to explain. We must then invent new semantic representations capturing regularities old frameworks could not. At these times, nonconformism plays an outsized role in knowledge creation.

Consider the shift from natural theology to evolution. The old paradigm assumed organisms were designed by a creator, species were fixed, Earth was young. As we learned to read deeper histories, through carbon dating, phylogeny and observing species change through selective breeding and extinction, we never witnessed the spontaneous formation of biological forms.

Deeper historical memory forces new descriptions to emerge. Evolution and geology revealed concepts of deep time, astronomy introduced concepts of deep space, and now, as historian Thomas Moynihan points out, we are entering an age revealing a universe deep in possibility. Our world does not suddenly change or get older, but our understanding does. We repeatedly find ourselves developing radically new words and concepts to reflect new meaning as we discover it in the world.

Philosopher of science Thomas Kuhn recognized these transitions as paradigm shifts, noting how abrupt periods of change force scientists to reconceptualize the way we see our field, what questions we ask, what methods we use, what we consider legitimate knowledge. What emerges are entirely new representations for describing the world, often including totally new descriptions of everyday objects we thought we understood.

Science, as Kuhn saw it, is messy, social and profoundly human. In an age where we are now worried about alignment, after alignment and re-alignment with our own technological creations, paradigm shifts might best be described as the representational alignment of our societal percepts, where we must find new ways for our representations to keep in sync with the changing structure of reality as presented to us across millennia of our cultural evolution.

Paradigm shifts reveal how the power of scientific thought does not lie in the literal truth of theories, but in our ability to identify new ways of describing the world and in how the structures we describe persist across different representational schemes. The culture of science helps distinguish between simulacra that approach causal mechanisms (sometimes called objective reality) and those that lead us astray. Crucially, discovering new features of reality requires building new descriptions. When frameworks fail to capture important worldly features, for example when we recognize patterns but cannot articulate them, new frameworks and representational maps must emerge.

Albert Einstein’s development of general relativity illustrates this. Seven years separated his realization that physics needed to transcend the linear Lorentz transformations (appearing in special relativity) to get to the general theory of relativity. In his own reflections, he comments on the reason being how “it is not so easy to free oneself from the idea that coordinates must have an immediate metrical meaning.” Mathematical structures imposed as models weren’t capturing meaning: They were missing features Einstein intuited must exist. Once he encoded his intuition, it became intersubjective and shareable between minds.

“Scientific ideas are not born solely of individual minds, but also of consensus interpretations of what those minds create.”

This brings us to why AI cannot replace human scientists. Controversy and debate over language and representation in science are not bugs; they are features of a societal system determining which models it wants. Stakes are high because our descriptive languages literally structure how we experience and interact with the world, forming the reality our descendants inherit.

AI will undoubtedly play a prominent role in “normal science,” something Kuhn defined as constituting the technical refinement of existing paradigms. Our world is growing increasingly complex, demanding correspondingly complex models. Scale is not all we need, but it will certainly help.

AlphaFold 3’s billions of parameters suggest parsimony and simplicity may not be science’s only path. If we want models mapping the world as tightly as possible, complexity may be necessary. This aligns with logical positivists Otto Neurath, Rudolph Carnap and the Vienna Circle’s view: “In science there are no ‘depths’; there is surface everywhere.” If we have accurate, predictive models of everything, maybe there are no deeper truths to be uncovered.

This surface view misses a profound feature of scientific knowledge creation. The simulacra change, but underlying patterns we uncover by manipulating symbols remain, inarticulable and persistent, independent of our languages. The concept of gravity was unknown to our species before science, despite direct sensorial contact throughout human history and an inherited memory from the nearly 4-billion-year lineage of life that preceded us. Every species is aware of gravity, and some microorganisms even use this awareness to navigate. We knew it as a regularity before Newton’s mathematical description, and this knowledge persisted through Einstein’s radical reconceptualization.

Prior to Newton’s generation, the model of Ptolemy was the most widely adopted for the study of planetary motions, as it had been for nearly 1,500 years. It included circular orbits for the planets, and to increase predictive power, epicycles were added for each planet, such that each planet in the model moved in a small circle while also moving in a larger circle around the Earth. Additional epicycles were added to increase predictive accuracy, not unlike adding nodes to a machine learning model with the accompanying risk of over-fitting.

We did not transition to the Newtonian model for its predictive power, but rather because it explained more. The modern concept of gravity was invented by this process of abstraction, and by the explanatory unification of our terrestrial experience of gravity with our celestial observations of it. It is likely that our species, and more specifically our species’ societies, will never forget gravity now that we have learned an abstraction to describe it, even as our symbols describing it may radically change.

It is this depth of meaning, inherent in our theories, that science discovers in the process of constructing new societal percepts. This cannot be captured by the surface level view, where science merely creates predictive maps, devoid of depth and meaning.

French literary critic Roland Barthes argued in his liberating 1967 essay “The Death of the Author” that texts contain multiple layers and meanings beyond their creators’ intentions. As with Feyerabend, this was a direct rebuttal “against method.” For Barthes, this rebuttal of method was in refute of literary criticism’s traditional methodological practice of relying on the identity of an author to interpret an ultimate meaning or truth for a text. Instead, Barthes argued for abandoning the idea of a definitive authorial meaning in favor of a more socially constructed and evolving one.

Similarly, it can be said the scientist “dies” in our writings. When we publish, we submit work to our peers’ interpretation, criticism and use. The peer review process is currently a target for AI automation, born from a misconception that peer review is strictly about fact-checking. In reality, peer review is about debate and discussion among peers and gives scholars an opportunity to cocreate how new scientific work is presented in the literature. That debate and cocreation are essential to the cultural system of science. It is only after peer review that we enter a method that allows reproducibility. Scientific ideas are not born solely of individual minds, but also of consensus interpretations of what those minds create.

The outputs of AI models arrive already “dead” in this crucial sense: They are produced without an embodied creative act of meaning-making that accompanies the modes of scientific discovery we have become accustomed to in the last 400 or so years. When a scientist develops a theory, even before peer review, there is an intentional act of explanation, and an internal act of wrestling with intuition and its representation. AI models, by contrast, generate predictions through statistical pattern recognition, a very different process.

“Will AI transform science? Certainly. Will it replace scientists? Certainly not.”

Science and AI are cultural technologies; both are systems societies use to organize knowledge. When considering the role of AI in science, we should not be comparing individual AI models to individual human scientists, or their minds, as these are incomparable.

Rather, we must ask how the cultural systems of AI technologies and science will interact. The death of the scientist is the loss of the inner world that creates an idea, but this is also when the idea can become shared, and the inner world of the societal system of debate and controversy comes alive. When human scientists die in their published work, they birth the possibility of shared understanding. Paradigm shifts are when this leads to entirely new ways for societies to understand the world, forcing us to collectively see new structure underneath our representational maps, structure we previously could not recognize was there.

An AI model can integrate an unprecedented number of observations. It can execute hypothesis testing, identify patterns in massive datasets and make predictions at scales an individual human cannot match. But current AI operates only within the representational schema humans give it, refining and extending them at scale. The creative act of recognizing that our maps are inadequate and building entirely new, social and symbolic frameworks to describe what was previously indescribable remains exceptionally challenging, impossible to reduce to method, and so far, uniquely human.

It is unclear how AI might participate in the intersubjective process of building scientific consensus. No one can yet foretell the role AI will play in a collective determination of which descriptions of reality a society will adopt, which new symbolic frameworks will replace those that have died, and which patterns matter enough to warrant new languages for their articulation.

The deeper question is not whether AI can do science, but whether societies can build shared representations and consensus meanings with algorithms that lack the intentional meaning creation that has always been at the heart of scientific explanation.

In essence, science itself is evolving, begging the question of what science after science will look like in an age where the cultural institution of science becomes radically transformed. We should be asking: When we find our species still craves meaning and understanding, beyond algorithmic instantiation, what will science become?

Will AI transform science? Certainly. Will it replace scientists? Certainly not. If we misunderstand what science is, mistaking automation of method for the human project of collectively constructing, debating and refining the symbolic representations through which we make sense of reality, AI may foretell the death of science: We will miss the true opportunity to integrate AI into the culture systems of science.

Science is not merely about prediction and automation; history tells us it is much more. It is about explanatory consensus, and an ongoing human negotiation of which descriptions of the world we will collectively adopt. That negotiation, the intersubjective binding of observations into shared meaning is irreducibly social and, for now, irreducibly human.