How an old idea quietly explains why remembering is often a search problem.

10 min readJust now

–

Introduction

Most of us talk about memory like it is storage. You learn something, you store it, you retrieve it later.

But everyday life feels different. You can be sure you know a name, a fact, or a detail, and still fail to pull it up. Then it pops out later in the shower, or when someone gives you a hint.

In a recent paper titled “Key-value memory in the brain” by Samuel J. Gershman, Ila Fiete, and Kazuki Irie, the authors argues that the bottleneck is often not storage. It is retrieval. And it proposes a clean computational way to think about retrieval: treat memory like a key value store, where the brain keeps an address for a memory separate from the content of that memory.

This post reviews the paper and explains the analogies and implications that the authors drive from neuroscience and brain theories like predictive processing and attractor networks.

What

The authors contrast two families of memory models.

1. Classical similarity based retrieval

Many psychological and neural models assume you store patterns, and retrieval is basically “find what looks most similar to the cue.” The paper points out a limitation: storage and retrieval are forced to share the same representation, even though they have different jobs.

2. Key value memory

A key value memory stores two representations for each experience.

Keys represent addresses. They are optimized to be discriminable, so you can find the right memory.

Values represent content. They are optimized for fidelity, so what you retrieve is useful.

Figure 1: Two architectures for key-value memory. Black symbols denote vectors and blue symbols denote matrices. (Left) Input x is mapped to key (k), query (q), and value (v) vectors. During memory writing, the weight matrix M is updated using Hebbian learning between the key and value vectors. During reading, the query is projected onto M to produce a retrieved value vˆ. (Right) The input vector is mapped to a hidden layer α, which is then mapped to an output layer vˆ. The input-to-hidden weights correspond to the stored keys; the hidden-to-output weights correspond to the stored values. Image from the paper https://arxiv.org/pdf/2501.02950v2

Retrieval then becomes a two step computation.

1. Match a query to stored keys to get attention weights

2. Use those weights to mix the stored values into the retrieved output.

One nice part of the paper is how directly it links classic associative memory to modern transformer attention. It shows the “dual form” of a correlation matrix memory, where the retrieved value is literally a weighted sum of stored values, with weights derived from query key similarity.

Why

The big payoff is a reframing: many memory failures look less like information decay and more like address failure.

The paper highlights evidence that “lost” memories can reappear under the right retrieval conditions, and that interference between memories is often the villain. This supports a retrieval oriented view of forgetting and amnesia.

Once you adopt key value memory as the lens, a bunch of puzzles become easier to talk about:

1. Interference

Too many similar keys make it hard to select the right value.

2. Robustness versus separability

A max like selector is great when cues are perfect, but brittle under noise. Softer selection (like softmax) trades perfect recall for stability.

3. Generalization

Sometimes you do not want an exact match. You want a blend, because the new situation is similar but not identical to past ones. Key value retrieval naturally supports that by design.

How

Below is a minimal Python demo of key value attention, close to what a transformer does for a single head.

It shows the core idea:

1. project inputs into queries, keys, values

2. compute attention weights from query key similarity

3. retrieve a weighted sum of values

import numpy as npdef softmax(x, axis=-1):    x = x - np.max(x, axis=axis, keepdims=True)    ex = np.exp(x)    return ex / np.sum(ex, axis=axis, keepdims=True)def scaled_dot_product_attention(Q, K, V):    """    Q: (T, d)    K: (T, d)    V: (T, dv) often dv == d    returns:      out: (T, dv)      attn: (T, T) attention weights    """    d = Q.shape[-1]    scores = (Q @ K.T) / np.sqrt(d)     # (T, T)    attn = softmax(scores, axis=-1)     # (T, T)    out = attn @ V                      # (T, dv)    return out, attn# Toy "sequence" of T tokens, each represented by an embedding vector of size d_modelrng = np.random.default_rng(0)T = 5d_model = 6d_k = 4d_v = 4X = rng.normal(size=(T, d_model))# Learned linear projections (in a real model these are trained)W_q = rng.normal(size=(d_model, d_k))W_k = rng.normal(size=(d_model, d_k))W_v = rng.normal(size=(d_model, d_v))# Build queries, keys, valuesQ = X @ W_qK = X @ W_kV = X @ W_v# Retrieve values using key matchingout, attn = scaled_dot_product_attention(Q, K, V)np.set_printoptions(precision=3, suppress=True)print("Attention weights (each row sums to 1):")print(attn)print("\nRetrieved representations:")print(out)# Optional: inspect which token each position attends to mosttop = np.argmax(attn, axis=-1)print("\nMost attended token index per position:", top.tolist())

How to read this like a neuroscientist

Keys are the “addresses.” They are what the query compares against.

Values are the “content.” They are what actually gets reconstructed.

Attention weights are the retrieval process. They decide which stored content gets reactivated, and how strongly.

That is exactly the computational separation the paper argues the brain benefits from: optimize keys for discriminability and values for fidelity.

Figure 3: Forgetting and reactivation of memory events. A one-layer feedforward neural network is trained on two tasks sequentially, Task 1 and 2, constructed using the MNIST and FashionMNIST datasets, respectively. (A) The evolution of the test classification accuracy for the two tasks as a function of training epochs. After epoch 5, the training dataset changes from Task 1 to Task 2; resulting in forgetting of Task 1 as the model learns Task 2. (B) The accuracy of the trained model on Task 1 as a function of the value of the artificial scaler β used to amplify the keys in all key-value memory pairs corresponding to Task 1 learning. Image from the paper https://arxiv.org/pdf/2501.02950v2

Memory formation and consolidation through a key value lens

One of the strengths of this paper is that it does not stop at retrieval. It also offers a useful way to think about how memories are formed and consolidated over time.

Classical memory consolidation theory proposes a division of labor. The hippocampus rapidly encodes new experiences, while the neocortex slowly integrates those experiences into long term semantic knowledge. Over time, memories become less dependent on the hippocampus and more embedded in cortical networks.

The key value framework provides a computational explanation for how this transition might happen.

Encoding: creating keys and values

During initial learning, an experience activates a rich, distributed pattern across sensory and associative cortex. In key value terms, this cortical activity corresponds to a value representation. It contains content: perceptual details, semantic associations, emotional tone.

At the same time, the hippocampus rapidly generates a key. This key does not need to preserve content in detail. Its job is to uniquely index the cortical pattern that was active during the experience.

This aligns well with hippocampal indexing theory. The hippocampus does not store the memory itself. It stores an address that can later reactivate the cortical pattern.

In the language of the paper, the hippocampus specializes in building discriminable keys, while the cortex specializes in storing expressive values.

Early retrieval: hippocampal driven reconstruction

In the early stages after learning, retrieval relies heavily on the hippocampus.

A partial cue generates a query that matches against hippocampal keys. The selected key then drives the reactivation of the associated cortical value pattern. This explains why early memories are vivid, context rich, and episodic.

From the key value perspective, retrieval is still indirect. You find the memory by looking up its address.

Consolidation: keys move into cortex

Over time, repeated reactivation changes the system.

Each hippocampal driven retrieval slightly reshapes cortical representations through plasticity. Associations between cortical features strengthen. Gradually, the cortex begins to build its own internal keys.

In other words, the cortex starts learning key value mappings internally.

Semantic knowledge emerges when cortical representations become mutually predictive enough that a partial cortical cue can retrieve the rest of the pattern without hippocampal help.

In the framework of the paper, consolidation corresponds to a shift from hippocampal keys pointing to cortical values, toward cortical keys directly indexing other cortical values.

This is not a copy operation. It is a restructuring of the retrieval geometry.

Why hippocampal dependence fades

This view helps explain why remote memories survive hippocampal damage while recent ones do not.

Recent memories still depend on hippocampal keys. Remove the index, and the value cannot be reliably retrieved.

Get Edgar Bermudez’s stories in your inbox

Join Medium for free to get updates from this writer.

Older memories have acquired redundant cortical keys. Even if the hippocampal index is gone, the cortex can still perform approximate lookup using its own learned associations.

The paper’s emphasis on retrieval interference also fits well here. Semantic memory trades specificity for robustness. Cortical keys are less precise than hippocampal ones, but they are more stable and less context dependent.

Implications for learning and semantic abstraction

Seen through this lens, learning is not just about strengthening representations. It is about reshaping the key space.

Episodic memory prioritizes distinct keys. Semantic memory prioritizes reusable keys.

Consolidation is therefore not simply a transfer of information. It is a transformation from address based access to structure based access.

This has implications beyond neuroscience.

It suggests that effective learning systems, biological or artificial, should explicitly separate fast indexing from slow representational refinement. It also suggests that abstraction emerges naturally when keys are optimized for generalization rather than discrimination.

The paper does not claim this is the only way the brain could work. But it offers a clean computational story that unifies indexing theory, consolidation theory, and modern attention based models under a single principle.

Memory becomes less about where information is stored and more about how access pathways are shaped over time.

Contrasting key value memory with predictive processing and attractor networks

The key value perspective is not the only way memory has been theorized in neuroscience. Two influential alternatives are predictive processing and attractor network theories. Each captures something real about brain function, but they emphasize different computational problems.

Putting them side by side helps clarify what the key value framework adds.

Key value memory versus predictive processing

Predictive processing frames the brain as a hierarchical inference machine. At every level, the brain generates predictions about incoming signals and updates its internal model based on prediction error.

In this view, memory is not a separate system. It is embedded in the generative model itself. Learning means adjusting parameters so future predictions become more accurate.

The strength of this approach is that it explains perception, action, and learning within a single framework. Memory is implicit. If the model predicts well, it has learned.

The limitation is that retrieval is under specified.

Predictive processing does not naturally distinguish between storing content and accessing it later. Remembering a past event becomes a form of reconstruction driven by the current generative model rather than a targeted lookup of a specific stored episode.

The key value framework makes a different commitment.

It treats retrieval as a first class computational problem. Keys exist specifically to solve the question: how do I find the right memory when many similar ones exist?

From this perspective, predictive processing explains how values are shaped and refined, while key value memory explains how those values are accessed when needed.

A useful way to reconcile them is to see predictive models as shaping the value space, while key value mechanisms shape the access paths through that space.

Key value memory versus attractor networks

Attractor network models assume that memories are stored as stable patterns in a recurrent system. Partial or noisy inputs cause the system to settle into the closest attractor basin, which corresponds to a stored memory.

This framework has been extremely influential, especially in explaining pattern completion in hippocampus and cortex.

Its core assumption is that storage and retrieval share the same representational substrate. The same neurons that store the memory also define the basin that retrieves it.

The key value framework relaxes that assumption.

Instead of asking the system to both store content and support retrieval dynamics in the same space, it splits the job.

Keys do the discrimination. Values do the representation.

This separation avoids a known tension in attractor models. Making attractors more distinct improves retrieval but reduces representational richness. Making representations richer increases interference between attractors.

Key value memory sidesteps this tradeoff by allowing keys to be simple and well separated while values remain expressive and overlapping.

Attractor dynamics may still play a role, but more as a refinement or stabilization mechanism within the value space, rather than as the primary retrieval engine.

What changes conceptually

The deepest difference is where each theory places the burden of memory.

Predictive processing places it in model parameters and error minimization. Attractor networks place it in stable dynamical states. Key value memory places it in the geometry of access.

This matters because many empirical memory failures look less like degraded representations and more like misaddressed retrieval. You know the information exists, but the system cannot reliably select it.

Key value memory gives that failure mode a clear computational interpretation.

A unifying perspective

These theories do not have to compete.

Predictive processing can explain how value representations are learned and shaped by experience.

Attractor dynamics can explain how representations stabilize and resist noise.

Key value memory explains how specific experiences and knowledge are accessed when the system is overloaded with similar alternatives.

Seen this way, the contribution of the paper is not to replace existing theories, but to highlight a missing axis: retrieval as a separable and optimizable process.

Once that axis is made explicit, many familiar phenomena in learning, consolidation, and forgetting fall into place.

Memory stops looking like a single mechanism and starts looking like a layered system where prediction, representation, and access each play distinct roles.

Conclusion

The key move in this paper is simple: stop forcing storage and retrieval to share the same representation.

Once you allow distinct keys and values, memory looks less like a fragile warehouse and more like a powerful indexing and lookup system. Forgetting becomes less mysterious. It is often a search failure.

And the hippocampus neocortex relationship gets a sharper computational interpretation: hippocampus as an index and query matcher, neocortex as the long term content store.

That is a small conceptual change with big implications, both for neuroscience theories of remembering and for how we think about intelligent systems more broadly.

References

• Gershman, S. J., Fiete, I., Irie, K. (2025). Key value memory in the brain. arXiv:2501.02950v2.
• Teyler, T. J., DiScenna, P. (1986). The hippocampal memory indexing theory. Behavioral Neuroscience, 100(2), 147–154.
• Goode, T. D., Tanaka, K. Z., Sahay, A., McHugh, T. J. (2020). An integrated index: engrams, place cells, and hippocampal memory. Neuron, 107(5), 805–820.
• McClelland, J. L., McNaughton, B. L., O’Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex. Psychological Review, 102(3), 419–457.