How LLMs Read Docs

Problem: During pretraining LLMs memorize documentation, which soon becomes outdated.

Solution: Just give LLMs relevant context!

Current Approaches to Providing Context

1) llms-full.txt

A large (sometimes >380k tokens) file that is stored somewhere on documentation website of the library provider.

Problem 1: It bloats the context. 200k tokens is what Claude Code currently has, so the doc that you are feeding it is becoming “losely compressed”. Also the majority of that documentation might not even be needed, if you are implementing a quickstart or using a specific small part of the system.

Problem 2: Who knows if it actually exists for that library provider, or no? So the agent needs to verify if that file exists, and have fallback scenarios if not.

2) Same page as .mdx/.md, “Copy as markdown”

This allows the user to copy a specific page and paste it into agent.

Problem 1: It is not controlled by the agent, so human is needed to do the manual work of copying and pasting.

Problem 2: The markdown is not ensured to be properly formatted and clean, so we end up getting pages like this one:

# CLI installation

> Use the CLI to preview and maintain your documentation locally

<img className="block dark:hidden my-0 pointer-events-none" src="https://mintcdn.com/mintlify/Y6rP0BmbzgwHuEoU/images/installation/local-development-light.png?fit=max&auto=format&n=Y6rP0BmbzgwHuEoU&q=85&s=ffd8bc88d87fc00fe1919a33a292fa01" alt="Decorative graphic representing the CLI." data-og-width="1184" width="1184" data-og-height="320" height="320" data-path="images/installation/…….

Use the CLI to preview your documentation locally as you write and edit. View changes in real-time before deploying, test your documentation site's appearance and functionality, and catch issues like broken links or accessibility problems.

(very large link to a logo image)

Or like this one:

---
title: Registry Directory
description: Discover community registries for shadcn/ui components and blocks.
---

These registries are built into the CLI with no additional configuration required. To add a component, run: `npx shadcn add @<registry>/<component>`.

<DirectoryList />

(Instead of the directory that is displayed on UI, agent gets a React component that tells nothing to it)

3) Using web fetching and parsing from real docs

This is what Claude Code currently does.

Problem 1: How well it is parsed is a mystery, so we are getting some context, but it can include ads or whatever the web scraper found.

Problem 2: The whole approach is weird in terms of assymetry, why do we need to spend compute (including llm compute for proper parsing and web scraping) many times per single chunk of docs? So the Company A has published Doc A, and now 1000s of agents are scraping the same page.

4) Searching for docs github

This has same problems as 3, as well as that some repositories don’t have the docs on github, some are react components with some routing for better rendering on UI, so extra compute is spent each time we need to get the real markdown representation of the doc.

5) MCPs for individual libraries

MCPs that library creators provide as a way to use their library in a coding agent.

Problem 1: It bloats your context. Every dependency will be an MCP, each llm call has the tools in the context, even if you don’t need them at all right now.

Problem 2: No standartization. The library providers decide which kind of read tools are available, so there is no room for the agent to optimize and he has different schemas for same operations.

6) MCPs for context search

Context7, etc.

Those are bloating your context as well, as they mostly operate in llms-full style format + apply vector search. So the llm is getting a large chunk of data that is somewhat around what it requested.

7) AIWiki

A combined and structured source of knowledge, available via single tool for the AI agent.

read(shadcn/directory/aceternity/components/cards) gives the AI agent an exact snippet needed to install the button, without all of the irrelevant context:

# Cards

A set of cards that can be used for different use cases

## Tags
`card`, `testimonials`, `features`

## Usage (Component Code)
npx shadcn@latest add @aceternity/cards-demo-1 @aceternity/cards-demo-2 @aceternity/cards-demo-3

## Installation
Run the following command:
```bash
npx shadcn@latest add @aceternity/cards-demo-1 @aceternity/cards-demo-2 @aceternity/cards-demo-3

[Aceternity UI](https://ui.aceternity.com/)

Side Note:

If it doesn’t know the exact route yet, it is able to use vectorized search to get to some relevant point on the graph, and move from there.

It’s also interesting to test out how “external links” would work. As we can specify the links in the markdown format, and let the AI agent explore and find all relevant info in-between different docs and providers.

It is compute/labor intensive, as it requires to create a unified library of docs, and post-process them to ensure proper formatting and token counts per chunk, but some of that work can be automated with AI once and let the coding agents operate with a clean and concrete context.