Behind the magic of LlamaIndex, LangChain, and Microsoft Agent Framework is a surprisingly simple pattern. Once you see it, you’ll never look at agents the same way.

5 min read22 hours ago

–

The Illusion of Intelligence

You build an agent. It reasons through problems. It calls the right tools at the right time. It feels… smart.

Here’s the uncomfortable truth: your agent isn’t thinking. It’s a while loop with a credit card.

Every framework — LlamaIndex, LangChain, Autogen, CrewAI, Microsoft Agent Framework— does the same thing under the hood. They wrap an LLM in a loop, inject tool descriptions into the prompt, parse the output, and repeat until done.

Once I understood this, I stopped fighting my agents and started designing them properly.

Let me show you exactly what happens when you write FunctionAgent(tools=[...]).

The ReAct Pattern: Reason, Act, Observe, Repeat

Every modern agent framework implements some variant of ReAct (Reasoning + Acting). The loop looks like this:

Press enter or click to view image in full size

Well, almost everything is in this illustration I made but still continue to read the article to have more details!

That’s it. That’s the entire “intelligence” of your agent.

The framework handles the plumbing. The LLM handles the reasoning. Your job is to design tools and prompts that make this loop productive.

What Happens When You Declare a Tool

Let’s trace through what LlamaIndex actually does when you write this:

from llama_index.core.tools import FunctionTool@FunctionTool.from_defaultsdef get_stock_price(ticker: str) -> str:    """Get the current stock price for a given ticker symbol."""    # Your API call here    return f"{ticker}: $142.50"

Step 1: Schema Extraction

The framework inspects your function and extracts:

• Name: get_stock_price
• Description: "Get the current stock price for a given ticker symbol."
• Parameters: {"ticker": {"type": "string", "description": "..."}}

This becomes a JSON schema that the LLM understands.

Step 2: Prompt Injection

When you create an agent, all tool schemas get injected into the system prompt. The LLM sees something like:

You have access to the following tools:Tool: get_stock_priceDescription: Get the current stock price for a given ticker symbol.Parameters:  - ticker (string, required): The stock ticker symbolTo use a tool, respond with:{"tool": "get_stock_price", "arguments": {"ticker": "AAPL"}}

The LLM doesn’t “know” about your tools. It just sees text describing them and learns the expected output format.

Step 3: The Model Decides

When a user asks “What’s the price of Apple stock?”, the LLM sees:

1. Your system prompt
2. The tool descriptions
3. The user’s question

It reasons: “I need stock price info. I have a tool for that. I should call it with ticker=’AAPL’.”

Then it outputs structured JSON (or uses native function calling if the model supports it).

The Loop in Action: A Real Example

Let me show you what actually happens when I ask my trading agent: “Analyze NVDA sentiment.”

Iteration 1: Initial Request

[SYSTEM] You are a financial analyst with access to tools...[TOOLS] get_reddit_sentiment, get_news_headlines, get_stock_price...[USER] Analyze NVDA sentiment.[LLM OUTPUT]I'll analyze NVDA sentiment by checking Reddit discussions.{"tool": "get_reddit_sentiment", "arguments": {"ticker": "NVDA"}}

Framework parses this, calls my function, gets result.

Iteration 2: First Tool Result

[SYSTEM] ...[TOOLS] ...[USER] Analyze NVDA sentiment.[ASSISTANT] I'll analyze NVDA sentiment...[TOOL RESULT] get_reddit_sentiment → "Bullish: 73%, Bearish: 15%, Neutral: 12%"[LLM OUTPUT]Reddit shows bullish sentiment. Let me also check recent news.{"tool": "get_news_headlines", "arguments": {"ticker": "NVDA"}}

Iteration 3: Second Tool Result

[... previous context ...][TOOL RESULT] get_news_headlines → "NVDA announces new AI chip..."[LLM OUTPUT]Based on my analysis:- Reddit sentiment: 73% bullish- Recent news: Positive (new AI chip announcement)- Overall: Strong bullish sentiment for NVDA

No tool call in the output → loop ends → user gets the response.

Three iterations. Three LLM calls. One answer.

Why Tool Descriptions Are Everything

Here’s what killed my early agents: bad tool descriptions.

Get Elliott Girard’s stories in your inbox

Join Medium for free to get updates from this writer.

The LLM only knows what you tell it. If your description is vague, the model will:

• Call the wrong tool
• Pass wrong arguments
• Skip tools it should use

Bad description:

def get_data(ticker: str) -> str:    """Get data for a stock."""

Good description:

def get_reddit_sentiment(ticker: str) -> str:    """    Analyze Reddit sentiment for a stock ticker.        Searches r/wallstreetbets, r/stocks, and r/investing.    Returns: Bullish/Bearish/Neutral percentages and top discussion themes.        Use this when: User asks about sentiment, social buzz, or retail investor opinion.    Do NOT use for: Price data, financial statements, or news.    """

The second description tells the LLM:

• What the tool does
• What data it returns
• When to use it
• When NOT to use it

This reduces wrong tool calls dramatically.

The Iteration Budget Problem

Remember: each loop iteration = one LLM call = tokens = money + latency.

Most frameworks default to 10–20 max iterations. Sounds like a lot until you realize:

5 stocks to analyze× 4 tools per stock (price, sentiment, news, filings)= 20 tool calls minimum+ reasoning steps between each= 25-30 iterations easily

Your agent hits the limit and returns incomplete garbage.

The fix: Meta-tools that do multiple things in one call.

def get_stock_snapshot(ticker: str) -> str:    """    Get complete stock analysis in ONE call.    Returns: price, sentiment, news, and key metrics combined.    """    # Fetch everything in parallel internally    results = await asyncio.gather(        get_price(ticker),        get_sentiment(ticker),        get_news(ticker)    )    return format_compact(results)

Before: 4 iterations per stock After: 1 iteration per stock

This is why understanding the loop matters. You can’t optimize what you don’t understand.

What This Means for You

Understanding the loop changes how you build agents:

1. Design tools for the loop, not for humans

Your tool returns data for the LLM to process, not for a user to read. Keep it compact. Include only what the model needs for the next reasoning step.

2. Descriptions are prompts

Tool descriptions are part of your system prompt. Write them like you’re explaining to a junior dev what the function does and when to use it.

3. Iteration budget is your constraint

Every complex query needs an iteration budget. Design meta-tools that consolidate common workflows. Don’t make your agent call 30 tools when 3 would do.

4. Debug by tracing the loop

When your agent misbehaves, don’t guess. Print each iteration:

• What prompt went in
• What the LLM returned
• What tool was called
• What result came back

The bug is always in one of these steps.

5. The LLM is just text prediction

It doesn’t “understand” your tools. It predicts the next token based on the prompt. Better prompts = better predictions = better tool selection.

The Bottom Line

Your AI agent is a loop that:

1. Stuffs tool descriptions into a prompt
2. Asks an LLM what to do
3. Parses the response
4. Executes tools if needed
5. Repeats until done

That’s not magic. That’s engineering.

Once you see the loop, you stop being impressed and start being effective. You design better tools, write better descriptions, and build agents that actually work in production.

The frameworks handle the plumbing. You handle the architecture.

Now go build something that loops intelligently.

Thanks for reading! I’m Elliott, a Python & Agentic AI consultant and Entrepreneur who builds practical AI tools and shares what actually works. Understanding this loop transformed how I build agents — hopefully it does the same for you.

If this clicked for you, smash that clap button 👏 and follow for more honest takes on AI development. Drop a comment if you want me to dive deeper into any part of the agent loop.