TL;DR
I worked on a coding agent harness that executes AI-generated code in Docker sandboxes, and learned that the real bottleneck isn't code generation, but everything after: environment setup, error handling, and integration with external services.

While this works great for local development, connecting to tools like GitHub, Slack, or databases meant building dozens of API integrations, managing OAuth flows, and handling edge cases. That's where Composio's ToolRouter came in, instead of building integrations myself, I now get tools through 6 simple meta-tools, with authentication and execution handled automatically.

The result? An agent that can write code, test it locally, create GitHub issues, and notify Slack, all through a single, observable execution loop. Turns out the coolest part wasn't just watching AI write code, but watching it interact with the real world safely and transparently.

Note: This is a learning project where I explored agent execution
patterns and integration approaches. The Composio integration shown
here demonstrates the concept, though a production implementation
would need additional error handling, cost controls, and testing.

Why Build a Coding Agent Harness

I didn’t start thinking about a coding agent harness because existing tools are bad. Quite the opposite. Tools like Claude Code and Codex are excellent, and I use them regularly for debugging and iteration.

What sparked this work wasn’t dissatisfaction, but curiosity.

While using these tools, I kept noticing that a lot of the most important work happens outside the model: code execution, error capture, retries, environment setup, and tool integration. These systems handle that complexity well, but largely invisibly. You get an answer, a fix, or a result, but not always a clear view into how the system arrived there.

I wanted to understand that execution loop more deeply.

Not just did it work, but:

• what code actually ran
• what error was thrown
• what context was passed back to the model
• what changed between attempts
• and where control really lives when things fail

Most modern AI coding tools optimize for speed and convenience. They hide complexity to make workflows feel smooth. That’s usually the right tradeoff. But it also means the execution layer, the part where code meets reality, is opaque.

The Real Bottleneck Isn’t Code Generation

Writing code is no longer the hard part. Models are already very good at producing first drafts, boilerplate, and obvious fixes.

The harder part is everything that happens after: running code in a real environment, capturing raw errors, installing missing dependencies, handling environment variables, and retrying with proper context when something fails.

Humans still do most of this manually. We run the code, read the error, decide what matters, and feed it back into the model. That makes the human the slowest and most expensive part of the loop.

If an AI system can write code but cannot directly observe failures and respond to them, it isn’t really an agent. It’s a suggestion engine with a human acting as the executor.

What I Mean by a “Harness”

A coding agent harness isn’t a model and it isn’t a framework. It’s the infrastructure around the model.

It’s the execution loop that:

• runs code in a controlled environment
• captures raw outputs and errors
• feeds that reality back to the model
• applies guardrails around what the system can touch
• and makes every step visible and debuggable

The language model provides reasoning and code.

The harness provides execution, feedback, and constraints.

Together, they turn “generate code” into “try, fail, observe, and improve.”

Why This Matters

This isn’t about replacing developers or claiming AI writes perfect code.

It’s about removing the least valuable parts of development work: rerunning commands, fixing missing imports, interpreting test failures, and repeating the same debug loop over and over.

By making execution and feedback explicit, AI systems become easier to trust, easier to debug, and easier to reason about.

That’s the motivation behind this work: not building a better model, but understanding and shaping the execution layer that makes these tools actually useful.

And honestly, the first time you watch an agent write code, see it fail, understand what broke, and fix itself, that’s genuinely fucking cool. Not because it’s magic, but because you can finally see the entire loop working.

Building the Harness

First part was the motivation. Next is the part where the idea becomes a real execution loop you can watch, debug, and trust.

Before writing any code, I forced myself to answer one question:

What does an "agent" actually do, step by step, when you strip away the UI?

Here's the loop in plain English:

1. Task comes in
2. Model decides what to do next
3. Model asks to use a tool
4. The harness runs something in the real world
5. The harness returns raw results
6. Model updates its plan
7. Repeat until done or we hit a cap

The important detail is the one people forget:

The model never touches your machine.

It only sees whatever the harness returns.

That boundary is the whole point. The harness is the interface between intelligence and execution. If you control that interface, you control the agent.

The Architecture

This harness has three main parts.

1. The workspace

A fresh directory on the host for every run.

This is where the agent writes and reads files. Think of it like a tiny project repo the agent can manipulate.

That choice matters because it's the difference between:

• "run a snippet of Python"
• and "build and iterate on an actual codebase"

2. The Docker sandbox

Every command runs inside a Docker container with the workspace mounted at /workspace.

That gives me isolation, repeatability, and a place to put guardrails.

By default:

• Resource limits are enforced (CPU + memory)
• Commands have a hard timeout
• Networking is disabled unless explicitly enabled

If you are going to let a model execute arbitrary code, you need a safety story. Docker isn't perfect, but it is an enormous step up from running things directly on the host.

The Image Setup:

I use a custom Docker image with pytest pre-installed:

FROM python:3.11-slim
RUN pip install --no-cache-dir pytest
WORKDIR /workspace
CMD ["bash"]

This solves a critical problem: if the agent has to install pytest every run, it wastes iterations and API calls. Pre-installing common dependencies in the image means every container starts ready to work.

3. The tool contract

The model does not run anything directly. It calls tools. The harness executes those tools and returns structured results.

I kept the tool surface area small but real:

• list_files()
• read_file(path)
• write_file(path, content)
• run_command(command, timeout, network_enabled)
• task_complete(summary)

That is enough for real developer workflows:

• Create modules
• Write tests
• Run pytest
• Read failures
• Patch code
• Rerun until green

Why Docker

Yes, I could have used subprocess.run() and called it a day.

But that's not "agent infrastructure." That's handing a model a loaded gun.

You do not want a tool-using model to have direct access to:

• Your filesystem
• Your SSH keys
• Your environment variables
• Your network
• Your ability to fork-bomb your laptop into a space heater

Docker gives me a baseline set of protections:

• Process isolation: Code runs in a container, not on my host
• Resource limits: CPU and memory caps prevent obvious abuse
• Network control: Networking can be off by default
• Reproducibility: Every run starts from a known image

The tradeoff is complexity. It adds friction and edge cases. But for this category of problem, it's the right trade.

The Tools

I used to think tool calling was the "agent" part.

It's not. Tool calling is just the API plumbing that makes the loop possible.

The model is effectively saying: "Please run this for me, and tell me what happened."

The harness is the one doing the doing.

Tool: write_file

This is how the model creates and edits code. It writes directly into the workspace.

In the simplest form, this tool is just an interface to a safe path resolver plus a size limit so the agent can't spam huge files.

Tool: run_command

This is the core tool. It's how the model runs tests, lints, scripts, whatever.

Two design choices here ended up being surprisingly important:

• Timeouts are enforced. If a command hangs, it dies.
• Networking is off by default. If the agent wants internet (pip install, curl, etc.), it has to explicitly ask for it.

That single boolean makes the system easier to reason about, and it's the kind of guardrail you never see from the outside in most agent products.

Here's what the tool schema looks like:

{
  "name": "run_command",
  "description": "Run a shell command in Docker in /workspace. Networking is off by default.",
  "parameters": {
    "type": "object",
    "properties": {
      "command": {"type": "string"},
      "timeout": {"type": "integer", "description": "Seconds"},
      "network_enabled": {"type": "boolean", "description": "Enable for pip install, etc."},
      "description": {"type": "string", "description": "What this is trying to do"}
    },
    "required": ["command", "description"]
  }
}

Tool: task_complete

This is the stop button. Without it, the harness just keeps looping until it hits max_iterations, even if the task is obviously done.

For an agent loop, you need a clear termination condition.

How tool calling actually works

The mechanics are simple:

1. You define tools using a JSON schema
2. You send them with your API request
3. The model can respond with tool calls rather than plain text
4. You execute the tool calls in your harness
5. You send the results back as tool messages
6. The model sees those results and continues

So the conversation looks like:

User: "Build X"

Model: calls write_file and run_command

Harness: executes and returns real output and errors

Model: uses that reality to decide what to do next

Repeat

The key point again:

The model is not executing. It's requesting.

You are the executor. The harness is the gate.

The code that matters

Most of the harness is glue code. The heart is the loop:

for i in range(max_iterations):
    # Get model response with tools
    resp = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        tools=TOOLS,
        tool_choice="auto",
    )
<span class="n">msg</span> <span class="o">=</span> <span class="n">resp</span><span class="p">.</span><span class="n">choices</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="n">message</span>

<span class="c1"># If model wants to use tools

    if msg.tool_calls:
# Store assistant message
        messages.append({
“role”: “assistant”,
“tool_calls”: […]
})
    <span class="c1"># Execute each tool call

        for tc in msg.tool_calls:
name = tc.function.name
args = json.loads(tc.function.arguments)
result = execute_tool(name, args)
        <span class="c1"># Add result back to conversation

            messages.append({
“role”: “tool”,
“tool_call_id”: tc.id,
“content”: json.dumps(result)
})
        <span class="c1"># Check for completion

            if name == “task_complete”:
return success

In practice, the "agent" behavior comes from one thing:

The model sees raw stdout, exit codes, and failures, and it learns what to do next from those signals.

It's the difference between:

• "I think this should work"
• and "I ran it, here is the traceback"

A Real Run

Here's what actually happened when I gave it a real task:

Task: Create a Python text normalization utility with pytest tests

Iteration 1: Create project structure

→ write_file("textutils/core.py")
→ write_file("tests/test_normalize.py")
✓ Files created

The agent writes a normalize() function and 6 test cases covering edge cases like empty strings, multiple spaces, tabs, and newlines.

Iteration 2: Run tests (first attempt)

→ run_command("pytest tests/test_normalize.py")
✗ exit_code=2
Error: "ERROR: file or directory not found: tests/test_normalize.py"

Pytest can't find the module. The agent doesn't understand why yet.

Iterations 3-7: The struggle

→ run_command("pytest --rootdir=/workspace")
✗ exit_code=2
→ run_command(“pytest –pythonpath=/workspace”)
✗ exit_code=4
→ run_command(“pytest –rootdir=/workspace”)
✗ exit_code=2

This is where you see real problem-solving behavior. The agent is stuck trying different pytest configuration flags, but that's not actually the problem.

Five iterations of trying variations before it figures out the real issue.

Iteration 8: The breakthrough

→ write_file("textutils/__init__.py", "")
→ run_command("pytest tests/test_normalize.py")
✓ exit_code=0
===== test session starts =====
collected 6 items
tests/test_normalize.py ……                [100%]
===== 6 passed in 0.02s =====

The agent realizes textutils needs to be a proper Python package. Adds the __init__.py file. All tests pass.

Iteration 9: Done

→ task_complete("Created a text normalization utility with tests. All tests passed successfully.")

Total time: ~25 seconds. Iterations: 9.

What This Teaches You

The satisfying part how it worked.

The agent:

• Created real files
• Ran real commands
• Saw real failures
• Got stuck for a bit (iterations 3-7)
• Had a realization (iteration 8)
• Fixed the actual problem
• Verified success

That's the loop working. Not magic, not cherry-picked—just a feedback cycle that eventually converges on the right answer.

And when something goes wrong, you can replay it:

{
  "iteration": 8,
  "tool": "run_command",
  "args": {"command": "pytest tests/test_normalize.py"},
  "result": {
    "exit_code": 0,
    "stdout": "===== 6 passed in 0.02s =====",
    "duration": 1.85
  }
}

Every tool call is logged. Every decision is traceable.

Error handling and retries

I don't do explicit "retry three times" logic.

The retry loop is implicit.

The model retries because it sees structured execution results like:

• exit_code
• timed_out
• stdout (often includes tracebacks)
• duration

When a command fails, the model gets the failure, thinks, changes code, and reruns.

This works well for:

• Syntax errors
• Missing imports
• Obvious mistakes in logic
• Test failures with clear assertions

It works less well when:

• Requirements are ambiguous
• The failure needs domain knowledge
• The fix is large and multi-step

That's not a model problem. That's a harness ergonomics problem, because the agent still lacks better tools like patch editing, diffing, and memory.

Logs: the feature you don’t appreciate until you need it

I log every tool call to a JSONL file inside the workspace.

That log includes:

• What tool was called
• Arguments
• The full result payload
• Timestamps

It sounds boring, but it changes everything.

When something goes wrong, you can answer:

• What command actually ran
• What the model actually saw
• Whether it tried to install dependencies
• Whether it timed out
• What changed right before it broke

Logs turn agent runs into something you can debug, replay, and trust.

What broke (and what I learned)

Let me be honest about the failures, because this is where the actual lessons are.

Problem 1: Docker dependency management

Initially, I tried letting the agent install pytest on every run. Bad idea.

Fresh containers mean fresh installs every time. The agent would waste 2-3 iterations just getting its environment ready before doing actual work.

The fix: Pre-install common dependencies (pytest, pip tools) in the Docker image. Every container starts with them available.

This is a general pattern: if you know the agent will need something frequently, bake it into the base image.

Problem 2: Context growth

Each iteration adds messages:

system → user → assistant → tool → assistant → tool → …

On longer runs you eventually run into context limits or degraded performance.

What helped in practice:

• Keep max_iterations sane (12-15)
• Make tasks more specific
• Reduce output size (truncate stdout to 20k chars)

The real fix (not fully implemented yet) is proper context management:

• Sliding windows
• Summarizing older tool results
• Storing full logs separately from working context

Problem 3: The agent can be too polite

Sometimes the model hits a wall and says "can't do that" instead of pushing.

Example: missing dependency, or a test failure it doesn't understand, and it decides to stop.

The fix is not "better model." It's better scaffolding:

• Stronger system prompt about persistence
• Better tools (like search within workspace, apply patch)
• Clearer success criteria (tests passing)

Problem 4: Networking policy

Some tasks need internet, often just for installing additional deps.

If network is on by default, the agent will use it constantly.

If it's off forever, the agent is stuck.

So I made it explicit: network is disabled by default, but the model can request it via network_enabled=true on run_command.

That's a good compromise because it forces you to see when and why the agent touches the network. In the logs, you can track every network-enabled command.

Beyond the Sandbox - Adding External Tools

The Integration Problem

When you start thinking about adding external tools, you realize the scope pretty quickly.

If you want your agent to create GitHub issues, you need:

• OAuth flow for GitHub
• Token management and refresh
• API wrapper for GitHub's REST API
• Error handling for rate limits
• Updates when GitHub's API changes

Now multiply that by every service you want: Slack, Gmail, Linear, Notion, databases...

You're looking at months of integration work. Or you could use something like Composio that already built those 1000+ integrations.

The important thing is that external tools don't change the agent loop, they stress it. Authentication failures, network timeouts, and rate limits are just another form of "reality" the harness has to surface back to the model. ToolRouter fits cleanly because it preserves the same contract: the model requests, the harness executes, and raw results come back.

What Composio’s ToolRouter Actually Is

Composio ToolRouter is an integration layer that gives your agent access to external tools through a simple API.

The core concept is sessions. Each session:

• Is scoped to a specific user
• Manages that user's connected apps
• Provides tools the agent can call
• Handles authentication automatically

Here's the basic setup:

from composio import Composio
# Initialize Composio
composio = Composio(api_key=COMPOSIO_API_KEY)
# Create a session for your user
session = composio.create(user_id=“alice@company.com”)
# Get tools for this session
tools = session.tools()

That's it. Now tools contains the ToolRouter meta-tools scoped to this user.

How ToolRouter Actually Works

Here's what I learned when integrating it: ToolRouter doesn't give you individual tools upfront.

Instead, it provides 6 meta-tools that handle everything:

1. COMPOSIO_SEARCH_TOOLS - Searches for relevant tools based on task description
2. COMPOSIO_MULTI_EXECUTE_TOOL - Executes discovered tools
3. COMPOSIO_MANAGE_CONNECTIONS - Handles authentication flows
4. COMPOSIO_REMOTE_WORKBENCH - Processes large responses
5. COMPOSIO_REMOTE_BASH_TOOL - Runs bash commands remotely
6. COMPOSIO_SEARCH_ENTITIES - Searches for entities in connected apps

So instead of calling GITHUB_CREATE_ISSUE directly, the workflow is:

1. Agent calls COMPOSIO_SEARCH_TOOLS with "create a GitHub issue"
2. ToolRouter finds the right tool
3. Agent calls COMPOSIO_MULTI_EXECUTE_TOOL with parameters
4. ToolRouter executes it using the user's connected GitHub account

This is actually smarter than loading every possible tool. The meta-tools handle discovery, authentication, and execution dynamically.

Two Ways to Use ToolRouter

Composio gives you two integration patterns:

Option 1: As Native Tools (What I Used)

You get the 6 meta-tools and pass them to your agent framework:

# Create session
session = composio.create(user_id="user_123")
# Get meta-tools
tools = session.tools()
# Pass to your agent (OpenAI, Anthropic, etc.)
response = client.chat.completions.create(
model=“gpt-4”,
messages=[…],
tools=tools  # The 6 ToolRouter meta-tools
)

The agent uses these meta-tools to discover and execute what it needs.

Option 2: As MCP Server

If you're using an MCP-compatible framework (like Claude Agent SDK), you can connect via the MCP protocol:

# Create session
session = composio.create(user_id="user_123")
# Get MCP endpoint
mcp_url = session.mcp.url
mcp_headers = session.mcp.headers
# Configure your MCP client
The MCP server handles tool routing automatically

The MCP approach is more dynamic but adds complexity. For the harness, native tools made more sense.

For this harness, I wanted to keep the execution boundary explicit in my own code rather than delegate it to an MCP runtime.

How It Integrates With the Harness

The integration pattern is straightforward:

Agent Loop
    ├─→ Local Tools (files, Docker commands)
    └─→ Composio Meta-Tools (discovery, execution)
    ↓
Execute tool
    ├─→ Local? Run in Docker
    └─→ Composio? Route to ToolRouter
    ↓
Results back to agent

The key is combining both tool types:

class AgentExecutor:
    def __init__(self, user_id, enable_composio=False):
        self.workspace = Workspace()
        self.sandbox = DockerSandbox(self.workspace)
    <span class="c1"># Composio setup

        self.composio_session = None
if enable_composio:
composio = Composio(api_key=COMPOSIO_API_KEY)
self.composio_session = composio.create(user_id=user_id)
<span class="k">def</span> <span class="nf">get_all_tools</span><span class="p">(</span><span class="n">self</span><span class="p">):</span>
    <span class="sh">"""</span><span class="s">Combine local and Composio tools</span><span class="sh">"""</span>
    <span class="n">tools</span> <span class="o">=</span> <span class="n">LOCAL_TOOLS</span>  <span class="c1"># write_file, run_command, etc.


        if self.composio_session:
            composio_tools = self.composio_session.tools()
            tools.extend(composio_tools)  # Add the 6 meta-tools

        return tools

The agent doesn't know or care where tools come from. It just calls them. We handle the routing.

Authentication Flow

The authentication part is what makes ToolRouter useful. Here's how it works:

First time a user needs an app:

1. Agent tries to use a tool (searches for "GitHub issue")
2. User hasn't connected GitHub yet
3. ToolRouter (via COMPOSIO_MANAGE_CONNECTIONS) returns an auth URL
4. User clicks URL, completes OAuth
5. Agent retries, now it works

After that:

• Composio manages the tokens
• Handles refresh automatically
• Agent just calls the meta-tools

You can pre-connect apps for users via the Composio dashboard (https://app.composio.dev/apps), or let them authenticate on-demand.

For the harness, pre-connecting apps is smoother. Less interruption during agent runs.

A Simple Example

Let me show you what a workflow looks like with ToolRouter.

Task: "Create a calculator module, test it, then create a GitHub issue."

The agent:

1. Writes calc.py with functions (local tools)
2. Writes test_calc.py (local tools)
3. Runs pytest (local tool: Docker execution)
4. Sees tests pass
5. Calls COMPOSIO_SEARCH_TOOLS to find GitHub tools
6. Calls COMPOSIO_MULTI_EXECUTE_TOOL to create the issue
7. Returns the issue URL

Steps 5-6 are where ToolRouter shines. The agent doesn't need to know GitHub's API. It just describes what it wants, and ToolRouter handles the rest.

What’s Available

Through ToolRouter's meta-tools, you get access to many integrations:

Development:

• GitHub: create issues, PRs, manage repos
• GitLab, Bitbucket
• Jira, Linear: task management

Communication:

• Slack: send messages, create channels
• Discord, Teams
• Gmail: send/read emails

Productivity:

• Notion, Google Docs
• Calendar, Drive
• Trello, Asana

Databases:

• PostgreSQL, MongoDB, MySQL
• Airtable, Supabase

The agent discovers these through COMPOSIO_SEARCH_TOOLS as needed.

What I Learned

What works well:

• The meta-tool approach is cleaner than loading individual tools
• Authentication handling is automatic
• Discovery works - agent finds the right tools
• Error messages are clear (e.g., "connect this app first")

What needs more work:

• Execution can be slow (network round trips)
• Error handling for transient failures could be better
• Cost tracking per workflow
• Pre-checking connection status before trying operations

The goal isn't to integrate everything. It's to integrate the 3-5 apps that matter for your specific workflows.

The Honest Assessment

Composio solves a real problem. Building and maintaining multiple integrations yourself isn't realistic.

But it's not magic:

• You still need to understand how the meta-tools work
• Authentication setup takes time (connecting apps)
• External APIs add latency and failure modes
• Costs go up (more API calls)

The value proposition: trade integration complexity for a simpler API. Instead of learning OAuth flows for 10 services, you learn how ToolRouter's 6 meta-tools work.

For most cases, that's the right trade.

Wrapping Up

At this point, the agent isn't just writing code in isolation. It can change a codebase, verify behavior, and take real actions on behalf of a user, all through explicit, inspectable boundaries.

That's the pattern that keeps showing up: models reason, harnesses execute, and tools expose reality. Once you get that separation right, adding integrations stops being scary and starts being composable.

The result: an agent that can write code, test it locally, and take actions through connected services.

It's not perfect. It's not production-ready. But it's functional and extensible.

The integration pattern works - the agent can discover and call external tools through ToolRouter's meta-tools. I've tested the basic workflow locally, and the potential for GitHub issues, Slack notifications, and other integrations is straightforward from here.

The integration patterns are straightforward. The APIs are manageable. The possibilities are interesting.

That's where this is at right now.