Towards a Disaggregated Agent Filesystem on Object Storage

The filesystem abstraction is an effective interface for AI agents, and AgentFS provides exactly that on top of a SQLite database file. But as agent workloads scale with more agents, longer-running tasks, and multi-agent collaboration, we need to move beyond the single-file-on-local-disk model. State needs to survive ephemeral compute, move between machines, and scale without requiring persistent volumes to be provisioned. This post explores how SQLite’s WAL-based architecture, combined with Turso’s S3-based diskless architecture, provides a path toward disaggregated agent filesystems on object storage.

Problem

Agents want a filesystem. The filesystem abstraction is an effective i…

Problem

Agents want a filesystem. The filesystem abstraction is an effective interface for AI agents. Foundation models have ingested enormous amounts of shell scripting, Unix documentation, and file-based workflows during training. Give an agent access to grep, sed, awk, cat, and git, and it becomes unreasonably capable and effective, requiring no custom tools. The everything-is-a-file philosophy that Unix established over 50 years ago created an ecosystem of thousands of specialized CLI tools, all built around a common abstraction. For agents, this represents a huge portion of their training data and a natural way to interact with the world.

The problem is that real filesystems require real infrastructure. Running agents with filesystem access means either spinning up containers, managing VMs, or accepting the security risk of letting agents run commands directly on your machine. Furthermore, in lightweight environments like serverless or browser-based agents, there is no filesystem to access to begin with. And even when you do have a filesystem, operating persistent volumes at scale to host them is just hard.

Agents need to be snapshotted and resumed. Autonomous systems are inherently unpredictable. Their chains of reasoning can lead to unforeseen outcomes, and when things go wrong, you need visibility into every decision, every file created, every state change. You need the ability to capture an agent’s entire runtime at any point for debugging, reproducibility, regulatory compliance, and rolling back mistakes.

Traditional approaches fall short because they involve cobbling together multiple tools: databases for state storage, logging systems for audit trails, local filesystems for artifacts, and version control for history. The inherent fragmentation creates complexity and leaves gaps in observability. What you actually want is the ability to snapshot an agent’s complete state by copying a single file, then restore it later to reproduce exact execution conditions or test what-if scenarios.

Agent state needs to be portable. Keeping the agent compute on a specific machine is cumbersome and expensive. Agents can be interrupted by timeouts, scaling decisions, or failures. Furthermore, serverless platforms are ephemeral by design: your code spins up, runs, and then disappears. If the agent state lives on a local filesystem or in memory, it disappears too.

State portability is especially needed when you need to snapshot and resume. An agent working on a long task might checkpoint its progress, get evicted, and need to resume on an entirely different machine hours later. Or you could pause an agent, inspect its state, then continue from exactly where it left off. None of this works if the state is tied to a specific instance or volume.

You need portability for debugging and collaboration as well. When something goes wrong, you want to grab the agent’s complete state files, context, tool call history—and reproduce the problem locally. You might want to share that state with a colleague or attach it to a bug report. You wish to diff two runs, or query exactly what happened with SQL. Having a state that can travel as a single, portable unit, not one scattered across filesystems, environment variables, and ephemeral containers, is the key.

The Agent Filesystem

AgentFS is a filesystem abstraction designed for agents. Rather than relying on the host operating system’s filesystem, AgentFS stores everything in a SQLite database: files, directories, metadata, key-value state, and an audit log of tool calls. The approach gives you the familiar filesystem semantics agents expect, while making the entire agent runtime portable, queryable, and snapshottable as a single file.

The design provides three core interfaces. The filesystem interface provides POSIX-like operations on files and directories—read, write, create, rename, and delete. Agents interact with it the same way they would with any filesystem. The key-value interface stores agent state and context as JSON-serialized values, needed for session data, configuration, or intermediate computation results. The toolcall interface records an append-only audit log of every tool invocation, capturing inputs, outputs, and timing for debugging and compliance.

Under the hood, the filesystem storage follows a classic inode/dentry design. The dentry (directory entry) table maps names to locations in the directory hierarchy, acting as a lookup cache. The inode table stores actual file contents and metadata (size, permissions, and timestamps). This separation mirrors how operating system kernels organize filesystem data, but implemented entirely in SQLite tables. Deletions in overlay scenarios use a whiteout mechanism: rather than modifying a base layer, the overlay records deleted paths in a whiteout table and filters them out on reads.

The key insight is that SQLite provides exactly the properties agents need. A single file contains the complete agent runtime, which you can copy to another machine, check into version control, or attach to a bug report. You can snapshot the state at any point by copying the database file, then restore it later to reproduce exact execution conditions. And because everything is structured data, you can query agent behavior with SQL—find all files modified in the last hour, trace the sequence of tool calls that led to an error, or extract metrics for analysis.

For agents that need native tool compatibility, AgentFS provides FUSE support on Linux (and NFS on macOS) that you mount the SQLite-backed filesystem as a real POSIX filesystem, allowing you to run git, grep, ffmpeg, or any Unix tool directly without integration code. As shown in Figure 1, the agent invokes bash, which executes commands. The commands interact with the operating system kernel via system calls. Filesystem operations such as open and close are delegated to a userspace daemon that AgentFS provides to access the filesystem metadata stored in a SQLite database file. Reading from files is very cheap because the kernel page cache amortizes the cost of reading from the FUSE userspace daemon. Writing to files is also done on the kernel page cache first, then written back asynchronously to the FUSE module.

Figure 1: AgentFS and FUSE architecture. The agent spawns a bash process that executes commands. System calls like open and close go through the kernel’s VFS layer to the FUSE module, which delegates to the AgentFS userspace daemon. The daemon stores filesystem metadata in Turso/SQLite. Reads benefit from the kernel page cache, avoiding repeated FUSE calls. Writes are buffered in the page cache and written back asynchronously.

AgentFS also supports other integration patterns depending on the environment. The SDK (available in TypeScript, Python, and Rust) provides programmatic access for agents built with frameworks like AI SDK, Mastra, or others. For lightweight environments like serverless or browsers where you can’t mount a filesystem, the bash-tool integration provides a TypeScript reimplementation of common shell commands that operate directly on AgentFS.

Towards a Disaggregated Agent Filesystem

The AgentFS design stores everything in a single SQLite database file on the local disk. The approach works well for single-machine scenarios: it provides atomicity, durability, queryability, and portability. However, as agent workloads scale with more agents, longer-running tasks, or multi-agent workloads, we need a disaggregated model in which the agent filesystem resides on an object store.

A decoupled-metadata, object-backed distributed filesystem, such as JuiceFS or HopsFS, solves the problem by separating filesystem metadata and data, storing them in a key-value or relational database and an object store, respectively. However, building AgentFS on top of SQLite enables disaggregation at a different level: the database itself. SQLite uses a write-ahead log (WAL) that captures all mutations as a sequence of changes before they’re checkpointed into the main database file. The database file represents the committed state; the WAL represents in-flight changes.

With Turso’s sync protocol, local changes are pushed to a remote as logical mutations, while remote changes are pulled as physical pages. The object store acts as the source of truth. This means you can checkpoint the database file to object storage, stream WAL entries through a coordination layer, and reconstruct agent state on any machine. As shown in Figure 2, an agent can work locally and sync incrementally to S3 via a coordinator. When there’s a need, for example, to migrate compute, we can sync all the changes to the cloud, start the agent on another machine, and seamlessly pull the changes and resume work.

Figure 2: Agent state portability with incremental sync. During work, WAL frames are batched and pushed to the coordinator, which immediately persists them to S3 for durability. When compute moves to a new instance, it pulls pages and WAL from the coordinator’s cache for fast restore. The compute is ephemeral; the state persists.

This hybrid approach gives you the best of both worlds. Reads and writes occur at local-disk speed against an in-memory or locally cached database. The WAL captures changes with minimal overhead. Background sync handles durability and replication to object storage without blocking the agent. And because the remote state is the source of truth, you get automatic conflict resolution when multiple agents modify the same filesystem, just like you would get with something like Dropbox, for example.

The approach also unlocks some additional capabilities agents need. Ephemeral compute, persistent state lets agents on serverless platforms spin up, pull their filesystem state from S3, do work, push changes, and disappear. No persistent volumes to provision, no EBS snapshots to manage, no storage costs while the agent is dormant. Time-travel and branching is easy because the WAL captures every mutation; you can roll back to any previous state or fork an agent filesystem to explore alternative paths. Want to see how an agent would handle a different approach? Branch from a checkpoint, let it run, and compare results. Traditional distributed filesystems only store the current state—this gives you history for free.

In addition, multi-agent collaboration means multiple agents can work on the same filesystem. The coordination layer handles conflicts—last-push-wins by default, or custom merge logic via transform hooks when needed. Offline execution means the agent doesn’t need continuous connectivity to object storage. Pull a checkpoint, work locally, push changes when you’re done. The sync protocol handles merging. This matters for intermittent connectivity or when you want to run experiments locally before committing to cloud resources.

Future Work

The disaggregated architecture described here is a direction, not a finished system. Several open questions remain.

Write concurrency. SQLite has a single-writer model, which limits write concurrency. This is particularly problematic for multi-threaded or multi-process agent workloads, as writes to the SQLite-based filesystem can easily become a bottleneck. The multi-version concurrency control implementation in Turso allows multiple writers, which mitigates this problem, but needs to be enabled in AgentFS.

Checkpointing overhead. Turso and SQLite need to periodically fold the WAL to the database file to retain good read performance. This checkpointing process introduces latency, which impacts filesystem performance. Although agents are typically latency sensitive, an incremental checkpoint approach might be needed.

Write amplification. SQLite writes happen at page granularity (typically 4KB). A single-byte change still writes a whole page to the WAL. FUSE and NFS mitigate this by batching writes through the kernel page cache, but the underlying amplification remains. Physiological logging, which combines logical operations with physical page references, could reduce the amplification by shipping only the actual changes rather than whole pages.

Large files. SQLite works well for many small files, but what about agents that produce huge artifacts, such as videos, datasets, or model weights? Storing multi-gigabyte blobs in SQLite pages is inefficient because the storage engine must split them into long chains of fixed-size pages, resulting in random I/O operations and greater write amplification, for example. A hybrid approach that stores large files directly in S3 while keeping metadata in SQLite might be necessary.

Summary

Agents need filesystems. They’ve been trained on decades of Unix tooling and file-based workflows. AgentFS provides that abstraction on top of SQLite, giving you portability, queryability, and snapshotting in a single file.

But a single file on the local disk only scales so far. When compute is ephemeral, when agents need to migrate between machines, when multiple agents collaborate on shared state, you need disaggregation. SQLite’s WAL-based architecture, combined with Turso’s sync protocol, provides a path forward: local-speed reads and writes, background sync to object storage, and state that can move from one machine to another.