Docs menu

Core ideas

Core ideas

LayerFS gives every agent a complete, writable filesystem over shared committed history. These are the ideas behind it, one at a time, using one project as the example: a web shop called shop.

The cards are the home page's. Their numbers and timings illustrate each idea; they are not measurements.

Core idea 1 Ephemeral Computing

Ephemeral Computing

Compute exists only while a command runs.

fs · atomic write → Storewriteeditrmsandbox · workspace per bashmountexeccommitunmountmountexeccommitunmountmountexeccommitunmountmountexeccommitunmountmountexeccommitunmount>_ mkdir out>_ cat log.txt>_ python train.py>_ node build.jsUpToDateStoreC41C42C43C44C45C46
  • Compute: Agents spend ~80% thinking, ~10% in Bash.
  • FS: Each file tool call is one atomic Store op.
  • Sandbox: Each Bash tool call mounts an ephemeral workspace, then commits.

An agent spends most of its time thinking, and a sandbox held for the whole task sits idle through all of it. LayerFS makes a complete Workspace cheap enough to mount for a single command, so compute exists only while a command runs.

How it works

  • File calls need no computer. fs.read, fs.edit, fs.write and the rest run in your own process, straight against the Store. A write is one atomic operation and one Commit.
  • Bash calls get a fresh Workspace. Mount the Branch, run the command, commit what changed, unmount. The mount copies and installs nothing; unmounting deletes the overlay and keeps the Commit.
  • Nothing happens on its own. LayerFS never commits, times out or unmounts by itself, so a Workspace can also live for a whole task.

Core idea 2 Decentralized Agents

Decentralized Agents

Agents on different computers share one serverless Store.

store.dbC41C42C43C44C45C46C47computer 1computer 2computer 3computer 4
  • Shared storage: Every agent commits to one Store, with no DB server.
  • Distributed computing: Each agent runs on its own computer.

There is no server. All of a project's history is one SQLite file, the Store, on a volume the agents share, and every agent opens it directly.

How it works

  • One Sandbox per agent. Each agent's container runs its own LayerFS daemon, which serves its Workspaces and reads and writes the Store in-process.
  • The SDK opens it too. fs, branch and project calls run in your process against the same file; only Workspace calls go to a daemon.
  • One writer at a time, briefly. Writers take turns through a short publish step (check the head, merge, publish). Reads never wait for it.

Today: every opener must share one Linux kernel, with the Store on a local ext4, xfs or btrfs filesystem. Elsewhere Store calls fail Unsupported.

Core idea 3 Multiagent Coordination

Multiagent Coordination

One project, many branches. One branch, many agents.

mainauthuilogin.tsapi.tssession.tsapp.tsxtoken.tsapp.tsxnav.tsxConflict: app.tsxapp.tsx · merge markers<<<<<<< head C44 <Cart />======= <Search />>>>>>>> workspaceblame app.tsxC12import NavC44<Header />C44<Cart />C46<Search />L3 · publishL4 · publish
  • Commits: Every tool call is a commit on its branch.
  • Conflict: A same-file clash is refused with Conflict.
  • Blame: Each line names the step that wrote it.

One project has many Branches, and one Branch can have many agents. Nobody locks files or takes turns: each tool call is a Commit that merges with the Branch as it is when the Commit lands.

How it works

  • Different files land on top of the newer head as Committed.
  • The same file is refused as Conflict, with the paths and the head it met. Nothing is committed; the agent reads the new head and edits again.
  • In a Workspace, a conflict writes marker blocks into the files and the Workspace follows the new head; fix them and commit again.
  • Blame and diffs name the Commit, and so the tool call, behind every line and change.

Core idea 4 Recursive Filesystem

Recursive Filesystem

Fork recursively any commit, at any depth.

publish0.620.480.210.400.780.660.300.710.910.69prunedprunedprunedprunedpruned★ best 0.91explorers ×3explorers ×3explorers ×4explorers ×3explorers ×1fork: no files copied
  • Zero cost: A fork is metadata and copies no files.
  • Recursive: A filesystem for RSI agents and recursive exploration for RL training.

A fork records where it starts and copies no files, so forking is cheap enough to use freely: from main, any Branch, any Commit or Layer, and from forks of forks at any depth.

How it works

  • A fork is one record. branch.fork(name, fromRef) points a new Branch at its start. Its files are the start's files until it changes them.
  • Changes are stored once. Each change a fork makes is shared with everything that already has it, so a losing fork costs only the bytes it changed.
  • Keep the winner. branch.merge folds it back (a fast-forward when one side holds the other), and branch.publish makes it main's next Layer.

Core idea 5 Ultra Storage Optimization

Ultra Storage Optimization

10 copies compress into 0.31.

commits1 · dedupe12 chunks13 chunks14 chunks15 chunks11 matched · +1 new11 matched · +1 new11 matched · +1 newdedup → ≈1.05 copies→2 · compress≈0.31 copiesillustrative · store.stats
  • Dedup: Each content chunk is stored once.
  • Compression: The stored chunks shrink further.
  • Ratio: 10 copies → 1.05 dedup → 0.31 compression wtf???

Agents make copies: a Branch per attempt, a Commit per step, the same node_modules in each. Copies of the same content cost almost nothing, and what the Store keeps is compressed.

How it works

  • Content addressing. Every object is named by the hash of its bytes, so identical content is stored once.
  • Chunking. Files under 128 KiB are kept whole; larger ones are cut by content into chunks of about 8 to 32 KiB, so an edit changes only nearby chunks.
  • Deltas and compression. A file or chunk can be stored as a delta against its last version, and everything is packed and compressed with zstd.
  • Measured per Store. The operator API's store.stats reports both ratios: dedupe (logical ÷ canonical) and compression (canonical ÷ stored).

Core idea 6 Instant Bootstrap

Instant Bootstrap

Dependencies come with every mount. No wasting time for re-installation.

1 · git / other AgentFS without .gitignore filesoops!src/package.json.git/node_modules/node_modules/.cache/.cache/$ npm test✗ Error: Cannot find module 'react'$ npm install$ npm test✓ passed⏱ 0:05⏱ 0:20⏱ 1:00⏱ 2:00⏱ 3:00⏱ 4:00⏱ 5:00⏱ ≈5 min before the first runclone + install, typical⚠ wasting time · waiting on install⌛2 · LayerFSnpm install committed once, back then (C12)src/package.json.git/node_modules/.cache/$ mount → workspace ready$ npm test✓ passedinstant · no install⚡ mount 0.05 smeasured mounts: 5–20 ms⚡⚡ ready the moment it mounts
  • Git / other AgentFS: Drops ignored files; reinstalls.
  • LayerFS: Ignored files stay in every commit.
  • Comparison: ~5 min to install vs 0.05 s to mount.

A fresh Git checkout isn't ready to run: ignored files like node_modules and caches are missing, so every new workspace installs first. LayerFS keeps them.

How it works

  • Init takes everything, once. project.init captures source, .git, dependencies, caches, symlinks and every ignored file as main's first Layer.
  • A mount binds the whole tree. No scan, copy or install. The only file it creates is the Workspace's small overlay database.
  • Content loads on demand. Files are read from the Store as they're opened, so the first read still pays real I/O.
  • Installs are just changes. An install that changes node_modules is committed, and every later mount has it.

Supporting ideas · they back the core ones

Supporting idea 1 Auto Checkpoints

Auto Checkpoints

Every step an agent takes is a checkpoint to return to.

every tool call → a checkpointcartwriteeditnpm testrm -rf srcUpToDate · no commitC41C42C43✗ tests failfork from C42cart.fixC44 ✓ tests pass
  • Every call: Each file call is a commit; each Bash call commits when it ends.
  • No noise: A call that changes nothing commits nothing.
  • Rewind: Fork from any checkpoint and read any file as it was.

You don't save checkpoints by hand. Every change an agent makes is already a Commit, so every step is a point to inspect, compare or start again from.

How it works

  • File calls commit as they write.
  • Bash calls commit when the orchestrator calls sandbox.commit after the command. It saves the whole Workspace, background processes' writes included.
  • No change, no Commit. A call that leaves the tree as it was ends UpToDate.
  • Every Commit is a ref. Read any file at it, diff any two, or fork from one to carry on.

Checkpoints hold files only, not running processes, memory, sockets or command output.

Supporting idea 2 Append-only Objects

Append-only Objects

Saved objects are never rewritten. The Store only grows.

store.db · append-onlynothing rewritten · nothing deletedwriterreaderreads never wait#a3f2#09bc#e71d#5c40#d2a8✗ overwrite refused
  • Content-addressed: An object is named by the hash of its bytes.
  • Sealed: A written pack is never updated or deleted.
  • Safe to share: Branches share objects; readers never wait for the writer.

Nothing the Store has saved is ever rewritten. Chunks, file trees, Commits and Layers are written once and then only read.

How it works

  • Named by content. Every object's id is the hash of its bytes; a Commit's id comes from its tree, parents and message.
  • Sealed packs. Objects are written into packs, and the Store refuses to update or delete a written pack.
  • Safe sharing. Forks, merges and other agents point at the same objects without copying them, and readers never wait for a writer.
  • History stays true. Deleting a file only stops later Commits from naming it; older ones keep it.

Immutable isn't durable: the Store runs WAL with synchronous=OFF, so a host crash can lose recent work. There's no garbage collection yet, so the Store only grows.

Supporting idea 3 Conflict Rules (.lww)

Conflict Rules (.lww)

Name the files that should never conflict. The newest edit wins.

/.lwwpackage-lock.jsondist/**10:0110:03package-lock.jsonsrc/cart.ts10:0110:0110:0310:03newer wins · 10:03 ✓⚠ Conflict · markers
  • /.lww: Gitignore-style rules at the root of the tree.
  • Newer wins: A matched file changed on both sides takes the newer edit.
  • For: Lockfiles, build output and caches.

Lockfiles, generated code, build output and caches are rewritten wholesale by tools, and a line merge of them is useless. A /.lww file names them, and for those paths the newest edit wins.

How it works

  • Rules at the root. /.lww holds gitignore-style patterns; %include <file> pulls in more.
  • Newer wins. When a matched path changed on both sides, the side with the newer modification time wins (ours on a tie), with no conflict. Other paths merge line by line.
  • Whose rules. A Workspace Commit reads the Branch head's; branch.merge(a, b) reads a's.
  • Recorded. Every decision is listed under lww in the result and kept on the Commit.

A rule drops the older side's change. Use it for files a tool can regenerate, never for code people write.

Supporting idea 4 Big File, Small Edit

Big File, Small Edit

An edit costs the size of the edit, not the size of the file.

big.log · 2 GiB · one extent treewrite 4 KiBwrite 1 KiBcommit: 1 chunk + 3 tree nodes newthe rest of the file is shared
  • Overlay: A write keeps only the bytes it wrote.
  • Commit: Only the edited extents and their tree path are rebuilt.
  • Storage: The rest of the file is shared with its old version.

Editing a few bytes of a large file shouldn't cost the whole file. An overwrite, append or truncate costs about the size of the change and the depth of the file's tree, not the file's size.

How it works

  • Files are trees of extents over their content chunks.
  • The overlay keeps only what was written, not a copy of the file.
  • The Commit edits the tree in place, rebuilding only the changed extents and their path to the root; the rest is shared with the previous version.
  • Ordinary writes, by any program on the mount. No special call, and no file size limit.

An insert or delete in the middle has no POSIX write, so most programs rewrite everything after it, and that rewrite is what gets saved.