Concepts

01 Project and Store

A project is one source tree under LayerFS. Its name is 1 to 63 lowercase ASCII letters, digits,_, - and ., starting and ending with a letter or digit.new LayerFS(name) selects it and does nothing else; the client opens files, loads its native addon and connects only when a call needs to.

project.init(sourcePath) reads the tree once and makes it the first Layer on main,main @ L0. It takes everything: source, .git and its index, node_modules, build caches, symlinks and every ignored file. Git ignore rules don't filter LayerFS state. A second Init of a name that exists is refused as Exists; later clients find the project without reading the source again.

import { LayerFS } from "@layerfs/sdk";

const layerfs = new LayerFS("shop");     // lazy: no file, addon or network I/O yet
await layerfs.project.init("./shop");    // main @ L0: the whole tree, once

The Store is one SQLite database file holding all of a project's committed state: content chunks, trees, Commits, Layers and Branches. There is no second database and nothing is stored outside it.

• Opened directly. Each Sandbox daemon opens the Store from the shared volume. So does the SDK, in your own process, for fs, branch and project calls. No database server sits in between.
• One writer at a time. Writes take turns through a short publication step. Reads never wait for it.
• Shared content. Content is split into chunks, each kept once and compressed, so Branches of one tree share almost all of it.

Init also records the Store's policy for its life: the size under which a file is kept whole (131072 bytes by default) and the delta depths for whole files, chunks and metadata (16, 8 and 8). Passnew LayerFS(name, { storePolicy }) to choose your own.

Every process that opens a Store must share one Linux kernel, and the Store must be on a local ext4, xfs or btrfs filesystem. Anywhere else Store calls fail Unsupported.

02 Branch

A Branch is a named line of Commits, and it's where agents work. main is different: it's read-only, a stack of Layers that changes only when a Branch is published.

branch.fork(name, fromRef) starts a Branch from fromRef, main by default. A fork records where it starts and copies no files, so a fork of a fork at any depth costs the same as the first. A fork from main has no Commit yet; its first write makes one.

await layerfs.branch.fork("cart");               // from main's newest Layer
await layerfs.branch.fork("cart.1", "cart");     // from cart's head
await layerfs.branch.fork("cart.old", "3f2a…");  // from a Commit or Layer id

Every read names a ref, the view of the tree to read:

A call resolves each live ref once, so one call reads one consistent snapshot. Two calls in a row may see two different heads if someone committed in between.

03 Commit and Layer

A Commit is one saved change on a Branch. Every fs write call makes one, and so does everysandbox.commit and every merge. A Commit names the tree it saved and its parent; a merge Commit has two. Commits never change, and a Commit id is a ref you can read, fork from or diff.

• Messages. Each write takes an optional message, up to 4096 bytes of UTF-8. Leave it out and LayerFS describes the operation. The message is part of the Commit's identity.
• No change, no Commit. A write that leaves the tree as it was answers UpToDate with the current head. A message alone never makes a Commit.
• Runs aren't Commits. sandbox.exec takes a message too, but a run creates no history; the Commit after it does.

A Layer is one published state of main. Init makes main @ L0, and eachbranch.publish(branch, commit) adds the next, from the Branch's head or the Commit you name.

const result = await layerfs.branch.publish("cart");
// { kind: "Added", publishedLayer, basedOn,
//   overwroteCount, overwrote: [first 50], overwroteHasMore }

Publishing overwrites. It doesn't merge with Layers published since the Branch forked: it skips them and lists them in the answer: overwroteCount and the first 50 in overwrote. To keep that work, fork from main, merge your Branch into the fork, then publish the fork.

04 Sandbox and Workspace

The two always come together. A Sandbox is the container where commands run; a Workspace is a Branch mounted inside it, the directory those commands run in. A Workspace exists only inside a Sandbox, and a Sandbox is useful only for the Workspaces it holds: one Sandbox, any number of Workspaces, each a mount of any Branch of the project.

const id = await layerfs.sandbox.create("dev", "shop", process.env.LAYERFS_IMAGE, {
  volumes: [{ volume: "pnpm-store", target: "/pnpm" }],   // optional, read-only
});

// One Workspace per Bash call: mount, run, commit, unmount.
const ws = await layerfs.sandbox.mount(id, "cart");       // "/workspaces/1"
await layerfs.sandbox.exec(id, ws, "npm install && npm test", "Run tests");
await layerfs.sandbox.status(id, ws);                     // its base, the Branch head, what changed
await layerfs.sandbox.commit(id, ws, "Fix cart total");
await layerfs.sandbox.unmount(id, ws);

await layerfs.sandbox.close(id);                          // once no Workspace is mounted

The Sandbox

A named Docker container running its own LayerFS daemon, bound to one project and one image for its life. The daemon opens the project's Store directly and serves the Workspaces' files; it doesn't supervise commands.

• Names. 1 to 63 lowercase letters, digits, _, - and .; the name is the Sandbox's id in every later call. Names are unique in the LayerFS home, so creating a live name again is Exists.
• Image. A local image digest, sha256: and 64 hex characters.
• Extra volumes. Existing volumes can be mounted read-only outside /workspaces, a shared package store say. A write there fails EROFS.
• One at a time to create. Creates share a lock per project, and a create that meets another is refused as Busy before it does anything. Once created, Sandboxes work independently and at the same time.
• Closing. sandbox.close stops and deletes the container. It's refused as MountsActive while any Workspace is mounted.

The Workspace

A complete, writable filesystem over a Branch's head, mounted at /workspaces/1,/workspaces/2 and so on. It holds the whole tree, ignored files and dependencies included, so commands run as they would in a fresh checkout that was already installed.

• Mounting is cheap. A mount doesn't scan, copy or install the tree. Its only new file is the Workspace's own small overlay database; content loads from the Store as it's read.
• Running commands. sandbox.exec runs bash -c in the Workspace and returns the exit code, stdout and stderr. Any other process that can see the mount works through ordinary file calls too, whether you started it with sandbox.exec or not.
• Commit saves everything so far. sandbox.commit saves the Workspace as it stands, which can include several commands' changes. Nothing happens on its own: no automatic Commit, timeout or unmount.
• One Workspace, many calls. Mount one per tool call, or keep one for a whole task and Commit as you go. Several calls can share one Workspace at once.
• Unmount ends it. sandbox.unmount deletes the overlay; the history stays. With changes not yet committed it's refused as Dirty, unless you pass discard = true.

05 Merge and conflicts

Changes meet in two places: when a Commit lands on a Branch whose head has moved, and whenbranch.merge folds one Branch into another. Both merge three ways, against what the two sides started from, and both decide a file changed on both sides the same way:

Overlapping changes are a conflict, and a conflict never half-lands: nothing is committed or created, and nothing is retried. There are two ways through one: update the files until the two sides agree, or name the paths in /.lww so the newer version wins. Each way is shown below as a scene and the calls that drive it.

Commit conflicts

Many agents can write to one Branch at once. Each write merges with the head it finds when it publishes: different files land on top as Committed; the same file is refused as Conflict, with the paths in detail.conflicts and the head it met in detail.head.

1 Resolve by updating the files

From a Workspace, the refused Commit leaves the merge in the Workspace, with a marker block (<<<<<<< workspace, =======, >>>>>>> cart@<head>) in each conflicted file, and the Workspace now follows the new head. Fix the files with ordinary commands and commit again; a Commit is refused while a conflicted file still holds a complete marker block. A file call such as fs.edit leaves nothing behind: read the file at the new head and make the edit again.

// A and B both mounted cart at C1 and changed the same line of src/cart.js. A committed first, as C2.

// 1. B commits. Nothing is published; the merge, with a marker block, is written into its Workspace, which now follows C2.

›

await layerfs.sandbox.commit(box, ws, "Rename qty")

LayerFSError: Conflict detail: { head: "c2…", conflicts: [{ path: "/src/cart.js", kind: "content", blocks: 1 }] }

// 2. The markers are in the file, for any command to read.

›

await layerfs.sandbox.exec(box, ws, "cat src/cart.js")

<<<<<<< workspace const total = price * quantity; ======= const total = cents(price) * qty; >>>>>>> cart@c2…

// 3. B writes the line both changes wanted, with an ordinary command.

›

await layerfs.sandbox.exec(box, ws,

"echo 'const total = cents(price) * quantity;' > src/cart.js")

{ kind: "exit:0", exitCode: 0, stdout: "", stderr: "" }

// 4. No marker block is left, so the Commit lands on C2.

›

await layerfs.sandbox.commit(box, ws, "Rename qty")

{ kind: "Committed", head: "c3…", commit: { parent: "c2…", message: "Rename qty" } }

›

2 Resolve with .lww

Tools rewrite some files wholesale (build output, lockfiles, caches), and a line merge of them is useless. Name them in /.lww at the root, with gitignore-style patterns (%include <file> pulls in more), and a path both sides changed goes to the side with the newer modification time, ours on a tie, with no conflict. A Workspace Commit reads the rules on the Branch head it merges with, and lists each decision underlww in the Commit.

// 1. Both agents rebuild, so both Commits will change dist/app.js. Name it in cart's /.lww first.

›

await layerfs.fs.write("cart", "/.lww", "dist/\n", "Build output: newest wins")

{ kind: "Committed", head: "c2…" }

// 2. A builds at 10:02 and commits first.

›

await layerfs.sandbox.exec(boxA, wsA, "npm run build")

await layerfs.sandbox.commit(boxA, wsA, "Build")

{ kind: "Committed", head: "c3…" }

// 3. B built at 10:05. Its Commit meets C3, reads the head's rules, and the newer file wins with no conflict.

›

await layerfs.sandbox.commit(boxB, wsB, "Build")

{ kind: "Committed", head: "c4…", commit: { mergedFrom: "c3…", lww: [{ path: "/dist/app.js", kept: "ours", deleted: false, rule: "dist/" }] } }

›

A rule drops the older side's change, so use it for files a tool can regenerate, never for code people write. It decides merges after it's on the head; it doesn't clear markers already written into a Workspace.

Branch merge conflicts

branch.merge(a, b, name, message) folds two Branches into a new Branch name and leavesa and b as they were. If one side already holds the other it fast-forwards; otherwise it merges against their merge base, and a clean result is one Commit with two parents. A conflict creates nothing; the refusal's detail names the conflicts and the mergeBase.

1 Resolve by updating the files

Change the conflicted files until the two sides agree, then merge again: where both hold the same text, the merge just takes it. To keep a and b as they are, make the changes on forks of each (branch.fork("cart.a.fix", "cart.a")) and merge the forks.

// 1. cart.a and cart.b both forked from C1 and changed the same line of src/cart.js.

›

await layerfs.branch.merge("cart.a", "cart.b", "cart.ab")

LayerFSError: Conflict detail: { mergeBase: "c1…", conflicts: [{ path: "/src/cart.js", kind: "content", blocks: 1 }] }

// 2. Nothing was created. Compare the two versions.

›

await layerfs.fs.diff_files("cart.a", "/src/cart.js", "cart.b", "/src/cart.js")

{ kind: "Text", contentEqual: false, hunks: [ { op: "-", text: "const total = cents(price) * qty;" }, { op: "+", text: "const total = price * quantity;" }] }

// 3. Write the line both wanted on each side, so the two agree.

›

await layerfs.fs.edit("cart.a", "/src/cart.js", "* qty;", "* quantity;")

await layerfs.fs.edit("cart.b", "/src/cart.js", "price *", "cents(price) *")

{ kind: "Committed", head: "a3…" } { kind: "Committed", head: "b3…" }

// 4. Merge again: the sides now hold the same line, so it merges clean, one Commit with two parents.

›

await layerfs.branch.merge("cart.a", "cart.b", "cart.ab")

{ kind: "Merged", fastForward: false, branch: "cart.ab", head: "m4…", conflicts: [], lww: [] }

›

2 Resolve with .lww

A Branch merge reads the rules from a, its first side. Add the paths to a's/.lww and merge again; each decision is listed under lww in the result.

// 1. Both Branches rebuilt, so dist/app.js differs on each side.

›

await layerfs.branch.merge("cart.a", "cart.b", "cart.ab")

LayerFSError: Conflict detail: { mergeBase: "c1…", conflicts: [{ path: "/dist/app.js", kind: "content", blocks: 3 }] }

// 2. A Branch merge reads a's rules, so name dist/ in cart.a's /.lww.

›

await layerfs.fs.write("cart.a", "/.lww", "dist/\n", "Build output: newest wins")

{ kind: "Committed", head: "a3…" }

// 3. Merge again: cart.b built later, so its dist/app.js is kept.

›

await layerfs.branch.merge("cart.a", "cart.b", "cart.ab")

{ kind: "Merged", branch: "cart.ab", head: "m4…", conflicts: [], lww: [{ path: "/dist/app.js", kept: "theirs", deleted: false, rule: "dist/" }] }

// The older build is dropped, not merged. Rebuild on cart.ab if dist/ should reflect both sides.

›

06 History

Every Commit and Layer stays readable, so the history answers what changed, when, and in which tool call. Each call below reads saved state; none of them writes.

await layerfs.branch.list_commits("cart", 20, 0);             // newest first
await layerfs.branch.diff_branches("main", "cart", 50, 0, { hunks: false });
await layerfs.fs.diff_files("main", "/app/cart/total.ts", "cart", "/app/cart/total.ts");
await layerfs.fs.blame("cart", "/app/cart/total.ts");          // the Commit behind each line
await layerfs.fs.read("3f2a…", "/app/cart/total.ts", 1, 40);   // any Commit, as it was

Lists come in pages of items and hasMore: 50 by default, up to 1000. Ask for the next page at pagePos + items.length. The two diffs also answer nextCursor; pass it back ascursor to resume after the last path, and pass hunks: false to list paths without their line changes.

History is the filesystem only. It doesn't save running processes, memory, sockets or command output.