Powered by

Creating a manifest without snapdir

A snapdir manifest is plain text: five space-separated columns, one line per file or directory, hashed with BLAKE3. There is no binary container, no database, and no proprietary format — which means you can recreate (or verify) every checksum in a manifest, including the snapshot ID itself, using nothing but b3sum and standard shell tools.

This guide does exactly that, by hand. It's worth doing once: it shows what snapdir manifest computes under the hood, and it demonstrates a property that matters in the long run — anyone you hand a snapshot to (a colleague, an external reviewer, an auditor, your future self after snapdir's binaries are long gone) can independently verify its contents with off-the-shelf tools, in any language, without trusting or even installing snapdir.

You'll need snapdir (install) and the BLAKE3 CLI, b3sum (cargo install b3sum, or brew install b3sum on macOS). Everything else is POSIX shell.

Build an example directory

First, create a small directory tree to snapshot. Setting umask 077 first makes the permission columns below reproducible (directories 700, files 600):

umask 077
mkdir -p example/a
echo a1 > example/a/a1
echo a2 > example/a/a2
echo base > example/base

Ask snapdir for its manifest:

snapdir manifest ./example
D 700 4257cc46336b9d0ae70a3104ae0382ac6a75da0ee49ffe69b423997e872276a7 11 ./
D 700 40bdff878af8e7ffbc40f1d4b5a72c892a0773df2d47cd164c2dc2e684299dfa 6 ./a/
F 600 92719755f8d6c804d44192bb5835654d27003fc8fdbb36a633b9063c7f9396a4 3 ./a/a1
F 600 ff3e86a123552d66c31eb3308916d76bf9d918b1f635aa39d00d3a3428bda536 3 ./a/a2
F 600 b9af5f26c46534d25add40a12c3f0b1ae926e39a2e669162664295040943f54a 5 ./base

Each line is PATH_TYPE PERMISSIONS CHECKSUM SIZE PATH: the entry type (F file, D directory), its octal permissions, its BLAKE3 checksum, its size in bytes (for a directory, the summed size of its contents), and its path (directories end with /). Lines are sorted by path, and lines starting with # are comments, ignored everywhere. The full column-by-column specification is in Manifests — the rest of this guide recreates the checksum column from scratch.

File checksums are just BLAKE3

A file's checksum is the plain BLAKE3 hash of its content, so b3sum produces it directly:

b3sum --no-names ./example/a/a1
92719755f8d6c804d44192bb5835654d27003fc8fdbb36a633b9063c7f9396a4

The same works for ./example/a/a2 (ff3e86a1…) and ./example/base (b9af5f26…) — compare each against the manifest above. Nothing snapdir-specific so far: any BLAKE3 implementation gives the same answers.

Directory checksums: a merkle hash of the children

A directory's checksum is computed from its direct children's checksums: sort them, drop duplicates, concatenate them into one long hex string with no separators, and hash that string. In shell, that recipe is sort -u | tr -d '\n' | b3sum.

Recreate the checksum of ./a/, whose children are the two files we just hashed:

b3sum ./example/a/* --no-names | sort -u | tr -d '\n' | b3sum --no-names
40bdff878af8e7ffbc40f1d4b5a72c892a0773df2d47cd164c2dc2e684299dfa

That matches the D … ./a/ line in the manifest. The root directory works the same way — its children are the directory ./a/ (using the checksum we just computed, not re-walking its contents) and the file ./base:

{
  b3sum ./example/a/* --no-names | sort -u | tr -d '\n' | b3sum --no-names
  b3sum --no-names ./example/base
} | sort -u | tr -d '\n' | b3sum --no-names
4257cc46336b9d0ae70a3104ae0382ac6a75da0ee49ffe69b423997e872276a7

— the checksum on the manifest's D … ./ line. Because each directory hashes its children's checksums, a change to any file ripples up through every parent directory to the root: a merkle tree, built with three lines of shell.

Note the -u in sort -u: duplicate checksums among a directory's children are dropped before hashing. Two identical files contribute one checksum, so a directory holding foo.txt and bar.txt — both empty, both hashing to af1349b9… — has the checksum of that single deduplicated value:

echo -n af1349b9f5f9a1a6a0404dea36dcc9499bcb25c9adc112b7cc9a93cae41f3262 | b3sum --no-names
dba5865c0d91b17958e4d2cac98c338f85cbbda07b71a020ab16c391b5e7af4b

This is the same identity-comes-from-content rule that gives snapdir its deduplication — a directory's checksum depends on what it contains, not how many copies.

The snapshot ID: hash of the manifest itself

The snapshot ID is not the root directory's checksum — it is the BLAKE3 hash of the entire manifest text, with comment lines stripped. That makes the ID cover everything the manifest records: permissions, sizes, and paths, not just content. The recipe:

snapdir manifest ./example | grep -v '^#' | b3sum --no-names
7ecd37f57f9d4b4128c4fe07c53e28e668c4f1df6bc6692155737d0ebdc81f8d

And the proof that we've reconstructed the whole pipeline:

snapdir id ./example
7ecd37f57f9d4b4128c4fe07c53e28e668c4f1df6bc6692155737d0ebdc81f8d

Why this matters

Everything snapdir later pushes, verifies, and restores is anchored to checksums you can recompute with a few lines of portable shell. In practice that means:

  • No lock-in. The manifest is the format. A third-party tool — or a 20-line script — can produce or consume snapdir manifests without linking any snapdir code.

  • Independent verification. Someone receiving a directory and its manifest can confirm, file by file and as a whole, that what they received is exactly what was recorded — using only standard tools they already trust. That's valuable wherever evidence has to hold up to outside scrutiny: handoffs to reviewers or authorities, archival storage, air-gapped environments.

  • Interop with existing tooling. snapdir manifest --checksum-bin md5sum (or sha256sum) emits the same five columns keyed on those digests for systems that already track them, and setting SNAPDIR_MANIFEST_CONTEXT switches to keyed BLAKE3 for domain-separated IDs — see Integrity.

Where to go next