Stores

A store is any backend that holds snapdir's content-addressed objects and manifests. You select one per command with --store <URI>, where the URI has the form protocol://location/path. The scheme alone decides which backend handles the request — the rest of every command stays identical whether you target a local directory or a cloud bucket. For the on-disk layout and how the local cache relates to remote stores, see Stores and cache.

New in 1.4.0, --store may be omitted when the SNAPDIR_STORE environment variable is set: snapdir falls back to $SNAPDIR_STORE for the store URI. The explicit --store flag, when given, always takes precedence over the environment.

Built-in backends

s3:// and b2:// are served by native, in-process adapters. gs:// maps to the GCS adapter (snapdir's hardcoded gs→gcs special case). All four use the same content-addressed layout, so a snapshot pushed to one can be synced to another without re-staging. Beyond the built-ins, snapdir also ships first-party ssh:// and sftp:// stores that turn any SSH-reachable host into a store.

# The same push/pull verbs, four different backends.
snapdir push --store "file://$PWD/store"        ./my-dir
snapdir push --store s3://my-bucket/snapshots   ./my-dir
snapdir push --store gs://my-bucket/snapshots   ./my-dir
snapdir push --store b2://my-bucket/snapshots   ./my-dir

Shared object pools

For scheduled captures or multi-tenant layouts, the manifest location and the object location can be split. Set --objects-store <URI> (or SNAPDIR_OBJECTS_STORE) alongside --store <URI>:

snapdir push \
  --store s3://inventory/manifests/host-a/2026-06-18 \
  --objects-store s3://inventory/object-pool \
  /var/lib/app/data

With that split, manifests are written under the --store location's .manifests/ tree, while content objects are written under the object pool's .objects/ tree. Repeating the same capture with a different --store path can publish a fresh manifest while reusing the same deduplicated object pool; objects that are already present in the pool are skipped.

Use the same split when reading the snapshot back:

snapdir pull \
  --store s3://inventory/manifests/host-a/2026-06-18 \
  --objects-store s3://inventory/object-pool \
  --id "$id" ./restore

If --objects-store is omitted, the store is colocated: manifests and objects both live under the --store URI. Both halves of a split store must be in-process backends (file://, s3://, gs://, or b2://); external snapdir-<scheme>-store shims are rejected for either side of the split.

Local file-store fast paths

For file:// stores and the local cache, snapdir can avoid rewriting file bytes when the source and destination are on the same copy-on-write-capable filesystem. macOS uses APFS clonefile(2), and Linux uses FICLONE reflink on supporting filesystems. If the clone/reflink path is unavailable, snapdir falls back to a normal copy.

Set SNAPDIR_CLONEFILE=0 to disable the clone/reflink attempt. Set SNAPDIR_VERIFY_COPIES=1 to force strict write-time re-hashing even when the fast path succeeds. These knobs only affect how bytes are copied locally; object bytes and manifest bytes are byte-identical, and snapshot IDs are unchanged.

For read-only local checkouts, pull --linked and checkout --linked skip the copy entirely and create symlinks into local content-addressed objects. That mode is local-only; a remote s3://, gs://, b2://, ssh://, or sftp:// source with --linked is refused.

Authentication and endpoints

The cloud backends authenticate using the standard credentials and environment of their underlying SDKs — there is no separate snapdir auth file to maintain.

• s3:// — uses the standard AWS credential chain (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, profiles, instance roles). To target an S3-compatible endpoint (MinIO, Ceph, R2, etc.), set SNAPDIR_S3_STORE_ENDPOINT_URL.

• b2:// — also speaks the S3-compatible protocol: it honors SNAPDIR_S3_STORE_ENDPOINT_URL for the endpoint and resolves its region from SNAPDIR_B2_REGION (falling back to AWS_REGION), with AWS-style credentials.

• gs:// — uses Google Cloud's standard application-default credentials (for example GOOGLE_APPLICATION_CREDENTIALS).

# Push to an S3-compatible endpoint (e.g. MinIO) with explicit auth.
export AWS_ACCESS_KEY_ID=...           # auth via the standard AWS chain
export AWS_SECRET_ACCESS_KEY=...
export SNAPDIR_S3_STORE_ENDPOINT_URL=https://minio.example.com:9000

snapdir push --store s3://my-bucket/snapshots ./my-dir

If credentials or region cannot be resolved, the command fails when it tries to construct the store, with an actionable error — snapdir never silently falls back.

SSH and SFTP stores

New in 1.5.0, snapdir can use any SSH-reachable host as a store. Two schemes are provided — they are distinct engines, not aliases:

Store URLs take the form ssh://[user@]host[:port]/abs/base/path (likewise sftp://). Use ssh:// when the remote gives you a shell — and it auto-accelerates when snapdir is installed remotely; use sftp:// for restricted/chroot accounts (it speaks pure SFTP, so it works even under ForceCommand internal-sftp with no remote shell at all). Embedded passwords (user:password@) are rejected — authenticate with a key or an agent.

snapdir push --store ssh://backup@nas.example.com/srv/snapdir   ./my-dir
snapdir push --store sftp://chrooted@host.example.com:2222/snaps ./my-dir

Both engines ship as external-store binaries in the snapdir-ssh-store crate (cargo install snapdir-ssh-store provides both; the prebuilt release archives bundle them). They must be on PATH where the snapdir CLI runs, and they drive your system ssh/sftp client — no SSH reimplementation, zero new crypto dependencies — so your ~/.ssh/config, keys, agent, and ProxyJump setups keep working. Every operation multiplexes over a single ControlMaster connection (one TCP + auth handshake per operation).

Configuration

Each scheme reads its own env family — SNAPDIR_SSH_STORE_* for ssh://, SNAPDIR_SFTP_STORE_* for sftp://:

The security floor

Both engines enforce an un-weakenable, modern-only security floor on every ssh/sftp invocation: modern-only key exchange (post-quantum hybrid sntrup761x25519 first, then X25519), AEAD-only ciphers (ChaCha20-Poly1305, AES-GCM), Ed25519/RSA-SHA-2/ECDSA host keys (SHA-1 ssh-rsa and DSS excluded), StrictHostKeyChecking=yes always, BatchMode=yes (never an interactive prompt), and no password or keyboard-interactive auth. OpenSSH ≥ 8.5 is required locally (checked via ssh -V, fail-closed). Because OpenSSH takes the first value obtained for each option and the floor is always emitted first, EXTRA_OPTS structurally cannot weaken the floor — e.g. EXTRA_OPTS="StrictHostKeyChecking=no" is inert; extras can only add options the floor doesn't set. The remote server is not version-gated — it only needs to offer at least one algorithm from each pinned list.

Acceleration (ssh:// only)

When the remote host has a wire-compatible snapdir on its PATH, ssh:// transfers automatically switch to a pack-stream protocol that diffs objects remotely and streams only what's missing in O(1) round trips, with every record BLAKE3-verified on arrival — falling back gracefully to the plain path otherwise; both paths produce byte-identical stores. Runtime toggles: SNAPDIR_SSH_NO_ACCEL=1 forces the plain path, SNAPDIR_SSH_FORCE_ACCEL=1 errors instead of falling back, and SNAPDIR_SSH_PULL_SENDALL=1 makes an accelerated fetch request the full object list.

When both local and remote snapdir binaries advertise the snappack-zstd capability, the accelerated path uses SNAPPACK 1Z: the same SNAPPACK 1 record grammar with the post-magic body carried in one zstd frame. The receiver still BLAKE3-verifies every decompressed record, and mixed-version peers keep using the plain v1 accelerated stream. Compression defaults to zstd level 3; tune it with SNAPDIR_SSH_ZSTD_LEVEL=1..19 (out-of-range values are clamped). Because the pack stream is already compressed above SSH, prefer an SSH Compression=no option for this store, for example via SNAPDIR_SSH_STORE_EXTRA_OPTS, to avoid double-compressing on WAN or HPN-SSH links.

On the receiving side of an accelerated push, hidden receive-pack honors SNAPDIR_FSYNC. The default batch mode fsyncs all received objects before the manifest is committed last, so a manifest that survives a crash is backed by durable objects. SNAPDIR_FSYNC=off skips that barrier for speed; any other value is a hard error. The cost is paid only by the receive-pack path, not by ordinary in-process file://, S3, GCS, or B2 pushes.

Limitation: snapdir sync does not support ssh:///sftp:// stores (they have no in-process streaming surface) — push, fetch, pull, and checkout all work.

Storage provider limits

To stay under each provider's published ceilings — and avoid provider-side throttling (HTTP 429, SlowDown, rateLimitExceeded) — snapdir paces requests and bytes per backend. The table below lists snapdir's default caps for each built-in scheme. These defaults are overridable with two global flags:

• --max-requests <N> (requests per second; env SNAPDIR_MAX_REQUESTS) caps the request rate.

• --limit-rate <RATE> (aggregate bytes per second, e.g. 50M; env SNAPDIR_LIMIT_RATE) caps bandwidth.

External snapdir-<scheme>-store shims carry no snapdir-imposed caps — pacing is the shim's own responsibility.

Custom backends: snapdir-<scheme>-store shims

Any scheme that is not one of the four built-ins is dispatched to an external store shim: a snapdir-<scheme>-store executable found on your PATH. This lets you add a backend without modifying snapdir itself — the ssh:// and sftp:// stores above ship as two such first-party shims. A webdav:// URL, for example, invokes snapdir-webdav-store:

# Routes to the snapdir-webdav-store binary on your PATH.
snapdir push --store webdav://host/snapshots ./my-dir

The shim is responsible for its own authentication. External stores are usable anywhere --store is accepted (push, fetch, pull, checkout), with one exception: store-to-store snapdir sync requires in-process stores on both ends, so snapdir-*-store URLs are rejected there.

Where to go next

• Pushing and pulling — the verbs that read and write stores.

• Syncing — copy a snapshot directly between two stores.

• History — list the stores and directories that hold snapshots.

• Quickstart — a file:// round-trip you can run with no cloud account.