Stores

A store is any backend that holds snapdir's content-addressed objects and manifests. You select one per command with --store <URI>, where the URI has the form protocol://location/path. The scheme alone decides which backend handles the request — the rest of every command stays identical whether you target a local directory or a cloud bucket. For the on-disk layout and how the local cache relates to remote stores, see Stores and cache.

New in 1.4.0, --store may be omitted when the SNAPDIR_STORE environment variable is set: snapdir falls back to $SNAPDIR_STORE for the store URI. The explicit --store flag, when given, always takes precedence over the environment.

Built-in backends

Scheme	Backend
`file://`	A local or mounted directory (also the cache shape)
`s3://`	Amazon S3 and S3-compatible object storage
`gs://`	Google Cloud Storage
`b2://`	Backblaze B2

s3:// and b2:// are served by native, in-process adapters. gs:// maps to the GCS adapter (snapdir's hardcoded gs→gcs special case). All four use the same content-addressed layout, so a snapshot pushed to one can be synced to another without re-staging.

# The same push/pull verbs, four different backends.
snapdir push --store "file://$PWD/store"        ./my-dir
snapdir push --store s3://my-bucket/snapshots   ./my-dir
snapdir push --store gs://my-bucket/snapshots   ./my-dir
snapdir push --store b2://my-bucket/snapshots   ./my-dir

Authentication and endpoints

The cloud backends authenticate using the standard credentials and environment of their underlying SDKs — there is no separate snapdir auth file to maintain.

s3:// — uses the standard AWS credential chain (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, profiles, instance roles). To target an S3-compatible endpoint (MinIO, Ceph, R2, etc.), set SNAPDIR_S3_STORE_ENDPOINT_URL.
b2:// — also speaks the S3-compatible protocol: it honors SNAPDIR_S3_STORE_ENDPOINT_URL for the endpoint and resolves its region from SNAPDIR_B2_REGION (falling back to AWS_REGION), with AWS-style credentials.
gs:// — uses Google Cloud's standard application-default credentials (for example GOOGLE_APPLICATION_CREDENTIALS).

# Push to an S3-compatible endpoint (e.g. MinIO) with explicit auth.
export AWS_ACCESS_KEY_ID=...           # auth via the standard AWS chain
export AWS_SECRET_ACCESS_KEY=...
export SNAPDIR_S3_STORE_ENDPOINT_URL=https://minio.example.com:9000

snapdir push --store s3://my-bucket/snapshots ./my-dir

If credentials or region cannot be resolved, the command fails when it tries to construct the store, with an actionable error — snapdir never silently falls back.

Storage provider limits

To stay under each provider's published ceilings — and avoid provider-side throttling (HTTP 429, SlowDown, rateLimitExceeded) — snapdir paces requests and bytes per backend. The table below lists snapdir's default caps for each built-in scheme. These defaults are overridable with two global flags:

--max-requests <N> (requests per second; env SNAPDIR_MAX_REQUESTS) caps the request rate.
--limit-rate <RATE> (aggregate bytes per second, e.g. 50M; env SNAPDIR_LIMIT_RATE) caps bandwidth.

Scheme	Default request caps (snapdir)	Default bandwidth caps	Max object / file size	Official limits
`s3://`	read 5500 / write 3500 req·s⁻¹	uncapped	5 TiB object (5 GB single PUT; larger via multipart)	AWS S3 performance
`gs://`	read 5000 / write 1000 req·s⁻¹ (autoscales)	uncapped	5 TiB	Cloud Storage request rate
`b2://`	read 20 / write 50 req·s⁻¹	25 MB·s⁻¹ down / 100 MB·s⁻¹ up	5 GB single upload (larger via large-file parts)	Backblaze B2 rate limits
`file://`	unlimited	unlimited	filesystem-dependent	—

External snapdir-<scheme>-store shims carry no snapdir-imposed caps — pacing is the shim's own responsibility.

Custom backends: `snapdir-<scheme>-store` shims

Any scheme that is not one of the four built-ins is dispatched to an external store shim: a snapdir-<scheme>-store executable found on your PATH. This lets you add a backend without modifying snapdir itself. A webdav:// URL, for example, invokes snapdir-webdav-store:

# Routes to the snapdir-webdav-store binary on your PATH.
snapdir push --store webdav://host/snapshots ./my-dir

The shim is responsible for its own authentication. External stores are usable anywhere --store is accepted (push, fetch, pull, checkout), with one exception: store-to-store snapdir sync requires in-process stores on both ends, so snapdir-*-store URLs are rejected there.

Where to go next