MinIO, Wasabi and other S3-compatible storage

Point DBTrail's archives and baselines at a store that speaks the S3 API but is not AWS, with uploads and reads aimed at the same place.

MinIO, Wasabi, Ceph, Garage, LocalStack: one variable, and both uploads and DuckDB reads follow it.

BINTRAIL_S3_ENDPOINT=https://minio.internal:9000
AWS_ACCESS_KEY_ID=<your key>
AWS_SECRET_ACCESS_KEY=<your secret>

Credentials come from the normal AWS chain (environment, ~/.aws/credentials, instance role). Covers --archive-s3, --baseline-s3, the archive reads behind query and recover, and the daemon's uploads.

With BINTRAIL_S3_ENDPOINT only, uploads and DuckDB reads both reach your store, with path-style addressing on and region us-east-1 if unset. With AWS_ENDPOINT_URL_S3 set as well, uploads go to one store and reads to another, silently: each half works on its own terms. Do not set both.

Why not the AWS variable

	`BINTRAIL_S3_ENDPOINT`	`AWS_ENDPOINT_URL_S3`
Validated at startup	Yes, a bad value fails the load	No
Path-style addressing	On	SDK default, virtual-hosted
Region fallback	`us-east-1`	Untouched

MinIO and LocalStack need host/bucket. A virtual-hosted URL against them fails at DNS resolution, which looks like a network outage.

Addressing style

BINTRAIL_S3_PATH_STYLE=0   # store wants bucket.host

Works without BINTRAIL_S3_ENDPOINT too; it is the only lever when the endpoint lives in ~/.aws/config.

Region

No region anywhere: us-east-1. MinIO accepts anything. A store with real regions (Wasabi) needs endpoint and region to match. The region is pinned into the DuckDB secret, so a mismatch fails at query time, not at upload.

BINTRAIL_S3_ENDPOINT=https://s3.eu-central-1.wasabisys.com
AWS_REGION=eu-central-1

In a compose file

services:
  dbtrail-console:
    image: ghcr.io/dbtrail/bintrail-console
    environment:
      BINTRAIL_S3_ENDPOINT: https://minio.internal:9000
      AWS_ACCESS_KEY_ID: ${MINIO_KEY}
      AWS_SECRET_ACCESS_KEY: ${MINIO_SECRET}

	Where	Scope
Which bucket	Archive to S3 in the server form	Per server
Which store	`BINTRAIL_S3_ENDPOINT`	Whole daemon

Two servers can use two buckets, in the same store. Two stores means two daemons; per-server endpoints are issue #1575.

Checking it works

Upload, then read back: bintrail status lists archives and coverage; bintrail archive reconcile scans the bucket against the index, the same read path a recovery uses. A wrong endpoint usually shows as an empty listing, not an error.

views.sql for DuckDB

The generated views.sql (Query in DuckDB) carries endpoint, addressing style and region, never keys: it uses PROVIDER credential_chain, so whoever opens it brings their own credentials.