dbtrail
Guides

Backup Strategy

How dbtrail's backup architecture works, from base snapshots and continuous binlog streaming to point-in-time recovery

dbtrail is your MySQL backup system. It combines full logical snapshots with continuous binary log streaming so every row change is captured, queryable, and restorable to any point in time within your retention window. This guide explains how the architecture works, how to configure it, and how it compares to a DIY mysqldump + cron setup.

This replaces your existing backup job

If you're running mysqldump + cron today, dbtrail replaces it. You get parallel logical dumps, continuous binlog streaming, per-row point-in-time recovery, and a queryable change history, all in one system.

How dbtrail backs up your database

dbtrail takes two things and combines them into a complete backup system:

  1. Full base snapshots: parallel logical dumps taken with mydumper. Each snapshot embeds the exact binlog position and GTID set at the time of the dump and is converted to Parquet baselines, optionally uploaded to S3.
  2. Continuous binlog streaming: the stream you started when you added the server registers as a MySQL replica and captures every INSERT, UPDATE, and DELETE in real time. Events are indexed for instant query and archived as Parquet for long-term retention.
Timeline: a base snapshot at T0, a continuous binlog stream capturing every change as it happens, a later base snapshot at T1, and capture continuing to now. A bracket under the timeline marks that you can restore to any point in between.

Together, these give you point-in-time recovery to any moment after the earliest available base snapshot, not just to the last nightly cron.

Why both pieces matter

A base snapshot by itself is stale the moment it's taken. A binlog stream by itself has no starting point: reconstructing state from a stream of deltas requires knowing what came before. dbtrail maintains both so you never have to think about the gap.

Snapshot frequency

More frequent snapshots = faster point-in-time recovery (less binlog to replay). A weekly snapshot is fine for most workloads; daily is a typical choice for production databases. Continuous streaming covers everything between snapshots.

Snapshots need one privilege capture does not

Base snapshots are point-consistent by default, and that needs LOCK TABLES on the source user (GRANT LOCK TABLES ON *.* TO 'bintrail'@'%';) — capture itself never locks your database and does not need it. If the grant is missing, capture keeps running and only the snapshot is refused, so it is worth checking before you rely on the schedule. On RDS and Aurora also set the lock mode to lock-all: the default needs BACKUP_ADMIN, which managed MySQL cannot grant. See the quickstart for the full grant list.

Taking snapshots

From the console

On the Backups page, Create backup runs the whole snapshot pipeline for the selected server (parallel mydumper dump, Parquet conversion, upload to the server's backup destination) entirely in-process (the console bundles mydumper; it never mounts the Docker socket, and the source credentials never leave the process). When it finishes, the new snapshot appears in the page's backup listing alongside each snapshot's timestamp, age, table count, and the binlog coordinates its deltas start from.

The button is on by default in the bundled Compose stack; the server needs a backup destination (local directory or s3:// prefix) configured under Manage servers → Edit → Advanced. One backup runs at a time per server.

The Backups page: source destination, backup count, latest snapshot with its binlog coordinates, the Create backup button, and the per-server snapshot listing.

On a schedule

Scheduled snapshots are a cron or systemd-timer job around the CLI pipeline (bintrail dumpbintrail baseline). The repository guide covers the full recipe, from scheduling and the mydumper flags dbtrail passes (including lock-free dumps, since the binlog stream already provides consistency) to upload retries after partial failures and pruning old local snapshots:

The `metadata` file is the PITR anchor

Every mydumper run writes a metadata file recording the dump's binlog position and GTID set. That's what lets PITR replay events from exactly the right point. If you customize the upload step, make sure metadata always lands next to the dump. Without it, PITR can only restore to the dump-finish moment, not to a precise later target time.

Encryption and retention

  • Encryption in transit/at rest. Use a customer-managed KMS key on the destination bucket (SSE-KMS) so encryption is automatic on upload. mydumper also supports on-disk encryption if you need defense-in-depth before upload.
  • Retention. Set an S3 lifecycle policy on the backups prefix (for example transition to Glacier after 30 days, expire after 365). dbtrail can also prune redundant local snapshots that already have a durable S3 copy.
  • Bucket layout. Keep snapshots and the Parquet binlog archives in different prefixes of the same bucket so a lifecycle rule on /backups/ doesn't accidentally expire your binlog archives.

Restore

The restore side of dbtrail is covered in two places depending on what you need:

  • Row-level recovery. Undo a specific DELETE, revert an UPDATE, or remove an unintended INSERT from the console's Restore view: dry-run SQL you review before applying. See the recovery guide.
  • Point-in-time restore of whole tables or the whole database. Reconstruct the full state at any past moment by combining a base snapshot with binlog events up to the target time. Single rows come back from the console's Restore view; full tables from the Backups page (Build a .sql backup for any moment). See the PITR guide.

dbtrail intentionally never executes writes against your MySQL; you always review and apply the output yourself.

Restoring with Claude

Ask Claude: "Undo the DELETE on user 12345". It calls the dbtrail recover tool and shows you the generated SQL for review. Claude is optional, and the console provides the same operations directly.

Compared to DIY mysqldump + cron

If you're migrating from a cron-driven dump setup, here's what dbtrail replaces and adds:

Capabilitymysqldump + crondbtrail
Full dumpsManual scriptOne click (console) or one scheduled command
Parallel dump engineSingle-threadedMydumper, multi-threaded
S3 uploadManual (aws s3 cp)Built into the baseline pipeline
Retention enforcementManual cleanup scriptSnapshots via S3 lifecycle policy; rotation prunes old change-history partitions (archiving them as Parquet), tunable from Settings → Rotation
Point-in-time recoveryManual binlog replayBaseline + indexed events combined for you: the Restore view (single row) or a downloadable .sql backup built for any moment (full tables)
Change history queriesEvery row change indexed and queryable (console, SQL, Claude)
Row-level recoveryGenerates inverse SQL for a single row
Forensics (who changed what)User/host/program attribution, a dbtrail EE capability; the free core records each change's raw connection_id
MonitoringDIYConsole status + Prometheus metrics (alerting is DIY on top)
Data stays in your infrastructureYou already own everythingAlways: self-hosted, and your S3 bucket for archives and snapshots

mysqldump still has its place for one-off exports, but the continuous-protection story it can't tell is where dbtrail lives.

Best practices

  1. Snapshot at least as often as your binlog retention. If MySQL purges binlogs after 7 days, snapshot at least weekly so you always have a baseline inside the retention window.

  2. Test your restores. A backup you've never restored is a backup you can't trust. The console's Verification page runs the check for you: it takes your two newest snapshots, replays the recorded changes from the older one forward, and confirms the result matches the newer one, per table. For a full dress rehearsal, use the PITR guide to validate end-to-end against a staging MySQL periodically.

    The Verification page after a run: compared two saved snapshots, three tables checked out clean, each marked MATCH, with a history of past runs.
  3. Monitor backup status. The console's Status view and Backups page show stream continuity and backup freshness. Wire them into your monitoring. A silently failing schedule is worse than no schedule: it creates a false sense of security.

  4. Keep at least two retention cycles. If you snapshot weekly with 30-day retention, you'll always have ~4 good baselines. If one is corrupted, you have fallbacks.

  5. Lock down your S3 bucket. Dumps are stored at rest in your backup bucket and may be accessed by multiple team members. Restrict the bucket to least-privilege IAM and enable SSE, preferably with a customer-managed KMS key.

Next steps

On this page