DBTrail

Data Verification

Prove your backups restore correctly before you need them. Three checks, one verdict per table, a row-level drill-down when something is off.

Verification starts from snapshot A, replays every recorded change up to where snapshot B was taken, fingerprints the rebuilt table and compares it with snapshot B: MATCH or MISMATCH. Both sides are files you already have; your database is not touched.

Web interface → Verification (under Protect): pick the server, pick a check, Run verification. Results arrive table by table. Nothing is written anywhere.

Three checks

Three checks on two paths. Rebuild path: compare two saved snapshots (recommended, no load on your database) and compare against your live database (needs a table nobody is writing, reads every row). Undo path: recovery inputs, index only. A clean rebuild says nothing about undo, and the reverse.
The Checks tab of the Snapshots page before any run: Never checked, a Run a check picker set to Compare two saved snapshots, the Run verification button with a What this check proves link, and empty Past checks and Current run panels.

Live comparison needs a quiet table

Any write during the read shows up as MISMATCH even when the chain is fine. Use a maintenance window or a replica; treat a live mismatch as "re-check".

Reading the result

Four verdicts per table. MATCH: proven, nothing to do. MISMATCH: would not restore; click Explain and check capture first, then the schema, then the backup. INCONCLUSIVE: not proven either way, usually setup. ERROR: the check failed; fix and re-run.

INCONCLUSIVE with the reason the newest backup keeps this table's previous file and stores its changes beside it (table deltas); verify compares table files, and this one did not change, so nothing was checked is not a fault (since v0.83.0): the newest backup wrote that table's changes as small files beside its file instead of rewriting it, so there is no new file to compare. The table is verified again the next time it is written in full, at most a day later: how the Parquet copy works.

When a table does not match

Explain rebuilds that table and lists the rows that differ, Recovered next to Backup (real). Large tables take minutes; it runs on the daemon.

The Explain drill-down for shop.orders: 2 rows differ, checked against binlog position mysql-bin.000003:11861. For id=2 the status column reads shipped in the recovered copy but cancelled in the real backup; for id=5, paid versus cancelled.

A real capture hole: two changes never reached the index, so rows 2 and 5 keep the old status. Check in this order:

  1. Capture continuity for the window: Status.
  2. A schema change the daemon did not re-snapshot: its events were skipped.
  3. A torn backup, taken with a weakened lock mode: taking snapshots.

Only then a stored row image that is genuinely wrong.

Recovery inputs

Walks each row's history and checks every before image equals what the previous change left. A break means an undo script would restore wrong values. Not findings: a row's first change in the window, and tables that only gained rows. On a young index every table reads INCONCLUSIVE until the index covers the lookback (CLI: verify --check recover --lookback 7d).

Schedule and alerts

Interval, tables, the button in the web interface: Verification settings. Webhook or Prometheus on a mismatch: Alerts.

What a MATCH proves

  • A 64-bit fingerprint of row content. Not cryptographic, not tamper-proof.
  • Row content only: not definitions, indexes, triggers, collations or AUTO_INCREMENT.
  • The chain between two anchor points, not every moment between them.
  • Rebuild checks do not test undo inputs, and the reverse.
  • Recovery inputs has no outside truth: consistently wrong capture passes.

All limits: limitations.

Headless

bintrail verify --index-dsn "$IDX" --baseline-dir /data/backups --explain
bintrail verify --index-dsn "$IDX" --check recover --lookback 7d --format json

Non-zero exit on any mismatch, any error, or when no table could be proven. CLI reference.

On this page