DBTrail
DBTrail EE

Forensics

Who changed this row. Evidence sources, confidence labels, audit-plugin setup, and the honest limits of MySQL-side evidence.

Enterprise feature

Ships in dbtrail-ee, active only when your license carries forensics (dbtrail-ee license lists features). Without it the commands below do not exist. The capture underneath (connection_id, query_text, query_hash in the index) is open-source core and always recorded; the license adds the surface that turns those columns into names.

The index always says what changed. Forensics answers who: database user, host, client program and, when the source records it, the statement.

Console → Forensics: pick schema, table and key, ask Who changed these rows. Same view: User activity, Connections, and a capture-status card.

The Forensics view: evidence-source chips, the attribution ledger summary, the Who changed form, honest coverage notes, and a list of changes with the attributed user or an explicit unattributed label.
$ dbtrail-ee who-changed --index-dsn "$IDX" --source-dsn "$SRC" \
    --schema shop --table orders --pk 42

How attribution works

A binlog event records only connection_id 8812. Forensics looks it up in three evidence sources: the audit log on the source with durable CONNECT and DISCONNECT records, live sessions in performance_schema, and the identity ledger in the index polled every 0.5 seconds and kept 7 days. The answer is a name such as app_rw at 10.0.3.7 with a confidence label, or a labeled unknown.

The binlog records a connection number, never a name. How well that number turns back into a name depends on one choice: is an audit plugin writing a log on your source?

What to expect, with and without an audit plugin

Three setups. A, no identity capture: names only sessions still connected. B, identity ledger from any capture daemon: names survive disconnect and are kept 7 days by default, days-old and very short sessions are partial, no proof. C, audit plugin: names and proof in every case, exact.
  • A: forensics unlicensed or BINTRAIL_ATTRIBUTION_RETENTION=0, no audit plugin.
  • B: any capture daemon (dbtrail-ee up, stream, agent, or the console's watch and monitor daemons) polls live sessions twice a second into the index. A session shorter than one poll can slip through.
  • C: an audit plugin writes a log. Days-old answers reach as far as the log's retention, and further through the ledger (below).

The statement itself is independent of all three: query_text. dbtrail-ee doctor prints how to enable what is missing.

How to read the confidence labels

Confidence labels. exact: the change falls inside the session's CONNECT and DISCONNECT, proven, id reuse cannot fool it. corroborated: the number matches but the session ends are unknown, strong evidence. heuristic: two candidate sessions in the same second, the likelier one is named and the answer says so.

Answers also carry notes: a truncated audit read, an unreachable source, a coverage gap. If a note says a source was NOT consulted, the result does not mean that source had nothing.

Without an audit plugin: what performance_schema can and cannot do

Four performance_schema limits. It forgets a session on disconnect, so the daemon polls twice a second. It keeps about 10,000 statements server-wide with no timestamps, so a time window cannot be applied. It is fixed at server startup, ON by default on MySQL 8.0+ and OFF on MariaDB. It cuts statement text at 1024 bytes by default.
  • Windows are not applied, and the answer says so. user-activity shows what the buffer still holds (--order ASC = oldest retained). With --since/--until a note says the filter was not applied, including on an empty result. connection-history reads sessions connected now; its command takes no window flags, and an empty answer ships the cumulative performance_schema.accounts query.
  • Startup switches. performance_schema and performance_schema_max_sql_text_length (default 1024) need a restart; on RDS and Aurora, parameter group plus reboot. Raising the ceiling cannot un-truncate history.
  • Cut statements are flagged. user-activity notes any statement at or near the ceiling. connection-history reads PROCESSLIST_INFO, fixed at 1024 bytes whatever the setting, and notes it too. The uncapped information_schema.PROCESSLIST.INFO needs PROCESS to see other accounts. Multi-byte text can be cut a few bytes short of the ceiling; MySQL 8.0's ... marker catches those, a flavor without a marker may not.
  • Full statement text, in order: DBTrail's index (query_text, 16 KiB cap), then mysql.general_log, then an audit log.

DBTrail only reads performance_schema and never changes your server.

With an audit plugin: the full answer

A durable record: sessions named long after disconnect, each connection id bracketed by CONNECT and DISCONNECT (the exact label), and evidence that survives DBTrail restarts.

ServerPluginNotes
Percona Serveraudit_log, audit_log_filterFree. JSON, CSV, OLD and NEW XML; audit_log_filter JSON and XML
MariaDBserver_auditFree; same dialect as RDS MySQL and MariaDB
RDS / Aurora MySQLMariaDB Audit Plugin (option group or Advanced Auditing)Read through the AWS API: rds:DescribeDBLogFiles and rds:DownloadDBLogFilePortion on the host's IAM role
MySQL Communitynone built inMySQL Enterprise Audit (commercial), or Percona Server as a free drop-in
$ dbtrail-ee doctor --source-dsn "$SRC"

Detects what you have and prints setup SQL or my.cnf per flavor. DBTrail never installs plugins or changes settings.

  • Retention is yours. Rotated or pruned records cannot be attributed; on RDS and Aurora instance storage that can be hours. Size rotation, or export to CloudWatch with explicit retention.
  • A read that understood nothing says so. Unrecognized records are counted and warned, never an empty "nothing happened". Every read states its reach and the matching remedy: rotated files not opened (include_rotated), tail mode (tail_lines=-1; tail is the default when since is set), a full page (offset), or the 100,000-matched-record scan cap, which offset cannot page past. A directory that cannot be listed reports its rotated scope as unknown. The CloudWatch reader, available to the agent's forensics_audit_log command with source: "cloudwatch", reports its floor from the log group's retention, or names the missing IAM action.
  • Zone-less timestamps (server_audit) are normalized with the source's OS clock offset; when it cannot be probed the read says so.

The statement itself (query_text)

Set binlog_rows_query_log_events=ON (MariaDB: binlog_annotate_row_events, on by default). The core indexes the originating SQL durably next to each change, even when nobody can name the session. who-changed shows it; doctor checks the flag.

What no tool can tell you

  • Behind a pooler or proxy (ProxySQL, RDS Proxy, app pools): the host is the proxy's and multiplexed users share one session. Use application logs.
  • Older than your evidence. No audit log, no history.
  • From a replica's binlog: ids belong to the replication applier. Capture from the primary.
  • A determined insider can set a fake pseudo_thread_id. Evidence, not courtroom proof; answers say so.
  • PostgreSQL sources: no connection identity in logical replication, no who-changed.

The commands

$ dbtrail-ee who-changed --index-dsn "$IDX" --source-dsn "$SRC" ...   # who changed these rows
$ dbtrail-ee user-activity --user X --source-dsn "$SRC"               # what a user runs right now
$ dbtrail-ee connection-history --source-dsn "$SRC"                   # who is connected now
$ dbtrail-ee attribution-status --index-dsn "$IDX"                    # is identity capture running

All accept --format json. who-changed tries audit log, live sessions, then ledger; without --source-dsn it uses index evidence only. DDL history comes from the index: dbtrail-ee query --event-type ddl.

The identity ledger. Every capture daemon for a MySQL-family source writes session_history in the index: one row per observed session interval per connection id, so a reused id appends instead of overwriting. An event strictly inside an interval attributes at exact. Old connection_cache rows are migrated once, capped at corroborated. Intervals are swept hourly.

  • BINTRAIL_ATTRIBUTION_RETENTION: Go duration, default 168h (7 days), 0 disables. Replaces the retired --attribution-retention flag.
  • With an audit plugin the daemon also reads the log every minute and stores each session's CONNECT..DISCONNECT lifetime (never statement text). Logs rotate in hours; with BINTRAIL_ATTRIBUTION_RETENTION=2160h a session from ten weeks ago still attributes at exact.

Is capture actually running?

Capture fails quietly by design so it never stops the stream. Check it: the console's capture-status card, or

dbtrail-ee attribution-status --index-dsn "$IDX"
Attribution capture (session_history)
  Status:            capturing (last sighting 1s ago)
  Retained:          1284 interval(s) across 412 connection id(s)
  Live sessions:     37 (of 40 interval(s) with no witnessed end)
  Collectors:        audit_log 84 (3s ago) · performance_schema 1200 (1s ago)
  Retention:         168h0m0s (BINTRAIL_ATTRIBUTION_RETENTION, as resolved by this command)

Ages use the index server's clock. STOPPED and NOT CAPTURING list the checks to run; an empty ledger (swept) reads differently from a missing one (never ran). The daemon log prints every five minutes:

level=INFO msg="session-history: capturing" window=5m0s polls_ok=600 polls_failed=0 open_intervals=37 consecutive_failures=0

polls_ok=0 polls_failed=600 means a running poller that cannot reach the source; silence means it exited, with the reason logged just above.

Asking from an AI client (MCP)

On the console's /mcp endpoint: who_changed, forensics_capabilities, user_activity, connection_history. Ask "Who changed row 42 in shop.orders this morning?".

  • Tools read the server selected through /mcp/{server} and take no DSN.
  • The three live-source tools refuse clearly when no source DSN is configured; who_changed still returns what changed, with notes.
  • Unlicensed binaries do not expose them. Setup: Claude integration.

Sub-second precision

event_timestamp has one-second resolution. DBTrail also stores the commit instant in microseconds (commit_ts_us, MySQL 8.0.1+, no source setting). Against Aurora Advanced Auditing's microsecond records, an id reused twice in one second resolves at exact. Against second-resolution sources (MariaDB, RDS audit logs, the ledger) comparisons use whole seconds, so no false exclusions. commit_ts_us is NULL on MariaDB, before MySQL 8.0.1, and for older events.

Other surfaces

  • Agent (BYOS): forensics_capabilities, forensics_enrich, forensics_activity, forensics_users, forensics_audit_log. Unlicensed: "unknown command type".
  • Console: the Forensics view needs analyst or above under RBAC. Running the console.

Privileges

SELECT on performance_schema for the --source-dsn user; filesystem access for a local audit log; the two IAM actions above for RDS and Aurora. Everything is read-only: doctor prints fixes, never runs them.

On this page