Backup & restore
What to back up (and what not to), how the Helm backup CronJobs work, the manual Compose/Ansible procedure, and how to restore — for both the settings database and the telemetry store.
Orbtrace keeps state in two stores, and they have very different backup needs. Get this right before you go to production.
What to back up
- PostgreSQL — back this upYour configuration and history: users and roles, SLO contracts, alert rules / channels / routing, dashboards, RCA incidents (with their pgvector embeddings), the license install record, and the audit log. This is small and irreplaceable — losing it means rebuilding every dashboard and alert by hand.
- Apache Doris — back up if you must keep historyYour raw logs, traces, and metrics. Large, and re-creatable in the sense that new telemetry keeps flowing in — but you can't recover the past once it's gone. Back it up if you're bound by a retention/compliance requirement; otherwise many teams accept that a disaster loses telemetry older than the last snapshot.
- Valkey — don't botherIt's a cache. It rebuilds itself. Nothing here needs backing up.
Rule of thumb
Back up PostgreSQL frequently (it's tiny and precious). Back up Doris on a schedule that matches your compliance window (it's large and partially self-healing). Skip Valkey.
On Kubernetes (Helm) — the built-in CronJobs
The chart ships backup CronJobs and restore Jobs, disabled by default. Turn them on in your values:
backup:
enabled: true
postgres:
schedule: "0 1 * * *" # nightly pg_dump
doris:
schedule: "0 2 * * *" # nightly Doris BACKUP SNAPSHOT
# destination (PVC / S3-compatible bucket) — see values.yaml comments- The Postgres CronJob runs
pg_dumpand writes a compressed dump to the configured destination. - The Doris CronJob issues a
BACKUP SNAPSHOTof the telemetry tables to the backup repository. - Restore is a one-shot Job (
restore.enabled: truewith the snapshot/dump to restore from) — it's wired to run as a post-install hook so you can restore into a fresh release. Always restore PostgreSQL and Doris from snapshots taken at a compatible time.
On Docker Compose / Ansible — manual
There's no scheduler in the Compose stack, so wire these into cron (or your backup tool) on the host.
PostgreSQL
A logical dump is simplest:
docker compose exec -T postgres \
pg_dump -U "$PGUSER" -d "$PGDATABASE" --format=custom \
> orbtrace-pg-$(date +%F).dumpRestore into a fresh stack:
docker compose exec -T postgres \
pg_restore -U "$PGUSER" -d "$PGDATABASE" --clean --if-exists \
< orbtrace-pg-YYYY-MM-DD.dumpApache Doris
Use Doris's native BACKUP to an S3-compatible repository (don't try to copy the BE data directories of a running cluster):
-- once: register a backup repository (S3/MinIO)
CREATE REPOSITORY orbtrace_backup WITH S3 (...);
-- snapshot the telemetry DB
BACKUP SNAPSHOT orbtrace.snapshot_YYYYMMDD
TO orbtrace_backup ON (otel_logs, otel_traces, otel_metrics_gauge, ...);Restore with RESTORE SNAPSHOT … FROM orbtrace_backup. See the Apache Doris backup-and-restore docs for the repository syntax; Orbtrace doesn't change it.
Restore the schema-compatible way
Restore PostgreSQL into a server running the same or a newer Orbtrace version (Flyway migrations are forward-only). For Doris, restore into a cluster on a compatible Doris major version — the 4.1 on-disk metadata is not readable by 3.x.
RPO / RTO in plain terms
- PostgreSQL is small; a nightly (or hourly) dump means your worst-case loss is one day (or hour) of configuration changes — usually nothing, since dashboards/alerts don't change constantly.
- Doris loss equals "telemetry since the last snapshot". If that matters to you, snapshot more often and ship the snapshots off-box.
- Recovery is: stand up a fresh Orbtrace (see Installation), restore PostgreSQL, restore Doris, point your Collector back at it. Your install fingerprint is derived from a value stored in PostgreSQL, so a PostgreSQL restore keeps your existing license valid (see Licensing).
Test your restore
A backup you've never restored is a hope, not a backup. At least once, restore both stores into a throwaway stack and confirm you can log in, see your dashboards, and query historical telemetry.
Next: Upgrades.