Replay
Pick a moment in the past. Change one fact ("what if we hadn't deployed?"). Orbtrace simulates the counterfactual outcome using past data and your AI provider. This page explains how to run one and how to read the result.
This is the feature that doesn't exist in any other observability tool. Read this page slowly — the concept is new even to experienced SREs.
What it is
A counterfactual simulator. You pick a real event from the past — usually an incident — and ask Orbtrace a question like:
- "What would checkout-api's p99 have been if we hadn't deployed
a8f21b?" - "What would total throughput have been if we'd doubled inventory-service's pod count?"
- "What was the user impact if email-service had failed for one extra hour?"
Orbtrace doesn't run a real simulator. It uses three sources:
- Historical data from your own telemetry around the chosen window.
- Similar past incidents retrieved by pgvector embedding search.
- Your configured LLM to synthesise a probabilistic outcome with a confidence interval.
The output is a side-by-side timeline: actual vs hypothesised — and a paragraph explaining the reasoning.
When to use it
- Post-mortemsThe "we'll never know if the deploy did it" debate is over. Replay gives a probabilistic answer.
- Pre-deploy canary calls"If we'd promoted this canary, what would have happened?" The canary becomes the actual; full rollout becomes the counterfactual.
- Capacity planning"If yesterday's traffic had been 2× higher, would we have held SLO?".
- Blast-radius estimation"If feature flag X had been off, would the bad query have run?".
How to run a replay
- 1
Open the Replay screen
Sidebar → Replay. The landing screen shows your saved replays plus a "+ New replay" button.
- 2
Pick the anchor
Three ways:
- From an incident — the most common. Click "Open in Replay" from the incident detail page. The anchor becomes the incident's start time.
- From a chart — right-click any spike on a metric chart and pick "Replay around this point".
- Manual — set a date/time and a service.
- 3
Choose the actual outcome
The "actual" is what really happened — a metric query that captures the thing that broke (typically
p99(http.server.duration{service=...})orerrors_per_second{service=...}). Pre-filled if you came from an incident. - 4
State the counterfactual
A short English sentence: "What if deploy a8f21b had not happened?" or "What if the inventory-service had 4 replicas instead of 2?". The page suggests common templates.
- 5
Pick context to include
Orbtrace lists relevant deploys, feature-flag changes, and similar past incidents. By default it includes everything within ±30 minutes. You can remove what you think isn't relevant — Replay's confidence rises when context is focused.
- 6
Run
Click Run replay. The LLM is called with the actual data, the counterfactual prompt, and the selected context. A spinner shows progress. Typical runs take 8-30 seconds.
Reading the output
The result page has six parts.
① Side-by-side timeline
Two charts stacked vertically:
- Top: ActualThe metric you chose, real values, over the window.
- Bottom: HypothesisedThe same metric, but as the LLM predicts it would have been under the counterfactual. The line is drawn with a confidence band — wider band = less certain.
Where the two charts diverge is where the counterfactual matters. If they're identical, the answer is "the change you proposed wouldn't have helped".
② Probabilistic summary
A sentence at the top: "Probably (87% confidence) checkout-api p99 would have stayed under 280 ms during the incident window if deploy a8f21b had not occurred."
The confidence number is the AI's own — anchored on the volume and quality of historical context. A run with 3 similar past incidents in context gets higher confidence than a run with zero.
③ Reasoning paragraph
Two to four paragraphs explaining the AI's logic. Cites the historical incidents it used, the spans it leaned on, and what it could not prove.
④ Sensitivity table
A small table: "if we removed assumption X, the predicted p99 changes by Y%". Use it to test how robust the answer is.
⑤ Similar past incidents
The list of incidents whose embedding was closest to this one. Click any for the original incident detail.
⑥ Save / share
Save the replay (it becomes a row on the Replay index). Replays are read-only; if you want to ask a different counterfactual, run a new one rather than editing the saved version.
What Replay is not
- Not a discrete-event simulator. Orbtrace does not literally re-execute requests; it synthesises a probable outcome from history.
- Not a forecast. Replay looks at the past — "what would have happened then". Forecasts are not a feature.
- Not deterministic. Re-running the same replay with the same context can produce subtly different wording (though the chart shape stays stable). For audit purposes, the saved replay freezes the result.
Cost and budget
Each replay is one LLM call against your configured AI provider. Typical cost: $0.10-$0.40 depending on context size and model. The cost is shown at the top of the result page (visible to administrators). Replay shares the monthly budget you set in Admin → AI; when budget is exhausted, Replay disables itself for the rest of the month.
Tips
- Run Replay during the post-mortem, not days later. Fresh context = better embedding matches = higher confidence.
- Constrain the counterfactual to one variable. "What if we hadn't deployed AND scaled up" mixes two effects; the AI's confidence will be lower than if you run two separate replays.
- Save replays as evidence. Attach them to the post-mortem document — they're the most concrete "we considered this counterfactual" record an SRE team can produce.
Next: Admin pages.