Capacity planning
Size the deployment for your load — tell the calculator your daily volume, retention, and service count and get back the Helm sizing, Doris FE/BE counts, vCPU, memory, and disk to provision. Grounded in Apache Doris's published deployment guidance.
How much hardware does Orbtrace need? It depends on three things: how much telemetry you send per day, how long you keep it, and how many services you run. Put those numbers in below and the calculator returns a recommended starting configuration — the Helm sizing, the Doris FE/BE node counts, total vCPU and memory, and the disk to provision — along with a my-values.yaml overlay you can paste straight into your deploy.
Recommended starting configuration
CPU & memory are per pod — exactly what goes in each component's Helm resources. Each cell reads request → limit: request is what Kubernetes reserves (and schedules on), limit is the burst ceiling (CPU is throttled above it, memory is OOM-killed). Multiply by Count for that component's total; the whole-cluster sum is below.
| Component | Count | vCPU/pod (req → limit) | Memory/pod GiB (req → limit) |
|---|---|---|---|
| Orbtrace app | 3 | 1 → 3 | 2 → 6 |
| Doris FE | 3 | 1 → 2 | 8 → 16 |
| Doris BE | 5 | 4 → 8 | 8 → 16 |
| PostgreSQL | 1 | 0.5 → 2 | 1 → 2 |
| Valkey | 1 | 0.25 → 1 | 0.25 → 1 |
Cluster compute total: 26.8–58 vCPU · 71.3–149 GiB (request → limit)
- Per-BE hot disk works out to ~60 TiB. That's a very large PVC — consider adding BE nodes or moving older data to a cold (S3) tier.
Drop these into your values.yaml on top of the matching preset, then tune against live telemetry.
# values-overlay.yaml — start from values-medium.yaml
orbtrace:
replicaCount: 3
doris:
profile:
activeProfile: custom
custom:
base: small # extend retention on a small/medium footprint
fe:
replicaCount: 3
be:
replicaCount: 5
storagePvc:
size: 60Ti
postgres:
pvc:
size: 50GiBack-of-envelope, correct to within ±25–50%. Doris-grounded (tablet ≤ 20K replicas/BE, 1–10 GB tablets, ≥ 3 FE for HA). Calibrate against real telemetry in the first operational week. · 7,384 tablets · 14,768 replicas @ RF2
These are starting points, not guarantees
The output is correct to within roughly ±25–50% — good enough to buy hardware and open a cluster, not a substitute for measuring your own workload. Orbtrace ships two monitors (CardinalityMonitor, TombstoneMonitor) that tell you within the first operational week whether the sizing holds; recalibrate against them. The numbers are deliberately conservative.
What the inputs mean
- Daily ingestTotal logs + traces + metrics per day, gzip-decompressed, measured at your OpenTelemetry Collector's egress — not at the application's emit side. If you only know the compressed wire volume, multiply by the rough compression ratios (logs 8–12×, traces 4–6×, metrics 3–5×).
- ServicesDistinct
service.namevalues. This is orthogonal to volume — 50 services at 5 TB/day and 5000 services at 50 GB/day are both valid and size differently. It drives cardinality-monitor calibration more than node count. - Hot retentionDays of data on fast BE disk, frequently queried. This is the single biggest lever on disk cost and on the Doris tablet count.
- Cold retentionExtra days parked on S3-compatible object storage. Set it to 0 for hot-only. Beyond ~30 days a cold tier saves a lot of expensive block storage.
CPU & memory are per pod; only storage is totalled
The per-component breakdown lists CPU and memory per pod — exactly what you put in each component's Helm resources.requests / resources.limits. Multiply by the Count column for that component's total, and the cluster compute total sits just under the table. The Total footprint cards only aggregate what is meaningful cluster-wide: the pod count and storage (a single summed "vCPU" across an app pod and a BE pod would be misleading, so it isn't shown as one number).
How the calculator is grounded in Apache Doris
Orbtrace stores telemetry in Apache Doris, and the sizing math follows Doris's own published deployment guidance rather than guesswork:
- Node shapes start from Doris's hardware floor — FE and BE need at least 4 vCPU / 8 GiB to boot, and production clusters are recommended at 16 vCPU / 64 GiB per node (Cluster planning). The shipped
values-large.yamlBE (8–16 vCPU / 32–64 GiB) sits right in that band. - FE count. Doris recommends at least 3 FE for a high-availability Raft quorum, and notes that one FE comfortably manages 10–20 BE — so FE count grows far more slowly than BE.
- The BE-count floor is a tablet-count budget, not just a disk number. Tablet metadata lives in FE memory, so Doris guidance is to keep tablet replicas per BE in the low tens of thousands. Orbtrace plans to a < 20,000 replicas/BE ceiling. The calculator computes the steady-state tablet count from your retention × the profile's bucket counts and partition cadence (the small/medium profiles partition raw tables by DAY, the large profile by HOUR — tablet & bucket design), then divides by that ceiling. This is why a high-volume cluster can need more BE than disk alone would suggest — and the calculator flags it when it happens.
- Tablet sizing. Doris's best practice keeps each tablet in the 1–10 GB band and bucket counts a multiple of the BE count. Orbtrace's shipped bucket counts (16 small, 128 large) are chosen so tablets land in that band at each tier's target volume.
The disk, replication, and cold-tier rules come from Orbtrace's own capacity-planning framework, which is layered on top of that Doris guidance.
The formulas
Every number above is back-of-envelope and verifiable by hand.
Hot disk
hot_disk_TiB = daily_ingest_TiB
× hot_retention_days
× replication_factor (2 by default; tolerates one BE loss)
× 1.5 (compaction working set)
× compression_ratio (0.5 conservative; 0.3 only on settled large tables)
× 1.3 (ingest-burst + compaction headroom)
per_BE_PVC = hot_disk_TiB / BE_countThe compression ratio is the riskiest input — measure it
The framework default is 0.5 (≈2:1), not the often-quoted 0.3. On a real OTel-Demo corpus the measured single-replica ratio was only ~2:1 at small volume — fixed per-column inverted-index overhead dominates until tables are large and compaction has settled. Size disk at 0.5, switch to 0.3 only after you've measured the ratio on your data at sustained volume (SHOW DATA FROM <db>.otel_traces ÷ loaded bytes ÷ replication).
Tablet count → minimum BE
tablets = Σ over tables ( live_partitions × buckets )
live_partitions = retention_days × (24 for HOUR, 1 for DAY)
replicas = tablets × replication_factor
min_BE = ceil( replicas / 20,000 )
BE_count = max( preset_BE, min_BE ) # the larger of disk- and tablet-floor winsCold tier (S3) and PostgreSQL
s3_bucket_TiB = daily_ingest_TiB × cold_retention_days × replication × compression × 1.2PostgreSQL holds metadata, audit, replay history, and scheduler locks — not telemetry. It scales with feature usage, not throughput, so the calculator maps it to the sizing tier (10 GiB small → 200 GiB large) rather than a formula.
After you deploy
- Start from the matching preset (
values-small.yaml/values-medium.yaml/values-large.yaml) and apply the overlay above. See Installation → Kubernetes with Helm. - Watch the first week. If
orbtrace.cardinality.trend.exceededororbtrace.tombstone.max_versionclimb, the sizing is tight — scale the BE count or extend the cold tier. - Stand up a Collector and prove data flows — Integration patterns.
If your workload is past 100 TB/day, the calculator returns a custom sketch on top of the large preset; hyperscale deploys need a dedicated engagement with the Orbtrace team for BE topology and S3 lifecycle planning.