# Benchmarks

Performance is a release gate, not a marketing page. Every selected report
keeps the host, toolchain, and OXC identity, corpus hash, raw timing or RSS
samples, sampling policy, distributions, and every assertion. Everything
below is generated at build time from the reports listed in
`docs/acceptance/performance-report.json`, so this page cannot silently pick
up a newer failed or incomplete run.

How to read the results:

- **MiB/s** is throughput: mebibytes of source processed per second. Higher
  is better.
- **p95** is the nearest-rank 95th percentile of the retained samples. It
  shows tail behavior, not just the average.
- **Ratios like 1.004×** divide by the denominator named in that row.
- **One chart per gate.** Every numeric release gate gets its own small
  chart with the gate's name on top and a one-line summary under it,
  including a plain "pass" or "FAIL" word.
- **The top strip** shows the checked value as a diamond on an axis that
  starts at zero. The dashed vertical line is the frozen budget, named in
  the strip's left label, and the lightly shaded area is the failing side
  of that budget. The release fails if the diamond ever lands in the
  shaded area.
- **The bottom strip** zooms in on the retained samples from the selected
  report, one dot per sample. Its axis is zoomed to the spread, so read
  its tick labels; it usually covers a much smaller range than the top
  strip. The solid vertical line is the median sample and the dotted
  vertical line is the p95 sample.
- **Ratio charts.** The diamond in the top strip is the asserted ratio,
  which is the value the release gate checks. The bottom strip shows the
  raw samples of the two runs that ratio divides, on one shared
  milliseconds (or MiB) axis, each run with its own median tick. The two
  runs are sampled independently, so dividing sample pairs would invent
  data; only the asserted ratio is plotted against the budget.
- **Single-value charts.** Some gates record exactly one number per
  report, like cold starts and editor memory. Those charts show only the
  top strip and say "single measurement per report".
- Hover or focus any table row for the exact result, the budget, the
  sample count, the median, and the p95.

Three results need extra care:

- The matched CLI lane gives ESLint, official Oxlint, and OXC for TSRX the
  same byte-identical 1,000-file TSX corpus, one rule, and identical run
  conditions. The separate mixed-file-types result only compares the product
  against itself on a different workload.
- The native cold-start ratios compare a direct Rust executable with the
  official npm launcher, so they are diagnostic, not tool-speed claims.
- The formatter's 16.6 MiB/s floor is a regression threshold derived from an
  older 1.66 MiB/s measurement on a different corpus, not a like-for-like
  Prettier comparison. The same-build canonical OXC controls are the
  comparable overhead measurements.

Noise is handled by a fail-closed rerun policy instead of picking a
favorable sample:

- Each invocation creates exactly one fresh report.
- A first report inside the 3% near-threshold band requires exactly two
  additional fresh reports. Only the assertions that triggered get
  two-of-three tolerance; everything else must pass in all three.
- Any failure more than 3% beyond its threshold is definitive.
- The representative report is selected by median normalized budget pressure
  with a stable report-path tie-break, never by picking the fastest run. The
  aggregate fails if the selected representative is red.

Make the authoritative release decision with:

```sh
node tests/acceptance/run-performance.mjs
```

The individual commands below produce diagnostic raw reports without the
aggregate admission, rerun, and selection policy:

```sh
cargo run --release --locked -p oxc_tsrx_benchmark -- \
  --assert benchmarks/native-lint/budgets.json
cargo run --release --locked -p oxc_tsrx_format_benchmark -- \
  --assert benchmarks/native-format/budgets.json
node benchmarks/vite/run.mjs
pnpm run benchmark:type-aware
pnpm run benchmark:editor
pnpm run benchmark:comparative
```

## Matched 1,000-file CLI comparison

ESLint, official Oxlint, and OXC for TSRX lint the same byte-identical TSX files with one no-debugger rule, the same explicit file list, zero-diagnostic default output, 5 warmups, and 20 measured processes. Every lane runs through its npm CLI entry point. The ordinary oxlint-tsrx lane imports the exact declared official Oxlint launcher in the same Node process; only the separate mixed-file-types lane enters the native TSRX path. That mixed row is a paired internal workload ratio, not a cross-tool comparison. Report: `benchmarks/comparative/results-1784321699288.json` (2026-07-17), every budget passed.

| Boundary | Result | Budget | Status |
| --- | ---: | ---: | --- |
| OXC for TSRX / official Oxlint (matched TSX lane) | 1.129 × | ≤ 1.250 × | pass |
| ESLint / OXC for TSRX (matched TSX lane) | 13.264 × | ≥ 2 × | pass |
| Paired mixed-file-types / all-TSX product workload | 1.489 × | ≤ 1.500 × | pass |

**Near-threshold adjudication.** `mixedNoBlowup` entered the unchanged 3% band, so the aggregate required exactly 2 additional fresh identity-matched reports. Only triggering assertions receive two-of-three tolerance; every other assertion and invariant must pass in every report. The representative is selected by median normalized budget pressure with a stable report-path tie-break, never by choosing the fastest passing sample. Any failure more than 3% beyond its threshold, or a red selected representative, fails the aggregate. Retained reports: `benchmarks/comparative/results-1784321699288.json`, `benchmarks/comparative/results-1784321719516.json`, `benchmarks/comparative/results-1784321739730.json`.

## Native lint

Frozen release gate: throughput, same-build ordinary-path overhead, cold start, memory, and batch invariants. The npm-launcher ratio is a diagnostic boundary, not a speed claim. Report: `benchmarks/native-lint/results-1784321646022.json` (2026-07-17), every budget passed.

| Boundary | Result | Budget | Status |
| --- | ---: | ---: | --- |
| P01 median standard-path latency ratio | 1.003 | ≤ 1.050 | pass |
| P01 p95 standard-path latency ratio | 1.020 | ≤ 1.080 | pass |
| P01 diagnostic parity | 1 | exactly 1 | pass |
| P01 direct standard bypass | 1 | exactly 1 | pass |
| P02 median scan+copy+parse throughput | 260.28 | ≥ 75 | pass |
| P02 p95 scan+copy+parse throughput | 251.05 | ≥ 60 | pass |
| P02 equivalent-TSX throughput ratio | 0.563 | ≥ 0.500 | pass |
| P02 warm 10 KiB p95 scan+copy+parse latency | 0.041 | ≤ 1 | pass |
| P03 in-process one-thread lint throughput | 134.91 | ≥ 35 | pass |
| P03 CLI one-thread lint throughput | 80.241 | ≥ 35 | pass |
| P03 end-to-end CLI latency ratio | 1.141 | ≤ 1.350 | pass |
| P03 diagnostic parity | 1 | exactly 1 | pass |
| P03 config compiled once | 1 | exactly 1 | pass |
| P03 config one parse per file | 1 | exactly 1 | pass |
| P03 configured rule applied | 1 | exactly 1 | pass |
| P05 fresh-process TSRX p95 latency | 3.155 | ≤ 50 | pass |
| Direct Rust / official Oxlint npm-launcher p95 ratio | 0.056 | ≤ 1.250 | pass |
| P05 startup-rule diagnostic parity | 1 | exactly 1 | pass |
| P07 TSRX peak RSS | 232980480 | ≤ 252690432 | pass |



## Native format

Frozen release gate: formatter throughput, convergence scaling, memory, and batch invariants. The historical 16.6 MiB/s value is an absolute cross-corpus-derived floor, not a speedup claim. Report: `benchmarks/native-format/results-1784321655592.json` (2026-07-17), every budget passed.

| Boundary | Result | Budget | Status |
| --- | ---: | ---: | --- |
| p04 direct median ratio | 0.982 | ≤ 1.050 | pass |
| p04 direct p95 ratio | 1.003 | ≤ 1.080 | pass |
| p04 direct output parity | 1 | exactly 1 | pass |
| p04 direct bypass | 1 | exactly 1 | pass |
| p04 sequential median mib s | 134.78 | ≥ 15 | pass |
| p04 sequential p95 mib s | 127.00 | ≥ 12 | pass |
| Sequential throughput vs absolute 16.6 MiB/s floor | 134.78 | ≥ 16.600 | pass |
| p04 default thread mib s | 823.10 | ≥ 100 | pass |
| p04 generalized control median mib s | 21.790 | ≥ 15 | pass |
| p04 generalized control p95 mib s | 21.135 | ≥ 12 | pass |
| p04 generalized control linear scaling | 1.000 | ≤ 1.350 | pass |
| p04 generalized control idempotent | 1 | exactly 1 | pass |
| p04 generalized control one parse | 1 | exactly 1 | pass |
| p04 generalized dynamic style coverage | 1 | exactly 1 | pass |
| p04 generalized style metadata | 1 | exactly 1 | pass |
| p04 generalized no hidden embedded parse | 1 | exactly 1 | pass |
| p04 idempotent | 1 | exactly 1 | pass |
| p04 one parse | 1 | exactly 1 | pass |
| p04 config compiled once | 1 | exactly 1 | pass |
| p04 config one parse per file | 1 | exactly 1 | pass |
| p04 config options applied | 1 | exactly 1 | pass |
| p05 stdin p95 ms | 3.156 | ≤ 110 | pass |
| Direct Rust / official Oxfmt npm-launcher p95 ratio | 0.036 | ≤ 1.250 | pass |
| p05 complete output | 1 | exactly 1 | pass |
| p07 rss ratio | 1.143 | ≤ 1.150 | pass |

**Near-threshold adjudication.** `p07_rss_ratio` entered the unchanged 3% band, so the aggregate required exactly 2 additional fresh identity-matched reports. Only triggering assertions receive two-of-three tolerance; every other assertion and invariant must pass in every report. The representative is selected by median normalized budget pressure with a stable report-path tie-break, never by choosing the fastest passing sample. Any failure more than 3% beyond its threshold, or a red selected representative, fails the aggregate. Retained reports: `benchmarks/native-format/results-1784321650912.json`, `benchmarks/native-format/results-1784321655592.json`, `benchmarks/native-format/results-1784321660260.json`.

## Opt-in type-aware lint

The opt-in TypeScript-Go lane. The default lane stays syntax-only with zero type processes. Report: `benchmarks/type-aware/results-1784321661795.json` (2026-07-17), every budget passed.

| Boundary | Result | Budget | Status |
| --- | ---: | ---: | --- |
| Default syntax lint p95 | 2.646 ms | ≤ 10 ms | pass |
| Single-file type-aware p95 | 24.777 ms | ≤ 60 ms | pass |
| Two-file project type-aware p95 | 24.040 ms | ≤ 70 ms | pass |
| Single-file type-aware cold start | 60.611 ms | ≤ 100 ms | pass |
| Two-file project cold start | 26.455 ms | ≤ 120 ms | pass |
| Type-aware vs default p95 ratio | 9.363 × | ≤ 12 × | pass |
| Type processes per batch | 1 | exactly 1 | pass |
| Default path parse count per file | 1 | exactly 1 | pass |

## Vite/Vite+ command boundary

Fresh companion processes at the ecosystem seam. Vite build and HMR carry zero OXC for TSRX transforms. Report: `benchmarks/vite/results-1784321678410.json` (2026-07-17), every budget passed.

| Boundary | Result | Budget | Status |
| --- | ---: | ---: | --- |
| Mixed companion lint p95 | 57.915 ms | ≤ 150 ms | pass |
| Mixed lint vs canonical p95 ratio | 1.813 × | ≤ 2.500 × | pass |
| Mixed companion format-check p95 | 127.11 ms | ≤ 220 ms | pass |
| Mixed format vs canonical p95 ratio | 1.234 × | ≤ 2 × | pass |
| Ordinary npm formatter p95 | 113.44 ms | ≤ 150 ms | pass |
| Ordinary npm formatter vs canonical p95 ratio | 1.101 × | ≤ 1.250 × | pass |
| Vite+ 0.2.4 mixed lint p95 | 237.08 ms | ≤ 750 ms | pass |
| Native TSRX parses per file | 1 | exactly 1 | pass |

## Native editor server

Syntax-only local language-server round trips on the retained Markless fixture. Report: `benchmarks/editor/results-1784321679056.json` (2026-07-17), every budget passed.

| Boundary | Result | Budget | Status |
| --- | ---: | ---: | --- |
| Server start to first diagnostics | 2.591 ms | ≤ 50 ms | pass |
| Edit-to-diagnostics p95 | 0.115 ms | ≤ 25 ms | pass |
| Formatting p95 | 0.125 ms | ≤ 15 ms | pass |
| Safe code-action p95 | 0.114 ms | ≤ 25 ms | pass |
| RSS after 1,000-edit soak | 10.984 MiB | ≤ 96 MiB | pass |
| RSS growth through soak | 0 MiB | ≤ 8 MiB | pass |

## Measurement hygiene

- Harnesses are release-only.
- `memory-stats` is linked only into benchmark executables, never the
  distributed CLI.
- Normal source metadata excludes configuration time; aggregate metadata and
  dedicated configuration lanes report it separately.
- Formatter reports keep scan, projection, canonical parse, canonical format,
  and checked-lift timing arrays separately.
- Thresholds never change during adjudication.