# The Server/Offline ratio is workload-specific

Released-data analysis and original visualization by Aryan Putta.

## What this bundle does

Recomputes within-submission Server/Offline throughput ratios from official MLPerf Inference v6.1 results. It is not a new hardware benchmark, vendor ranking, power measurement, or model accuracy rerun.

- Full source: 492 rows, 476 closed and 16 open.
- Scope: available, closed, datacenter, `inferred = 0`, zero reported errors, Tokens/s, Offline or Server, `has_power = false`, null `design_power_watts`.
- Matching: all 27 metadata fields other than Scenario, Result, Accuracy, and Location must match. The same equality is checked against the raw CSV fields, including distinct missing-value spellings.
- Coverage: 305 eligible rows, 152 complete pairs, one unmatched Offline row (JSON ID 264).
- Highlighted B300 ratios: 85.36%, 94.37%, 97.87%; span 12.51 percentage points.
- Separate GB200 GPT-OSS pair: 101.72%. Eleven of 152 matched pairs exceed 100%.
- All highlighted source logs are VALID and rates agree with summary rounding. Accuracy thresholds and actual Server TTFT/TPOT limits are checked.

The 99.9 in Llama 2's label is its target relative to reference quality metrics, not 99.9% question-answer accuracy. No cross-system absolute throughput ranking is made. Each figure bar uses only one matched pair as its numerator and denominator.

## Run offline

Use Python 3.9+ from this extracted folder. All runtime imports are standard-library modules.

```
python analyze.py
python make_plot.py
python build_article.py
python execute_notebook.py
python validate.py
```

`analysis.ipynb` includes executed outputs. It was executed sequentially using the supplied Python helper, not a Jupyter service. It can also be run in an already-installed Python 3 Jupyter kernel. No package installation, network requests, API keys, model weights or GPU are required. Keep the notebook beside `analyze.py`, `make_plot.py`, `data/`, and `sources/`.

For a browser Python runtime, the same analysis is standard-library-only: place the supplied files in its virtual filesystem, then call `analyze(root)` with the extracted root path. A Pyodide runtime is not bundled and a browser execution has not been claimed. The article itself needs no runtime or remote script.

`build_notebook.py` recreates the unexecuted notebook source; run `execute_notebook.py` afterward to populate genuine outputs. `build_provenance.py` checks saved acquisition records against exact upstream Git blob hashes and builds the source manifest. It does not fetch anything.

## Files

- `article.html`: standalone article content, styles, and inline SVG. Author/profile links are site-root paths for integration into aryanputta.com; those site-owned resources are not copied into this bundle.
- `analysis.ipynb`: executed notebook with six code cells.
- `analyze.py`: data validation, exact matching, numerical analysis, and adversarial fixtures.
- `data/summary.csv`, `data/summary.json`: byte-identical full official summary snapshots (under 1 MB combined).
- `data/selected_rows.json`: derived eight-row subset retaining every source field and original row ID.
- `results.json`: all matched pairs, highlighted results, achieved accuracy, target thresholds, log checks, counts and field list.
- `scenario-ratios.svg`, `make_plot.py`: original data-derived plot and generator.
- `sources/`: upstream Apache-2.0 license, README files, eight performance logs, scenario-definition header and submission-checker constants.
- `provenance.json`: immutable repository revisions, upstream URLs, exact file sizes, Git blob SHA-1 and SHA-256 digests.
- `validate.py`, `qa/offline-validation.json`: local static checks.
- `reproducibility.zip`, `SHA256SUMS`: the portable deliverable and file inventory.

## Sources and licensing

Results: https://github.com/mlcommons/inference_results_v6.1/tree/4bb63cd28eb136be221bc801a9e38c66c8a7553f

Documentation/checker snapshot: https://github.com/mlcommons/inference/tree/3fbc329939999c13d0a7b5e67fb2092287e06047

Upstream files are byte-identical to their Git blobs. Source acquisition date is 2026-11-06 UTC, not an article publication date. The two repositories have the Apache License 2.0; its complete text is retained in `sources/LICENSE-MLCommons.md`. Attribution and provenance are preserved. The derived subset, results and original article/figure are not original MLCommons outputs and should not be presented as such. MLPerf is a registered trademark of MLCommons. This analysis is not endorsed by MLCommons or any submitter.

The original benchmark paper requested by the source README is Reddi et al., *MLPerf Inference Benchmark*, 2019, https://arxiv.org/abs/1911.02549.

## Interpretation limits

These are submitter-provided released measurements. Matching summary metadata does not establish identical model-serving configurations across scenarios, an optimal implementation, repeated-run uncertainty, or causal effects of hardware or arrival process. The full released summary and all pairs are supplied to expose selection choices. All 492 rows report no measured-power result; TGP notes must not be converted into energy claims. No price, cost, independent hardware performance, or deployment-specific latency claim is made.