# Metrics methodology

How the figures Azoth OS cites in marketing and product material are computed, where they come from, and — above all — what each one is not. Use this page to check any figure you read on the site or hear in a sales call.

## Confidence tags

- **Measured** — Derived from production telemetry over a stated window, or from a deterministic benchmark committed to the repository (`reports/benchmark-report.json`, `npm run benchmark`).
- **Modelled** — Computed by simulation against a price list and a representative workload; it depends on those assumptions. Older copy may say “simulated”.
- **Projected** — A forward-looking figure: a published price list applied to a typical-customer profile. Not a guarantee.

> A figure in customer-facing material must carry its tag. A figure without a tag should not appear.

## The figures, and where each comes from

- **Cost per call · Measured, per call** — From the tokens the provider counted and the price in force in the model catalog; returned in the `X-Azoth-Cost-Usd` header and sealed with the decision. Your own figures are in your dashboard.
- **Energy per call · Modelled, per call** — An estimate under the GreenOps methodology, returned in the `X-Azoth-Energy-Wh` header. Absolute grams are advisory; the avoided fraction is the reportable figure.
- **Reused answers · Measured on the benchmark harness** — The cache hit rate of `npm run benchmark` on its offline scenarios (`reports/benchmark-report.md`). Your own rate depends on your workload.
- **Latency Azoth adds · Not published** — No figure yet: a p50 and p95 measurement is on the roadmap.
- **Availability · On the status page** — The observed state is on the status page. No uptime figure is published here; where a contract sets an SLA, it states its own.

## The quality the router weighs · Measured, per workspace

- The quality term of routing is the model’s share of the workspace’s own reference cases that passed, over the last 90 days, once at least 10 cases were judged.
- Otherwise it is 0.5, neutral: what the workspace has not measured weighs the same for every model, and the choice falls to latency and cost.
- A quality floor eliminates a model only when its measured pass rate is below it, never an unmeasured one. The cost term reads the model catalog’s prices.
- Each routing decision says which it was: “N% of M reference cases passed”, or “nothing measured yet”. It is a statement about this workspace’s cases, on its own criteria, not about a model in general.

## What these figures are not

- Not a promise about your bill: say “up to” or “median”, and cite the tag.
- Not an average over Azoth’s customers: no customer workload is aggregated into a public figure.
- Not fixed: provider prices, new models and changes to the engines move them, and the benchmark runs again with each release.
- Not an uptime promise: the observed state is on the status page.

## The source

The repository reference is `docs/METRICS_METHODOLOGY.md`, kept in step with this page. A figure on the site that does not trace back to it is a bug — please tell us.
