Enrichments
Metrics methodology
How the figures Azoth OS cites in marketing and product material are computed, where they come from, and — above all — what each one is not. Use this page to check any figure you read on the site or hear in a sales call.
Confidence tags
Measured
Derived from production telemetry over a stated window, or from a deterministic benchmark committed to the repository (
reports/benchmark-report.json, npm run benchmark).Modelled
Computed by simulation against a price list and a representative workload; it depends on those assumptions. Older copy may say “simulated”.
Projected
A forward-looking figure: a published price list applied to a typical-customer profile. Not a guarantee.
A figure in customer-facing material must carry its tag. A figure without a tag should not appear.
The figures, and where each comes from
Cost per call · Measured, per call
From the tokens the provider counted and the price in force in the model catalog; returned in the
X-Azoth-Cost-Usd header and sealed with the decision. Your own figures are in your dashboard.Energy per call · Modelled, per call
An estimate under the GreenOps methodology, returned in the
X-Azoth-Energy-Wh header. Absolute grams are advisory; the avoided fraction is the reportable figure.Reused answers · Measured on the benchmark harness
The cache hit rate of
npm run benchmark on its offline scenarios (reports/benchmark-report.md). Your own rate depends on your workload.Latency Azoth adds · Not published
No figure yet: a p50 and p95 measurement is on the roadmap.
Availability · On the status page
The observed state is on the status page. No uptime figure is published here; where a contract sets an SLA, it states its own.
The quality the router weighs · Measured, per workspace
- The quality term of routing is the model’s share of the workspace’s own reference cases that passed, over the last 90 days, once at least 10 cases were judged.
- Otherwise it is 0.5, neutral: what the workspace has not measured weighs the same for every model, and the choice falls to latency and cost.
- A quality floor eliminates a model only when its measured pass rate is below it, never an unmeasured one. The cost term reads the model catalog’s prices.
- Each routing decision says which it was: “N% of M reference cases passed”, or “nothing measured yet”. It is a statement about this workspace’s cases, on its own criteria, not about a model in general.
What these figures are not
- Not a promise about your bill: say “up to” or “median”, and cite the tag.
- Not an average over Azoth’s customers: no customer workload is aggregated into a public figure.
- Not fixed: provider prices, new models and changes to the engines move them, and the benchmark runs again with each release.
- Not an uptime promise: the observed state is on the status page.
The source
The repository reference is docs/METRICS_METHODOLOGY.md, kept in step with this page. A figure on the site that does not trace back to it is a bug — please tell us.