Getting started

Quick start

Get Azoth OS routing your AI calls in under 5 minutes. No changes to your prompts or application logic required.

Installation

npm install @azoth/sdk

import OpenAI from 'openai';
import { AzothClient, wrapOpenAI } from '@azoth/sdk';

const azoth = new AzothClient({ apiKey: process.env.AZOTH_API_KEY });
const openai = wrapOpenAI(new OpenAI(), azoth);

// Use exactly as before — Azoth tracks cost, latency, and usage automatically
const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello' }],
});

How it works

Every LLM call flows through the Azoth runtime, which applies five optimization layers before forwarding to your provider:

1. Semantic cache — semantically equivalent queries return cached responses instantly, eliminating redundant model calls (avg 30% elimination rate).
2. Loop detection — recursive patterns and runaway agents are detected and terminated before cost compounds.
3. Intelligent Model Arbiter — the request is scored against all models in your tier for cost, quality, and latency. The cheapest model that meets your SLA is selected.
4. Prompt Intelligence Engine — prompts are analyzed and compressed (avg 40% token reduction) without loss of semantic meaning.
5. FinOps attribution — cost, tokens, latency, and routing decision are attributed to agent, workspace, and workflow for full financial visibility.

Model Arbiter

For automatic model selection across cost/quality/latency, route calls through the governed chat endpoint (it takes the same OpenAI message shape but lets the Arbiter pick the best-fit model):

// POST https://www.azothos.ai/api/v1/chat
const res = await fetch('https://www.azothos.ai/api/v1/chat', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.AZOTH_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    provider: 'openai',           // required hint — Arbiter may pick a cheaper provider
    model:    'gpt-4o',
    messages: [{ role: 'user', content: 'Hello' }],
    mode:     'cost',             // 'cost' | 'quality' | 'latency' | 'balanced'
  }),
});

Semantic cache

Cache is configured at the workspace level in Dashboard → Settings → Routing. Cache hits are reported in the usage trace returned by /api/v1/usage withcacheHit: true.

Budget enforcement

Set hard spend limits per agent or workspace in Dashboard → Budgets. When a budget is exceeded, the runtime returns a structured error:

// Budget exceeded — the call is rejected before reaching your provider
{
  error: 'BudgetExceeded',
  code: 'BUDGET_EXCEEDED',
  message: 'Monthly budget of $500.00 exceeded. Resets on 2026-06-01.',
  budget: { limit: 500, spent: 501.23, currency: 'USD' }
}

Loop Termination Engine

// Loop terminated — returned by the proxy before the provider call
{
  error: 'LoopDetected',
  loopDetected: true,
  pattern: 'cost_runaway',  // 'repetition' | 'high_frequency' | 'circular'
  terminatedAt: '2026-05-12T10:23:44Z',
}

Examples

import OpenAI from 'openai';
import { AzothClient, wrapOpenAI } from '@azoth/sdk';

const azoth = new AzothClient({ apiKey: process.env.AZOTH_API_KEY });
// agentId/workflowId labels every span + usage event for this client
const openai = wrapOpenAI(new OpenAI(), azoth, {
  provider: 'openai',
  model:    'gpt-4o',
  agentId:  'support-bot',
});

const res = await fetch('https://www.azothos.ai/api/v1/chat', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.AZOTH_API_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({
    provider: 'openai',
    model:    'gpt-4o-mini',
    messages: [{ role: 'user', content: 'Summarise this ticket: ...' }],
    mode:     'latency',
  }),
});