$worker
llm-router
v1.4.23One front door + provider protocol in front of every LLM provider.
iiiverifiedinstall
$iii trigger compose::add worker=llm-router@1.4.23
- macOS: arm64
- Linux: arm64 · armv7 · x64
- Windows: arm64 · x64
repoiii-hq/workers
exact versions are immutable; binary and bundle artifacts are digest-pinned.
agent-ready brief for v1.4.23
install + config + dependencies + readme + api reference, all in one place. fetch as agent-context.md for an llm to consume.
full markdown
the same content rendered as discrete blocks below is exposed as a single markdown document at
/workers/llm-router.md?version=1.4.23. paste it into an llm prompt or pipe it through curl from a worker.install
install
$iii trigger compose::add worker=llm-router@1.4.23
dependencies
readme
README.md
llm-router
One front door for every LLM provider. The router owns routing, the provider registry, credential resolution, the model catalog, streaming relay, retries, and a single failure contract — consumers call one chat surface and never talk to a provider directly.
llm-router is a standalone iii worker. Providers plug in as separate workers
at runtime through a self-registration protocol
(iii trigger compose::add worker=provider-); the router never compiles against a provider, and removing a
provider worker removes the provider.
Install
iii trigger compose::add worker=llm-routerQuickstart
A consumer streams a turn by creating an iii channel, handing the router the
channel's write endpoint, and reading frames from the read endpoint
while router::chat runs. Any SDK works; Node shown:
import { createChannel } from 'iii-sdk';
const { reader, writerRef } = await createChannel(iii);
reader.onMessage((frame) => {
const event = JSON.parse(frame); // AssistantMessageEvent
if (event.type === 'text_delta') process.stdout.write(event.delta);
});
const res = await iii.trigger('router::chat', {
writer_ref: writerRef, // direction "write"
model: 'claude-sonnet-4',
messages: [{ role: 'user', content: [{ type: 'text', text: 'Hello' }], timestamp: Date.now() }],
}, { timeout_ms: 620_000 }); // outer timeout ≥ the router's 600s stream budget
// res: { ok, provider, model, stop_reason, usage }The streaming contract: every stream ends with exactly one terminal frame
(done or error). When the router has to kill a stream itself (idle
timeout, provider crash), it synthesizes the terminal frame and attaches the
partial content, so consumers never hang on a half-open stream.
Functions
Consumer surface
| Function | Purpose |
|---|---|
router::chat |
Stream a turn into the caller's channel; returns the turn summary. |
router::complete |
Non-streaming convenience over the same pipeline; returns the final message. |
router::abort |
Cancel an in-flight turn by request_id. |
router::route |
Read-only routing preview: {model, provider?} → {provider, candidates}, same rules and error codes as router::chat. Pin the result as the explicit provider on the chat call when you need the provider before streaming. |
router::models::list |
List catalog models, filterable by provider / capability / modality (chat by default, stt, tts, any). |
router::models::get |
Fetch one model record (null when unknown). |
router::models::supports |
Check one capability flag for one model. |
router::embed |
Batch text embeddings through the first embed-capable provider in the registry ({model?, provider?, input[]} → {provider, model, embeddings[][]}); providers without an embed surface are skipped. |
router::transcribe |
Speech to text: {model?, provider?, audio_base64, mime?, language?, prompt?} → {provider, model, text, segments[]?, language?, duration_secs?}. The provider comes from the named provider, else from the catalog owner of the stt model, else the first provider that declared one. |
router::speak |
Text to speech: {model?, provider?, text, voice?, format?, language?, speed?} → {provider, model, audio_base64, mime, voice?, duration_secs?}. Same resolution over tts models. |
router::count_tokens |
Count prompt tokens: {model, provider?, system_prompt?, tools?, messages} → {provider, model, tokens, estimator}, resolved with the same routing rules as router::chat and forwarded to provider::. Never runs the model and costs nothing; estimator is provider (metering API) or tiktoken (local tokenizer). A provider without the surface is a typed router/no_token_counter error, so callers can fall back to their own estimate. |
router::provider::list |
Registered providers with configured / available status. |
Only the read surface is agent-callable (router::models::list / get /
supports, router::provider::list); everything else is denied to in-run
agents — see Security model.
Error conventions
Errors use stable router/ identifiers. Streaming failures return an
error object with a user-facing message, the shared kind, retryable,
and an optional diagnostic detail. User interfaces should show message
and keep code / detail behind technical details; automation should branch
on code and retryable, never parse the prose.
The main recovery codes are router/provider_unavailable,
router/capacity_exceeded, router/stream_idle_timeout,
router/stream_incomplete, router/provider_auth_expired,
router/provider_rate_limited, router/context_overflow, and
router/provider_rejected.
Provider protocol
Token-gated after the first declare: the response to register carries a
registration token, and every later protocol call must present it.
| Function | Purpose |
|---|---|
router::provider::register |
Self-declaration at attach time; idempotent re-declare with the token. |
router::provider::resolve |
Per-request credential + endpoint resolution (config > env > none). |
router::provider::update_credential |
Persist a refreshed credential (OAuth write-back). |
router::models::reconcile |
Replace the provider's catalog slice in one write. |
The provider worker itself exposes provider:: and, when
capable, provider:: (model discovery),
provider:: (prompt token counting),
provider:: (embeddings), and the speech pair
provider:: / provider::.
Configuration
All operator configuration lives in the engine's llm-router configuration
entry — no env vars, no config file. The entry schema is composed at runtime
from each registered provider's declaration. The router fetches the complete
entry at boot and keeps an in-memory snapshot synchronized by the
configuration:updated trigger; request handlers never poll
configuration::get.
{
"default_provider": "anthropic",
"providers": {
"anthropic": {
"api_key": "sk-…",
"api_url": "https://api.anthropic.com/v1/messages",
"max_tokens": 8192
}
},
"routing_heuristics": [{ "pattern": "^gpt-", "provider": "openai" }],
"settings": {
"stream_timeout_ms": 600000,
"idle_timeout_ms": 120000,
"retry_max": 2,
"output_token_max": 32000
}
}| Setting | Default | Meaning |
|---|---|---|
stream_timeout_ms |
600000 |
Hard budget for one streamed turn. |
idle_timeout_ms |
120000 |
Max silence between provider frames before the attempt is cut. |
retry_max |
2 |
Retries per turn for retryable failures before the first forwarded frame (0–10). |
output_token_max |
unset | Optional ceiling on max_output_tokens. Unset forwards each model's own output ceiling, so providers run without an artificial limit; set it only to cap spend. |
Pasting a key into a provider's slice is the whole onboarding flow: the
router diffs the changed slice, debounces ~2 s, and kicks that provider's
provider:: discovery; discovered models land in the
catalog via router::models::reconcile and show up in router::models::list
within seconds — no restart.
Operational notes
- Env-var credential fallback resolves in the router's process. A
provider's
credential_env_var(e.g.ANTHROPIC_API_KEY) is read by the llm-router binary, not by the provider worker — launch the router with those variables set, or put keys in the entry. A key present only in another worker's environment shows up asconfigured: false. - Registration-token recovery. Re-registering a provider id without its
original token is rejected (anti-takeover). If a provider durably lost its
token, delete the router's registry state (state scope
llm-router, keyregistry) and restart the affected providers to re-bind; pasted credentials in the configuration entry are unaffected.
Security model
The agent-callable surface is governed by the engine-wide permission policy
(repo-root iii-permissions.yaml), not a per-worker
file. In-run agents may only read the catalog and provider list:
- Allowed:
router::models::list,router::models::get,router::models::supports,router::provider::list. - Denied to agents: the chat/spend surface (
router::chat,router::complete,router::abort,router::route), the whole provider/credential protocol (router::provider::resolve/register/update_credential,router::models::reconcile), and directprovider::*calls.
Worker-to-worker calls bypass the agent gate, so the harness, context-manager, and provider workers reach the full surface — only in-run agents are restricted. Provider credentials live in the configuration entry and are never readable back through any allowed function.
Events
The router registers three custom trigger types and fans out to every bound handler. Bind with the standard two-step pattern; the handler receives the payload verbatim (no envelope).
| Trigger type | Fires when | Payload |
|---|---|---|
router::models::changed |
a provider reconciles its catalog slice | { "provider": " |
router::provider::changed |
the registry changes (declare / availability flip) | { "provider": " |
router::ready |
the router finishes booting; providers re-declare on it | {} |
iii.registerFunction('my-worker::on-models-changed', async (payload) => {
console.log('catalog changed:', payload); // { provider, count }
return {};
});
iii.registerTrigger({
type: 'router::models::changed',
function_id: 'my-worker::on-models-changed',
config: {},
});Writing a provider worker
A provider worker must:
- Register
provider::honouring the channel-writer contract: forward upstream output as::stream AssistantMessageEventframes into thewriter_refit receives, ending with one terminal frame. - Declare itself at startup via
router::provider::register— retrying with backoff until acknowledged (covers provider-before-router boot order) — and re-declare on therouter::readyevent after a router restart. The declaration may carryicon_svg: one self-contained, monochromedocument (squareviewBox, no fixed size, at most 32 KiB). The router echoes it inrouter::provider::listand the console paints it as acurrentColormask beside the provider's models; a missing or malformed mark falls back to the provider's initial. - Resolve credentials per request via
router::provider::resolve; never read keys directly. - Treat closure of its stream channel as cancellation: abort the upstream request and stop writing frames.
- Map upstream failures to the shared
ErrorKindtaxonomy on itserrorframes. Transport retries (429 / 5xx / connect) are the router's job, not the provider's.
A provider may also register provider:: ({model, system_prompt?, tools?, messages} → {model, tokens, estimator}) to serve
router::count_tokens: an exact count from a provider metering API
(estimator: "provider") or a local tokenizer estimate (estimator: "tiktoken"). Providers without it simply make router::count_tokens return
a typed router/no_token_counter error for that provider.
Speech providers
A speech provider is the same kind of worker with no chat stream: it
declares its models with a speech block and serves one or both speech
surfaces. router::models::list hides speech models from chat pickers
unless asked (modality: "stt" | "tts" | "any"), and router::chat
refuses them with a message that names the right door.
{ "id": "scribe-v1", "provider": "elevenlabs", "context_window": 0, "max_output_tokens": 0,
"speech": { "modality": "stt", "languages": ["en", "hi"], "streaming": true } }| Function | Request | Response |
|---|---|---|
provider:: |
{model?, audio_base64, mime, language?, prompt?} |
{model, text, segments[{text, start_secs?, end_secs?}]?, language?, duration_secs?} |
provider:: |
{model?, text, voice?, format, language?, speed?} |
{model, audio_base64, mime, voice?, duration_secs?} |
model arrives as the id the caller named, or null for the provider's
default. format is what the caller asked for (mp3, wav, pcm16,
opus); answer with the container you produced and say so in mime.
Credentials resolve through router::provider::resolve exactly as for
chat providers.
The first real provider implementing this protocol is
provider-anthropic/ — useful as a reference
implementation alongside the scripted provider in the integration tests.
provider-openai/ follows the same structure for the
OpenAI Chat Completions API (native structured output, reasoning_effort).
provider-opencode-go/ follows the same structure for the
OpenCode Go Chat Completions API (curated model metadata, reasoning_effort).
Local development & testing
cargo test # unit suite, no engine needed
cargo test --test integration # engine-backed suite; self-skips without an engineThe integration suite spawns a throwaway engine per test when iii is on
PATH (or III_ENGINE_BIN points at a binary) and covers the chat relay,
cancellation, abort, restart recovery, registration token gating, paste-a-key
discovery, and event delivery end to end.
To run the worker locally against an engine:
cargo run -- --url ws://127.0.0.1:49134--url defaults to ws://127.0.0.1:49134 and honours the III_WS_URL
environment variable when the flag is not set. --config is accepted per
the standard worker CLI but ignored with a warning — operator config lives
in the engine's llm-router configuration entry (see Configuration above).
api reference (json)
agent-api-reference.json
{
"functions": [
{
"description": "Serve the llm-router worker's injected console UI assets (content function for its console:script / console:style triggers).",
"metadata": {
"internal": true
},
"name": "llm-router::ui-content",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of the content function: the console asks for one asset by path.",
"properties": {
"path": {
"description": "The asset path from the trigger config (e.g. `state/page.js`).",
"type": "string"
}
},
"required": [
"path"
],
"title": "UiContentInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of the content function.",
"properties": {
"content": {
"description": "The asset source, verbatim.",
"type": "string"
},
"content_type": {
"description": "MIME type the console should serve the asset with.",
"type": "string"
}
},
"required": [
"content",
"content_type"
],
"title": "UiContentResult",
"type": "object"
}
},
{
"description": "Abort an in-flight router::chat/complete by request_id; reports whether a live request was cancelled.",
"metadata": {
"internal": true
},
"name": "router::abort",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of the `router::abort` iii function.",
"properties": {
"request_id": {
"type": "string"
}
},
"required": [
"request_id"
],
"title": "AbortRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"aborted": {
"type": "boolean"
}
},
"required": [
"aborted"
],
"title": "AbortResponse",
"type": "object"
}
},
{
"description": "Stream a chat completion: route {model, provider?} to a provider, relay assistant frames to writer_ref, and return the terminal response.",
"metadata": {
"internal": true
},
"name": "router::chat",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ChannelDirection": {
"enum": [
"read",
"write"
],
"type": "string"
},
"StreamChannelRef": {
"properties": {
"access_key": {
"type": "string"
},
"channel_id": {
"type": "string"
},
"direction": {
"$ref": "#/definitions/ChannelDirection"
}
},
"required": [
"access_key",
"channel_id",
"direction"
],
"type": "object"
}
},
"description": "Input of the `router::chat` iii function: a [`ChatCall`] plus the caller's write channel. The handler relays assistant frames to `writer_ref` and also returns the terminal [`ChatResponse`].",
"properties": {
"max_output_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"messages": true,
"metadata": {
"default": null
},
"model": {
"type": "string"
},
"provider": {
"default": null,
"type": [
"string",
"null"
]
},
"provider_options": {
"default": null
},
"request_id": {
"default": null,
"type": [
"string",
"null"
]
},
"response_format": {
"default": null
},
"session_id": {
"default": null,
"description": "Stable conversation identity, forwarded separately from the per-turn request id.",
"type": [
"string",
"null"
]
},
"system_prompt": {
"default": null,
"type": [
"string",
"null"
]
},
"thinking_level": {
"default": null
},
"tools": {
"default": null
},
"writer_ref": {
"allOf": [
{
"$ref": "#/definitions/StreamChannelRef"
}
],
"description": "The caller's write channel (direction \"write\"); frames are relayed here."
}
},
"required": [
"messages",
"model",
"writer_ref"
],
"title": "ChatFnInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ErrorKind": {
"enum": [
"auth_expired",
"rate_limited",
"context_overflow",
"transient",
"permanent"
],
"type": "string"
},
"ErrorShape": {
"properties": {
"code": {
"type": "string"
},
"detail": {
"description": "Provider/transport diagnostics for logs and expandable UI details. `message` remains the stable, user-facing explanation.",
"type": [
"string",
"null"
]
},
"kind": {
"anyOf": [
{
"$ref": "#/definitions/ErrorKind"
},
{
"type": "null"
}
]
},
"message": {
"type": "string"
},
"retryable": {
"type": [
"boolean",
"null"
]
}
},
"required": [
"code",
"message"
],
"type": "object"
},
"StopReason": {
"enum": [
"end",
"length",
"function_call",
"aborted",
"error"
],
"type": "string"
},
"Usage": {
"properties": {
"cache_read": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"reasoning": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
}
},
"properties": {
"error": {
"anyOf": [
{
"$ref": "#/definitions/ErrorShape"
},
{
"type": "null"
}
]
},
"model": {
"type": "string"
},
"ok": {
"type": "boolean"
},
"provider": {
"type": "string"
},
"stop_reason": {
"anyOf": [
{
"$ref": "#/definitions/StopReason"
},
{
"type": "null"
}
]
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
}
},
"required": [
"model",
"ok",
"provider"
],
"title": "ChatResponse",
"type": "object"
}
},
{
"description": "Non-streaming convenience over router::chat: run the turn on an internal channel and return the final assistant message + usage.",
"metadata": {
"internal": true
},
"name": "router::complete",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "ChatRequest minus writer_ref (the sink arrives separately — the function handler wraps the looked-up writer, complete wraps its own).\n\n`messages` / `tools` / `response_format` / `thinking_level` / `provider_options` stay `Value`: they are forwarded to the provider verbatim, so the router intentionally does not re-validate their shape. The struct still derives `JsonSchema` so the SDK emits a real request schema (the freeform sub-fields surface as permissive sub-schemas).",
"properties": {
"max_output_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"messages": true,
"metadata": {
"default": null
},
"model": {
"type": "string"
},
"provider": {
"default": null,
"type": [
"string",
"null"
]
},
"provider_options": {
"default": null
},
"request_id": {
"default": null,
"type": [
"string",
"null"
]
},
"response_format": {
"default": null
},
"session_id": {
"default": null,
"description": "Stable conversation identity, forwarded separately from the per-turn request id.",
"type": [
"string",
"null"
]
},
"system_prompt": {
"default": null,
"type": [
"string",
"null"
]
},
"thinking_level": {
"default": null
},
"tools": {
"default": null
}
},
"required": [
"messages",
"model"
],
"title": "ChatCall",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"AssistantMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"error_kind": {
"anyOf": [
{
"$ref": "#/definitions/ErrorKind"
},
{
"type": "null"
}
]
},
"error_message": {
"type": [
"string",
"null"
]
},
"model": {
"type": "string"
},
"native_stop_reason": {
"type": [
"string",
"null"
]
},
"provider": {
"type": "string"
},
"role": {
"$ref": "#/definitions/AssistantRoleTag"
},
"stop_reason": {
"$ref": "#/definitions/StopReason"
},
"timestamp": {
"format": "int64",
"type": "integer"
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
},
"warnings": {
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
}
},
"required": [
"content",
"model",
"provider",
"role",
"stop_reason",
"timestamp"
],
"type": "object"
},
"AssistantRoleTag": {
"enum": [
"assistant"
],
"type": "string"
},
"ContentBlock": {
"description": "Content blocks — the atomic units of message content (README § Content blocks).",
"oneOf": [
{
"properties": {
"text": {
"type": "string"
},
"type": {
"enum": [
"text"
],
"type": "string"
}
},
"required": [
"text",
"type"
],
"type": "object"
},
{
"properties": {
"attachment_id": {
"description": "Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.",
"type": [
"string",
"null"
]
},
"data": {
"type": "string"
},
"mime": {
"type": "string"
},
"type": {
"enum": [
"image"
],
"type": "string"
}
},
"required": [
"data",
"mime",
"type"
],
"type": "object"
},
{
"description": "Reference to an original upload in the session-manager attachment store. A REFERENCE ONLY (never inline bytes); the harness strips it before `router::chat`. Tolerated here so a stray block deserializes instead of failing the request; every counter/mapping emits nothing for it.",
"properties": {
"attachment_id": {
"type": "string"
},
"mime": {
"type": "string"
},
"name": {
"type": "string"
},
"size": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": {
"enum": [
"file"
],
"type": "string"
}
},
"required": [
"attachment_id",
"mime",
"name",
"size",
"type"
],
"type": "object"
},
{
"properties": {
"signature": {
"type": [
"string",
"null"
]
},
"text": {
"type": "string"
},
"type": {
"enum": [
"thinking"
],
"type": "string"
}
},
"required": [
"text",
"type"
],
"type": "object"
},
{
"description": "Opaque redacted thinking payload — replayed verbatim on the Anthropic wire.",
"properties": {
"data": {
"type": "string"
},
"type": {
"enum": [
"redacted_thinking"
],
"type": "string"
}
},
"required": [
"data",
"type"
],
"type": "object"
},
{
"properties": {
"arguments": true,
"function_id": {
"type": "string"
},
"id": {
"type": "string"
},
"type": {
"enum": [
"function_call"
],
"type": "string"
}
},
"required": [
"arguments",
"function_id",
"id",
"type"
],
"type": "object"
},
{
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"function_call_id": {
"type": "string"
},
"is_error": {
"type": [
"boolean",
"null"
]
},
"type": {
"enum": [
"function_result"
],
"type": "string"
}
},
"required": [
"content",
"function_call_id",
"type"
],
"type": "object"
}
]
},
"ErrorKind": {
"enum": [
"auth_expired",
"rate_limited",
"context_overflow",
"transient",
"permanent"
],
"type": "string"
},
"StopReason": {
"enum": [
"end",
"length",
"function_call",
"aborted",
"error"
],
"type": "string"
},
"Usage": {
"properties": {
"cache_read": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"reasoning": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
}
},
"description": "Output of the `router::complete` iii function (non-streaming convenience).",
"properties": {
"message": {
"$ref": "#/definitions/AssistantMessage"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
}
},
"required": [
"message",
"model",
"provider"
],
"title": "CompleteResponse",
"type": "object"
}
},
{
"description": "Count prompt tokens for {model, provider?, system_prompt?, tools?, messages} through the resolved provider's provider::<id>::count_tokens surface; never runs the model and costs nothing.",
"metadata": {
"internal": true
},
"name": "router::count_tokens",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"AgentFunction": {
"description": "Function invocation schema — what a provider sees as a `tools` array entry (README § Function invocation schema; adapter boundary). These describe iii functions exposed to the model, not provider-native tools.",
"properties": {
"description": {
"type": "string"
},
"execution_mode": {
"type": [
"string",
"null"
]
},
"label": {
"type": [
"string",
"null"
]
},
"name": {
"type": "string"
},
"parameters": true
},
"required": [
"description",
"name",
"parameters"
],
"type": "object"
},
"AgentMessage": {
"anyOf": [
{
"$ref": "#/definitions/AssistantMessage"
},
{
"$ref": "#/definitions/FunctionResultMessage"
},
{
"$ref": "#/definitions/CustomMessage"
},
{
"$ref": "#/definitions/UserMessage"
}
],
"description": "The canonical transcript message union. Untagged: the single-variant role tags disambiguate deserialization."
},
"AssistantMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"error_kind": {
"anyOf": [
{
"$ref": "#/definitions/ErrorKind"
},
{
"type": "null"
}
]
},
"error_message": {
"type": [
"string",
"null"
]
},
"model": {
"type": "string"
},
"native_stop_reason": {
"type": [
"string",
"null"
]
},
"provider": {
"type": "string"
},
"role": {
"$ref": "#/definitions/AssistantRoleTag"
},
"stop_reason": {
"$ref": "#/definitions/StopReason"
},
"timestamp": {
"format": "int64",
"type": "integer"
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
},
"warnings": {
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
}
},
"required": [
"content",
"model",
"provider",
"role",
"stop_reason",
"timestamp"
],
"type": "object"
},
"AssistantRoleTag": {
"enum": [
"assistant"
],
"type": "string"
},
"ContentBlock": {
"description": "Content blocks — the atomic units of message content (README § Content blocks).",
"oneOf": [
{
"properties": {
"text": {
"type": "string"
},
"type": {
"enum": [
"text"
],
"type": "string"
}
},
"required": [
"text",
"type"
],
"type": "object"
},
{
"properties": {
"attachment_id": {
"description": "Optional link to the stored original in the session-manager attachment store. Transcript bookkeeping for lazy readers only: the harness clears it before `router::chat`, and no provider mapping reads it. Tolerated here so a block that still carries it deserializes instead of failing the request; omitted on the wire when absent.",
"type": [
"string",
"null"
]
},
"data": {
"type": "string"
},
"mime": {
"type": "string"
},
"type": {
"enum": [
"image"
],
"type": "string"
}
},
"required": [
"data",
"mime",
"type"
],
"type": "object"
},
{
"description": "Reference to an original upload in the session-manager attachment store. A REFERENCE ONLY (never inline bytes); the harness strips it before `router::chat`. Tolerated here so a stray block deserializes instead of failing the request; every counter/mapping emits nothing for it.",
"properties": {
"attachment_id": {
"type": "string"
},
"mime": {
"type": "string"
},
"name": {
"type": "string"
},
"size": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": {
"enum": [
"file"
],
"type": "string"
}
},
"required": [
"attachment_id",
"mime",
"name",
"size",
"type"
],
"type": "object"
},
{
"properties": {
"signature": {
"type": [
"string",
"null"
]
},
"text": {
"type": "string"
},
"type": {
"enum": [
"thinking"
],
"type": "string"
}
},
"required": [
"text",
"type"
],
"type": "object"
},
{
"description": "Opaque redacted thinking payload — replayed verbatim on the Anthropic wire.",
"properties": {
"data": {
"type": "string"
},
"type": {
"enum": [
"redacted_thinking"
],
"type": "string"
}
},
"required": [
"data",
"type"
],
"type": "object"
},
{
"properties": {
"arguments": true,
"function_id": {
"type": "string"
},
"id": {
"type": "string"
},
"type": {
"enum": [
"function_call"
],
"type": "string"
}
},
"required": [
"arguments",
"function_id",
"id",
"type"
],
"type": "object"
},
{
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"function_call_id": {
"type": "string"
},
"is_error": {
"type": [
"boolean",
"null"
]
},
"type": {
"enum": [
"function_result"
],
"type": "string"
}
},
"required": [
"content",
"function_call_id",
"type"
],
"type": "object"
}
]
},
"CustomMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"custom_type": {
"type": "string"
},
"details": true,
"display": {
"type": [
"string",
"null"
]
},
"role": {
"$ref": "#/definitions/CustomRoleTag"
},
"timestamp": {
"format": "int64",
"type": "integer"
}
},
"required": [
"content",
"custom_type",
"role",
"timestamp"
],
"type": "object"
},
"CustomRoleTag": {
"enum": [
"custom"
],
"type": "string"
},
"ErrorKind": {
"enum": [
"auth_expired",
"rate_limited",
"context_overflow",
"transient",
"permanent"
],
"type": "string"
},
"FunctionResultMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"details": true,
"function_call_id": {
"type": "string"
},
"function_id": {
"type": "string"
},
"is_error": {
"type": "boolean"
},
"role": {
"$ref": "#/definitions/FunctionResultRoleTag"
},
"timestamp": {
"format": "int64",
"type": "integer"
}
},
"required": [
"content",
"details",
"function_call_id",
"function_id",
"is_error",
"role",
"timestamp"
],
"type": "object"
},
"FunctionResultRoleTag": {
"enum": [
"function_result"
],
"type": "string"
},
"StopReason": {
"enum": [
"end",
"length",
"function_call",
"aborted",
"error"
],
"type": "string"
},
"Usage": {
"properties": {
"cache_read": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"reasoning": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"UserMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"role": {
"$ref": "#/definitions/UserRoleTag"
},
"timestamp": {
"format": "int64",
"type": "integer"
}
},
"required": [
"content",
"role",
"timestamp"
],
"type": "object"
},
"UserRoleTag": {
"description": "Single-variant role tags: exact-match on deserialize, correct wire string on serialize, and they let `AgentMessage` be an untagged union.",
"enum": [
"user"
],
"type": "string"
}
},
"properties": {
"messages": {
"description": "Wire agent messages, the same shape `router::chat` accepts. Must be non-empty.",
"items": {
"$ref": "#/definitions/AgentMessage"
},
"type": "array"
},
"model": {
"default": null,
"description": "Model id the prompt targets; routes exactly like `router::chat` and selects the provider's tokenizer. Optional only when `provider` pins the destination.",
"type": [
"string",
"null"
]
},
"provider": {
"default": null,
"description": "Pin an explicit provider, bypassing heuristics (optional).",
"type": [
"string",
"null"
]
},
"system_prompt": {
"default": null,
"description": "System prompt counted as part of the request (optional).",
"type": [
"string",
"null"
]
},
"tools": {
"default": null,
"description": "Function invocation schemas the turn would carry; their serialized schemas count toward the total (optional).",
"items": {
"$ref": "#/definitions/AgentFunction"
},
"type": [
"array",
"null"
]
}
},
"required": [
"messages"
],
"title": "RouterCountTokensRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"estimator": {
"description": "`provider` when a provider metering API produced the count, `tiktoken` when a local tokenizer estimated it.",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"tokens": {
"description": "Prompt tokens the provider counted for the assembled request.",
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"estimator",
"model",
"provider",
"tokens"
],
"title": "RouterCountTokensResponse",
"type": "object"
}
},
{
"description": "Batch text embeddings through a provider's provider::<id>::embed surface. Names a provider or discovers the first embed-capable one from the live registry; one vector per input, order preserved.",
"metadata": {
"internal": true
},
"name": "router::embed",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"input": {
"description": "Texts to embed, one vector returned per input, order preserved.",
"items": {
"type": "string"
},
"type": "array"
},
"model": {
"default": null,
"description": "Embedding model id; the provider's default when omitted.",
"type": [
"string",
"null"
]
},
"provider": {
"default": null,
"description": "Provider id (e.g. `openai`); the first embed-capable provider in the live registry when omitted.",
"type": [
"string",
"null"
]
}
},
"required": [
"input"
],
"title": "RouterEmbedRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"embeddings": {
"description": "One embedding per input, in input order.",
"items": {
"items": {
"format": "float",
"type": "number"
},
"type": "array"
},
"type": "array"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
}
},
"required": [
"embeddings",
"model",
"provider"
],
"title": "RouterEmbedResponse",
"type": "object"
}
},
{
"description": "Resolve the effective model output budget using the same precedence as router::chat.",
"metadata": {
"internal": true
},
"name": "router::models::budget",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::models::budget`.",
"properties": {
"id": {
"default": "",
"description": "Model id to budget.",
"type": "string"
},
"max_output_tokens": {
"description": "Optional caller-requested output budget. When absent, the same provider and router defaults used by `router::chat` apply.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"provider": {
"default": "",
"description": "Provider id that owns the model. Empty resolves an unambiguous model id.",
"type": "string"
}
},
"title": "ModelBudgetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"anyOf": [
{
"$ref": "#/definitions/ModelBudgetResponse"
},
{
"type": "null"
}
],
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"reasoning_efforts": {
"items": {
"$ref": "#/definitions/ReasoningEffort"
},
"type": [
"array",
"null"
]
},
"speech": {
"anyOf": [
{
"$ref": "#/definitions/SpeechModel"
},
{
"type": "null"
}
],
"description": "Set on speech models only; see [`SpeechModel`]."
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"ModelBudgetResponse": {
"description": "Effective limits `router::chat` will use for this model and request.",
"properties": {
"effective_max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"model": {
"$ref": "#/definitions/Model"
}
},
"required": [
"effective_max_output_tokens",
"model"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"ReasoningEffort": {
"description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.",
"properties": {
"description": {
"type": [
"string",
"null"
]
},
"effort": {
"type": "string"
}
},
"required": [
"effort"
],
"type": "object"
},
"SpeechModality": {
"description": "What a speech model does. Chat models carry no `speech` block.",
"oneOf": [
{
"description": "Speech to text, served through `router::transcribe`.",
"enum": [
"stt"
],
"type": "string"
},
{
"description": "Text to speech, served through `router::speak`.",
"enum": [
"tts"
],
"type": "string"
}
]
},
"SpeechModel": {
"description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.",
"properties": {
"languages": {
"description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.",
"items": {
"type": "string"
},
"type": "array"
},
"modality": {
"$ref": "#/definitions/SpeechModality"
},
"streaming": {
"default": false,
"description": "Realtime input (stt) or streamed audio output (tts) is available.",
"type": "boolean"
}
},
"required": [
"modality"
],
"type": "object"
}
},
"title": "Nullable_ModelBudgetResponse"
}
},
{
"description": "Read one catalog model by {provider, id}; null when the model is not registered.",
"metadata": {},
"name": "router::models::get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::models::get`.",
"properties": {
"id": {
"default": "",
"description": "Model id to look up.",
"type": "string"
},
"provider": {
"default": "",
"description": "Provider id that owns the model.",
"type": "string"
}
},
"title": "ModelGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"anyOf": [
{
"$ref": "#/definitions/ModelGetResponse"
},
{
"type": "null"
}
],
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"reasoning_efforts": {
"items": {
"$ref": "#/definitions/ReasoningEffort"
},
"type": [
"array",
"null"
]
},
"speech": {
"anyOf": [
{
"$ref": "#/definitions/SpeechModel"
},
{
"type": "null"
}
],
"description": "Set on speech models only; see [`SpeechModel`]."
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"ModelGetResponse": {
"description": "Output of `router::models::get` (the function returns `null` when the model is not registered — the cold-window signal).",
"properties": {
"model": {
"$ref": "#/definitions/Model"
}
},
"required": [
"model"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"ReasoningEffort": {
"description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.",
"properties": {
"description": {
"type": [
"string",
"null"
]
},
"effort": {
"type": "string"
}
},
"required": [
"effort"
],
"type": "object"
},
"SpeechModality": {
"description": "What a speech model does. Chat models carry no `speech` block.",
"oneOf": [
{
"description": "Speech to text, served through `router::transcribe`.",
"enum": [
"stt"
],
"type": "string"
},
{
"description": "Text to speech, served through `router::speak`.",
"enum": [
"tts"
],
"type": "string"
}
]
},
"SpeechModel": {
"description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.",
"properties": {
"languages": {
"description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.",
"items": {
"type": "string"
},
"type": "array"
},
"modality": {
"$ref": "#/definitions/SpeechModality"
},
"streaming": {
"default": false,
"description": "Realtime input (stt) or streamed audio output (tts) is available.",
"type": "boolean"
}
},
"required": [
"modality"
],
"type": "object"
}
},
"title": "Nullable_ModelGetResponse"
}
},
{
"description": "List catalog models, optionally filtered by provider, a capability flag, and modality (chat by default; stt, tts, or any).",
"metadata": {},
"name": "router::models::list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ModalityFilter": {
"description": "Model family selector for `router::models::list`. `chat` is the default so pickers built before speech models existed keep listing only what `router::chat` can run.",
"enum": [
"chat",
"stt",
"tts",
"any"
],
"type": "string"
}
},
"description": "Input of `router::models::list`.",
"properties": {
"capability": {
"description": "Keep only models that support this capability flag (optional).",
"type": [
"string",
"null"
]
},
"modality": {
"anyOf": [
{
"$ref": "#/definitions/ModalityFilter"
},
{
"type": "null"
}
],
"description": "Model family: `chat` (default), `stt`, `tts`, or `any`."
},
"provider": {
"description": "Filter to a single provider id (optional).",
"type": [
"string",
"null"
]
}
},
"title": "ModelsListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"reasoning_efforts": {
"items": {
"$ref": "#/definitions/ReasoningEffort"
},
"type": [
"array",
"null"
]
},
"speech": {
"anyOf": [
{
"$ref": "#/definitions/SpeechModel"
},
{
"type": "null"
}
],
"description": "Set on speech models only; see [`SpeechModel`]."
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"ReasoningEffort": {
"description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.",
"properties": {
"description": {
"type": [
"string",
"null"
]
},
"effort": {
"type": "string"
}
},
"required": [
"effort"
],
"type": "object"
},
"SpeechModality": {
"description": "What a speech model does. Chat models carry no `speech` block.",
"oneOf": [
{
"description": "Speech to text, served through `router::transcribe`.",
"enum": [
"stt"
],
"type": "string"
},
{
"description": "Text to speech, served through `router::speak`.",
"enum": [
"tts"
],
"type": "string"
}
]
},
"SpeechModel": {
"description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.",
"properties": {
"languages": {
"description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.",
"items": {
"type": "string"
},
"type": "array"
},
"modality": {
"$ref": "#/definitions/SpeechModality"
},
"streaming": {
"default": false,
"description": "Realtime input (stt) or streamed audio output (tts) is available.",
"type": "boolean"
}
},
"required": [
"modality"
],
"type": "object"
}
},
"description": "Output of `router::models::list`.",
"properties": {
"models": {
"items": {
"$ref": "#/definitions/Model"
},
"type": "array"
}
},
"required": [
"models"
],
"title": "ModelsListResponse",
"type": "object"
}
},
{
"description": "Replace a provider's catalog slice — the only catalog write path (token-gated).",
"metadata": {
"internal": true
},
"name": "router::models::reconcile",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"reasoning_efforts": {
"items": {
"$ref": "#/definitions/ReasoningEffort"
},
"type": [
"array",
"null"
]
},
"speech": {
"anyOf": [
{
"$ref": "#/definitions/SpeechModel"
},
{
"type": "null"
}
],
"description": "Set on speech models only; see [`SpeechModel`]."
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"ReasoningEffort": {
"description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.",
"properties": {
"description": {
"type": [
"string",
"null"
]
},
"effort": {
"type": "string"
}
},
"required": [
"effort"
],
"type": "object"
},
"SpeechModality": {
"description": "What a speech model does. Chat models carry no `speech` block.",
"oneOf": [
{
"description": "Speech to text, served through `router::transcribe`.",
"enum": [
"stt"
],
"type": "string"
},
{
"description": "Text to speech, served through `router::speak`.",
"enum": [
"tts"
],
"type": "string"
}
]
},
"SpeechModel": {
"description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.",
"properties": {
"languages": {
"description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.",
"items": {
"type": "string"
},
"type": "array"
},
"modality": {
"$ref": "#/definitions/SpeechModality"
},
"streaming": {
"default": false,
"description": "Realtime input (stt) or streamed audio output (tts) is available.",
"type": "boolean"
}
},
"required": [
"modality"
],
"type": "object"
}
},
"description": "Input of `router::models::reconcile` — the only catalog write path.",
"properties": {
"models": {
"default": [],
"description": "The full replacement set of models for this provider.",
"items": {
"$ref": "#/definitions/Model"
},
"type": "array"
},
"provider": {
"default": "",
"description": "Provider whose catalog slice is being replaced.",
"type": "string"
},
"token": {
"description": "Registration token gating the write (optional).",
"type": [
"string",
"null"
]
}
},
"title": "ModelsReconcileRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of the `router::models::reconcile` iii function.",
"properties": {
"count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"provider": {
"type": "string"
}
},
"required": [
"count",
"provider"
],
"title": "ModelsReconcileResponse",
"type": "object"
}
},
{
"description": "Check whether a model supports a capability flag (fails open for unknown models).",
"metadata": {},
"name": "router::models::supports",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::models::supports`.",
"properties": {
"capability": {
"default": "",
"description": "Capability flag to check (e.g. `structured_output`, `vision`).",
"type": "string"
},
"id": {
"default": "",
"description": "Model id to check.",
"type": "string"
},
"provider": {
"default": "",
"description": "Provider id that owns the model.",
"type": "string"
}
},
"title": "ModelsSupportsRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of `router::models::supports`.",
"properties": {
"supported": {
"type": "boolean"
}
},
"required": [
"supported"
],
"title": "ModelsSupportsResponse",
"type": "object"
}
},
{
"description": "Internal: reactively reload the in-memory configuration snapshot and fan out provider model discovery after configuration changes.",
"metadata": {
"internal": true,
"trace_hidden": true
},
"name": "router::on_config_changed",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Advisory configuration-change event delivered to `router::on_config_changed`. The handler ignores event values and re-fetches the authoritative entry before replacing its in-memory snapshot.",
"properties": {
"id": {
"description": "Configuration id that changed (advisory).",
"type": [
"string",
"null"
]
}
},
"title": "ConfigChangedEvent",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Generic acknowledgement returned by trigger-bound handlers whose result is not consumed by callers (kept typed so the response schema is concrete).",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "RouterAck",
"type": "object"
}
},
{
"description": "Internal: a worker's function registrations changed — re-discover live providers and nudge them to re-declare, so a provider that reconnected is resolvable again without waiting for its own catalog timer.",
"metadata": {
"internal": true,
"trace_hidden": true
},
"name": "router::on_functions_changed",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Advisory function-registry change event delivered to `router::on_functions_changed`. The handler ignores event values and re-fetches the authoritative registry before nudging live providers.",
"properties": {
"event": {
"description": "Engine event tag (advisory).",
"type": [
"string",
"null"
]
},
"worker_id": {
"description": "Worker whose registered functions changed (advisory).",
"type": [
"string",
"null"
]
}
},
"title": "FunctionsChangedEvent",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Generic acknowledgement returned by trigger-bound handlers whose result is not consumed by callers (kept typed so the response schema is concrete).",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "RouterAck",
"type": "object"
}
},
{
"description": "List registered providers with their configured/available status.",
"metadata": {},
"name": "router::provider::list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::provider::list` — takes no arguments. A struct (rather than `Value`) keeps the request schema concrete; unknown fields (e.g. the engine-injected `_caller_worker_id`) are ignored.",
"title": "ProviderListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ProviderInfo": {
"properties": {
"available": {
"type": "boolean"
},
"configured": {
"type": "boolean"
},
"credential_env_var": {
"description": "Environment variable declared by API-key providers. `None` means the provider owns authentication (OAuth, local app login, device flow).",
"type": [
"string",
"null"
]
},
"display_name": {
"type": "string"
},
"icon_svg": {
"description": "The provider's mark as inline SVG, copied from its declaration. Absent when the provider declared none.",
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"supports_model_listing": {
"type": "boolean"
}
},
"required": [
"available",
"configured",
"display_name",
"id",
"supports_model_listing"
],
"type": "object"
}
},
"properties": {
"providers": {
"items": {
"$ref": "#/definitions/ProviderInfo"
},
"type": "array"
}
},
"required": [
"providers"
],
"title": "ProviderListResponse",
"type": "object"
}
},
{
"description": "Provider self-declaration at attach time (token-gated upsert); composes the configuration entry schema and reconciles static models.",
"metadata": {
"internal": true
},
"name": "router::provider::register",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"reasoning_efforts": {
"items": {
"$ref": "#/definitions/ReasoningEffort"
},
"type": [
"array",
"null"
]
},
"speech": {
"anyOf": [
{
"$ref": "#/definitions/SpeechModel"
},
{
"type": "null"
}
],
"description": "Set on speech models only; see [`SpeechModel`]."
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"ProviderDefaults": {
"additionalProperties": true,
"properties": {
"api_url": {
"type": [
"string",
"null"
]
},
"max_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"ReasoningEffort": {
"description": "One provider-native reasoning effort advertised for a specific model.\n\nValues intentionally remain strings: provider catalogs can add efforts without requiring a router-wide enum release first.",
"properties": {
"description": {
"type": [
"string",
"null"
]
},
"effort": {
"type": "string"
}
},
"required": [
"effort"
],
"type": "object"
},
"SpeechModality": {
"description": "What a speech model does. Chat models carry no `speech` block.",
"oneOf": [
{
"description": "Speech to text, served through `router::transcribe`.",
"enum": [
"stt"
],
"type": "string"
},
{
"description": "Text to speech, served through `router::speak`.",
"enum": [
"tts"
],
"type": "string"
}
]
},
"SpeechModel": {
"description": "Facts about a speech model. Present only on models served through `router::transcribe` / `router::speak`; such models report `context_window` and `max_output_tokens` as 0.",
"properties": {
"languages": {
"description": "BCP-47 tags of the languages the model handles; empty when the provider does not say.",
"items": {
"type": "string"
},
"type": "array"
},
"modality": {
"$ref": "#/definitions/SpeechModality"
},
"streaming": {
"default": false,
"description": "Realtime input (stt) or streamed audio output (tts) is available.",
"type": "boolean"
}
},
"required": [
"modality"
],
"type": "object"
}
},
"description": "Input of `router::provider::register` — a provider worker's declaration plus the optional re-registration token.",
"properties": {
"config_schema": true,
"credential_env_var": {
"type": [
"string",
"null"
]
},
"defaults": {
"anyOf": [
{
"$ref": "#/definitions/ProviderDefaults"
},
{
"type": "null"
}
]
},
"display_name": {
"type": [
"string",
"null"
]
},
"icon_svg": {
"description": "The provider's mark: one self-contained `<svg>` document, monochrome, drawn with a square `viewBox` and no fixed size. Consoles paint it as a `currentColor` mask next to the provider's models, so colour and scripts are ignored. Keep it under [`PROVIDER_ICON_SVG_MAX_BYTES`]; the router drops anything larger or not starting with `<svg`.",
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"models": {
"items": {
"$ref": "#/definitions/Model"
},
"type": [
"array",
"null"
]
},
"supports_model_listing": {
"type": [
"boolean",
"null"
]
},
"token": {
"description": "Registration token proving ownership on re-register (omit on first declare).",
"type": [
"string",
"null"
]
},
"worker_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"id"
],
"title": "ProviderRegisterRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "`registration_token` is the provider ownership credential; only its sha256 hash is persisted. Engine caller metadata is not an authorization identity because worker names are self-reported.",
"properties": {
"id": {
"type": "string"
},
"ok": {
"type": "boolean"
},
"registration_token": {
"type": "string"
}
},
"required": [
"id",
"ok",
"registration_token"
],
"title": "ProviderRegisterResponse",
"type": "object"
}
},
{
"description": "Resolve a provider's effective credential + api_url + max_tokens (token-gated).",
"metadata": {
"internal": true
},
"name": "router::provider::resolve",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::provider::resolve`.",
"properties": {
"id": {
"default": "",
"description": "Provider id to resolve credentials/config for.",
"type": "string"
},
"token": {
"description": "Registration token gating the resolve (optional).",
"type": [
"string",
"null"
]
}
},
"title": "ProviderResolveRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Credential": {
"oneOf": [
{
"properties": {
"key": {
"type": "string"
},
"type": {
"enum": [
"api_key"
],
"type": "string"
}
},
"required": [
"key",
"type"
],
"type": "object"
},
{
"properties": {
"access_token": {
"type": "string"
},
"expires_at": {
"format": "int64",
"type": [
"integer",
"null"
]
},
"provider_extra": true,
"refresh_token": {
"type": [
"string",
"null"
]
},
"scopes": {
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
},
"type": {
"enum": [
"oauth"
],
"type": "string"
}
},
"required": [
"access_token",
"type"
],
"type": "object"
}
]
},
"CredentialSource": {
"enum": [
"config",
"env",
"none"
],
"type": "string"
}
},
"description": "Output of the `router::provider::resolve` iii function.",
"properties": {
"api_url": {
"type": [
"string",
"null"
]
},
"configured": {
"type": "boolean"
},
"credential": {
"anyOf": [
{
"$ref": "#/definitions/Credential"
},
{
"type": "null"
}
]
},
"max_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"source": {
"$ref": "#/definitions/CredentialSource"
}
},
"required": [
"configured",
"source"
],
"title": "ProviderResolveResponse",
"type": "object"
}
},
{
"description": "OAuth write-back: store a provider credential in the configuration entry under the entry write lock (token-gated).",
"metadata": {
"internal": true
},
"name": "router::provider::update_credential",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::provider::update_credential` (OAuth write-back).",
"properties": {
"credential": {
"default": null,
"description": "The credential object to store (provider-specific shape)."
},
"id": {
"default": "",
"description": "Provider id whose credential slice is being written.",
"type": "string"
},
"token": {
"description": "Registration token gating the write (optional).",
"type": [
"string",
"null"
]
}
},
"title": "UpdateCredentialRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of `router::provider::update_credential`.",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "UpdateCredentialResponse",
"type": "object"
}
},
{
"description": "Read-only routing preview: resolve {model, provider?} to the chosen provider + ordered candidates without streaming.",
"metadata": {
"internal": true
},
"name": "router::route",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::route` — the read-only routing preview.",
"properties": {
"model": {
"default": "",
"description": "Model id to route.",
"type": "string"
},
"provider": {
"description": "Pin an explicit provider, bypassing heuristics (optional).",
"type": [
"string",
"null"
]
}
},
"title": "RouteRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of `router::route`.",
"properties": {
"candidates": {
"items": {
"type": "string"
},
"type": "array"
},
"provider": {
"type": "string"
}
},
"required": [
"candidates",
"provider"
],
"title": "RouteResponse",
"type": "object"
}
},
{
"description": "Text to speech through provider::<id>::speak: {model?, provider?, text, voice?, format?} to {provider, model, audio_base64, mime}. Resolves the provider from the catalog's tts models when none is named.",
"metadata": {
"internal": true
},
"name": "router::speak",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"format": {
"default": null,
"description": "Audio container wanted: `mp3` (default), `wav`, `pcm16`, `opus`. Providers answer with what they can and say so in `mime`.",
"type": [
"string",
"null"
]
},
"language": {
"default": null,
"description": "BCP-47 language hint for multilingual voices.",
"type": [
"string",
"null"
]
},
"model": {
"default": null,
"description": "Text-to-speech model id (a `tts` model from `router::models::list`); the provider's default when omitted.",
"type": [
"string",
"null"
]
},
"provider": {
"default": null,
"description": "Provider id; resolved from the model's catalog owner when omitted.",
"type": [
"string",
"null"
]
},
"speed": {
"default": null,
"description": "Speaking-rate multiplier; 1.0 is the voice's own pace.",
"format": "double",
"type": [
"number",
"null"
]
},
"text": {
"description": "Text to speak.",
"type": "string"
},
"voice": {
"default": null,
"description": "Voice id or name as the provider knows it; the provider's default when omitted.",
"type": [
"string",
"null"
]
}
},
"required": [
"text"
],
"title": "RouterSpeakRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"audio_base64": {
"description": "The audio, base64.",
"type": "string"
},
"duration_secs": {
"format": "double",
"type": [
"number",
"null"
]
},
"mime": {
"description": "MIME type of the audio, e.g. `audio/mpeg`.",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"voice": {
"description": "Voice the provider used.",
"type": [
"string",
"null"
]
}
},
"required": [
"audio_base64",
"mime",
"model",
"provider"
],
"title": "RouterSpeakResponse",
"type": "object"
}
},
{
"description": "Speech to text through provider::<id>::transcribe: {model?, provider?, audio_base64, mime?, language?} to {provider, model, text, segments?}. Resolves the provider from the catalog's stt models when none is named.",
"metadata": {
"internal": true
},
"name": "router::transcribe",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"audio_base64": {
"description": "The audio, base64. WAV unless `mime` says otherwise.",
"type": "string"
},
"language": {
"default": null,
"description": "BCP-47 language hint; the model detects the language when omitted.",
"type": [
"string",
"null"
]
},
"mime": {
"default": null,
"description": "MIME type of the audio: `audio/wav` (default), `audio/mpeg`, `audio/webm`, `audio/ogg`, `audio/flac`.",
"type": [
"string",
"null"
]
},
"model": {
"default": null,
"description": "Speech-to-text model id (an `stt` model from `router::models::list`); the provider's default when omitted.",
"type": [
"string",
"null"
]
},
"prompt": {
"default": null,
"description": "Vocabulary or context hint for the recognizer, when the provider takes one.",
"type": [
"string",
"null"
]
},
"provider": {
"default": null,
"description": "Provider id; resolved from the model's catalog owner when omitted.",
"type": [
"string",
"null"
]
}
},
"required": [
"audio_base64"
],
"title": "RouterTranscribeRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"TranscriptSegment": {
"description": "One timed span of a transcript; times are absent when the provider gives none.",
"properties": {
"end_secs": {
"format": "double",
"type": [
"number",
"null"
]
},
"start_secs": {
"format": "double",
"type": [
"number",
"null"
]
},
"text": {
"type": "string"
}
},
"required": [
"text"
],
"type": "object"
}
},
"properties": {
"duration_secs": {
"format": "double",
"type": [
"number",
"null"
]
},
"language": {
"description": "Language the provider detected or was told (BCP-47).",
"type": [
"string",
"null"
]
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"segments": {
"description": "Timed spans when the provider produces them.",
"items": {
"$ref": "#/definitions/TranscriptSegment"
},
"type": "array"
},
"text": {
"description": "The whole transcript.",
"type": "string"
}
},
"required": [
"model",
"provider",
"text"
],
"title": "RouterTranscribeResponse",
"type": "object"
}
}
],
"triggers": [
{
"description": "A provider reconciled its catalog slice. Payload: { provider, count }.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by all three router event bindings. No filters in v1.",
"title": "EventBindingConfig",
"type": "object"
},
"metadata": {},
"name": "router::models::changed",
"return_schema": {}
},
{
"description": "The provider registry changed (declare / availability flip). Payload: { provider, op }.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by all three router event bindings. No filters in v1.",
"title": "EventBindingConfig",
"type": "object"
},
"metadata": {},
"name": "router::provider::changed",
"return_schema": {}
},
{
"description": "The router finished booting; providers bind here and re-declare after a restart.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"description": "Config accepted by all three router event bindings. No filters in v1.",
"title": "EventBindingConfig",
"type": "object"
},
"metadata": {},
"name": "router::ready",
"return_schema": {}
}
]
}