$worker
llm-router
v1.0.0One front door + provider protocol in front of every LLM provider.
- macOS: arm64 · x64
- Linux: arm64 · armv7 · x64
- Windows: arm64 · x64 · x86
agent-ready brief for v1.0.0
install + config + dependencies + readme + api reference, all in one place. fetch as agent-context.md for an llm to consume.
full markdown
the same content rendered as discrete blocks below is exposed as a single markdown document at
/workers/llm-router.md. paste it into an llm prompt or pipe it through curl from a worker.install
install
$iii worker add llm-router@1.0.0
dependencies
readme
README.md
llm-router
One front door for every LLM provider. The router owns routing, the provider registry, credential resolution, the model catalog, streaming relay, retries, and a single failure contract — consumers call one chat surface and never talk to a provider directly.
llm-router is a standalone iii worker. Providers plug in as separate workers
at runtime through a self-registration protocol (iii worker add provider-); the router never compiles against a provider, and removing a
provider worker removes the provider.
Install
iii worker add llm-routerQuickstart
A consumer streams a turn by creating an iii channel, handing the router the
channel's write endpoint, and reading frames from the read endpoint
while router::chat runs. Any SDK works; Node shown:
import { createChannel } from 'iii-sdk';
const { reader, writerRef } = await createChannel(iii);
reader.onMessage((frame) => {
const event = JSON.parse(frame); // AssistantMessageEvent
if (event.type === 'text_delta') process.stdout.write(event.delta);
});
const res = await iii.trigger('router::chat', {
writer_ref: writerRef, // direction "write"
model: 'claude-sonnet-4',
messages: [{ role: 'user', content: [{ type: 'text', text: 'Hello' }], timestamp: Date.now() }],
}, { timeout_ms: 320_000 }); // outer timeout ≥ the router's 300s stream budget
// res: { ok, provider, model, stop_reason, usage }The streaming contract: every stream ends with exactly one terminal frame
(done or error). When the router has to kill a stream itself (idle
timeout, provider crash), it synthesizes the terminal frame and attaches the
partial content, so consumers never hang on a half-open stream.
Functions
Consumer surface
| Function | Purpose |
|---|---|
router::chat |
Stream a turn into the caller's channel; returns the turn summary. |
router::complete |
Non-streaming convenience over the same pipeline; returns the final message. |
router::abort |
Cancel an in-flight turn by request_id. |
router::route |
Read-only routing preview: {model, provider?} → {provider, candidates}, same rules and error codes as router::chat. Pin the result as the explicit provider on the chat call when you need the provider before streaming. |
router::models::list |
List catalog models, filterable by provider / capability. |
router::models::get |
Fetch one model record (null when unknown). |
router::models::supports |
Check one capability flag for one model. |
router::provider::list |
Registered providers with configured / available status. |
Agent exposure is restricted per iii-permissions.yaml to the read surface
(router::models::*, router::provider::list).
Provider protocol
Token-gated after the first declare: the response to register carries a
registration token, and every later protocol call must present it.
| Function | Purpose |
|---|---|
router::provider::register |
Self-declaration at attach time; idempotent re-declare with the token. |
router::provider::resolve |
Per-request credential + endpoint resolution (config > env > none). |
router::provider::update_credential |
Persist a refreshed credential (OAuth write-back). |
router::models::reconcile |
Replace the provider's catalog slice in one write. |
The provider worker itself exposes provider:: and, when it
supports model discovery, provider::.
Configuration
All operator configuration lives in the engine's llm-router configuration
entry — no env vars, no config file. The entry schema is composed at runtime
from each registered provider's declaration:
{
"default_provider": "anthropic",
"providers": {
"anthropic": { "api_key": "sk-…", "api_url": "https://api.anthropic.com/v1/messages", "max_tokens": 8192 }
},
"routing_heuristics": [{ "pattern": "^gpt-", "provider": "openai" }],
"settings": {
"stream_timeout_ms": 300000,
"idle_timeout_ms": 120000,
"retry_max": 2,
"output_token_max": 32000
}
}| Setting | Default | Meaning |
|---|---|---|
stream_timeout_ms |
300000 |
Hard budget for one streamed turn. |
idle_timeout_ms |
120000 |
Max silence between provider frames before the attempt is cut. |
retry_max |
2 |
Retries per turn for retryable failures before the first forwarded frame. |
output_token_max |
32000 |
Ceiling on max_output_tokens forwarded to providers. |
Pasting a key into a provider's slice is the whole onboarding flow: the
router diffs the changed slice, debounces ~2 s, and kicks that provider's
provider:: discovery; discovered models land in the
catalog via router::models::reconcile and show up in router::models::list
within seconds — no restart.
Operational notes
- Env-var credential fallback resolves in the router's process. A
provider's
credential_env_var(e.g.ANTHROPIC_API_KEY) is read by the llm-router binary, not by the provider worker — launch the router with those variables set, or put keys in the entry. A key present only in another worker's environment shows up asconfigured: false. - Registration-token recovery. Re-registering a provider id without its
original token is rejected (anti-takeover). If a provider durably lost its
token, delete the router's registry state (iii-state scope
llm-router, keyregistry) and restart the affected providers to re-bind; pasted credentials in the configuration entry are unaffected.
Events
The router registers three custom trigger types and fans out to every bound handler. Bind with the standard two-step pattern; the handler receives the payload verbatim (no envelope).
| Trigger type | Fires when | Payload |
|---|---|---|
router::models::changed |
a provider reconciles its catalog slice | { "provider": " |
router::provider::changed |
the registry changes (declare / availability flip) | { "provider": " |
router::ready |
the router finishes booting; providers re-declare on it | {} |
iii.registerFunction({ id: 'my-worker::onModelsChanged' }, async (payload) => {
console.log('catalog changed:', payload); // { provider, count }
return {};
});
iii.registerTrigger({
type: 'router::models::changed',
function_id: 'my-worker::onModelsChanged',
config: {},
});Writing a provider worker
A provider worker must:
- Register
provider::honouring the channel-writer contract: forward upstream output as::stream AssistantMessageEventframes into thewriter_refit receives, ending with one terminal frame. - Declare itself at startup via
router::provider::register— retrying with backoff until acknowledged (covers provider-before-router boot order) — and re-declare on therouter::readyevent after a router restart. - Resolve credentials per request via
router::provider::resolve; never read keys directly. - Treat closure of its stream channel as cancellation: abort the upstream request and stop writing frames.
- Map upstream failures to the shared
ErrorKindtaxonomy on itserrorframes. Transport retries (429 / 5xx / connect) are the router's job, not the provider's.
The first real provider implementing this protocol is
provider-anthropic/ — useful as a reference
implementation alongside the scripted provider in the integration tests.
provider-openai/ follows the same structure for the
OpenAI Chat Completions API (native structured output, reasoning_effort).
Local development & testing
cargo test # unit suite, no engine needed
cargo test --test integration # engine-backed suite; self-skips without an engineThe integration suite spawns a throwaway engine per test when iii is on
PATH (or III_ENGINE_BIN points at a binary) and covers the chat relay,
cancellation, abort, restart recovery, registration token gating, paste-a-key
discovery, and event delivery end to end.
To run the worker locally against an engine:
cargo run -- --url ws://127.0.0.1:49134--url defaults to ws://127.0.0.1:49134 and honours the III_WS_URL
environment variable when the flag is not set. --config is accepted per
the standard worker CLI but ignored with a warning — operator config lives
in the engine's llm-router configuration entry (see Configuration above).
api reference (json)
agent-api-reference.json
{
"functions": [
{
"description": "Abort an in-flight router::chat/complete by request_id; reports whether a live request was cancelled.",
"metadata": {},
"name": "router::abort",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of the `router::abort` iii function.",
"properties": {
"request_id": {
"type": "string"
}
},
"required": [
"request_id"
],
"title": "AbortRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"aborted": {
"type": "boolean"
}
},
"required": [
"aborted"
],
"title": "AbortResponse",
"type": "object"
}
},
{
"description": "Stream a chat completion: route {model, provider?} to a provider, relay assistant frames to writer_ref, and return the terminal response.",
"metadata": {},
"name": "router::chat",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ChannelDirection": {
"enum": [
"read",
"write"
],
"type": "string"
},
"StreamChannelRef": {
"properties": {
"access_key": {
"type": "string"
},
"channel_id": {
"type": "string"
},
"direction": {
"$ref": "#/definitions/ChannelDirection"
}
},
"required": [
"access_key",
"channel_id",
"direction"
],
"type": "object"
}
},
"description": "Input of the `router::chat` iii function: a [`ChatCall`] plus the caller's write channel. The handler relays assistant frames to `writer_ref` and also returns the terminal [`ChatResponse`].",
"properties": {
"max_output_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"messages": true,
"metadata": {
"default": null
},
"model": {
"type": "string"
},
"provider": {
"default": null,
"type": [
"string",
"null"
]
},
"provider_options": {
"default": null
},
"request_id": {
"default": null,
"type": [
"string",
"null"
]
},
"response_format": {
"default": null
},
"system_prompt": {
"default": null,
"type": [
"string",
"null"
]
},
"thinking_level": {
"default": null
},
"tools": {
"default": null
},
"writer_ref": {
"allOf": [
{
"$ref": "#/definitions/StreamChannelRef"
}
],
"description": "The caller's write channel (direction \"write\"); frames are relayed here."
}
},
"required": [
"messages",
"model",
"writer_ref"
],
"title": "ChatFnInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ErrorShape": {
"properties": {
"code": {
"type": "string"
},
"message": {
"type": "string"
}
},
"required": [
"code",
"message"
],
"type": "object"
},
"StopReason": {
"enum": [
"end",
"length",
"function_call",
"aborted",
"error"
],
"type": "string"
},
"Usage": {
"properties": {
"cache_read": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"reasoning": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
}
},
"properties": {
"error": {
"anyOf": [
{
"$ref": "#/definitions/ErrorShape"
},
{
"type": "null"
}
]
},
"model": {
"type": "string"
},
"ok": {
"type": "boolean"
},
"provider": {
"type": "string"
},
"stop_reason": {
"anyOf": [
{
"$ref": "#/definitions/StopReason"
},
{
"type": "null"
}
]
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
}
},
"required": [
"model",
"ok",
"provider"
],
"title": "ChatResponse",
"type": "object"
}
},
{
"description": "Non-streaming convenience over router::chat: run the turn on an internal channel and return the final assistant message + usage.",
"metadata": {},
"name": "router::complete",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "ChatRequest minus writer_ref (the sink arrives separately — the function handler wraps the looked-up writer, complete wraps its own).\n\n`messages` / `tools` / `response_format` / `thinking_level` / `provider_options` stay `Value`: they are forwarded to the provider verbatim, so the router intentionally does not re-validate their shape. The struct still derives `JsonSchema` so the SDK emits a real request schema (the freeform sub-fields surface as permissive sub-schemas).",
"properties": {
"max_output_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"messages": true,
"metadata": {
"default": null
},
"model": {
"type": "string"
},
"provider": {
"default": null,
"type": [
"string",
"null"
]
},
"provider_options": {
"default": null
},
"request_id": {
"default": null,
"type": [
"string",
"null"
]
},
"response_format": {
"default": null
},
"system_prompt": {
"default": null,
"type": [
"string",
"null"
]
},
"thinking_level": {
"default": null
},
"tools": {
"default": null
}
},
"required": [
"messages",
"model"
],
"title": "ChatCall",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"AssistantMessage": {
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"error_kind": {
"anyOf": [
{
"$ref": "#/definitions/ErrorKind"
},
{
"type": "null"
}
]
},
"error_message": {
"type": [
"string",
"null"
]
},
"model": {
"type": "string"
},
"native_stop_reason": {
"type": [
"string",
"null"
]
},
"provider": {
"type": "string"
},
"role": {
"$ref": "#/definitions/AssistantRoleTag"
},
"stop_reason": {
"$ref": "#/definitions/StopReason"
},
"timestamp": {
"format": "int64",
"type": "integer"
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
},
"warnings": {
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
}
},
"required": [
"content",
"model",
"provider",
"role",
"stop_reason",
"timestamp"
],
"type": "object"
},
"AssistantRoleTag": {
"enum": [
"assistant"
],
"type": "string"
},
"ContentBlock": {
"description": "Content blocks — the atomic units of message content (README § Content blocks).",
"oneOf": [
{
"properties": {
"text": {
"type": "string"
},
"type": {
"enum": [
"text"
],
"type": "string"
}
},
"required": [
"text",
"type"
],
"type": "object"
},
{
"properties": {
"data": {
"type": "string"
},
"mime": {
"type": "string"
},
"type": {
"enum": [
"image"
],
"type": "string"
}
},
"required": [
"data",
"mime",
"type"
],
"type": "object"
},
{
"properties": {
"signature": {
"type": [
"string",
"null"
]
},
"text": {
"type": "string"
},
"type": {
"enum": [
"thinking"
],
"type": "string"
}
},
"required": [
"text",
"type"
],
"type": "object"
},
{
"description": "Opaque redacted thinking payload — replayed verbatim on the Anthropic wire.",
"properties": {
"data": {
"type": "string"
},
"type": {
"enum": [
"redacted_thinking"
],
"type": "string"
}
},
"required": [
"data",
"type"
],
"type": "object"
},
{
"properties": {
"arguments": true,
"function_id": {
"type": "string"
},
"id": {
"type": "string"
},
"type": {
"enum": [
"function_call"
],
"type": "string"
}
},
"required": [
"arguments",
"function_id",
"id",
"type"
],
"type": "object"
},
{
"properties": {
"content": {
"items": {
"$ref": "#/definitions/ContentBlock"
},
"type": "array"
},
"function_call_id": {
"type": "string"
},
"is_error": {
"type": [
"boolean",
"null"
]
},
"type": {
"enum": [
"function_result"
],
"type": "string"
}
},
"required": [
"content",
"function_call_id",
"type"
],
"type": "object"
}
]
},
"ErrorKind": {
"enum": [
"auth_expired",
"rate_limited",
"context_overflow",
"transient",
"permanent"
],
"type": "string"
},
"StopReason": {
"enum": [
"end",
"length",
"function_call",
"aborted",
"error"
],
"type": "string"
},
"Usage": {
"properties": {
"cache_read": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"reasoning": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
}
},
"description": "Output of the `router::complete` iii function (non-streaming convenience).",
"properties": {
"message": {
"$ref": "#/definitions/AssistantMessage"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
}
},
"required": [
"message",
"model",
"provider"
],
"title": "CompleteResponse",
"type": "object"
}
},
{
"description": "Read one catalog model by {provider, id}; null when the model is not registered.",
"metadata": {},
"name": "router::models::get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::models::get`.",
"properties": {
"id": {
"default": "",
"description": "Model id to look up.",
"type": "string"
},
"provider": {
"default": "",
"description": "Provider id that owns the model.",
"type": "string"
}
},
"title": "ModelGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"anyOf": [
{
"$ref": "#/definitions/ModelGetResponse"
},
{
"type": "null"
}
],
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"ModelGetResponse": {
"description": "Output of `router::models::get` (the function returns `null` when the model is not registered — the cold-window signal).",
"properties": {
"model": {
"$ref": "#/definitions/Model"
}
},
"required": [
"model"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
}
},
"title": "Nullable_ModelGetResponse"
}
},
{
"description": "List catalog models, optionally filtered by provider and/or a capability flag.",
"metadata": {},
"name": "router::models::list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::models::list`.",
"properties": {
"capability": {
"description": "Keep only models that support this capability flag (optional).",
"type": [
"string",
"null"
]
},
"provider": {
"description": "Filter to a single provider id (optional).",
"type": [
"string",
"null"
]
}
},
"title": "ModelsListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
}
},
"description": "Output of `router::models::list`.",
"properties": {
"models": {
"items": {
"$ref": "#/definitions/Model"
},
"type": "array"
}
},
"required": [
"models"
],
"title": "ModelsListResponse",
"type": "object"
}
},
{
"description": "Replace a provider's catalog slice — the only catalog write path (token-gated).",
"metadata": {},
"name": "router::models::reconcile",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
}
},
"description": "Input of `router::models::reconcile` — the only catalog write path.",
"properties": {
"models": {
"default": [],
"description": "The full replacement set of models for this provider.",
"items": {
"$ref": "#/definitions/Model"
},
"type": "array"
},
"provider": {
"default": "",
"description": "Provider whose catalog slice is being replaced.",
"type": "string"
},
"token": {
"description": "Registration token gating the write (optional).",
"type": [
"string",
"null"
]
}
},
"title": "ModelsReconcileRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of the `router::models::reconcile` iii function.",
"properties": {
"count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"provider": {
"type": "string"
}
},
"required": [
"count",
"provider"
],
"title": "ModelsReconcileResponse",
"type": "object"
}
},
{
"description": "Check whether a model supports a capability flag (fails open for unknown models).",
"metadata": {},
"name": "router::models::supports",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::models::supports`.",
"properties": {
"capability": {
"default": "",
"description": "Capability flag to check (e.g. `structured_output`, `vision`).",
"type": "string"
},
"id": {
"default": "",
"description": "Model id to check.",
"type": "string"
},
"provider": {
"default": "",
"description": "Provider id that owns the model.",
"type": "string"
}
},
"title": "ModelsSupportsRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of `router::models::supports`.",
"properties": {
"supported": {
"type": "boolean"
}
},
"required": [
"supported"
],
"title": "ModelsSupportsResponse",
"type": "object"
}
},
{
"description": "Internal: configuration-change handler (paste-a-key) that refreshes settings and fans out provider model discovery.",
"metadata": {},
"name": "router::on_config_changed",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Configuration-change event delivered to `router::on_config_changed` via the engine's `configuration` trigger (paste-a-key flow).",
"properties": {
"id": {
"description": "Configuration id; the handler only acts on `llm-router`.",
"type": [
"string",
"null"
]
},
"new_value": {
"default": null,
"description": "The new authoritative configuration value."
}
},
"title": "ConfigChangedEvent",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Generic acknowledgement returned by trigger-bound handlers whose result is not consumed by callers (kept typed so the response schema is concrete).",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "RouterAck",
"type": "object"
}
},
{
"description": "List registered providers with their configured/available status.",
"metadata": {},
"name": "router::provider::list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::provider::list` — takes no arguments. A struct (rather than `Value`) keeps the request schema concrete; unknown fields (e.g. the engine-injected `_caller_worker_id`) are ignored.",
"title": "ProviderListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ProviderInfo": {
"properties": {
"available": {
"type": "boolean"
},
"configured": {
"type": "boolean"
},
"display_name": {
"type": "string"
},
"id": {
"type": "string"
},
"supports_model_listing": {
"type": "boolean"
}
},
"required": [
"available",
"configured",
"display_name",
"id",
"supports_model_listing"
],
"type": "object"
}
},
"properties": {
"providers": {
"items": {
"$ref": "#/definitions/ProviderInfo"
},
"type": "array"
}
},
"required": [
"providers"
],
"title": "ProviderListResponse",
"type": "object"
}
},
{
"description": "Provider self-declaration at attach time (token-gated upsert); composes the configuration entry schema and reconciles static models.",
"metadata": {},
"name": "router::provider::register",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Model": {
"description": "The capability record (README § Model descriptor).",
"properties": {
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"input_limit": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"pricing": {
"anyOf": [
{
"$ref": "#/definitions/Pricing"
},
{
"type": "null"
}
]
},
"provider": {
"type": "string"
},
"supports_cache": {
"type": [
"boolean",
"null"
]
},
"supports_structured_output": {
"type": [
"boolean",
"null"
]
},
"supports_thinking": {
"type": [
"boolean",
"null"
]
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
},
"supports_xhigh": {
"type": [
"boolean",
"null"
]
},
"thinking_budgets": {
"additionalProperties": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": [
"object",
"null"
]
}
},
"required": [
"context_window",
"id",
"max_output_tokens",
"provider"
],
"type": "object"
},
"Pricing": {
"properties": {
"cache_read": {
"format": "double",
"type": [
"number",
"null"
]
},
"cache_write": {
"format": "double",
"type": [
"number",
"null"
]
},
"input": {
"format": "double",
"type": [
"number",
"null"
]
},
"output": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"ProviderDefaults": {
"additionalProperties": true,
"properties": {
"api_url": {
"type": [
"string",
"null"
]
},
"max_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
}
},
"description": "Input of `router::provider::register` — a provider worker's declaration plus the optional re-registration token.",
"properties": {
"config_schema": true,
"credential_env_var": {
"type": [
"string",
"null"
]
},
"defaults": {
"anyOf": [
{
"$ref": "#/definitions/ProviderDefaults"
},
{
"type": "null"
}
]
},
"display_name": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"models": {
"items": {
"$ref": "#/definitions/Model"
},
"type": [
"array",
"null"
]
},
"supports_model_listing": {
"type": [
"boolean",
"null"
]
},
"token": {
"description": "Registration token proving ownership on re-register (omit on first declare).",
"type": [
"string",
"null"
]
},
"worker_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"id"
],
"title": "ProviderRegisterRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "registration_token: spec adaptation — the engine exposes no caller identity, so identity binding is a bearer token; only its sha256 hash is persisted.",
"properties": {
"id": {
"type": "string"
},
"ok": {
"type": "boolean"
},
"registration_token": {
"type": "string"
}
},
"required": [
"id",
"ok",
"registration_token"
],
"title": "ProviderRegisterResponse",
"type": "object"
}
},
{
"description": "Resolve a provider's effective credential + api_url + max_tokens (token-gated).",
"metadata": {},
"name": "router::provider::resolve",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::provider::resolve`.",
"properties": {
"id": {
"default": "",
"description": "Provider id to resolve credentials/config for.",
"type": "string"
},
"token": {
"description": "Registration token gating the resolve (optional).",
"type": [
"string",
"null"
]
}
},
"title": "ProviderResolveRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Credential": {
"oneOf": [
{
"properties": {
"key": {
"type": "string"
},
"type": {
"enum": [
"api_key"
],
"type": "string"
}
},
"required": [
"key",
"type"
],
"type": "object"
},
{
"properties": {
"access_token": {
"type": "string"
},
"expires_at": {
"format": "int64",
"type": [
"integer",
"null"
]
},
"provider_extra": true,
"refresh_token": {
"type": [
"string",
"null"
]
},
"scopes": {
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
},
"type": {
"enum": [
"oauth"
],
"type": "string"
}
},
"required": [
"access_token",
"type"
],
"type": "object"
}
]
},
"CredentialSource": {
"enum": [
"config",
"env",
"none"
],
"type": "string"
}
},
"description": "Output of the `router::provider::resolve` iii function.",
"properties": {
"api_url": {
"type": [
"string",
"null"
]
},
"configured": {
"type": "boolean"
},
"credential": {
"anyOf": [
{
"$ref": "#/definitions/Credential"
},
{
"type": "null"
}
]
},
"max_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"source": {
"$ref": "#/definitions/CredentialSource"
}
},
"required": [
"configured",
"source"
],
"title": "ProviderResolveResponse",
"type": "object"
}
},
{
"description": "OAuth write-back: store a provider credential in the configuration entry under the entry write lock (token-gated).",
"metadata": {},
"name": "router::provider::update_credential",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::provider::update_credential` (OAuth write-back).",
"properties": {
"credential": {
"default": null,
"description": "The credential object to store (provider-specific shape)."
},
"id": {
"default": "",
"description": "Provider id whose credential slice is being written.",
"type": "string"
},
"token": {
"description": "Registration token gating the write (optional).",
"type": [
"string",
"null"
]
}
},
"title": "UpdateCredentialRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of `router::provider::update_credential`.",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "UpdateCredentialResponse",
"type": "object"
}
},
{
"description": "Read-only routing preview: resolve {model, provider?} to the chosen provider + ordered candidates without streaming.",
"metadata": {},
"name": "router::route",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of `router::route` — the read-only routing preview.",
"properties": {
"model": {
"default": "",
"description": "Model id to route.",
"type": "string"
},
"provider": {
"description": "Pin an explicit provider, bypassing heuristics (optional).",
"type": [
"string",
"null"
]
}
},
"title": "RouteRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of `router::route`.",
"properties": {
"candidates": {
"items": {
"type": "string"
},
"type": "array"
},
"provider": {
"type": "string"
}
},
"required": [
"candidates",
"provider"
],
"title": "RouteResponse",
"type": "object"
}
}
],
"triggers": [
{
"description": "A provider reconciled its catalog slice. Payload: { provider, count }.",
"invocation_schema": {},
"metadata": {},
"name": "router::models::changed",
"return_schema": {}
},
{
"description": "The provider registry changed (declare / availability flip). Payload: { provider, op }.",
"invocation_schema": {},
"metadata": {},
"name": "router::provider::changed",
"return_schema": {}
},
{
"description": "The router finished booting; providers bind here and re-declare after a restart.",
"invocation_schema": {},
"metadata": {},
"name": "router::ready",
"return_schema": {}
}
]
}