# llm-router

> One front door + provider protocol in front of every LLM provider.

| field | value |
|-------|-------|
| version | 1.0.0-rc.1 |
| type | binary |
| repo | https://github.com/iii-hq/workers |
| supported_targets | x86_64-apple-darwin, aarch64-apple-darwin, i686-pc-windows-msvc, x86_64-pc-windows-msvc, aarch64-pc-windows-msvc, x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-unknown-linux-musl, armv7-unknown-linux-gnueabihf |
| author | iii |

## installation

```sh
iii worker add llm-router@1.0.0-rc.1
```

## dependencies

- `iii-state` @ `^0.19.0`
- `configuration` @ `^0.19.0`

## readme

# llm-router

One front door for every LLM provider. The router owns routing, the provider
registry, credential resolution, the model catalog, streaming relay, retries,
and a single failure contract — consumers call one chat surface and never talk
to a provider directly.

llm-router is a standalone iii worker. Providers plug in as separate workers
at runtime through a self-registration protocol (`iii worker add
provider-<x>`); the router never compiles against a provider, and removing a
provider worker removes the provider.

## Install

```bash
iii worker add llm-router
```

## Quickstart

A consumer streams a turn by creating an iii channel, handing the router the
channel's **write** endpoint, and reading frames from the **read** endpoint
while `router::chat` runs. Any SDK works; Node shown:

```ts
import { createChannel } from 'iii-sdk';

const { reader, writerRef } = await createChannel(iii);
reader.onMessage((frame) => {
  const event = JSON.parse(frame); // AssistantMessageEvent
  if (event.type === 'text_delta') process.stdout.write(event.delta);
});

const res = await iii.trigger('router::chat', {
  writer_ref: writerRef, // direction "write"
  model: 'claude-sonnet-4',
  messages: [{ role: 'user', content: [{ type: 'text', text: 'Hello' }], timestamp: Date.now() }],
}, { timeout_ms: 320_000 }); // outer timeout ≥ the router's 300s stream budget
// res: { ok, provider, model, stop_reason, usage }
```

The streaming contract: every stream ends with exactly one terminal frame
(`done` or `error`). When the router has to kill a stream itself (idle
timeout, provider crash), it synthesizes the terminal frame and attaches the
partial content, so consumers never hang on a half-open stream.

## Functions

### Consumer surface

| Function | Purpose |
|---|---|
| `router::chat` | Stream a turn into the caller's channel; returns the turn summary. |
| `router::complete` | Non-streaming convenience over the same pipeline; returns the final message. |
| `router::abort` | Cancel an in-flight turn by `request_id`. |
| `router::route` | Read-only routing preview: `{model, provider?}` → `{provider, candidates}`, same rules and error codes as `router::chat`. Pin the result as the explicit `provider` on the chat call when you need the provider before streaming. |
| `router::models::list` | List catalog models, filterable by `provider` / `capability`. |
| `router::models::get` | Fetch one model record (`null` when unknown). |
| `router::models::supports` | Check one capability flag for one model. |
| `router::provider::list` | Registered providers with `configured` / `available` status. |

Agent exposure is restricted per `iii-permissions.yaml` to the read surface
(`router::models::*`, `router::provider::list`).

### Provider protocol

Token-gated after the first declare: the response to `register` carries a
registration token, and every later protocol call must present it.

| Function | Purpose |
|---|---|
| `router::provider::register` | Self-declaration at attach time; idempotent re-declare with the token. |
| `router::provider::resolve` | Per-request credential + endpoint resolution (config > env > none). |
| `router::provider::update_credential` | Persist a refreshed credential (OAuth write-back). |
| `router::models::reconcile` | Replace the provider's catalog slice in one write. |

The provider worker itself exposes `provider::<id>::stream` and, when it
supports model discovery, `provider::<id>::refresh_models`.

## Configuration

All operator configuration lives in the engine's `llm-router` configuration
entry — no env vars, no config file. The entry schema is composed at runtime
from each registered provider's declaration:

```json
{
  "default_provider": "anthropic",
  "providers": {
    "anthropic": { "api_key": "sk-…", "api_url": "https://api.anthropic.com/v1/messages", "max_tokens": 8192 }
  },
  "routing_heuristics": [{ "pattern": "^gpt-", "provider": "openai" }],
  "settings": {
    "stream_timeout_ms": 300000,
    "idle_timeout_ms": 120000,
    "retry_max": 2,
    "output_token_max": 32000
  }
}
```

| Setting | Default | Meaning |
|---|---|---|
| `stream_timeout_ms` | `300000` | Hard budget for one streamed turn. |
| `idle_timeout_ms` | `120000` | Max silence between provider frames before the attempt is cut. |
| `retry_max` | `2` | Retries per turn for retryable failures before the first forwarded frame. |
| `output_token_max` | `32000` | Ceiling on `max_output_tokens` forwarded to providers. |

Pasting a key into a provider's slice is the whole onboarding flow: the
router diffs the changed slice, debounces ~2 s, and kicks that provider's
`provider::<id>::refresh_models` discovery; discovered models land in the
catalog via `router::models::reconcile` and show up in `router::models::list`
within seconds — no restart.

### Operational notes

- **Env-var credential fallback resolves in the router's process.** A
  provider's `credential_env_var` (e.g. `ANTHROPIC_API_KEY`) is read by the
  llm-router binary, not by the provider worker — launch the router with
  those variables set, or put keys in the entry. A key present only in
  another worker's environment shows up as `configured: false`.
- **Registration-token recovery.** Re-registering a provider id without its
  original token is rejected (anti-takeover). If a provider durably lost its
  token, delete the router's registry state (iii-state scope `llm-router`,
  key `registry`) and restart the affected providers to re-bind; pasted
  credentials in the configuration entry are unaffected.

## Events

The router registers three custom trigger types and fans out to every bound
handler. Bind with the standard two-step pattern; the handler receives the
payload verbatim (no envelope).

| Trigger type | Fires when | Payload |
|---|---|---|
| `router::models::changed` | a provider reconciles its catalog slice | `{ "provider": "<id>", "count": <n> }` |
| `router::provider::changed` | the registry changes (declare / availability flip) | `{ "provider": "<id>", "op": "register" \| "available" \| "unavailable" }` |
| `router::ready` | the router finishes booting; providers re-declare on it | `{}` |

```ts
iii.registerFunction({ id: 'my-worker::onModelsChanged' }, async (payload) => {
  console.log('catalog changed:', payload); // { provider, count }
  return {};
});

iii.registerTrigger({
  type: 'router::models::changed',
  function_id: 'my-worker::onModelsChanged',
  config: {},
});
```

## Writing a provider worker

A provider worker must:

1. Register `provider::<id>::stream` honouring the channel-writer contract:
   forward upstream output as `AssistantMessageEvent` frames into the
   `writer_ref` it receives, ending with one terminal frame.
2. Declare itself at startup via `router::provider::register` — retrying with
   backoff until acknowledged (covers provider-before-router boot order) —
   and re-declare on the `router::ready` event after a router restart.
3. Resolve credentials per request via `router::provider::resolve`; never
   read keys directly.
4. Treat closure of its stream channel as cancellation: abort the upstream
   request and stop writing frames.
5. Map upstream failures to the shared `ErrorKind` taxonomy on its `error`
   frames. Transport retries (429 / 5xx / connect) are the router's job, not
   the provider's.

The first real provider implementing this protocol is
[`provider-anthropic/`](../provider-anthropic/) — useful as a reference
implementation alongside the scripted provider in the integration tests.
[`provider-openai/`](../provider-openai/) follows the same structure for the
OpenAI Chat Completions API (native structured output, reasoning_effort).

## Local development & testing

```bash
cargo test                       # unit suite, no engine needed
cargo test --test integration    # engine-backed suite; self-skips without an engine
```

The integration suite spawns a throwaway engine per test when `iii` is on
`PATH` (or `III_ENGINE_BIN` points at a binary) and covers the chat relay,
cancellation, abort, restart recovery, registration token gating, paste-a-key
discovery, and event delivery end to end.

To run the worker locally against an engine:

```bash
cargo run -- --url ws://127.0.0.1:49134
```

`--url` defaults to `ws://127.0.0.1:49134` and honours the `III_WS_URL`
environment variable when the flag is not set. `--config` is accepted per
the standard worker CLI but ignored with a warning — operator config lives
in the engine's `llm-router` configuration entry (see Configuration above).

## api reference

```json
{
  "functions": [
    {
      "description": "Abort an in-flight router::chat/complete by request_id; reports whether a live request was cancelled.",
      "metadata": {},
      "name": "router::abort",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of the `router::abort` iii function.",
        "properties": {
          "request_id": {
            "type": "string"
          }
        },
        "required": [
          "request_id"
        ],
        "title": "AbortRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "aborted": {
            "type": "boolean"
          }
        },
        "required": [
          "aborted"
        ],
        "title": "AbortResponse",
        "type": "object"
      }
    },
    {
      "description": "Stream a chat completion: route {model, provider?} to a provider, relay assistant frames to writer_ref, and return the terminal response.",
      "metadata": {},
      "name": "router::chat",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "ChannelDirection": {
            "enum": [
              "read",
              "write"
            ],
            "type": "string"
          },
          "StreamChannelRef": {
            "properties": {
              "access_key": {
                "type": "string"
              },
              "channel_id": {
                "type": "string"
              },
              "direction": {
                "$ref": "#/definitions/ChannelDirection"
              }
            },
            "required": [
              "access_key",
              "channel_id",
              "direction"
            ],
            "type": "object"
          }
        },
        "description": "Input of the `router::chat` iii function: a [`ChatCall`] plus the caller's write channel. The handler relays assistant frames to `writer_ref` and also returns the terminal [`ChatResponse`].",
        "properties": {
          "max_output_tokens": {
            "default": null,
            "format": "uint64",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          },
          "messages": true,
          "metadata": {
            "default": null
          },
          "model": {
            "type": "string"
          },
          "provider": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "provider_options": {
            "default": null
          },
          "request_id": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "response_format": {
            "default": null
          },
          "system_prompt": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "thinking_level": {
            "default": null
          },
          "tools": {
            "default": null
          },
          "writer_ref": {
            "allOf": [
              {
                "$ref": "#/definitions/StreamChannelRef"
              }
            ],
            "description": "The caller's write channel (direction \"write\"); frames are relayed here."
          }
        },
        "required": [
          "messages",
          "model",
          "writer_ref"
        ],
        "title": "ChatFnInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "ErrorShape": {
            "properties": {
              "code": {
                "type": "string"
              },
              "message": {
                "type": "string"
              }
            },
            "required": [
              "code",
              "message"
            ],
            "type": "object"
          },
          "StopReason": {
            "enum": [
              "end",
              "length",
              "function_call",
              "aborted",
              "error"
            ],
            "type": "string"
          },
          "Usage": {
            "properties": {
              "cache_read": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cache_write": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "input": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "output": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "reasoning": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          }
        },
        "properties": {
          "error": {
            "anyOf": [
              {
                "$ref": "#/definitions/ErrorShape"
              },
              {
                "type": "null"
              }
            ]
          },
          "model": {
            "type": "string"
          },
          "ok": {
            "type": "boolean"
          },
          "provider": {
            "type": "string"
          },
          "stop_reason": {
            "anyOf": [
              {
                "$ref": "#/definitions/StopReason"
              },
              {
                "type": "null"
              }
            ]
          },
          "usage": {
            "anyOf": [
              {
                "$ref": "#/definitions/Usage"
              },
              {
                "type": "null"
              }
            ]
          }
        },
        "required": [
          "model",
          "ok",
          "provider"
        ],
        "title": "ChatResponse",
        "type": "object"
      }
    },
    {
      "description": "Non-streaming convenience over router::chat: run the turn on an internal channel and return the final assistant message + usage.",
      "metadata": {},
      "name": "router::complete",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "ChatRequest minus writer_ref (the sink arrives separately — the function handler wraps the looked-up writer, complete wraps its own).\n\n`messages` / `tools` / `response_format` / `thinking_level` / `provider_options` stay `Value`: they are forwarded to the provider verbatim, so the router intentionally does not re-validate their shape. The struct still derives `JsonSchema` so the SDK emits a real request schema (the freeform sub-fields surface as permissive sub-schemas).",
        "properties": {
          "max_output_tokens": {
            "default": null,
            "format": "uint64",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          },
          "messages": true,
          "metadata": {
            "default": null
          },
          "model": {
            "type": "string"
          },
          "provider": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "provider_options": {
            "default": null
          },
          "request_id": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "response_format": {
            "default": null
          },
          "system_prompt": {
            "default": null,
            "type": [
              "string",
              "null"
            ]
          },
          "thinking_level": {
            "default": null
          },
          "tools": {
            "default": null
          }
        },
        "required": [
          "messages",
          "model"
        ],
        "title": "ChatCall",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "AssistantMessage": {
            "properties": {
              "content": {
                "items": {
                  "$ref": "#/definitions/ContentBlock"
                },
                "type": "array"
              },
              "error_kind": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ErrorKind"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "error_message": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "model": {
                "type": "string"
              },
              "native_stop_reason": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "provider": {
                "type": "string"
              },
              "role": {
                "$ref": "#/definitions/AssistantRoleTag"
              },
              "stop_reason": {
                "$ref": "#/definitions/StopReason"
              },
              "timestamp": {
                "format": "int64",
                "type": "integer"
              },
              "usage": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/Usage"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "warnings": {
                "items": {
                  "type": "string"
                },
                "type": [
                  "array",
                  "null"
                ]
              }
            },
            "required": [
              "content",
              "model",
              "provider",
              "role",
              "stop_reason",
              "timestamp"
            ],
            "type": "object"
          },
          "AssistantRoleTag": {
            "enum": [
              "assistant"
            ],
            "type": "string"
          },
          "ContentBlock": {
            "description": "Content blocks — the atomic units of message content (README § Content blocks).",
            "oneOf": [
              {
                "properties": {
                  "text": {
                    "type": "string"
                  },
                  "type": {
                    "enum": [
                      "text"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "text",
                  "type"
                ],
                "type": "object"
              },
              {
                "properties": {
                  "data": {
                    "type": "string"
                  },
                  "mime": {
                    "type": "string"
                  },
                  "type": {
                    "enum": [
                      "image"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "data",
                  "mime",
                  "type"
                ],
                "type": "object"
              },
              {
                "properties": {
                  "signature": {
                    "type": [
                      "string",
                      "null"
                    ]
                  },
                  "text": {
                    "type": "string"
                  },
                  "type": {
                    "enum": [
                      "thinking"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "text",
                  "type"
                ],
                "type": "object"
              },
              {
                "description": "Opaque redacted thinking payload — replayed verbatim on the Anthropic wire.",
                "properties": {
                  "data": {
                    "type": "string"
                  },
                  "type": {
                    "enum": [
                      "redacted_thinking"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "data",
                  "type"
                ],
                "type": "object"
              },
              {
                "properties": {
                  "arguments": true,
                  "function_id": {
                    "type": "string"
                  },
                  "id": {
                    "type": "string"
                  },
                  "type": {
                    "enum": [
                      "function_call"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "arguments",
                  "function_id",
                  "id",
                  "type"
                ],
                "type": "object"
              },
              {
                "properties": {
                  "content": {
                    "items": {
                      "$ref": "#/definitions/ContentBlock"
                    },
                    "type": "array"
                  },
                  "function_call_id": {
                    "type": "string"
                  },
                  "is_error": {
                    "type": [
                      "boolean",
                      "null"
                    ]
                  },
                  "type": {
                    "enum": [
                      "function_result"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "content",
                  "function_call_id",
                  "type"
                ],
                "type": "object"
              }
            ]
          },
          "ErrorKind": {
            "enum": [
              "auth_expired",
              "rate_limited",
              "context_overflow",
              "transient",
              "permanent"
            ],
            "type": "string"
          },
          "StopReason": {
            "enum": [
              "end",
              "length",
              "function_call",
              "aborted",
              "error"
            ],
            "type": "string"
          },
          "Usage": {
            "properties": {
              "cache_read": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cache_write": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "cost_usd": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "input": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "output": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "reasoning": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          }
        },
        "description": "Output of the `router::complete` iii function (non-streaming convenience).",
        "properties": {
          "message": {
            "$ref": "#/definitions/AssistantMessage"
          },
          "model": {
            "type": "string"
          },
          "provider": {
            "type": "string"
          },
          "usage": {
            "anyOf": [
              {
                "$ref": "#/definitions/Usage"
              },
              {
                "type": "null"
              }
            ]
          }
        },
        "required": [
          "message",
          "model",
          "provider"
        ],
        "title": "CompleteResponse",
        "type": "object"
      }
    },
    {
      "description": "Read one catalog model by {provider, id}; null when the model is not registered.",
      "metadata": {},
      "name": "router::models::get",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of `router::models::get`.",
        "properties": {
          "id": {
            "default": "",
            "description": "Model id to look up.",
            "type": "string"
          },
          "provider": {
            "default": "",
            "description": "Provider id that owns the model.",
            "type": "string"
          }
        },
        "title": "ModelGetRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "anyOf": [
          {
            "$ref": "#/definitions/ModelGetResponse"
          },
          {
            "type": "null"
          }
        ],
        "definitions": {
          "Model": {
            "description": "The capability record (README § Model descriptor).",
            "properties": {
              "context_window": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "display_name": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "id": {
                "type": "string"
              },
              "input_limit": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "max_output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "pricing": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/Pricing"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "provider": {
                "type": "string"
              },
              "supports_cache": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_structured_output": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_thinking": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_tools": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_vision": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_xhigh": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "thinking_budgets": {
                "additionalProperties": {
                  "format": "uint64",
                  "minimum": 0,
                  "type": "integer"
                },
                "type": [
                  "object",
                  "null"
                ]
              }
            },
            "required": [
              "context_window",
              "id",
              "max_output_tokens",
              "provider"
            ],
            "type": "object"
          },
          "ModelGetResponse": {
            "description": "Output of `router::models::get` (the function returns `null` when the model is not registered — the cold-window signal).",
            "properties": {
              "model": {
                "$ref": "#/definitions/Model"
              }
            },
            "required": [
              "model"
            ],
            "type": "object"
          },
          "Pricing": {
            "properties": {
              "cache_read": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "cache_write": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "input": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "output": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "type": "object"
          }
        },
        "title": "Nullable_ModelGetResponse"
      }
    },
    {
      "description": "List catalog models, optionally filtered by provider and/or a capability flag.",
      "metadata": {},
      "name": "router::models::list",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of `router::models::list`.",
        "properties": {
          "capability": {
            "description": "Keep only models that support this capability flag (optional).",
            "type": [
              "string",
              "null"
            ]
          },
          "provider": {
            "description": "Filter to a single provider id (optional).",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "ModelsListRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "Model": {
            "description": "The capability record (README § Model descriptor).",
            "properties": {
              "context_window": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "display_name": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "id": {
                "type": "string"
              },
              "input_limit": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "max_output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "pricing": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/Pricing"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "provider": {
                "type": "string"
              },
              "supports_cache": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_structured_output": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_thinking": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_tools": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_vision": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_xhigh": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "thinking_budgets": {
                "additionalProperties": {
                  "format": "uint64",
                  "minimum": 0,
                  "type": "integer"
                },
                "type": [
                  "object",
                  "null"
                ]
              }
            },
            "required": [
              "context_window",
              "id",
              "max_output_tokens",
              "provider"
            ],
            "type": "object"
          },
          "Pricing": {
            "properties": {
              "cache_read": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "cache_write": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "input": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "output": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "type": "object"
          }
        },
        "description": "Output of `router::models::list`.",
        "properties": {
          "models": {
            "items": {
              "$ref": "#/definitions/Model"
            },
            "type": "array"
          }
        },
        "required": [
          "models"
        ],
        "title": "ModelsListResponse",
        "type": "object"
      }
    },
    {
      "description": "Replace a provider's catalog slice — the only catalog write path (token-gated).",
      "metadata": {},
      "name": "router::models::reconcile",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "Model": {
            "description": "The capability record (README § Model descriptor).",
            "properties": {
              "context_window": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "display_name": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "id": {
                "type": "string"
              },
              "input_limit": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "max_output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "pricing": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/Pricing"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "provider": {
                "type": "string"
              },
              "supports_cache": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_structured_output": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_thinking": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_tools": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_vision": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_xhigh": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "thinking_budgets": {
                "additionalProperties": {
                  "format": "uint64",
                  "minimum": 0,
                  "type": "integer"
                },
                "type": [
                  "object",
                  "null"
                ]
              }
            },
            "required": [
              "context_window",
              "id",
              "max_output_tokens",
              "provider"
            ],
            "type": "object"
          },
          "Pricing": {
            "properties": {
              "cache_read": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "cache_write": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "input": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "output": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "type": "object"
          }
        },
        "description": "Input of `router::models::reconcile` — the only catalog write path.",
        "properties": {
          "models": {
            "default": [],
            "description": "The full replacement set of models for this provider.",
            "items": {
              "$ref": "#/definitions/Model"
            },
            "type": "array"
          },
          "provider": {
            "default": "",
            "description": "Provider whose catalog slice is being replaced.",
            "type": "string"
          },
          "token": {
            "description": "Registration token gating the write (optional).",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "ModelsReconcileRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Output of the `router::models::reconcile` iii function.",
        "properties": {
          "count": {
            "format": "uint",
            "minimum": 0,
            "type": "integer"
          },
          "provider": {
            "type": "string"
          }
        },
        "required": [
          "count",
          "provider"
        ],
        "title": "ModelsReconcileResponse",
        "type": "object"
      }
    },
    {
      "description": "Check whether a model supports a capability flag (fails open for unknown models).",
      "metadata": {},
      "name": "router::models::supports",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of `router::models::supports`.",
        "properties": {
          "capability": {
            "default": "",
            "description": "Capability flag to check (e.g. `structured_output`, `vision`).",
            "type": "string"
          },
          "id": {
            "default": "",
            "description": "Model id to check.",
            "type": "string"
          },
          "provider": {
            "default": "",
            "description": "Provider id that owns the model.",
            "type": "string"
          }
        },
        "title": "ModelsSupportsRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Output of `router::models::supports`.",
        "properties": {
          "supported": {
            "type": "boolean"
          }
        },
        "required": [
          "supported"
        ],
        "title": "ModelsSupportsResponse",
        "type": "object"
      }
    },
    {
      "description": "Internal: configuration-change handler (paste-a-key) that refreshes settings and fans out provider model discovery.",
      "metadata": {},
      "name": "router::on_config_changed",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Configuration-change event delivered to `router::on_config_changed` via the engine's `configuration` trigger (paste-a-key flow).",
        "properties": {
          "id": {
            "description": "Configuration id; the handler only acts on `llm-router`.",
            "type": [
              "string",
              "null"
            ]
          },
          "new_value": {
            "default": null,
            "description": "The new authoritative configuration value."
          }
        },
        "title": "ConfigChangedEvent",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Generic acknowledgement returned by trigger-bound handlers whose result is not consumed by callers (kept typed so the response schema is concrete).",
        "properties": {
          "ok": {
            "type": "boolean"
          }
        },
        "required": [
          "ok"
        ],
        "title": "RouterAck",
        "type": "object"
      }
    },
    {
      "description": "List registered providers with their configured/available status.",
      "metadata": {},
      "name": "router::provider::list",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of `router::provider::list` — takes no arguments. A struct (rather than `Value`) keeps the request schema concrete; unknown fields (e.g. the engine-injected `_caller_worker_id`) are ignored.",
        "title": "ProviderListRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "ProviderInfo": {
            "properties": {
              "available": {
                "type": "boolean"
              },
              "configured": {
                "type": "boolean"
              },
              "display_name": {
                "type": "string"
              },
              "id": {
                "type": "string"
              },
              "supports_model_listing": {
                "type": "boolean"
              }
            },
            "required": [
              "available",
              "configured",
              "display_name",
              "id",
              "supports_model_listing"
            ],
            "type": "object"
          }
        },
        "properties": {
          "providers": {
            "items": {
              "$ref": "#/definitions/ProviderInfo"
            },
            "type": "array"
          }
        },
        "required": [
          "providers"
        ],
        "title": "ProviderListResponse",
        "type": "object"
      }
    },
    {
      "description": "Provider self-declaration at attach time (token-gated upsert); composes the configuration entry schema and reconciles static models.",
      "metadata": {},
      "name": "router::provider::register",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "Model": {
            "description": "The capability record (README § Model descriptor).",
            "properties": {
              "context_window": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "display_name": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "id": {
                "type": "string"
              },
              "input_limit": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "max_output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "pricing": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/Pricing"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "provider": {
                "type": "string"
              },
              "supports_cache": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_structured_output": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_thinking": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_tools": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_vision": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "supports_xhigh": {
                "type": [
                  "boolean",
                  "null"
                ]
              },
              "thinking_budgets": {
                "additionalProperties": {
                  "format": "uint64",
                  "minimum": 0,
                  "type": "integer"
                },
                "type": [
                  "object",
                  "null"
                ]
              }
            },
            "required": [
              "context_window",
              "id",
              "max_output_tokens",
              "provider"
            ],
            "type": "object"
          },
          "Pricing": {
            "properties": {
              "cache_read": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "cache_write": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "input": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              },
              "output": {
                "format": "double",
                "type": [
                  "number",
                  "null"
                ]
              }
            },
            "type": "object"
          },
          "ProviderDefaults": {
            "additionalProperties": true,
            "properties": {
              "api_url": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "max_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          }
        },
        "description": "Input of `router::provider::register` — a provider worker's declaration plus the optional re-registration token.",
        "properties": {
          "config_schema": true,
          "credential_env_var": {
            "type": [
              "string",
              "null"
            ]
          },
          "defaults": {
            "anyOf": [
              {
                "$ref": "#/definitions/ProviderDefaults"
              },
              {
                "type": "null"
              }
            ]
          },
          "display_name": {
            "type": [
              "string",
              "null"
            ]
          },
          "id": {
            "type": "string"
          },
          "models": {
            "items": {
              "$ref": "#/definitions/Model"
            },
            "type": [
              "array",
              "null"
            ]
          },
          "supports_model_listing": {
            "type": [
              "boolean",
              "null"
            ]
          },
          "token": {
            "description": "Registration token proving ownership on re-register (omit on first declare).",
            "type": [
              "string",
              "null"
            ]
          },
          "worker_id": {
            "type": [
              "string",
              "null"
            ]
          }
        },
        "required": [
          "id"
        ],
        "title": "ProviderRegisterRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "registration_token: spec adaptation — the engine exposes no caller identity, so identity binding is a bearer token; only its sha256 hash is persisted.",
        "properties": {
          "id": {
            "type": "string"
          },
          "ok": {
            "type": "boolean"
          },
          "registration_token": {
            "type": "string"
          }
        },
        "required": [
          "id",
          "ok",
          "registration_token"
        ],
        "title": "ProviderRegisterResponse",
        "type": "object"
      }
    },
    {
      "description": "Resolve a provider's effective credential + api_url + max_tokens (token-gated).",
      "metadata": {},
      "name": "router::provider::resolve",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of `router::provider::resolve`.",
        "properties": {
          "id": {
            "default": "",
            "description": "Provider id to resolve credentials/config for.",
            "type": "string"
          },
          "token": {
            "description": "Registration token gating the resolve (optional).",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "ProviderResolveRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "Credential": {
            "oneOf": [
              {
                "properties": {
                  "key": {
                    "type": "string"
                  },
                  "type": {
                    "enum": [
                      "api_key"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "key",
                  "type"
                ],
                "type": "object"
              },
              {
                "properties": {
                  "access_token": {
                    "type": "string"
                  },
                  "expires_at": {
                    "format": "int64",
                    "type": [
                      "integer",
                      "null"
                    ]
                  },
                  "provider_extra": true,
                  "refresh_token": {
                    "type": [
                      "string",
                      "null"
                    ]
                  },
                  "scopes": {
                    "items": {
                      "type": "string"
                    },
                    "type": [
                      "array",
                      "null"
                    ]
                  },
                  "type": {
                    "enum": [
                      "oauth"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "access_token",
                  "type"
                ],
                "type": "object"
              }
            ]
          },
          "CredentialSource": {
            "enum": [
              "config",
              "env",
              "none"
            ],
            "type": "string"
          }
        },
        "description": "Output of the `router::provider::resolve` iii function.",
        "properties": {
          "api_url": {
            "type": [
              "string",
              "null"
            ]
          },
          "configured": {
            "type": "boolean"
          },
          "credential": {
            "anyOf": [
              {
                "$ref": "#/definitions/Credential"
              },
              {
                "type": "null"
              }
            ]
          },
          "max_tokens": {
            "format": "uint64",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          },
          "source": {
            "$ref": "#/definitions/CredentialSource"
          }
        },
        "required": [
          "configured",
          "source"
        ],
        "title": "ProviderResolveResponse",
        "type": "object"
      }
    },
    {
      "description": "OAuth write-back: store a provider credential in the configuration entry under the entry write lock (token-gated).",
      "metadata": {},
      "name": "router::provider::update_credential",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of `router::provider::update_credential` (OAuth write-back).",
        "properties": {
          "credential": {
            "default": null,
            "description": "The credential object to store (provider-specific shape)."
          },
          "id": {
            "default": "",
            "description": "Provider id whose credential slice is being written.",
            "type": "string"
          },
          "token": {
            "description": "Registration token gating the write (optional).",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "UpdateCredentialRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Output of `router::provider::update_credential`.",
        "properties": {
          "ok": {
            "type": "boolean"
          }
        },
        "required": [
          "ok"
        ],
        "title": "UpdateCredentialResponse",
        "type": "object"
      }
    },
    {
      "description": "Read-only routing preview: resolve {model, provider?} to the chosen provider + ordered candidates without streaming.",
      "metadata": {},
      "name": "router::route",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of `router::route` — the read-only routing preview.",
        "properties": {
          "model": {
            "default": "",
            "description": "Model id to route.",
            "type": "string"
          },
          "provider": {
            "description": "Pin an explicit provider, bypassing heuristics (optional).",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "RouteRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Output of `router::route`.",
        "properties": {
          "candidates": {
            "items": {
              "type": "string"
            },
            "type": "array"
          },
          "provider": {
            "type": "string"
          }
        },
        "required": [
          "candidates",
          "provider"
        ],
        "title": "RouteResponse",
        "type": "object"
      }
    }
  ],
  "triggers": [
    {
      "description": "A provider reconciled its catalog slice. Payload: { provider, count }.",
      "invocation_schema": {},
      "metadata": {},
      "name": "router::models::changed",
      "return_schema": {}
    },
    {
      "description": "The provider registry changed (declare / availability flip). Payload: { provider, op }.",
      "invocation_schema": {},
      "metadata": {},
      "name": "router::provider::changed",
      "return_schema": {}
    },
    {
      "description": "The router finished booting; providers bind here and re-declare after a restart.",
      "invocation_schema": {},
      "metadata": {},
      "name": "router::ready",
      "return_schema": {}
    }
  ]
}
```