$worker

llm-router

v0.2.4

One front door + provider protocol in front of every LLM provider.

macOS: arm64 · x64
Linux: arm64 · armv7 · x64
Windows: arm64 · x64 · x86

functions

router::abort

function

Abort an in-flight router::chat/complete by request_id; reports whether a live request was cancelled.

request

request_idstringrequired

response

abortedbooleanrequired

router::chat

function

Stream a chat completion: route {model, provider?} to a provider, relay assistant frames to writer_ref, and return the terminal response.

request

max_output_tokensinteger· uint64min 0
messagesunknownrequired
metadataunknown
modelstringrequired
providerstring
provider_optionsunknown
request_idstring
response_formatunknown
system_promptstring
thinking_levelunknown
toolsunknown
writer_refall ofrequired
The caller's write channel (direction "write"); frames are relayed here.
all of (1)
variant 1
access_keystringrequired
channel_idstringrequired
directionstringrequiredenum: read, write

response

errorany of
any of (2)
variant 1
codestringrequired
messagestringrequired
variant 2
valuenull
modelstringrequired
okbooleanrequired
providerstringrequired
stop_reasonany of
any of (2)
variant 1
valuestringenum: end, length, function_call, aborted, error
variant 2
valuenull
usageany of
any of (2)
variant 1
cache_readinteger· uint64min 0
cache_writeinteger· uint64min 0
cost_usdnumber· double
inputinteger· uint64min 0
outputinteger· uint64min 0
reasoninginteger· uint64min 0
variant 2
valuenull

router::complete

function

Non-streaming convenience over router::chat: run the turn on an internal channel and return the final assistant message + usage.

request

max_output_tokensinteger· uint64min 0
messagesunknownrequired
metadataunknown
modelstringrequired
providerstring
provider_optionsunknown
request_idstring
response_formatunknown
system_promptstring
thinking_levelunknown
toolsunknown

response

messageobjectrequired
- contentone of[]required
  one of (6)
  variant 1
  textstringrequired
  typestringrequiredenum: text
  variant 2
  datastringrequired
  mimestringrequired
  typestringrequiredenum: image
  variant 3
  signaturestring
  textstringrequired
  typestringrequiredenum: thinking
  variant 4
  datastringrequired
  typestringrequiredenum: redacted_thinking
  variant 5
  argumentsunknownrequired
  function_idstringrequired
  idstringrequired
  typestringrequiredenum: function_call
  variant 6
  contentunknown[]required
  function_call_idstringrequired
  is_errorboolean
  typestringrequiredenum: function_result
- error_kindany of
  any of (2)
  variant 1
  valuestringenum: auth_expired, rate_limited, context_overflow, transient, permanent
  variant 2
  valuenull
- error_messagestring
- modelstringrequired
- native_stop_reasonstring
- providerstringrequired
- rolestringrequiredenum: assistant
- stop_reasonstringrequiredenum: end, length, function_call, aborted, error
- timestampinteger· int64required
- usageany of
  any of (2)
  variant 1
  cache_readinteger· uint64min 0
  cache_writeinteger· uint64min 0
  cost_usdnumber· double
  inputinteger· uint64min 0
  outputinteger· uint64min 0
  reasoninginteger· uint64min 0
  variant 2
  valuenull
- warningsstring[]
modelstringrequired
providerstringrequired
usageany of
any of (2)
variant 1
cache_readinteger· uint64min 0
cache_writeinteger· uint64min 0
cost_usdnumber· double
inputinteger· uint64min 0
outputinteger· uint64min 0
reasoninginteger· uint64min 0
variant 2
valuenull

router::models::get

function

Read one catalog model by {provider, id}; null when the model is not registered.

request

idstring
Model id to look up.
providerstring
Provider id that owns the model.

response

any of (2)

variant 1

modelobjectrequired
The capability record (README § Model descriptor).
- context_windowinteger· uint64requiredmin 0
- display_namestring
- idstringrequired
- input_limitinteger· uint64min 0
- max_output_tokensinteger· uint64requiredmin 0
- pricingany of
  … expand 2 nestedcollapse
  any of (2)
  variant 1
  cache_readnumber· double
  cache_writenumber· double
  inputnumber· double
  outputnumber· double
  variant 2
  valuenull
- providerstringrequired
- supports_cacheboolean
- supports_structured_outputboolean
- supports_thinkingboolean
- supports_toolsboolean
- supports_visionboolean
- supports_xhighboolean
- thinking_budgetsobject

variant 2

valuenull

router::models::list

function

List catalog models, optionally filtered by provider and/or a capability flag.

request

capabilitystring
Keep only models that support this capability flag (optional).
providerstring
Filter to a single provider id (optional).

response

modelsobject[]required
- context_windowinteger· uint64requiredmin 0
- display_namestring
- idstringrequired
- input_limitinteger· uint64min 0
- max_output_tokensinteger· uint64requiredmin 0
- pricingany of
  any of (2)
  variant 1
  cache_readnumber· double
  cache_writenumber· double
  inputnumber· double
  outputnumber· double
  variant 2
  valuenull
- providerstringrequired
- supports_cacheboolean
- supports_structured_outputboolean
- supports_thinkingboolean
- supports_toolsboolean
- supports_visionboolean
- supports_xhighboolean
- thinking_budgetsobject

router::models::reconcile

function

Replace a provider's catalog slice — the only catalog write path (token-gated).

request

modelsobject[]
The full replacement set of models for this provider.
- context_windowinteger· uint64requiredmin 0
- display_namestring
- idstringrequired
- input_limitinteger· uint64min 0
- max_output_tokensinteger· uint64requiredmin 0
- pricingany of
  any of (2)
  variant 1
  cache_readnumber· double
  cache_writenumber· double
  inputnumber· double
  outputnumber· double
  variant 2
  valuenull
- providerstringrequired
- supports_cacheboolean
- supports_structured_outputboolean
- supports_thinkingboolean
- supports_toolsboolean
- supports_visionboolean
- supports_xhighboolean
- thinking_budgetsobject
providerstring
Provider whose catalog slice is being replaced.
tokenstring
Registration token gating the write (optional).

response

countinteger· uintrequiredmin 0
providerstringrequired

router::models::supports

function

Check whether a model supports a capability flag (fails open for unknown models).

request

capabilitystring
Capability flag to check (e.g. `structured_output`, `vision`).
idstring
Model id to check.
providerstring
Provider id that owns the model.

response

supportedbooleanrequired

router::on_config_changed

function

Internal: configuration-change handler (paste-a-key) that refreshes settings and fans out provider model discovery.

request

idstring
Configuration id; the handler only acts on `llm-router`.
new_valueunknown
The new authoritative configuration value.

response

okbooleanrequired

router::on_worker_available

function

Internal: engine::workers-available subscriber that flips provider availability on connect/disconnect.

request

eventstring
Event name; one containing "disconnect" marks the worker(s) down.
worker_idstring
The worker id the event concerns (absent shapes are ignored).

response

okbooleanrequired

router::provider::list

function

List registered providers with their configured/available status.

request

empty object

response

providersobject[]required
- availablebooleanrequired
- configuredbooleanrequired
- display_namestringrequired
- idstringrequired
- supports_model_listingbooleanrequired

router::provider::register

function

Provider self-declaration at attach time (token-gated upsert); composes the configuration entry schema and reconciles static models.

request

config_schemaunknown
credential_env_varstring
defaultsany of
any of (2)
variant 1
api_urlstring
max_tokensinteger· uint64min 0
variant 2
valuenull
display_namestring
idstringrequired
modelsobject[]
- context_windowinteger· uint64requiredmin 0
- display_namestring
- idstringrequired
- input_limitinteger· uint64min 0
- max_output_tokensinteger· uint64requiredmin 0
- pricingany of
  any of (2)
  variant 1
  cache_readnumber· double
  cache_writenumber· double
  inputnumber· double
  outputnumber· double
  variant 2
  valuenull
- providerstringrequired
- supports_cacheboolean
- supports_structured_outputboolean
- supports_thinkingboolean
- supports_toolsboolean
- supports_visionboolean
- supports_xhighboolean
- thinking_budgetsobject
supports_model_listingboolean
tokenstring
Registration token proving ownership on re-register (omit on first declare).
worker_idstring

response

idstringrequired
okbooleanrequired
registration_tokenstringrequired

router::provider::resolve

function

Resolve a provider's effective credential + api_url + max_tokens (token-gated).

request

idstring
Provider id to resolve credentials/config for.
tokenstring
Registration token gating the resolve (optional).

response

api_urlstring
configuredbooleanrequired
credentialany of
any of (2)
variant 1
one of (2)
variant 1
keystringrequired
typestringrequiredenum: api_key
variant 2
access_tokenstringrequired
expires_atinteger· int64
provider_extraunknown
refresh_tokenstring
scopesstring[]
typestringrequiredenum: oauth
variant 2
valuenull
max_tokensinteger· uint64min 0
sourcestringrequiredenum: config, env, none

router::provider::update_credential

function

OAuth write-back: store a provider credential in the configuration entry under the entry write lock (token-gated).

request

credentialunknown
The credential object to store (provider-specific shape).
idstring
Provider id whose credential slice is being written.
tokenstring
Registration token gating the write (optional).

response

okbooleanrequired

router::route

function

Read-only routing preview: resolve {model, provider?} to the chosen provider + ordered candidates without streaming.

request

modelstring
Model id to route.
providerstring
Pin an explicit provider, bypassing heuristics (optional).

response

candidatesstring[]required
providerstringrequired

triggers

no triggers registered