$worker

llm-router

Name: llm-router
Author: iii

v1.1.3

One front door + provider protocol in front of every LLM provider.

iiiverified

631 installs172 in 7d1 today

install

$iii worker add llm-router@1.1.3

binaryllm router models providers

macOS: arm64 · x64
Linux: arm64 · armv7 · x64
Windows: arm64 · x64 · x86

16functions 3triggers 0skills 2deps v1.1.3version

repoiii-hq/workers

exact versions are immutable; binary and bundle artifacts are digest-pinned.

functions

router::abort

function

Abort an in-flight router::chat/complete by request_id; reports whether a live request was cancelled.

request

request_idstringrequired

response

abortedbooleanrequired

router::chat

function

Stream a chat completion: route {model, provider?} to a provider, relay assistant frames to writer_ref, and return the terminal response.

request

max_output_tokensinteger· uint64min 0
messagesunknownrequired
metadataunknown
modelstringrequired
providerstring
provider_optionsunknown
request_idstring
response_formatunknown
system_promptstring
thinking_levelunknown
toolsunknown
writer_refall ofrequired
The caller's write channel (direction "write"); frames are relayed here.
all of (1)
variant 1
access_keystringrequired
channel_idstringrequired
directionstringrequiredenum: read, write

response

errorany of
any of (2)
variant 1
codestringrequired
messagestringrequired
variant 2
valuenull
modelstringrequired
okbooleanrequired
providerstringrequired
stop_reasonany of
any of (2)
variant 1
valuestringenum: end, length, function_call, aborted, error
variant 2
valuenull
usageany of
any of (2)
variant 1
cache_readinteger· uint64min 0
cache_writeinteger· uint64min 0
cost_usdnumber· double
inputinteger· uint64min 0
outputinteger· uint64min 0
reasoninginteger· uint64min 0
variant 2
valuenull

router::complete

function

Non-streaming convenience over router::chat: run the turn on an internal channel and return the final assistant message + usage.

request

max_output_tokensinteger· uint64min 0
messagesunknownrequired
metadataunknown
modelstringrequired
providerstring
provider_optionsunknown
request_idstring
response_formatunknown
system_promptstring
thinking_levelunknown
toolsunknown

response

messageobjectrequired
- contentone of[]required
  one of (6)
  variant 1
  textstringrequired
  typestringrequiredenum: text
  variant 2
  datastringrequired
  mimestringrequired
  typestringrequiredenum: image
  variant 3
  signaturestring
  textstringrequired
  typestringrequiredenum: thinking
  variant 4
  datastringrequired
  typestringrequiredenum: redacted_thinking
  variant 5
  argumentsunknownrequired
  function_idstringrequired
  idstringrequired
  typestringrequiredenum: function_call
  variant 6
  contentunknown[]required
  function_call_idstringrequired
  is_errorboolean
  typestringrequiredenum: function_result
- error_kindany of
  any of (2)
  variant 1
  valuestringenum: auth_expired, rate_limited, context_overflow, transient, permanent
  variant 2
  valuenull
- error_messagestring
- modelstringrequired
- native_stop_reasonstring
- providerstringrequired
- rolestringrequiredenum: assistant
- stop_reasonstringrequiredenum: end, length, function_call, aborted, error
- timestampinteger· int64required
- usageany of
  any of (2)
  variant 1
  cache_readinteger· uint64min 0
  cache_writeinteger· uint64min 0
  cost_usdnumber· double
  inputinteger· uint64min 0
  outputinteger· uint64min 0
  reasoninginteger· uint64min 0
  variant 2
  valuenull
- warningsstring[]
modelstringrequired
providerstringrequired
usageany of
any of (2)
variant 1
cache_readinteger· uint64min 0
cache_writeinteger· uint64min 0
cost_usdnumber· double
inputinteger· uint64min 0
outputinteger· uint64min 0
reasoninginteger· uint64min 0
variant 2
valuenull

router::embed

function

Batch text embeddings through a provider's provider::<id>::embed surface. Names a provider or discovers the first embed-capable one from the live registry; one vector per input, order preserved.

request

inputstring[]required
Texts to embed, one vector returned per input, order preserved.
modelstring
Embedding model id; the provider's default when omitted.
providerstring
Provider id (e.g. `openai`); the first embed-capable provider in the live registry when omitted.

response

embeddingsnumber[][]required
One embedding per input, in input order.
modelstringrequired
providerstringrequired

router::models::budget

function

Resolve the effective model output budget using the same precedence as router::chat.

request

idstring
Model id to budget.
max_output_tokensinteger· uint64min 0
Optional caller-requested output budget. When absent, the same provider and router defaults used by `router::chat` apply.
providerstring
Provider id that owns the model. Empty resolves an unambiguous model id.

response

any of (2)

variant 1

effective_max_output_tokensinteger· uint64requiredmin 0
modelobjectrequired
The capability record (README § Model descriptor).
- context_windowinteger· uint64requiredmin 0
- display_namestring
- idstringrequired
- input_limitinteger· uint64min 0
- max_output_tokensinteger· uint64requiredmin 0
- pricingany of
  … expand 2 nestedcollapse
  any of (2)
  variant 1
  cache_readnumber· double
  cache_writenumber· double
  inputnumber· double
  outputnumber· double
  variant 2
  valuenull
- providerstringrequired
- reasoning_effortsobject[]
  … expand 2 nestedcollapse
  descriptionstring
  effortstringrequired
- supports_cacheboolean
- supports_structured_outputboolean
- supports_thinkingboolean
- supports_toolsboolean
- supports_visionboolean
- supports_xhighboolean
- thinking_budgetsobject

variant 2

valuenull

router::models::get

function

Read one catalog model by {provider, id}; null when the model is not registered.

request

idstring
Model id to look up.
providerstring
Provider id that owns the model.

response

any of (2)

variant 1

modelobjectrequired
The capability record (README § Model descriptor).
- context_windowinteger· uint64requiredmin 0
- display_namestring
- idstringrequired
- input_limitinteger· uint64min 0
- max_output_tokensinteger· uint64requiredmin 0
- pricingany of
  … expand 2 nestedcollapse
  any of (2)
  variant 1
  cache_readnumber· double
  cache_writenumber· double
  inputnumber· double
  outputnumber· double
  variant 2
  valuenull
- providerstringrequired
- reasoning_effortsobject[]
  … expand 2 nestedcollapse
  descriptionstring
  effortstringrequired
- supports_cacheboolean
- supports_structured_outputboolean
- supports_thinkingboolean
- supports_toolsboolean
- supports_visionboolean
- supports_xhighboolean
- thinking_budgetsobject

variant 2

valuenull

router::models::list

function

List catalog models, optionally filtered by provider and/or a capability flag.

request

capabilitystring
Keep only models that support this capability flag (optional).
providerstring
Filter to a single provider id (optional).

response

modelsobject[]required
- context_windowinteger· uint64requiredmin 0
- display_namestring
- idstringrequired
- input_limitinteger· uint64min 0
- max_output_tokensinteger· uint64requiredmin 0
- pricingany of
  any of (2)
  variant 1
  cache_readnumber· double
  cache_writenumber· double
  inputnumber· double
  outputnumber· double
  variant 2
  valuenull
- providerstringrequired
- reasoning_effortsobject[]
  descriptionstring
  effortstringrequired
- supports_cacheboolean
- supports_structured_outputboolean
- supports_thinkingboolean
- supports_toolsboolean
- supports_visionboolean
- supports_xhighboolean
- thinking_budgetsobject

router::models::reconcile

function

Replace a provider's catalog slice — the only catalog write path (token-gated).

request

modelsobject[]
The full replacement set of models for this provider.
- context_windowinteger· uint64requiredmin 0
- display_namestring
- idstringrequired
- input_limitinteger· uint64min 0
- max_output_tokensinteger· uint64requiredmin 0
- pricingany of
  any of (2)
  variant 1
  cache_readnumber· double
  cache_writenumber· double
  inputnumber· double
  outputnumber· double
  variant 2
  valuenull
- providerstringrequired
- reasoning_effortsobject[]
  descriptionstring
  effortstringrequired
- supports_cacheboolean
- supports_structured_outputboolean
- supports_thinkingboolean
- supports_toolsboolean
- supports_visionboolean
- supports_xhighboolean
- thinking_budgetsobject
providerstring
Provider whose catalog slice is being replaced.
tokenstring
Registration token gating the write (optional).

response

countinteger· uintrequiredmin 0
providerstringrequired

router::models::supports

function

Check whether a model supports a capability flag (fails open for unknown models).

request

capabilitystring
Capability flag to check (e.g. `structured_output`, `vision`).
idstring
Model id to check.
providerstring
Provider id that owns the model.

response

supportedbooleanrequired

router::on_config_changed

function

Internal: reactively reload the in-memory configuration snapshot and fan out provider model discovery after configuration changes.

request

idstring
Configuration id that changed (advisory).

response

okbooleanrequired

router::provider::list

function

List registered providers with their configured/available status.

request

empty object

response

providersobject[]required
- availablebooleanrequired
- configuredbooleanrequired
- display_namestringrequired
- idstringrequired
- supports_model_listingbooleanrequired

router::provider::register

function

Provider self-declaration at attach time (token-gated upsert); composes the configuration entry schema and reconciles static models.

request

config_schemaunknown
credential_env_varstring
defaultsany of
any of (2)
variant 1
api_urlstring
max_tokensinteger· uint64min 0
variant 2
valuenull
display_namestring
idstringrequired
modelsobject[]
- context_windowinteger· uint64requiredmin 0
- display_namestring
- idstringrequired
- input_limitinteger· uint64min 0
- max_output_tokensinteger· uint64requiredmin 0
- pricingany of
  any of (2)
  variant 1
  cache_readnumber· double
  cache_writenumber· double
  inputnumber· double
  outputnumber· double
  variant 2
  valuenull
- providerstringrequired
- reasoning_effortsobject[]
  descriptionstring
  effortstringrequired
- supports_cacheboolean
- supports_structured_outputboolean
- supports_thinkingboolean
- supports_toolsboolean
- supports_visionboolean
- supports_xhighboolean
- thinking_budgetsobject
supports_model_listingboolean
system_promptstring
tokenstring
Registration token proving ownership on re-register (omit on first declare).
worker_idstring

response

idstringrequired
okbooleanrequired
registration_tokenstringrequired

router::provider::resolve

function

Resolve a provider's effective credential + api_url + max_tokens (token-gated).

request

idstring
Provider id to resolve credentials/config for.
tokenstring
Registration token gating the resolve (optional).

response

api_urlstring
configuredbooleanrequired
credentialany of
any of (2)
variant 1
one of (2)
variant 1
keystringrequired
typestringrequiredenum: api_key
variant 2
access_tokenstringrequired
expires_atinteger· int64
provider_extraunknown
refresh_tokenstring
scopesstring[]
typestringrequiredenum: oauth
variant 2
valuenull
max_tokensinteger· uint64min 0
sourcestringrequiredenum: config, env, none

router::provider::update_credential

function

OAuth write-back: store a provider credential in the configuration entry under the entry write lock (token-gated).

request

credentialunknown
The credential object to store (provider-specific shape).
idstring
Provider id whose credential slice is being written.
tokenstring
Registration token gating the write (optional).

response

okbooleanrequired

router::route

function

Read-only routing preview: resolve {model, provider?} to the chosen provider + ordered candidates without streaming.

request

modelstring
Model id to route.
providerstring
Pin an explicit provider, bypassing heuristics (optional).

response

candidatesstring[]required
providerstringrequired

router::system_prompt::get

function

Effective identity prompt for {provider?} (default_provider when omitted): operator override when set, else provider-declared; null when absent.

request

providerstring

response

providerstring
system_promptstring

triggers

router::models::changed

trigger

A provider reconciled its catalog slice. Payload: { provider, count }.

invocation

empty object

return

valueunknown

router::provider::changed

trigger

The provider registry changed (declare / availability flip). Payload: { provider, op }.

invocation

empty object

return

valueunknown

router::ready

trigger

The router finished booting; providers bind here and re-declare after a restart.

invocation

empty object

return

valueunknown