provider-kimi

Moonshot (Kimi) Chat Completions provider worker behind llm-router. Moonshot's API is OpenAI Chat Completions–compatible, so this worker forks the shared provider scaffolding and adds Kimi's reasoning_content thinking stream.

Implements the provider protocol from tech-specs/2026-06-agentic/llm-router.md: provider::kimi::stream (SSE chunks → AssistantMessageEvent frames into a router-owned channel) and provider::kimi::refresh_models (live GET /v1/models filtered to Kimi / Moonshot chat families, enriched with a curated capability snapshot → router::models::reconcile).

provider::kimi::count_tokens counts a prompt through Moonshot's own estimator endpoint behind router::count_tokens, so the number is the upstream's rather than a local reconstruction of it. Counting never runs the model.

Behavior

• Registration: self-declares via router::provider::register with backoff until acked, and re-declares on the router::ready trigger type. The declaration carries no static models slice — GET /v1/models is the source of truth and a refresh fires right after registration. Defaults: api_url: https://api.moonshot.ai/v1/chat/completions, credential_env_var: MOONSHOT_API_KEY.
• Identity binding: the router returns a registration_token on first registration; it is persisted in state (scope provider-kimi, key registration_token) and presented on every later register/resolve/reconcile. If that state is lost the router rejects re-registration — the operator must clear the binding on the router side.
• Credentials: resolved per request via router::provider::resolve (config slice → MOONSHOT_API_KEY env on the router → none). Both api_key and oauth credential shapes are sent as Authorization: Bearer; v1 performs no OAuth refresh.
• Liveness: ping at least every 30s of upstream silence; a failed channel write (caller gone / router::abort) drops the SSE receiver and aborts the in-flight HTTP request.
• Errors: 401/403 → auth_expired, 429 → rate_limited (except a quota wall, exceeded_current_quota_error → permanent), context-length errors → context_overflow, 5xx/engine_overloaded_error / network → transient, other 4xx → permanent. No transport retries here — the router owns retry policy.
• Request shape vs OpenAI: Moonshot accepts the classic max_tokens param (not max_completion_tokens) and has no reasoning_effort knob.
• Thinking: Kimi thinking models (Kimi K2 Thinking, kimi-thinking-preview) stream their reasoning in delta.reasoning_content before the answer text. This worker surfaces it as a Thinking content block via ThinkingStart/ThinkingDelta/ThinkingEnd (no replay signature). The model itself decides whether it thinks; thinking_level is advisory.
• Structured output: Moonshot supports JSON mode only (response_format: {"type":"json_object"}), not OpenAI's strict json_schema mode. A requested schema is mapped to json_object and a report-and-continue warning rides the final message (the caller must mention "JSON" in the prompt). Curated records declare supports_structured_output: true.
• Prompt caching: Moonshot context caching is automatic — no request markers. prompt_tokens_details.cached_tokens lands on usage.cache_read.
• Curated snapshot: src/curated.rs carries display names / context windows / output ceilings / capability flags / pricing for known Kimi and Moonshot families; conservative defaults for unknown ones. Context windows and pricing are best-effort placeholders — verify against platform.moonshot.ai before release. Discovery supplies only bare ids.

Tests

cargo test                                            # unit (pure modules + TCP stubs)
III_ENGINE_BIN=$(which iii) cargo test --test integration -- --test-threads=1

The integration suite spawns a real engine, the real router (path dep), this provider, and a local stub upstream — no external API calls anywhere.

Running

The binary takes the standard worker CLI flags: --url (engine WebSocket, default ws://127.0.0.1:49134, falls back to the III_WS_URL environment variable), --manifest (print the registry manifest and exit), and --config (accepted but ignored with a warning — provider config comes from the llm-router configuration entry).