judge
v0.2.10Provider-neutral evaluation hub (judge::evaluate, judge::cancel, and the internal judge::models::list) that forwards typed Noul, Choice and Score questions to the selected judge-<provider> worker; the default provider is a Console setting seeded from JUDGE_PROVIDER (typesafe). Also resolves worker-defined @<name>(id=…) chat mentions for agents (judge::mentions::resolve and a harness pre-generate hook).
Install
bash
iii trigger compose::add worker=judge@nextRun in your project with the iii engine running. Exact versions are immutable and digest-pinned.
Agent-ready brief for v0.2.10
Install, config, dependencies, readme, skills and API reference, all in one place. Fetch as agent-context.md for an LLM to consume.
Full markdown
The same content rendered as discrete blocks below is exposed as a single markdown document at
/workers/judge.md?version=next. Paste it into an LLM prompt or pipe it through curl from a worker.Install
iii trigger compose::add worker=judge@0.2.10Configuration
iii-config.yaml
- provider: typesafeDependencies
Dependency · 2Version
Readme
README.md
judge
judge turns JSON state and Noul, Choice or Score questions into typed,
validated answers for ticket triage, content checks and relevance judgments.
It is a provider-neutral hub: every call is forwarded to a judge-
worker that owns the model, credentials and transport. The default provider is
judge-typesafe (TypeSafe JEV); other strategies plug in
by registering the same three functions under their own judge- id.
Install
iii trigger compose::add worker=judgeCompose starts the hub with its judge-typesafe dependency, which in turn needs
configuration. Use engine iii/v0.24.0-rc.2, the verified release used by
this repository's CI. Wait for judge::evaluate to register before calling it.
Quickstart
Set the TypeSafe key in the Console under Settings → Workers → judge-typesafe
or supply TYPESAFE_API_KEY in that worker's environment (see the
provider README). Then ask whether
a support ticket needs urgent attention:
iii trigger judge::evaluate --timeout-ms 65000 --json '{
"timeout_ms": 60000,
"evaluations": [{
"id": "ticket-42",
"state": {"ticket": "Production checkout is down for all customers."},
"questions": {
"urgent": {
"type": "noul",
"instructions": "Does this describe an urgent production outage?",
"criteria": {"true": "A critical production workflow is blocked."}
}
}
}]
}'A successful response looks like this; scores, usage and timing are illustrative:
{
"status": "ok", "model": "jev-1.13.0",
"results": {"ticket-42": {
"answers": {"urgent": {"type": "noul", "noul": 0.98}},
"usage": {"input_tokens": 120, "output_tokens": 8}
}},
"stats": {"attempts": 1, "requests": 1, "questions": 1,
"input_tokens": 120, "output_tokens": 8, "elapsed_ms": 250,
"usage_complete": true}
}Check status before reading answers. Errors return status: "error" and a
code with no partial results; provider_unavailable means the selected
judge- worker is not registered on the engine (or, for a local
provider, its model failed to load). A bus invocation
failure is handled separately. The hub adds 5 s of bus slack on top of your
timeout_ms; the provider enforces the deadline itself.
See the mixed Noul/Choice/Score example, result and error handling, model listing and cancellation.
Providers
Settings → Workers → judge selects the default provider from the workers
registered as judge- (seeded from JUDGE_PROVIDER, else typesafe).
Six ship today: judge-typesafe (TypeSafe's hosted JEV),
judge-openai (OpenAI's hosted Decisions API),
judge-decider, judge-semif,
judge-laya and judge-clef (open models running inside the worker); a request may name its own with a top-level
provider. Between the two sits
the calling session's provider: the console's composer (beside the model
picker) stores it as the session's judge_provider metadata, the harness
stamps it on every turn as the iii.judge.provider OTel baggage, and callers
in that turn (function search, call reconciliation, browser::run) send it as
provider; a request without one falls back to that baggage, then to the
default. Each session routes on its own; no session changes another's judge
or the default. A local provider keeps its model loaded while it is the
default (all of them with Keep every local provider loaded, preload_all);
a request naming any other one loads its model on first use, which that request
may not outlast, and the model is released after 10 idle minutes. Picking a
provider in the composer starts that load. The composer's judge picker is
laid out like the model picker: each provider shows the model its settings
name, Configure edits a provider's settings (or the hub's) inside the
picker, and + adds a judge worker from the registry. A choice that would
not be used (its worker is not running, or it has no usable key) shows a
warning on the composer and beside it in the list. A new provider is a
worker that registers judge-, ::models::list and
::cancel (plus ::status when it needs a key) with the
judge-contract types,
marked metadata.internal: true so default discovery shows only the hub, and
accepts request ids up to 512 bytes; the hub needs no change. See
Configuration and
Cancellation.
Chat mentions
Because the harness always runs the judge, the judge also delivers chat
mentions to agents: a user's @kanban(id="…") (or any worker-defined
mention) reaches the agent with a one-line summary of the item and a
pre-verified call for its full details, and agents learn which mention names
they may write in replies. See Chat mentions;
JUDGE_MENTIONS=false turns it off.
The hub holds no credentials; its configuration entry (judge, or
III_CONFIG_NAME) carries only the default provider and preload_all. For the full API,
read reference.md; for the provider's build, configuration and
tests, read judge-typesafe.
Skills
API reference (JSON)
agent-api-reference.json
{
"functions": [
{
"description": "Signal cancellation of an active evaluation or model listing started by the calling worker through the same provider. Returns whether a signal was accepted; does not roll back provider work.",
"metadata": {},
"name": "judge::cancel",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"properties": {
"provider": {
"description": "Provider worker suffix (judge-<provider>). Defaults to the calling session's provider (OTel baggage iii.judge.provider), then the hub's JUDGE_PROVIDER.",
"pattern": "^[a-z0-9-]{1,64}$",
"type": "string"
},
"request_id": {
"maxLength": 128,
"minLength": 1,
"type": "string"
}
},
"required": [
"request_id"
],
"title": "CancelRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ErrorCode": {
"oneOf": [
{
"enum": [
"invalid_request",
"missing_key",
"payload_too_large",
"deadline",
"attempt_timeout",
"cancelled",
"http",
"transport",
"invalid_response"
],
"type": "string"
},
{
"description": "The selected `judge-<provider>` worker is not registered on the engine, or (a local provider) its model failed to load.",
"enum": [
"provider_unavailable"
],
"type": "string"
}
]
}
},
"oneOf": [
{
"additionalProperties": false,
"properties": {
"cancelled": {
"type": "boolean"
},
"status": {
"enum": [
"ok"
],
"type": "string"
}
},
"required": [
"cancelled",
"status"
],
"type": "object"
},
{
"additionalProperties": false,
"properties": {
"code": {
"$ref": "#/definitions/ErrorCode"
},
"status": {
"enum": [
"error"
],
"type": "string"
}
},
"required": [
"code",
"status"
],
"type": "object"
}
],
"title": "CancelResponse"
}
},
{
"description": "Returns this worker instance's effective configuration entry ID, without its value.",
"metadata": {
"internal": true
},
"name": "judge::configuration-id",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Empty user request. Ignore transport metadata such as `_caller_worker_id`, which the engine adds to routed calls; no payload field selects the entry.",
"title": "ConfigurationIdentityRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Only the entry ID is exposed; configuration values and secrets stay in the service.",
"properties": {
"id": {
"type": "string"
}
},
"required": [
"id"
],
"title": "ConfigurationIdentityResponse",
"type": "object"
}
},
{
"description": "Evaluate Noul, Choice and Score questions against arbitrary JSON state through the selected judge-<provider> worker. Results are atomic; stats retain known accepted usage. No credentials are accepted in the request.",
"metadata": {},
"name": "judge::evaluate",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"definitions": {
"Content": {
"anyOf": [
{
"type": "string"
},
{
"additionalProperties": true,
"type": "object"
},
{
"items": true,
"type": "array"
},
{
"type": "null"
}
],
"description": "Structured provider descriptions; top-level numbers/bools are not descriptions."
},
"Evaluation": {
"additionalProperties": false,
"properties": {
"id": {
"type": "string"
},
"questions": {
"additionalProperties": {
"$ref": "#/definitions/Question"
},
"minProperties": 1,
"type": "object"
},
"state": {
"$ref": "#/definitions/ScoreLevel"
}
},
"required": [
"id",
"questions",
"state"
],
"type": "object"
},
"Question": {
"oneOf": [
{
"additionalProperties": false,
"properties": {
"criteria": {
"anyOf": [
{
"additionalProperties": false,
"properties": {
"false": {
"$ref": "#/definitions/Content"
},
"true": {
"$ref": "#/definitions/Content"
}
},
"type": "object"
},
{
"type": "null"
}
]
},
"instructions": {
"allOf": [
{
"$ref": "#/definitions/Content"
}
],
"default": null
},
"type": {
"enum": [
"noul"
],
"type": "string"
}
},
"required": [
"type"
],
"type": "object"
},
{
"additionalProperties": false,
"properties": {
"criteria": {
"additionalProperties": {
"$ref": "#/definitions/Content"
},
"maxProperties": 255,
"minProperties": 1,
"type": "object"
},
"instructions": {
"allOf": [
{
"$ref": "#/definitions/Content"
}
],
"default": null
},
"type": {
"enum": [
"choice"
],
"type": "string"
}
},
"required": [
"criteria",
"type"
],
"type": "object"
},
{
"additionalProperties": false,
"properties": {
"criteria": {
"items": {
"$ref": "#/definitions/ScoreLevel"
},
"maxItems": 10,
"minItems": 2,
"type": "array"
},
"instructions": {
"allOf": [
{
"$ref": "#/definitions/Content"
}
],
"default": null
},
"type": {
"enum": [
"score"
],
"type": "string"
}
},
"required": [
"criteria",
"type"
],
"type": "object"
}
]
},
"RequestOptions": {
"additionalProperties": false,
"properties": {
"attempt_timeout_ms": {
"description": "Bounds each network attempt separately from the whole-call deadline.",
"format": "uint64",
"minimum": 1,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"ScoreLevel": {
"anyOf": [
{
"type": "string"
},
{
"additionalProperties": true,
"type": "object"
},
{
"items": true,
"type": "array"
}
],
"description": "Score levels require a description. Unlike instructions, Choice option descriptions and Noul criteria, the provider rejects a null level (HTTP422)."
}
},
"properties": {
"evaluations": {
"items": {
"$ref": "#/definitions/Evaluation"
},
"maxItems": 512,
"minItems": 1,
"type": "array"
},
"expires_at_unix_ms": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"model": {
"type": [
"string",
"null"
]
},
"options": {
"allOf": [
{
"$ref": "#/definitions/RequestOptions"
}
],
"default": {}
},
"provider": {
"description": "Provider worker suffix (judge-<provider>). Defaults to the calling session's provider (OTel baggage iii.judge.provider), then the hub's JUDGE_PROVIDER.",
"pattern": "^[a-z0-9-]{1,64}$",
"type": "string"
},
"request_id": {
"maxLength": 128,
"minLength": 1,
"type": [
"string",
"null"
]
},
"timeout_ms": {
"format": "uint64",
"minimum": 1,
"type": "integer"
}
},
"required": [
"evaluations",
"timeout_ms"
],
"title": "EvaluateRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Answer": {
"oneOf": [
{
"additionalProperties": false,
"properties": {
"noul": {
"format": "double",
"type": "number"
},
"type": {
"enum": [
"noul"
],
"type": "string"
}
},
"required": [
"noul",
"type"
],
"type": "object"
},
{
"additionalProperties": false,
"properties": {
"choice": {
"type": "string"
},
"confidence": {
"format": "double",
"type": "number"
},
"probabilities": {
"additionalProperties": {
"format": "double",
"type": "number"
},
"type": "object"
},
"type": {
"enum": [
"choice"
],
"type": "string"
}
},
"required": [
"choice",
"confidence",
"probabilities",
"type"
],
"type": "object"
},
{
"additionalProperties": false,
"properties": {
"confidence": {
"format": "double",
"type": "number"
},
"legend": {
"additionalProperties": {
"$ref": "#/definitions/ScoreLevel"
},
"type": "object"
},
"probabilities": {
"additionalProperties": {
"format": "double",
"type": "number"
},
"type": "object"
},
"score": {
"format": "double",
"type": "number"
},
"type": {
"enum": [
"score"
],
"type": "string"
}
},
"required": [
"confidence",
"legend",
"probabilities",
"score",
"type"
],
"type": "object"
}
]
},
"ErrorCode": {
"oneOf": [
{
"enum": [
"invalid_request",
"missing_key",
"payload_too_large",
"deadline",
"attempt_timeout",
"cancelled",
"http",
"transport",
"invalid_response"
],
"type": "string"
},
{
"description": "The selected `judge-<provider>` worker is not registered on the engine, or (a local provider) its model failed to load.",
"enum": [
"provider_unavailable"
],
"type": "string"
}
]
},
"EvaluationResult": {
"additionalProperties": false,
"properties": {
"answers": {
"additionalProperties": {
"$ref": "#/definitions/Answer"
},
"type": "object"
},
"usage": {
"anyOf": [
{
"$ref": "#/definitions/Usage"
},
{
"type": "null"
}
]
}
},
"required": [
"answers"
],
"type": "object"
},
"ProviderError": {
"additionalProperties": false,
"description": "Bounded provider diagnostics, with credentials and supplied header values redacted.",
"properties": {
"detail": true,
"message": {
"type": [
"string",
"null"
]
},
"truncated": {
"default": false,
"type": "boolean"
}
},
"type": "object"
},
"ScoreLevel": {
"anyOf": [
{
"type": "string"
},
{
"additionalProperties": true,
"type": "object"
},
{
"items": true,
"type": "array"
}
],
"description": "Score levels require a description. Unlike instructions, Choice option descriptions and Noul criteria, the provider rejects a null level (HTTP422)."
},
"Stats": {
"additionalProperties": false,
"description": "Known accepted usage, never a billing estimate. Unreported counters or failure/cancellation make usage_complete false.",
"properties": {
"attempts": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"elapsed_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"input_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"questions": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"requests": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"usage_complete": {
"type": "boolean"
}
},
"required": [
"attempts",
"elapsed_ms",
"input_tokens",
"output_tokens",
"questions",
"requests",
"usage_complete"
],
"type": "object"
},
"Usage": {
"additionalProperties": false,
"description": "Each absent/null counter is unknown, independently of the other counter.",
"properties": {
"input_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
}
},
"oneOf": [
{
"additionalProperties": false,
"properties": {
"model": {
"type": "string"
},
"results": {
"additionalProperties": {
"$ref": "#/definitions/EvaluationResult"
},
"type": "object"
},
"stats": {
"$ref": "#/definitions/Stats"
},
"status": {
"enum": [
"ok"
],
"type": "string"
}
},
"required": [
"model",
"results",
"stats",
"status"
],
"type": "object"
},
{
"additionalProperties": false,
"properties": {
"code": {
"$ref": "#/definitions/ErrorCode"
},
"http_status": {
"format": "uint16",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"provider_error": {
"anyOf": [
{
"$ref": "#/definitions/ProviderError"
},
{
"type": "null"
}
]
},
"retry_after_ms": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stats": {
"$ref": "#/definitions/Stats"
},
"status": {
"enum": [
"error"
],
"type": "string"
}
},
"required": [
"code",
"stats",
"status"
],
"type": "object"
}
],
"title": "EvaluateResponse"
}
},
{
"description": "Internal: harness pre-generate hook — tells agents which @<name>(id=…) mentions they may write and resolves the ones users wrote into summaries plus pre-verified details calls.",
"metadata": {
"internal": true,
"trace_hidden": true
},
"name": "judge::mentions::pre-generate",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"GenerateInput": {
"properties": {
"messages": {
"default": [],
"description": "The window this step sends, earlier hook notices replayed in place.",
"items": true,
"type": "array"
},
"tools": {
"default": [],
"description": "The concrete functions this session may call (`{ name, … }`).",
"items": true,
"type": "array"
}
},
"type": "object"
}
},
"description": "The envelope the harness posts to pre-generate hooks (only what this hook reads; everything else is ignored).",
"properties": {
"generate": {
"anyOf": [
{
"$ref": "#/definitions/GenerateInput"
},
{
"type": "null"
}
]
},
"session_id": {
"default": "",
"type": "string"
},
"step": {
"default": 0,
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"title": "PreGenerateInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Mutations": {
"properties": {
"append_messages": {
"items": true,
"type": "array"
}
},
"required": [
"append_messages"
],
"type": "object"
}
},
"description": "Always `continue`: mentions enrich a turn, they never gate it.",
"properties": {
"annotations": {
"additionalProperties": true,
"type": [
"object",
"null"
]
},
"decision": {
"type": "string"
},
"mutations": {
"anyOf": [
{
"$ref": "#/definitions/Mutations"
},
{
"type": "null"
}
]
}
},
"required": [
"decision"
],
"title": "HookResponse",
"type": "object"
}
},
{
"description": "Resolve @<name>(id=\"…\") chat mentions (found in `text`, or listed) through the workers that provide them: each one's status, one-line summary and the call that returns its full details.",
"metadata": {},
"name": "judge::mentions::resolve",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"MentionKey": {
"properties": {
"id": {
"type": "string"
},
"name": {
"type": "string"
}
},
"required": [
"id",
"name"
],
"type": "object"
}
},
"properties": {
"mentions": {
"description": "Mentions to resolve, in addition to those in `text`.",
"items": {
"$ref": "#/definitions/MentionKey"
},
"type": "array"
},
"text": {
"default": null,
"description": "Text to find mentions in (markdown code is skipped).",
"type": [
"string",
"null"
]
}
},
"title": "ResolveRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"DetailsCall": {
"description": "The pre-verified call that returns a mentioned item's full details.",
"properties": {
"function_id": {
"type": "string"
},
"payload": true
},
"required": [
"function_id",
"payload"
],
"type": "object"
},
"MentionStatus": {
"description": "What became of one mention.",
"enum": [
"resolved",
"not_found",
"unknown_provider",
"error"
],
"type": "string"
},
"ResolvedMention": {
"description": "One mention, resolved (or not).",
"properties": {
"details": {
"anyOf": [
{
"$ref": "#/definitions/DetailsCall"
},
{
"type": "null"
}
]
},
"error": {
"description": "Why it did not resolve, when it errored.",
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"label": {
"description": "What the provider calls this kind of item (\"Tickets\").",
"type": [
"string",
"null"
]
},
"name": {
"type": "string"
},
"status": {
"$ref": "#/definitions/MentionStatus"
},
"summary": {
"description": "The agent-facing line, when resolved.",
"type": [
"string",
"null"
]
},
"token": {
"description": "The canonical token (`@<name>(id=\"<id>\")`).",
"type": "string"
}
},
"required": [
"id",
"name",
"status",
"token"
],
"type": "object"
}
},
"properties": {
"mentions": {
"items": {
"$ref": "#/definitions/ResolvedMention"
},
"type": "array"
}
},
"required": [
"mentions"
],
"title": "ResolveResponse",
"type": "object"
}
},
{
"description": "List the selected provider's available model names, descriptions and release dates; performs no inference.",
"metadata": {
"internal": true
},
"name": "judge::models::list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"definitions": {
"RequestOptions": {
"additionalProperties": false,
"properties": {
"attempt_timeout_ms": {
"description": "Bounds each network attempt separately from the whole-call deadline.",
"format": "uint64",
"minimum": 1,
"type": [
"integer",
"null"
]
}
},
"type": "object"
}
},
"properties": {
"expires_at_unix_ms": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"options": {
"allOf": [
{
"$ref": "#/definitions/RequestOptions"
}
],
"default": {}
},
"provider": {
"description": "Provider worker suffix (judge-<provider>). Defaults to the calling session's provider (OTel baggage iii.judge.provider), then the hub's JUDGE_PROVIDER.",
"pattern": "^[a-z0-9-]{1,64}$",
"type": "string"
},
"request_id": {
"maxLength": 128,
"minLength": 1,
"type": [
"string",
"null"
]
},
"timeout_ms": {
"default": 30000,
"format": "uint64",
"minimum": 1,
"type": "integer"
}
},
"title": "ModelsRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ErrorCode": {
"oneOf": [
{
"enum": [
"invalid_request",
"missing_key",
"payload_too_large",
"deadline",
"attempt_timeout",
"cancelled",
"http",
"transport",
"invalid_response"
],
"type": "string"
},
{
"description": "The selected `judge-<provider>` worker is not registered on the engine, or (a local provider) its model failed to load.",
"enum": [
"provider_unavailable"
],
"type": "string"
}
]
},
"ModelCard": {
"properties": {
"context_window": {
"description": "Tokens one evaluation row can hold (question, options and state), when the provider has a fixed window. Callers size their states from it.",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"description": {
"type": "string"
},
"max_options": {
"description": "Most options one Choice question (or levels one Score) can offer, when the provider has a fixed limit below the contract's 255. Callers size their option groups from it.",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"name": {
"type": "string"
},
"release_date": {
"type": "string"
}
},
"required": [
"description",
"name",
"release_date"
],
"type": "object"
},
"ProviderError": {
"additionalProperties": false,
"description": "Bounded provider diagnostics, with credentials and supplied header values redacted.",
"properties": {
"detail": true,
"message": {
"type": [
"string",
"null"
]
},
"truncated": {
"default": false,
"type": "boolean"
}
},
"type": "object"
},
"Stats": {
"additionalProperties": false,
"description": "Known accepted usage, never a billing estimate. Unreported counters or failure/cancellation make usage_complete false.",
"properties": {
"attempts": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"elapsed_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"input_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"questions": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"requests": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"usage_complete": {
"type": "boolean"
}
},
"required": [
"attempts",
"elapsed_ms",
"input_tokens",
"output_tokens",
"questions",
"requests",
"usage_complete"
],
"type": "object"
}
},
"oneOf": [
{
"additionalProperties": false,
"properties": {
"models": {
"items": {
"$ref": "#/definitions/ModelCard"
},
"type": "array"
},
"stats": {
"$ref": "#/definitions/Stats"
},
"status": {
"enum": [
"ok"
],
"type": "string"
}
},
"required": [
"models",
"stats",
"status"
],
"type": "object"
},
{
"additionalProperties": false,
"properties": {
"code": {
"$ref": "#/definitions/ErrorCode"
},
"http_status": {
"format": "uint16",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"provider_error": {
"anyOf": [
{
"$ref": "#/definitions/ProviderError"
},
{
"type": "null"
}
]
},
"retry_after_ms": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stats": {
"$ref": "#/definitions/Stats"
},
"status": {
"enum": [
"error"
],
"type": "string"
}
},
"required": [
"code",
"stats",
"status"
],
"type": "object"
}
],
"title": "ModelsResponse"
}
},
{
"description": "Internal: reload judge configuration from authoritative storage.",
"metadata": {
"internal": true
},
"name": "judge::on-config-change",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Trigger payload for `<worker>::on-config-change`. Advisory only: handlers re-fetch the authoritative value and ignore it, so a direct call can never inject config.",
"properties": {
"id": {
"default": null,
"description": "Configuration id that changed.",
"type": [
"string",
"null"
]
}
},
"title": "OnConfigChangeEvent",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Ack returned by the internal `<worker>::on-config-change` handler.",
"properties": {
"ok": {
"type": "boolean"
}
},
"required": [
"ok"
],
"title": "OnConfigChangeResponse",
"type": "object"
}
},
{
"description": "Serve the judge worker's injected console UI assets (content function for its console:script / console:style / console:module triggers).",
"metadata": {
"internal": true
},
"name": "judge::ui-content",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Input of the content function: the console asks for one asset by path.",
"properties": {
"path": {
"description": "The asset path from the trigger config (e.g. `state/page.js`).",
"type": "string"
}
},
"required": [
"path"
],
"title": "UiContentInput",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Output of the content function.",
"properties": {
"content": {
"description": "The asset source, verbatim.",
"type": "string"
},
"content_type": {
"description": "MIME type the console should serve the asset with.",
"type": "string"
}
},
"required": [
"content",
"content_type"
],
"title": "UiContentResult",
"type": "object"
}
}
],
"triggers": []
}