# judge

> Provider-neutral evaluation hub (judge::evaluate, judge::cancel, and the internal judge::models::list) that forwards typed Noul, Choice and Score questions to the selected judge-<provider> worker; the default provider is a Console setting seeded from JUDGE_PROVIDER (typesafe). Also resolves worker-defined @<name>(id=…) chat mentions for agents (judge::mentions::resolve and a harness pre-generate hook).

| field | value |
|-------|-------|
| version | 0.2.0 |
| type | binary |
| license | Apache-2.0 |
| repo | https://github.com/iii-hq/workers |
| supported_targets | x86_64-apple-darwin, aarch64-apple-darwin, x86_64-pc-windows-msvc, aarch64-pc-windows-msvc, x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-unknown-linux-musl, armv7-unknown-linux-gnueabihf |
| author | iii |

## installation

```sh
iii trigger compose::add worker=judge@0.2.0
```

## configuration

```yaml
- provider: typesafe
```

## dependencies

- `configuration` @ `latest`
- `judge-typesafe` @ `latest`

## readme

# judge

`judge` turns JSON state and Noul, Choice or Score questions into typed,
validated answers for ticket triage, content checks and relevance judgments.
It is a provider-neutral hub: every call is forwarded to a `judge-<provider>`
worker that owns the model, credentials and transport. The default provider is
[`judge-typesafe`](../judge-typesafe/) (TypeSafe JEV); other strategies plug in
by registering the same three functions under their own `judge-<provider>` id.

## Install

```bash
iii trigger compose::add worker=judge
```

Compose starts the hub with its `judge-typesafe` dependency, which in turn needs
`configuration`. Use engine **`iii/v0.24.0-rc.2`**, the verified release used by
this repository's CI. Wait for `judge::evaluate` to register before calling it.

## Quickstart

Set the TypeSafe key in the Console under **Settings → Workers → judge-typesafe**
or supply `TYPESAFE_API_KEY` in that worker's environment (see the
[provider README](../judge-typesafe/README.md#configuration)). Then ask whether
a support ticket needs urgent attention:

```bash
iii trigger judge::evaluate --timeout-ms 65000 --json '{
  "timeout_ms": 60000,
  "evaluations": [{
    "id": "ticket-42",
    "state": {"ticket": "Production checkout is down for all customers."},
    "questions": {
      "urgent": {
        "type": "noul",
        "instructions": "Does this describe an urgent production outage?",
        "criteria": {"true": "A critical production workflow is blocked."}
      }
    }
  }]
}'
```

A successful response looks like this; scores, usage and timing are illustrative:

```json
{
  "status": "ok", "model": "jev-1.13.0",
  "results": {"ticket-42": {
    "answers": {"urgent": {"type": "noul", "noul": 0.98}},
    "usage": {"input_tokens": 120, "output_tokens": 8}
  }},
  "stats": {"attempts": 1, "requests": 1, "questions": 1,
    "input_tokens": 120, "output_tokens": 8, "elapsed_ms": 250,
    "usage_complete": true}
}
```

Check `status` before reading answers. Errors return `status: "error"` and a
`code` with no partial results; `provider_unavailable` means the selected
`judge-<provider>` worker is not registered on the engine. A bus invocation
failure is handled separately. The hub adds 5 s of bus slack on top of your
`timeout_ms`; the provider enforces the deadline itself.

See the [mixed Noul/Choice/Score example](reference.md#evaluate),
[result and error handling](reference.md#handle-results-and-failures),
[model listing](reference.md#list-models) and [cancellation](reference.md#cancellation).

## Providers

**Settings → Workers → judge** selects the default provider from the workers
registered as `judge-<provider>` (seeded from `JUDGE_PROVIDER`, else `typesafe`);
a request may name its own with a top-level `provider`. A new provider is a
worker that registers `judge-<provider>::evaluate`, `::models::list` and
`::cancel` with the [`judge-contract`](../crates/judge-contract/) types and
accepts request ids up to 512 bytes; the hub needs no change. See
[Configuration](reference.md#configuration) and
[Cancellation](reference.md#cancellation).

The hub holds no credentials; its configuration entry (`judge`, or
`III_CONFIG_NAME`) carries only the default provider. For the full API,
read [reference.md](reference.md); for the provider's build, configuration and
tests, read [judge-typesafe](../judge-typesafe/README.md).

## skills

This version ships 1 skill file for agents, also browsable on the worker page:

- [`SKILL.md`](https://workers.iii.dev/workers/judge?tab=skills&skill=SKILL.md&version=0.2.0)

### SKILL.md

# judge

`judge::evaluate` evaluates supplied state with mixed Noul, Choice and Score
questions. `judge::models::list` lists provider models using the same credentials
and transport. `judge::cancel` signals cancellation of an identified evaluation or
listing owned by the same caller. Each call is forwarded to `judge-<provider>`:
the **Default provider** under Console Settings → Workers → judge (seeded from
`JUDGE_PROVIDER`, default `typesafe`) or a top-level `provider` field selects it. Credentials and the default model live with the
provider: for TypeSafe, configure `api_key` in the `judge-typesafe` entry under
Console Settings → Workers or supply `TYPESAFE_API_KEY` to that worker process.
An unregistered provider returns `code: "provider_unavailable"`.

## Invocation

1. Read the registered contract with
   `iii trigger engine::functions::info --json '{"function_id":"judge::evaluate"}'`.
   Use `judge::models::list` or `judge::cancel` as `function_id` for those contracts.
   For unfamiliar question/reply shapes,
   read [the mixed request and typed response examples](../reference.md#evaluate).
2. For evaluation, supply 1–512 evaluations with unique nonblank IDs, JSON state
   and a nonempty map of distinct question IDs. Choose primitives using the table
   below. Supply a positive `timeout_ms` within the operator's `max_timeout_ms`
   (default 300000). For queued work, also set `expires_at_unix_ms`, an absolute
   Unix deadline in milliseconds. Omit `model` to use the worker default.
   `options.attempt_timeout_ms` bounds each network attempt; retries are the
   provider's fixed policy, see [Request options and retries](../reference.md#request-options-and-retries).
3. Invoke using the positional function name and `--json`, then inspect `status`
   before reading results. Completion requires all requested typed answers or a
   reported failure; keep `stats.usage_complete` with the observed usage.

| Primitive | Question criteria | Typed answer |
| --- | --- | --- |
| `noul` | Optional `true` and/or `false` descriptions; omitted, null or empty criteria are accepted. | `noul` in `[0, 1]`. |
| `choice` | 1–255 named options. | `choice`, `probabilities`, `confidence`. |
| `score` | 2–10 ordered, non-null levels. | `score` in `[0, levels - 1]`, `probabilities`, `confidence`, `legend`. |

For all primitives, instructions may be omitted or contain strings, objects,
arrays or null. Noul and Choice descriptions accept those same forms. Score
levels accept strings, objects or arrays; **null Score levels are invalid**
(confirmed by a provider 422 and the SDK schema, despite the advanced guide).
Structured values remain structured in Score legends. Noul can describe just one
endpoint. Bare numbers and booleans are not top-level descriptions. A minimal mixed call:

```bash
iii trigger judge::evaluate --json '{
  "timeout_ms": 3000,
  "evaluations": [{
    "id": "ticket-42",
    "state": {"ticket": "Production checkout is down for all customers."},
    "questions": {
      "urgent": {
        "type": "noul",
        "instructions": "Does this ticket describe an urgent production outage?"
      },
      "department": {"type": "choice", "criteria": {"billing": null, "technical": null}},
      "severity": {"type": "score", "criteria": ["Routine", {"impact": "Production blocked"}]}
    }
  }]
}'
```

On `status: "ok"`, read `results["ticket-42"].answers`, selecting the field matching
each answer's `type`. Choice probabilities use option IDs; Score probabilities
and legends use zero-based index strings. Treat confidence as returned by the
provider; it is not a caller-defined threshold or a formula to recompute.

To list models, invoke `iii trigger judge::models::list --json '{}'`. This uses
`GET /v1/models`; the default timeout is 30000 ms, with an optional smaller or
larger `timeout_ms` within the operator ceiling, and optional `expires_at_unix_ms`.
Both evaluation and listing accept `options` and an optional `request_id`.
Success supplies `models` containing `name`, `description` and `release_date`,
plus `stats`. A catalog may contain aliases without every accepted versioned ID.
Model listing claims no inference token usage.

## Cancel an active call

Start the evaluation or listing with a top-level `request_id` (≤128 printable
ASCII characters, unique among the caller's active calls), then invoke
`judge::cancel` with `{"request_id":"..."}` **from the same caller connection**:
ownership comes from the engine's `_caller_worker_id`, and separate CLI
invocations may get different ids. `cancelled: true` means the signal was
accepted, not that the provider rolled anything back; still await the original
call, which answers `code: "cancelled"`. Absent, finished or other callers' ids
return `cancelled: false`. Details and the bus example:
[cancellation](../reference.md#cancellation).

## Result and operational boundaries

- Success carries the effective `model`, all `results` and `stats`; errors carry
  `status: "error"`, `code`, optional `http_status`, a bounded and key-redacted
  `provider_error`, `retry_after_ms` and `stats`, with no partial answers. Bus
  invocation failures are separate. See
  [results and failures](../reference.md#handle-results-and-failures).
- `usage` counters are independently nullable; `stats.usage_complete` is false
  whenever any counter is unknown, so zero never means free.
- Deadlines, retries, permits and byte limits are in
  [limits and compatibility](../reference.md#limits-and-compatibility).
  Credentials stay in the provider's configuration or environment; requests
  never carry keys, provider URLs or extra headers.

## api reference

```json
{
  "functions": [
    {
      "description": "Signal cancellation of an active evaluation or model listing started by the calling worker through the same provider. Returns whether a signal was accepted; does not roll back provider work.",
      "metadata": {},
      "name": "judge::cancel",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "properties": {
          "provider": {
            "description": "Provider worker suffix (judge-<provider>). Defaults to the hub's JUDGE_PROVIDER.",
            "pattern": "^[a-z0-9-]{1,64}$",
            "type": "string"
          },
          "request_id": {
            "maxLength": 128,
            "minLength": 1,
            "type": "string"
          }
        },
        "required": [
          "request_id"
        ],
        "title": "CancelRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "ErrorCode": {
            "oneOf": [
              {
                "enum": [
                  "invalid_request",
                  "missing_key",
                  "payload_too_large",
                  "deadline",
                  "attempt_timeout",
                  "cancelled",
                  "http",
                  "transport",
                  "invalid_response"
                ],
                "type": "string"
              },
              {
                "description": "The selected `judge-<provider>` worker is not registered on the engine.",
                "enum": [
                  "provider_unavailable"
                ],
                "type": "string"
              }
            ]
          }
        },
        "oneOf": [
          {
            "additionalProperties": false,
            "properties": {
              "cancelled": {
                "type": "boolean"
              },
              "status": {
                "enum": [
                  "ok"
                ],
                "type": "string"
              }
            },
            "required": [
              "cancelled",
              "status"
            ],
            "type": "object"
          },
          {
            "additionalProperties": false,
            "properties": {
              "code": {
                "$ref": "#/definitions/ErrorCode"
              },
              "status": {
                "enum": [
                  "error"
                ],
                "type": "string"
              }
            },
            "required": [
              "code",
              "status"
            ],
            "type": "object"
          }
        ],
        "title": "CancelResponse"
      }
    },
    {
      "description": "Returns this worker instance's effective configuration entry ID, without its value.",
      "metadata": {
        "internal": true
      },
      "name": "judge::configuration-id",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Empty user request. Ignore transport metadata such as `_caller_worker_id`, which the engine adds to routed calls; no payload field selects the entry.",
        "title": "ConfigurationIdentityRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Only the entry ID is exposed; configuration values and secrets stay in the service.",
        "properties": {
          "id": {
            "type": "string"
          }
        },
        "required": [
          "id"
        ],
        "title": "ConfigurationIdentityResponse",
        "type": "object"
      }
    },
    {
      "description": "Evaluate Noul, Choice and Score questions against arbitrary JSON state through the selected judge-<provider> worker. Results are atomic; stats retain known accepted usage. No credentials are accepted in the request.",
      "metadata": {},
      "name": "judge::evaluate",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "Content": {
            "anyOf": [
              {
                "type": "string"
              },
              {
                "additionalProperties": true,
                "type": "object"
              },
              {
                "items": true,
                "type": "array"
              },
              {
                "type": "null"
              }
            ],
            "description": "Structured provider descriptions; top-level numbers/bools are not descriptions."
          },
          "Evaluation": {
            "additionalProperties": false,
            "properties": {
              "id": {
                "type": "string"
              },
              "questions": {
                "additionalProperties": {
                  "$ref": "#/definitions/Question"
                },
                "minProperties": 1,
                "type": "object"
              },
              "state": {
                "$ref": "#/definitions/ScoreLevel"
              }
            },
            "required": [
              "id",
              "questions",
              "state"
            ],
            "type": "object"
          },
          "Question": {
            "oneOf": [
              {
                "additionalProperties": false,
                "properties": {
                  "criteria": {
                    "anyOf": [
                      {
                        "additionalProperties": false,
                        "properties": {
                          "false": {
                            "$ref": "#/definitions/Content"
                          },
                          "true": {
                            "$ref": "#/definitions/Content"
                          }
                        },
                        "type": "object"
                      },
                      {
                        "type": "null"
                      }
                    ]
                  },
                  "instructions": {
                    "allOf": [
                      {
                        "$ref": "#/definitions/Content"
                      }
                    ],
                    "default": null
                  },
                  "type": {
                    "enum": [
                      "noul"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "type"
                ],
                "type": "object"
              },
              {
                "additionalProperties": false,
                "properties": {
                  "criteria": {
                    "additionalProperties": {
                      "$ref": "#/definitions/Content"
                    },
                    "maxProperties": 255,
                    "minProperties": 1,
                    "type": "object"
                  },
                  "instructions": {
                    "allOf": [
                      {
                        "$ref": "#/definitions/Content"
                      }
                    ],
                    "default": null
                  },
                  "type": {
                    "enum": [
                      "choice"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "criteria",
                  "type"
                ],
                "type": "object"
              },
              {
                "additionalProperties": false,
                "properties": {
                  "criteria": {
                    "items": {
                      "$ref": "#/definitions/ScoreLevel"
                    },
                    "maxItems": 10,
                    "minItems": 2,
                    "type": "array"
                  },
                  "instructions": {
                    "allOf": [
                      {
                        "$ref": "#/definitions/Content"
                      }
                    ],
                    "default": null
                  },
                  "type": {
                    "enum": [
                      "score"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "criteria",
                  "type"
                ],
                "type": "object"
              }
            ]
          },
          "RequestOptions": {
            "additionalProperties": false,
            "properties": {
              "attempt_timeout_ms": {
                "description": "Bounds each network attempt separately from the whole-call deadline.",
                "format": "uint64",
                "minimum": 1,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          },
          "ScoreLevel": {
            "anyOf": [
              {
                "type": "string"
              },
              {
                "additionalProperties": true,
                "type": "object"
              },
              {
                "items": true,
                "type": "array"
              }
            ],
            "description": "Score levels require a description. Unlike instructions, Choice option descriptions and Noul criteria, the provider rejects a null level (HTTP422)."
          }
        },
        "properties": {
          "evaluations": {
            "items": {
              "$ref": "#/definitions/Evaluation"
            },
            "maxItems": 512,
            "minItems": 1,
            "type": "array"
          },
          "expires_at_unix_ms": {
            "format": "uint64",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          },
          "model": {
            "type": [
              "string",
              "null"
            ]
          },
          "options": {
            "allOf": [
              {
                "$ref": "#/definitions/RequestOptions"
              }
            ],
            "default": {}
          },
          "provider": {
            "description": "Provider worker suffix (judge-<provider>). Defaults to the hub's JUDGE_PROVIDER.",
            "pattern": "^[a-z0-9-]{1,64}$",
            "type": "string"
          },
          "request_id": {
            "maxLength": 128,
            "minLength": 1,
            "type": [
              "string",
              "null"
            ]
          },
          "timeout_ms": {
            "format": "uint64",
            "minimum": 1,
            "type": "integer"
          }
        },
        "required": [
          "evaluations",
          "timeout_ms"
        ],
        "title": "EvaluateRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "Answer": {
            "oneOf": [
              {
                "additionalProperties": false,
                "properties": {
                  "noul": {
                    "format": "double",
                    "type": "number"
                  },
                  "type": {
                    "enum": [
                      "noul"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "noul",
                  "type"
                ],
                "type": "object"
              },
              {
                "additionalProperties": false,
                "properties": {
                  "choice": {
                    "type": "string"
                  },
                  "confidence": {
                    "format": "double",
                    "type": "number"
                  },
                  "probabilities": {
                    "additionalProperties": {
                      "format": "double",
                      "type": "number"
                    },
                    "type": "object"
                  },
                  "type": {
                    "enum": [
                      "choice"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "choice",
                  "confidence",
                  "probabilities",
                  "type"
                ],
                "type": "object"
              },
              {
                "additionalProperties": false,
                "properties": {
                  "confidence": {
                    "format": "double",
                    "type": "number"
                  },
                  "legend": {
                    "additionalProperties": {
                      "$ref": "#/definitions/ScoreLevel"
                    },
                    "type": "object"
                  },
                  "probabilities": {
                    "additionalProperties": {
                      "format": "double",
                      "type": "number"
                    },
                    "type": "object"
                  },
                  "score": {
                    "format": "double",
                    "type": "number"
                  },
                  "type": {
                    "enum": [
                      "score"
                    ],
                    "type": "string"
                  }
                },
                "required": [
                  "confidence",
                  "legend",
                  "probabilities",
                  "score",
                  "type"
                ],
                "type": "object"
              }
            ]
          },
          "ErrorCode": {
            "oneOf": [
              {
                "enum": [
                  "invalid_request",
                  "missing_key",
                  "payload_too_large",
                  "deadline",
                  "attempt_timeout",
                  "cancelled",
                  "http",
                  "transport",
                  "invalid_response"
                ],
                "type": "string"
              },
              {
                "description": "The selected `judge-<provider>` worker is not registered on the engine.",
                "enum": [
                  "provider_unavailable"
                ],
                "type": "string"
              }
            ]
          },
          "EvaluationResult": {
            "additionalProperties": false,
            "properties": {
              "answers": {
                "additionalProperties": {
                  "$ref": "#/definitions/Answer"
                },
                "type": "object"
              },
              "usage": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/Usage"
                  },
                  {
                    "type": "null"
                  }
                ]
              }
            },
            "required": [
              "answers"
            ],
            "type": "object"
          },
          "ProviderError": {
            "additionalProperties": false,
            "description": "Bounded provider diagnostics, with credentials and supplied header values redacted.",
            "properties": {
              "detail": true,
              "message": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "truncated": {
                "default": false,
                "type": "boolean"
              }
            },
            "type": "object"
          },
          "ScoreLevel": {
            "anyOf": [
              {
                "type": "string"
              },
              {
                "additionalProperties": true,
                "type": "object"
              },
              {
                "items": true,
                "type": "array"
              }
            ],
            "description": "Score levels require a description. Unlike instructions, Choice option descriptions and Noul criteria, the provider rejects a null level (HTTP422)."
          },
          "Stats": {
            "additionalProperties": false,
            "description": "Known accepted usage, never a billing estimate. Unreported counters or failure/cancellation make usage_complete false.",
            "properties": {
              "attempts": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "elapsed_ms": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "questions": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "requests": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "usage_complete": {
                "type": "boolean"
              }
            },
            "required": [
              "attempts",
              "elapsed_ms",
              "input_tokens",
              "output_tokens",
              "questions",
              "requests",
              "usage_complete"
            ],
            "type": "object"
          },
          "Usage": {
            "additionalProperties": false,
            "description": "Each absent/null counter is unknown, independently of the other counter.",
            "properties": {
              "input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          }
        },
        "oneOf": [
          {
            "additionalProperties": false,
            "properties": {
              "model": {
                "type": "string"
              },
              "results": {
                "additionalProperties": {
                  "$ref": "#/definitions/EvaluationResult"
                },
                "type": "object"
              },
              "stats": {
                "$ref": "#/definitions/Stats"
              },
              "status": {
                "enum": [
                  "ok"
                ],
                "type": "string"
              }
            },
            "required": [
              "model",
              "results",
              "stats",
              "status"
            ],
            "type": "object"
          },
          {
            "additionalProperties": false,
            "properties": {
              "code": {
                "$ref": "#/definitions/ErrorCode"
              },
              "http_status": {
                "format": "uint16",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "provider_error": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ProviderError"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "retry_after_ms": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "stats": {
                "$ref": "#/definitions/Stats"
              },
              "status": {
                "enum": [
                  "error"
                ],
                "type": "string"
              }
            },
            "required": [
              "code",
              "stats",
              "status"
            ],
            "type": "object"
          }
        ],
        "title": "EvaluateResponse"
      }
    },
    {
      "description": "List the selected provider's available model names, descriptions and release dates; performs no inference.",
      "metadata": {},
      "name": "judge::models::list",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "definitions": {
          "RequestOptions": {
            "additionalProperties": false,
            "properties": {
              "attempt_timeout_ms": {
                "description": "Bounds each network attempt separately from the whole-call deadline.",
                "format": "uint64",
                "minimum": 1,
                "type": [
                  "integer",
                  "null"
                ]
              }
            },
            "type": "object"
          }
        },
        "properties": {
          "expires_at_unix_ms": {
            "format": "uint64",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          },
          "options": {
            "allOf": [
              {
                "$ref": "#/definitions/RequestOptions"
              }
            ],
            "default": {}
          },
          "provider": {
            "description": "Provider worker suffix (judge-<provider>). Defaults to the hub's JUDGE_PROVIDER.",
            "pattern": "^[a-z0-9-]{1,64}$",
            "type": "string"
          },
          "request_id": {
            "maxLength": 128,
            "minLength": 1,
            "type": [
              "string",
              "null"
            ]
          },
          "timeout_ms": {
            "default": 30000,
            "format": "uint64",
            "minimum": 1,
            "type": "integer"
          }
        },
        "title": "ModelsRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "ErrorCode": {
            "oneOf": [
              {
                "enum": [
                  "invalid_request",
                  "missing_key",
                  "payload_too_large",
                  "deadline",
                  "attempt_timeout",
                  "cancelled",
                  "http",
                  "transport",
                  "invalid_response"
                ],
                "type": "string"
              },
              {
                "description": "The selected `judge-<provider>` worker is not registered on the engine.",
                "enum": [
                  "provider_unavailable"
                ],
                "type": "string"
              }
            ]
          },
          "ModelCard": {
            "properties": {
              "description": {
                "type": "string"
              },
              "name": {
                "type": "string"
              },
              "release_date": {
                "type": "string"
              }
            },
            "required": [
              "description",
              "name",
              "release_date"
            ],
            "type": "object"
          },
          "ProviderError": {
            "additionalProperties": false,
            "description": "Bounded provider diagnostics, with credentials and supplied header values redacted.",
            "properties": {
              "detail": true,
              "message": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "truncated": {
                "default": false,
                "type": "boolean"
              }
            },
            "type": "object"
          },
          "Stats": {
            "additionalProperties": false,
            "description": "Known accepted usage, never a billing estimate. Unreported counters or failure/cancellation make usage_complete false.",
            "properties": {
              "attempts": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "elapsed_ms": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "input_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "output_tokens": {
                "format": "uint64",
                "minimum": 0,
                "type": "integer"
              },
              "questions": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "requests": {
                "format": "uint",
                "minimum": 0,
                "type": "integer"
              },
              "usage_complete": {
                "type": "boolean"
              }
            },
            "required": [
              "attempts",
              "elapsed_ms",
              "input_tokens",
              "output_tokens",
              "questions",
              "requests",
              "usage_complete"
            ],
            "type": "object"
          }
        },
        "oneOf": [
          {
            "additionalProperties": false,
            "properties": {
              "models": {
                "items": {
                  "$ref": "#/definitions/ModelCard"
                },
                "type": "array"
              },
              "stats": {
                "$ref": "#/definitions/Stats"
              },
              "status": {
                "enum": [
                  "ok"
                ],
                "type": "string"
              }
            },
            "required": [
              "models",
              "stats",
              "status"
            ],
            "type": "object"
          },
          {
            "additionalProperties": false,
            "properties": {
              "code": {
                "$ref": "#/definitions/ErrorCode"
              },
              "http_status": {
                "format": "uint16",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "provider_error": {
                "anyOf": [
                  {
                    "$ref": "#/definitions/ProviderError"
                  },
                  {
                    "type": "null"
                  }
                ]
              },
              "retry_after_ms": {
                "format": "uint64",
                "minimum": 0,
                "type": [
                  "integer",
                  "null"
                ]
              },
              "stats": {
                "$ref": "#/definitions/Stats"
              },
              "status": {
                "enum": [
                  "error"
                ],
                "type": "string"
              }
            },
            "required": [
              "code",
              "stats",
              "status"
            ],
            "type": "object"
          }
        ],
        "title": "ModelsResponse"
      }
    },
    {
      "description": "Internal: reload judge configuration from authoritative storage.",
      "metadata": {
        "internal": true
      },
      "name": "judge::on-config-change",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Trigger payload for `<worker>::on-config-change`. Advisory only: handlers re-fetch the authoritative value and ignore it, so a direct call can never inject config.",
        "properties": {
          "id": {
            "default": null,
            "description": "Configuration id that changed.",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "OnConfigChangeEvent",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Ack returned by the internal `<worker>::on-config-change` handler.",
        "properties": {
          "ok": {
            "type": "boolean"
          }
        },
        "required": [
          "ok"
        ],
        "title": "OnConfigChangeResponse",
        "type": "object"
      }
    },
    {
      "description": "Serve the judge worker's injected console UI assets (content function for its console:script / console:style triggers).",
      "metadata": {
        "internal": true
      },
      "name": "judge::ui-content",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of the content function: the console asks for one asset by path.",
        "properties": {
          "path": {
            "description": "The asset path from the trigger config (e.g. `state/page.js`).",
            "type": "string"
          }
        },
        "required": [
          "path"
        ],
        "title": "UiContentInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Output of the content function.",
        "properties": {
          "content": {
            "description": "The asset source, verbatim.",
            "type": "string"
          },
          "content_type": {
            "description": "MIME type the console should serve the asset with.",
            "type": "string"
          }
        },
        "required": [
          "content",
          "content_type"
        ],
        "title": "UiContentResult",
        "type": "object"
      }
    }
  ],
  "triggers": []
}
```