$worker
harness-e2e
v0.11.19-experimentalMeasure Harness capability, retain reproducible evidence, and expose the E2E dashboard.
iiiverifiedinstall
$iii trigger compose::add worker=harness-e2e@0.11.19-experimental
- macOS: arm64
- Linux: arm64 · armv7 · x64
exact versions are immutable; binary and bundle artifacts are digest-pinned.
agent-ready brief for v0.11.19-experimental
install + config + dependencies + readme + api reference, all in one place. fetch as agent-context.md for an llm to consume.
full markdown
the same content rendered as discrete blocks below is exposed as a single markdown document at
/workers/harness-e2e.md?version=0.11.19-experimental. paste it into an llm prompt or pipe it through curl from a worker.install
install
$iii trigger compose::add worker=harness-e2e@0.11.19-experimental
configuration
iii-config.yaml
- control_database: harness_e2e
control_namespace: harness-e2e-control
data_dir: ~/.iii/data/harness-e2edependencies
readme
README.md
Harness E2E
harness-e2e measures what a Harness stack can execute: correct deliverables,
structural integrity, bounded work, and repeatable outcomes.
The repository stays independent of the workers source tree. Discovery,
execution, observation, state access, and cleanup go through functions
registered in iii. The product input is an already-running iii stack.
One binary, harness-e2e, does both jobs. Compose starts it as the
asynchronous e2e::* control plane and the injectable Console page. The same
binary also runs scenarios and inspects reports from the command line.
Score
A run's score is the sum of the points its evaluated criteria awarded.
- A criterion nobody evaluated adds nothing.
- Scores are not normalized or rescaled.
- Criterion points stay independent of completion and resource limits.
- Criteria do not veto the score or approve a run.
- Completion, technical validity, artifact evidence, and runtime controls are reported separately.
- Infrastructure and execution failures still fail the CLI.
Native criteria keep awards that were already measured when a later check
cannot run. That later check has no award and stays not_evaluated. An
incomplete criterion set has no total score. A product failure remains
technically valid. An infrastructure failure invalidates the run and keeps the
observations already recorded.
Every score and every audit flag is deterministic. No scenario calls a second model to judge the first.
Requirements
shell_coder_sandbox, chess_engine_build, and trend_blog prepare their
reviewed fixture from an embedded Git bundle. They need Git. They do not need
a fixture checkout or HARNESS_E2E_FIXTURE_PATH. Each attempt uses a private
workspace, and temporary source checkouts are removed after their contents are
read.
typescript_chat_service carries its own frozen skeleton. It needs Node 22.6
or newer on the runner host: the subject's TypeScript application runs through
Node type stripping, both in the public suite and in the runner-owned
behavioral probe.
The trending-topics build needs Linux amd64, Docker, Git, Python 3, Node, and access to its pinned fixture. See runtime and controls.
The engineering handoff uses a protected disposable checkout of a pinned
revision of iii-hq/e2e-fixture.
Develop
From the repository root:
pnpm --dir dashboard install --frozen-lockfile
pnpm --dir dashboard typecheck
pnpm --dir dashboard lint
pnpm --dir dashboard test
pnpm --dir dashboard build
cargo test --locked --all-targets
cargo clippy --locked --all-targets -- -D warnings
node --test tests/dashboard/*.test.cjs
HARNESS_E2E_BIN="$PWD/target/debug/harness-e2e" python3 -m unittest discover -s tests/python -p 'test_*.py'The Rust build writes dashboard/dist-console/page.js and styles.css, then
embeds both in the worker. Node and pnpm must be on PATH.
List scenarios and their definition digests:
cargo run --locked --bin harness-e2e -- list
cargo run --locked --bin harness-e2e -- catalogCommands
| Command | What it does |
|---|---|
(none) / worker |
Run the Compose-managed control-plane service. |
list |
Print every scenario id. |
catalog |
Print the materialized scenario catalog. |
run |
Execute one or more scenarios against a running stack. |
report |
Summarize a saved results.json. |
models |
List models registered in the running stack. |
test-plan list |
List profile templates and their coverage. |
test-plan materialize |
Expand one profile into campaigns, groups, and cases. |
--manifest |
Print the Registry worker manifest as JSON. |
Run one scenario against an existing stack:
cargo run --locked --bin harness-e2e -- run \
--url ws://127.0.0.1:49134 \
--model codex/gpt-5.6-luna \
--provider openai-codex \
--scenario todo_worker_simpleValidate a checked-in campaign without executing it:
python3 scripts/run_e2e_campaign.py config/campaigns/endurance.json --e2e-bin target/debug/harness-e2e --validate-only
python3 scripts/run_e2e_campaign.py config/campaigns/endurance.json --e2e-bin target/debug/harness-e2e --dry-runScenarios
Every scenario is a built-in module under src/scenarios/. The module owns
its prompt, setup, deterministic evaluator, and cleanup. Its id is the same
in the CLI, the worker catalog, the campaign runner, the dashboard, and the
canonical result artifacts. Plans name the scenarios they run.
Featured suites:
- Registry:
registry_planning,registry_implementation,registry_environment, andregistry_verification. Each has its own atomic validations. - Linkly tutorial:
linkly_tutorialruns the seven tutorial chapters plus a project-restart guard as one scripted dialogue on one Harness session, against thelinkly-agenticscaffold's Compose stack, and scores twenty-two deterministic checks.scripts/linkly_stack.pyprepares that stack. - Trending topics: an isolated per-attempt Git remote and an independent Playwright acceptance check against the delivered SHA. Design is free. Screenshots are evidence, not an aesthetic score.
config/test-plan.json groups the catalog into modules (state, context,
wakes, coordination, software engineering, integration, security, adaptive
operations, continuous engineering, and the incremental Kanban application)
plus a diagnostic set that profiles can opt into.
Test plans
config/test-plan.json is the executable plan. Rust materializes templates and execution rules from that source and from the native scenario contracts. There is no generated catalog to keep in sync.
| Profile | Purpose |
|---|---|
smoke |
Essential behavior on the published stack. |
regression |
Representative capabilities, with one technical retry. |
capability |
Coverage, repeatability, and difficulty across domains. |
evolution |
Quality and resource use across fixed Harness versions. Three repetitions. |
endurance |
Sustained correct work and the accepted capability boundary. |
software-engineering |
Incremental Kanban, Registry, the trending-topics blog, and the Linkly tutorial. |
In the Console these profiles are starting templates for the same plan form and the same baseline/candidate view used by saved plans. Choose New plan, optionally pick a template, edit the scope, and select the model. Save draft, Save and run, and Duplicate plan share one lifecycle and keep native evidence. See executable profile plans.
cargo run --locked -- test-plan list
cargo run --locked -- test-plan materialize --profile software-engineeringThe software-engineering profile runs each selected case once, with no
technical retries: the seven Kanban cases, four Registry cases, the
trending-topics build, and the Linkly tutorial. That is 13 cases and 13
planned runs, in 12 execution groups. Registry implementation and verification
share a group, in that order, so verification receives the implementation
delivery. Trending topics runs in case-trending-topics-build. Linkly runs
its eight exchanges in case-linkly-tutorial. The executor creates a fresh
pinned linkly-agentic scaffold as that group's Compose project, with
baseline worker versions taken from the resolved stack contract.
Release Control
Operational campaigns are dispatched only by Release Control, through
.github/workflows/exact-stack-e2e.yml.
This repository does not publish independent daily, weekly, post-deploy, or
fault-stress dispatch workflows.
Release Control passes five inputs and makes no further decisions:
execution_id, the plan (one profile id from config/test-plan.json), a
stack policy ({"policy":"latest"} or exact versions), the executor commit
runner_sha, and the cli_version to install.
This repository resolves the rest from the commit pinned by runner_sha:
harness-e2e test-plan materialize --profileexpands the profile into campaigns, groups, and cases, with aprofile_sha256over the result.scripts/resolve_stack_lock.pyturns the stack policy into one exactrc-e2e/v2contract per campaign. Every Registry version is resolved;latestdoes not survive into a contract.scripts/exact_stack_campaign.pyvalidates that contract.- Each group runs in an isolated ephemeral stack. Fault groups go to the protected runner. One root bundle is produced without rebuilding the native Harness artifacts.
Campaign manifests never select or rotate seeds. They keep replay-safe turns separate from scripted dialogue and composite flows, persist a summary for every group, and are advisory while their longitudinal history is being calibrated. Release Control owns scheduling. The executor keeps the result advisory and archives each materialized group through the environment-owned durable archiver.
Partial GitHub reruns reuse the contract artifact produced by prepare, kept
for 90 days. The finalizer matches group artifacts to the completed jobs'
execution times and keeps successful groups from earlier attempts. A rerun
that produces no artifact stays missing evidence; an older artifact does not
replace it.
scripts/report_execution.py posts observations to Release Control's run
ledger over OIDC: materialized before anything runs, one shard per
campaign group, and a summary from the finalizer. Runs come from
results.json, or from journal checkpoints when a group died before writing
one. A group that produced neither still reports that fact.
workers supplies versioned stack components. It does not orchestrate
campaigns.
Agent profile and project template
Release Control can name an existing Directory agent profile in the frozen plan:
{ "agent_profile": "console-ui" }console-ui is Console UI Engineer. The runner reads that profile from
the group's Directory and sends its id as agent to e2e::run. The profile,
its parents, skills, functions, and model or provider must exist in that
stack. The runner downloads the stack's versioned skill bundles into an
isolated Directory and waits up to 120 seconds. A profile that is still
missing fails resolution. A profile model overrides the plan model;
provider::model also selects its provider. Results record the resolved
subject model and the profile configuration hash. Run again resolves the
same profile id in its test stack. Comparisons stay manual in Release Control.
Omitting agent_profile keeps the built-in agent.
A project template is independent of the test-plan profile and the agent profile:
{ "template": "harness", "agent_profile": "tech-lead" }The runner reads iii/template.yaml in iii-hq/templates from main,
resolves it to one commit before sharding, and records that commit as
identity.template.revision. Every group uses that source. The selected
Compose project supplies the base. Test-stack versions override its package
selectors, extra packages enter the stack lock, and local workers stay local.
Template skills replace whole downloaded namespaces, and its agent files take
precedence over downloaded profiles. Machine-global profiles and skills are
unused when a template or agent override is selected. The runner also enables
the campaign's provider when the project does not already have it, honoring
any provider version override in the stack contract.
Scenarios, prompts, permissions, fixtures, seeds, and repetitions stay the same. The evaluated agent applies to ordinary sessions and to workflow or adaptive steps. Evaluators stay the same. Linkly keeps its pinned task scaffold and container roles; the selected template's base and agent assets are applied separately. With no template, the run keeps the existing generated stack (or the required fixture).
To measure a profile, compare the same plan, template commit, model, and stack
with and without agent_profile. Changing the template as well measures the
combined effect. Non-Compose templates and templates that ask for an
interactive language choice are rejected before boot.
Console
Build the worker and its page from the repository root:
cargo build --locked --bin harness-e2eWhen Console connects to the same iii namespace, the worker registers the page
assets and the e2e::dashboard::* functions for read, plan, run, status, and
cancellation. The page lives under #/ext/harness-e2e and exposes Overview,
Tests, Executions, and Plans. See dashboard/README.md.
The running Harness must publish request and response schemas compatible with the current typed surface. Missing or incompatible fields fail preflight. There is no payload-version compatibility mode.
The page loads through iii: 25 compact summaries on the first overview page, one complete report when an execution is opened, only the selected pair for comparison, and the model and scenario catalog when the run dialog opens. Filtering and cursor pagination run on the server. Transport failures stay visible. The trusted publisher still writes the bounded JSON report archive used by CI. It does not publish a standalone Harness E2E web application.
Compare a local change with Release Control
On Plans, Reference: Release Control browses RC history through the authenticated Release Control browser bridge. Keep the RC tab open, enable its local Harness connection, and connect it to the same personal Engine as the Console. The bridge needs the E2E read functions from the companion Release Control change. No GitHub token or artifact sync is required.
Open a plan to see remote and local executions together, with their origin. Select a reference and a local result to compare measurements. Scenario links open the existing A → B comparison with both executions selected. Missing reports and metrics stay visible as unavailable. Reading history does not create a local plan, and the comparison sends no local results to Release Control.
Run locally on a remote reference saves that execution's materialized test parameters as a local plan and runs them against the current Harness. Repeating the action creates a new local plan from the current scenario contracts and keeps earlier plans and results. Scenarios, rounds, repetitions, and retries come from the execution's materialization, not from a current profile of the same name. The local scenario implementations and Harness are used on purpose: this is a personal experiment, not an exact-stack certification. Fault-injection groups still require the protected executor. References without a shard seed for every scenario cannot be reproduced. Differences in the local scenario definition or case identity are shown as advisory information.
Results stay in the local plan store. The RC execution remains a reference. Native result validation stays strict; remote data is read through the RC API and is not installed as a native report. Full remote evidence is the execution's GitHub link, subject to retention. This flow does not download an evidence archive.
Worker
Release Control names the exact project roots. This repository writes only the
root configuration and passes worker@version references to compose::add.
iii resolves the Registry graph, writes the project topology, and reconciles
containers. Every execution starts an empty Engine and a dedicated Compose
daemon, then runs compose::add, compose::up, compose::status, and
compose::down. Each execution uses one isolated namespace for Compose and
for the project functions it starts.
Compose supplies III_URL, III_NAMESPACE, III_WORKER_NAME, and
III_CONFIG. All four are mandatory. The configuration holds the
execution-specific evidence directory and the separate control-plane database
namespace. Start worker-compose.control.yaml before worker-compose.yaml.
The control file provisions the single-connection harness_e2e SQLite pool
and disables SQL history. The Harness exits when that database or its schema
is unavailable. The subject namespace never receives the database client or
its filesystem path.
Publication validates the locally built binary through a path:// Compose
container before upload. Published campaigns use only exact Registry package
versions. Provider secrets go to temporary permission-restricted env_file
files and never appear in contract, Compose, evidence, or archive artifacts.
The worker exposes:
e2e::run,e2e::status,e2e::cancele2e::results-get,e2e::results-list,e2e::comparee2e::scenarios-liste2e::archive,e2e::archive-head,e2e::archive-restoree2e::history-list,e2e::retention-sweep
Subject policies deny e2e::*.
Live worker check
tests/live_worker.rs talks to a worker registered in a real engine, so those
tests are ignored by default. They check that the worker publishes this
revision's catalog (ids, seeds, cases, definition digests), that it refuses
malformed requests, and, when a subject is named, that one minimal_path
execution passes admission, setup, the subject turn, capture, evaluation, and
persistence. The resulting report is readable through e2e::results-get:
HARNESS_E2E_LIVE_URL=ws://127.0.0.1:49134 HARNESS_E2E_LIVE_NAMESPACE=my-project \
HARNESS_E2E_LIVE_PROVIDER=deepseek HARNESS_E2E_LIVE_MODEL=deepseek-flash \
cargo test --test live_worker -- --ignoredSeven scenarios state host paths in their prompts, so their definition digests
match only when the test process has the worker's HARNESS_E2E_RUN_DIR,
TMPDIR, and HARNESS_E2E_*_FIXTURE_PATH values. Export the same environment
the Compose file gives the worker. The scenario run leaves one execution
labelled live worker validation in the worker's storage.
Storage
Durable artifacts are chunked through storage::*. Admissions, executions,
runs, attempts, and artifact references are written through the control-plane
database::* worker. Execution records keep compact dashboard summaries and
observations, so lists and history do not load native reports.
Storage has no version number and no migration step. Every table records the fingerprint of the statements that create it. At start, the worker recreates tables whose fingerprint moved, in one transaction. It keeps the execution records, local plans, and receipts it can still read, and rebuilds run projections from the native bundles. Rows it cannot read, missing bundles, and imported Release Control history in a recreated table are logged as warnings. That history returns by importing it again. Nothing is reconstructed as a scored result. A report or plan written under another results contract is read with a warning.
Plan definitions and composed execution receipts live in saved_plans and
saved_plan_executions. A saved plan or receipt this binary cannot read is
deleted on the next read. Plans written by another binary are not migrated.
Release Control history imports use harness-e2e-history, wrapped as
{json, sha256} with a sha256: digest of the exact UTF-8 JSON. Import
history in the Console imports a file or fetches a plan from the RC bridge.
Plans and executions keep source identities, revisions, and every retained
report. Repeated imports do not create duplicates. Imported active work never
enters local admission or recovery. History stays readable without RC.
Evidence uses local gh credentials and Python 3 to verify the GitHub bundle
manifest, execution and attempt identity, and file checksums. Missing,
expired, inaccessible, and invalid evidence are separate states. Native
bundles keep full reports, manifests, and transcripts, loaded on demand. The
runner has no S3, GCS, R2, SQL-driver, or Harness dependency.
Weekly Stress materializes deterministic fault plans and evaluates journals
from a protected supervisor. Lane promotion is governed by
config/policies/cutover.json.
Observation
The runner waits for a session tree to finish by binding
harness::turn-completed to an internal sink, e2e::on-turn-completed, before
harness::send. That sink is not a control-plane verb: it is not registered
with e2e::run, e2e::status, or e2e::cancel, and it does not appear in
e2e::scenarios-list. Subject policies already deny e2e::*.
A 15-second watchdog samples harness::metrics and one root
harness::status for stuck detection, heartbeat logs, and e2e::cancel. If
the trigger type is missing from engine::triggers::list, the run is
unsupported infrastructure. There is no fallback that polls harness::status
or harness::metrics. After the tree completes, the runner still collects
terminal status, metrics, transcripts, and deliverables.
Comparison
Every completed execution records the subject and E2E revisions, observed wire
contracts, definition digest, materialized inputs, seed, policies, artifacts,
and raw structural evidence. e2e::compare takes two distinct completed
execution ids (from_execution_id and to_execution_id) and writes
comparisons/ plus e2e-summary.md. Numeric
deltas stay disabled when the case set or the canonical contract differs.
Deliverable, structural, technical, cost, latency, turn, and retry deltas stay independent. A case is repeatable after five local runs meet the deliverable, structural, and technical thresholds. Cost and wall time are observed metrics and are compared only inside a compatible baseline and candidate cohort.
Deterministic assessment has one payload shape, written only to results.json.
Scenario contracts are the only versioned domain. Before cleanup, asset
capture applies explicit safety limits and writes an unversioned sidecar with
the canonical deterministic validation portion, which is aggregated into
results.json.
Repository
| Path | Owns |
|---|---|
src/ |
Runner, wire adapters, scenarios, evaluation, longitudinal comparison, and the E2E control worker. |
config/ |
Comparison and cutover policies, fault profiles, and the master test plan. |
tests/ |
Test-only fixtures, golden wire schemas, and the Node and Python suites. |
schemas/ |
Public contracts for generated E2E artifacts. |
dashboard/ |
React, TypeScript, Vite, and Tailwind Console page embedded in the worker. |
Generated reports, transcripts, logs, and deliverables stay outside Git.
The crate may depend on the iii SDK and on generic libraries. It must not
declare a path or Git dependency on workers, Harness, or another product
crate. Contract compatibility is established at runtime from
engine::functions::list and engine::functions::info. The checked-in schemas
are parity fixtures, not a linked product API.
Package boundary
This repository executes exact-stack Test Plans and publishes immutable
harness-e2e Registry releases through the manually dispatched release.yml
workflow. Release Control supplies a stack policy and an immutable executor
SHA to exact-stack-e2e.yml; the contract this repository assembles from them
pins every Registry version to an exact one, including historical candidates,
because a campaign has to be able to say afterwards what it ran.
The root iii.worker.yaml is the public manifest for local iii worker
development and package compatibility. The root worker-compose.yaml is a
normal public Compose document. Release Control and post-prepare workflow
phases read neither source contract.
api reference (json)
agent-api-reference.json
{
"functions": [
{
"description": "Persist a completed execution through private iii storage and idempotent history.",
"metadata": {
"contract": {
"capabilities": [
"archive"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::archive",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"RetentionClass": {
"enum": [
"temporary",
"pull_request",
"longitudinal",
"canonical"
],
"type": "string"
}
},
"properties": {
"execution_id": {
"type": "string"
},
"retention_class": {
"$ref": "#/definitions/RetentionClass"
}
},
"required": [
"execution_id",
"retention_class"
],
"title": "ArchiveExecutionRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"DurableArchiveReference": {
"additionalProperties": false,
"properties": {
"archive_id": {
"type": "string"
},
"created_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"expires_at": {
"type": [
"string",
"null"
]
},
"identity_sha256": {
"type": "string"
},
"manifest": {
"$ref": "#/definitions/StorageObjectReference"
},
"manifest_backup": {
"$ref": "#/definitions/StorageObjectReference"
},
"retention_class": {
"$ref": "#/definitions/RetentionClass"
}
},
"required": [
"archive_id",
"created_at",
"execution_id",
"identity_sha256",
"manifest",
"manifest_backup",
"retention_class"
],
"type": "object"
},
"HistoryRecord": {
"additionalProperties": false,
"properties": {
"archive": {
"$ref": "#/definitions/DurableArchiveReference"
},
"baseline_comparable": {
"type": "boolean"
},
"case_count": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"e2e_repository": {
"type": "string"
},
"e2e_revision": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"identity_sha256": {
"type": "string"
},
"ingestion_id": {
"type": "string"
},
"lane": {
"type": "string"
},
"occurred_at": {
"type": "string"
},
"passed": {
"type": "boolean"
},
"result_contract_sha256": {
"type": "string"
},
"stack_mode": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"subject_revision": {
"type": "string"
}
},
"required": [
"archive",
"baseline_comparable",
"case_count",
"e2e_repository",
"e2e_revision",
"execution_id",
"identity_sha256",
"ingestion_id",
"lane",
"occurred_at",
"passed",
"result_contract_sha256",
"stack_mode",
"subject_model",
"subject_provider",
"subject_revision"
],
"type": "object"
},
"RetentionClass": {
"enum": [
"temporary",
"pull_request",
"longitudinal",
"canonical"
],
"type": "string"
},
"StorageObjectReference": {
"properties": {
"bucket": {
"type": "string"
},
"key": {
"type": "string"
},
"media_type": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"uri": {
"type": "string"
}
},
"required": [
"bucket",
"key",
"media_type",
"sha256",
"size_bytes",
"uri"
],
"type": "object"
}
},
"properties": {
"archive": {
"$ref": "#/definitions/DurableArchiveReference"
},
"duplicate_ingestion": {
"type": "boolean"
},
"history": {
"$ref": "#/definitions/HistoryRecord"
}
},
"required": [
"archive",
"duplicate_ingestion",
"history"
],
"title": "ArchiveResponse",
"type": "object"
}
},
{
"description": "Verify availability and immutable metadata for an archived execution.",
"metadata": {
"contract": {
"capabilities": [
"archive-head"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::archive-head",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ArchiveAvailability": {
"enum": [
"available",
"expired"
],
"type": "string"
},
"DurableArchiveReference": {
"additionalProperties": false,
"properties": {
"archive_id": {
"type": "string"
},
"created_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"expires_at": {
"type": [
"string",
"null"
]
},
"identity_sha256": {
"type": "string"
},
"manifest": {
"$ref": "#/definitions/StorageObjectReference"
},
"manifest_backup": {
"$ref": "#/definitions/StorageObjectReference"
},
"retention_class": {
"$ref": "#/definitions/RetentionClass"
}
},
"required": [
"archive_id",
"created_at",
"execution_id",
"identity_sha256",
"manifest",
"manifest_backup",
"retention_class"
],
"type": "object"
},
"RetentionClass": {
"enum": [
"temporary",
"pull_request",
"longitudinal",
"canonical"
],
"type": "string"
},
"StorageObjectReference": {
"properties": {
"bucket": {
"type": "string"
},
"key": {
"type": "string"
},
"media_type": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"uri": {
"type": "string"
}
},
"required": [
"bucket",
"key",
"media_type",
"sha256",
"size_bytes",
"uri"
],
"type": "object"
}
},
"properties": {
"archive": {
"$ref": "#/definitions/DurableArchiveReference"
},
"availability": {
"$ref": "#/definitions/ArchiveAvailability"
},
"verified_at": {
"type": "string"
}
},
"required": [
"archive",
"availability",
"verified_at"
],
"title": "ArchiveHeadResponse",
"type": "object"
}
},
{
"description": "Restore and verify an archived execution from its immutable manifest.",
"metadata": {
"contract": {
"capabilities": [
"archive-restore"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::archive-restore",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ArchiveAvailability": {
"enum": [
"available",
"expired"
],
"type": "string"
},
"DurableArchiveReference": {
"additionalProperties": false,
"properties": {
"archive_id": {
"type": "string"
},
"created_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"expires_at": {
"type": [
"string",
"null"
]
},
"identity_sha256": {
"type": "string"
},
"manifest": {
"$ref": "#/definitions/StorageObjectReference"
},
"manifest_backup": {
"$ref": "#/definitions/StorageObjectReference"
},
"retention_class": {
"$ref": "#/definitions/RetentionClass"
}
},
"required": [
"archive_id",
"created_at",
"execution_id",
"identity_sha256",
"manifest",
"manifest_backup",
"retention_class"
],
"type": "object"
},
"RetentionClass": {
"enum": [
"temporary",
"pull_request",
"longitudinal",
"canonical"
],
"type": "string"
},
"StorageObjectReference": {
"properties": {
"bucket": {
"type": "string"
},
"key": {
"type": "string"
},
"media_type": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"uri": {
"type": "string"
}
},
"required": [
"bucket",
"key",
"media_type",
"sha256",
"size_bytes",
"uri"
],
"type": "object"
}
},
"properties": {
"archive": {
"$ref": "#/definitions/DurableArchiveReference"
},
"availability": {
"$ref": "#/definitions/ArchiveAvailability"
},
"restored_files": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"restored_from_backup": {
"type": "boolean"
},
"restored_root": {
"type": [
"string",
"null"
]
}
},
"required": [
"archive",
"availability",
"restored_files",
"restored_from_backup"
],
"title": "ArchiveRestoreResponse",
"type": "object"
}
},
{
"description": "Request idempotent cancellation and cleanup of an E2E execution.",
"metadata": {
"contract": {
"capabilities": [
"cancel"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::cancel",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ExecutionPhase": {
"enum": [
"requested",
"admitted",
"preflighting",
"materializing",
"setting_up",
"executing",
"collecting",
"evaluating",
"persisting",
"cleaning_up",
"finalizing",
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
}
},
"properties": {
"accepted": {
"type": "boolean"
},
"execution_id": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
}
},
"required": [
"accepted",
"execution_id",
"phase"
],
"title": "CancelResponse",
"type": "object"
}
},
{
"description": "Compare two completed executions when their identities are eligible.",
"metadata": {
"contract": {
"capabilities": [
"compare"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::compare",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ComparisonPolicy": {
"properties": {
"baseline_id": {
"description": "Identifier of the reviewed baseline these thresholds came from, when one was loaded. Absent when the code-default thresholds are in effect.",
"type": [
"string",
"null"
]
},
"regression": {
"allOf": [
{
"$ref": "#/definitions/RegressionThresholds"
}
],
"default": {
"cost_increase_ratio": 0.2,
"deliverable_success_drop": 0.03,
"p95_turns_increase_ratio": 0.25,
"structural_integrity_drop": 0.02,
"technical_failure_increase": 0.02,
"wall_time_increase_ratio": 0.2
}
},
"scenario_overrides": {
"additionalProperties": {
"$ref": "#/definitions/ThresholdOverrides"
},
"description": "Per-scenario partial threshold overrides carried from the reviewed baseline.",
"type": "object"
}
},
"type": "object"
},
"RegressionThresholds": {
"properties": {
"cost_increase_ratio": {
"format": "double",
"type": "number"
},
"deliverable_success_drop": {
"format": "double",
"type": "number"
},
"p95_turns_increase_ratio": {
"format": "double",
"type": "number"
},
"structural_integrity_drop": {
"format": "double",
"type": "number"
},
"technical_failure_increase": {
"format": "double",
"type": "number"
},
"wall_time_increase_ratio": {
"format": "double",
"type": "number"
}
},
"required": [
"cost_increase_ratio",
"deliverable_success_drop",
"p95_turns_increase_ratio",
"structural_integrity_drop",
"technical_failure_increase",
"wall_time_increase_ratio"
],
"type": "object"
},
"ThresholdOverrides": {
"additionalProperties": false,
"description": "Partial per-scenario replacement for [`RegressionThresholds`]: a named field replaces the baseline value, an omitted field inherits it.",
"properties": {
"cost_increase_ratio": {
"format": "double",
"type": [
"number",
"null"
]
},
"deliverable_success_drop": {
"format": "double",
"type": [
"number",
"null"
]
},
"p95_turns_increase_ratio": {
"format": "double",
"type": [
"number",
"null"
]
},
"structural_integrity_drop": {
"format": "double",
"type": [
"number",
"null"
]
},
"technical_failure_increase": {
"format": "double",
"type": [
"number",
"null"
]
},
"wall_time_increase_ratio": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
}
},
"properties": {
"baseline": {
"default": null,
"description": "Reviewed baseline file to load thresholds from. Omit to use the checked-in `config/baselines/default.json` when it exists, falling back to the code-default thresholds. Ignored when `policy` is set.",
"type": [
"string",
"null"
]
},
"from_execution_id": {
"type": "string"
},
"policy": {
"anyOf": [
{
"$ref": "#/definitions/ComparisonPolicy"
},
{
"type": "null"
}
],
"default": null,
"description": "Explicit policy override. Omit to gate on the reviewed baseline."
},
"to_execution_id": {
"type": "string"
}
},
"required": [
"from_execution_id",
"to_execution_id"
],
"title": "CompareRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ArtifactReference": {
"properties": {
"id": {
"type": "string"
},
"kind": {
"type": "string"
},
"media_type": {
"type": "string"
},
"path": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"id",
"kind",
"media_type",
"path",
"sha256",
"size_bytes"
],
"type": "object"
},
"BenchmarkDelta": {
"properties": {
"consumption": {
"additionalProperties": {
"$ref": "#/definitions/DeltaValue"
},
"default": {},
"description": "Descriptive only; these deltas never create a regression gate.",
"type": "object"
},
"deliverable_success_rate": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"flaky_rate": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"mean_score": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"p50_cost_usd": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"p50_turns": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"p50_wall_time_ms": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"p95_cost_usd": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"p95_turns": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"p95_wall_time_ms": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"retry_rate": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"structural_integrity_rate": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"technical_failure_rate": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"type": "object"
},
"CaseCohortKey": {
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"execution": {
"$ref": "#/definitions/ExecutionCohortIdentity"
},
"inputs_sha256": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"contract_sha256",
"execution",
"inputs_sha256",
"scenario_id",
"seed"
],
"type": "object"
},
"CaseComparison": {
"properties": {
"delta": {
"$ref": "#/definitions/BenchmarkDelta"
},
"from": {
"$ref": "#/definitions/CaseMetrics"
},
"from_run_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"key": {
"$ref": "#/definitions/CaseCohortKey"
},
"regressions": {
"items": {
"$ref": "#/definitions/RegressionSignal"
},
"type": "array"
},
"to": {
"$ref": "#/definitions/CaseMetrics"
},
"to_run_ids": {
"items": {
"type": "string"
},
"type": "array"
}
},
"required": [
"delta",
"from",
"from_run_ids",
"key",
"regressions",
"to",
"to_run_ids"
],
"type": "object"
},
"CaseMetrics": {
"properties": {
"consumption": {
"allOf": [
{
"$ref": "#/definitions/ConsumptionMetrics"
}
],
"default": {
"completed_runs": 0,
"observed_runs": 0,
"p50_function_calls": null,
"p50_total_tokens": null,
"p95_function_calls": null,
"p95_total_tokens": null,
"tokens_per_completion": null,
"total_tokens_consumed": null,
"unavailable": {}
},
"description": "Includes every observed run, even those excluded from quality comparison."
},
"deliverable_success": {
"anyOf": [
{
"$ref": "#/definitions/RateEstimate"
},
{
"type": "null"
}
]
},
"excluded_infrastructure_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"flaky_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"included_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"p50_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"p50_turns": {
"format": "double",
"type": [
"number",
"null"
]
},
"p50_wall_time_ms": {
"format": "double",
"type": [
"number",
"null"
]
},
"p95_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"p95_turns": {
"format": "double",
"type": [
"number",
"null"
]
},
"p95_wall_time_ms": {
"format": "double",
"type": [
"number",
"null"
]
},
"retry_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"structural_integrity": {
"anyOf": [
{
"$ref": "#/definitions/RateEstimate"
},
{
"type": "null"
}
]
},
"technical_failure": {
"anyOf": [
{
"$ref": "#/definitions/RateEstimate"
},
{
"type": "null"
}
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"required": [
"excluded_infrastructure_runs",
"included_runs"
],
"type": "object"
},
"CohortAudit": {
"properties": {
"excluded_cases": {
"items": {
"$ref": "#/definitions/ExcludedCase"
},
"type": "array"
},
"excluded_runs": {
"items": {
"$ref": "#/definitions/ExcludedRun"
},
"type": "array"
},
"included_case_ids": {
"items": {
"type": "string"
},
"type": "array"
}
},
"required": [
"excluded_cases",
"excluded_runs",
"included_case_ids"
],
"type": "object"
},
"ComparisonSide": {
"enum": [
"from",
"to"
],
"type": "string"
},
"ComparisonSummary": {
"properties": {
"baseline_id": {
"description": "Reviewed baseline the regression thresholds came from; absent when the code-default thresholds gated this comparison.",
"type": [
"string",
"null"
]
},
"cases": {
"items": {
"$ref": "#/definitions/CaseComparison"
},
"type": "array"
},
"cohort": {
"$ref": "#/definitions/CohortAudit"
},
"comparable": {
"type": "boolean"
},
"comparison_id": {
"type": "string"
},
"from_execution_id": {
"type": "string"
},
"from_revision": {
"type": [
"string",
"null"
]
},
"gate_passed": {
"type": "boolean"
},
"reasons": {
"items": {
"type": "string"
},
"type": "array"
},
"regressions": {
"items": {
"$ref": "#/definitions/RegressionSignal"
},
"type": "array"
},
"to_execution_id": {
"type": "string"
},
"to_revision": {
"type": [
"string",
"null"
]
}
},
"required": [
"cases",
"cohort",
"comparable",
"comparison_id",
"from_execution_id",
"gate_passed",
"reasons",
"regressions",
"to_execution_id"
],
"type": "object"
},
"ConsumptionMetrics": {
"description": "Subject-side usage across all observed attempts.",
"properties": {
"completed_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"observed_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"p50_function_calls": {
"format": "double",
"type": [
"number",
"null"
]
},
"p50_total_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"p95_function_calls": {
"format": "double",
"type": [
"number",
"null"
]
},
"p95_total_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"tokens_per_completion": {
"format": "double",
"type": [
"number",
"null"
]
},
"total_tokens_consumed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"required": [
"completed_runs",
"observed_runs",
"unavailable"
],
"type": "object"
},
"DeltaValue": {
"properties": {
"absolute": {
"format": "double",
"type": "number"
},
"relative_ratio": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"required": [
"absolute"
],
"type": "object"
},
"ExcludedCase": {
"properties": {
"case_id": {
"type": "string"
},
"reasons": {
"items": {
"type": "string"
},
"type": "array"
},
"scenario_id": {
"type": "string"
},
"side": {
"$ref": "#/definitions/ComparisonSide"
}
},
"required": [
"case_id",
"reasons",
"scenario_id",
"side"
],
"type": "object"
},
"ExcludedRun": {
"properties": {
"attempt_id": {
"type": "string"
},
"case_id": {
"type": "string"
},
"reason": {
"type": "string"
},
"run_id": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"side": {
"$ref": "#/definitions/ComparisonSide"
}
},
"required": [
"attempt_id",
"case_id",
"reason",
"run_id",
"scenario_id",
"side"
],
"type": "object"
},
"ExecutionCohortIdentity": {
"properties": {
"e2e_repository": {
"type": [
"string",
"null"
]
},
"lane": {
"type": "string"
},
"stack_mode": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
}
},
"required": [
"lane",
"stack_mode",
"subject_model",
"subject_provider"
],
"type": "object"
},
"RateEstimate": {
"properties": {
"ci95_lower": {
"format": "double",
"type": "number"
},
"ci95_upper": {
"format": "double",
"type": "number"
},
"rate": {
"format": "double",
"type": "number"
},
"sample_size": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"successes": {
"format": "uint32",
"minimum": 0,
"type": "integer"
}
},
"required": [
"ci95_lower",
"ci95_upper",
"rate",
"sample_size",
"successes"
],
"type": "object"
},
"RegressionDimension": {
"enum": [
"deliverable",
"structural_integrity",
"technical_reliability",
"cost",
"wall_time",
"turns"
],
"type": "string"
},
"RegressionSignal": {
"properties": {
"blocking": {
"type": "boolean"
},
"case_id": {
"type": "string"
},
"dimension": {
"$ref": "#/definitions/RegressionDimension"
},
"maturity": {
"default": "",
"type": "string"
},
"n_from": {
"default": 0,
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"n_to": {
"default": 0,
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"observed_delta": {
"format": "double",
"type": "number"
},
"reason": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"statistic": {
"default": "",
"type": "string"
},
"threshold": {
"format": "double",
"type": "number"
}
},
"required": [
"blocking",
"case_id",
"dimension",
"observed_delta",
"reason",
"scenario_id",
"threshold"
],
"type": "object"
}
},
"properties": {
"artifacts": {
"items": {
"$ref": "#/definitions/ArtifactReference"
},
"type": "array"
},
"comparison": {
"$ref": "#/definitions/ComparisonSummary"
}
},
"required": [
"artifacts",
"comparison"
],
"title": "ComparisonResponse",
"type": "object"
}
},
{
"description": "Read the persisted projection for one logical-run attempt.",
"metadata": {
"contract": {
"capability": "attempt-get",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::attempt-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"attempt_id": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"run_id": {
"type": "string"
}
},
"required": [
"attempt_id",
"execution_id",
"run_id"
],
"title": "AttemptGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"CompletionState": {
"enum": [
"completed",
"task_incomplete",
"undetermined"
],
"type": "string"
},
"ContaminationScope": {
"enum": [
"none",
"case",
"execution"
],
"type": "string"
},
"CriterionReport": {
"properties": {
"awarded": {
"format": "uint8",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"description": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"possible": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"reason": {
"type": "string"
}
},
"required": [
"id",
"possible",
"reason"
],
"type": "object"
},
"DimensionReport": {
"properties": {
"dimension": {
"$ref": "#/definitions/EvaluationDimension"
},
"passed": {
"type": [
"boolean",
"null"
]
},
"signals": true
},
"required": [
"dimension",
"signals"
],
"type": "object"
},
"EfficiencyReport": {
"properties": {
"child_sessions": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"child_turns": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"critical_path_ms": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"effective_fan_out": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"function_call_errors": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"function_calls": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"input_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"observed_complexity": {
"$ref": "#/definitions/ObservedComplexityReport"
},
"observed_work": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"root_turns": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_attempts": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"total_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"transient_resumes": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"validation_retries": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wake_resumes": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wall_time_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"observed_complexity",
"technical_attempts",
"wall_time_ms"
],
"type": "object"
},
"EvaluationDimension": {
"enum": [
"deliverable",
"structural_integrity",
"efficiency",
"robustness",
"e2e_infrastructure"
],
"type": "string"
},
"FailureDomain": {
"enum": [
"subject",
"resource",
"e2e_infrastructure"
],
"type": "string"
},
"FailurePhase": {
"enum": [
"setup",
"execute",
"collect",
"evaluate",
"cleanup"
],
"type": "string"
},
"FailureRecord": {
"properties": {
"code": {
"default": "",
"type": "string"
},
"contamination": {
"allOf": [
{
"$ref": "#/definitions/ContaminationScope"
}
],
"default": "none"
},
"domain": {
"$ref": "#/definitions/FailureDomain"
},
"message": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/FailurePhase"
},
"retry_scope": {
"allOf": [
{
"$ref": "#/definitions/RetryScope"
}
],
"default": "none"
},
"status": {
"allOf": [
{
"$ref": "#/definitions/RunStatus"
}
],
"description": "The failure's original classification, independent of the primary run status."
}
},
"required": [
"domain",
"message",
"phase",
"status"
],
"type": "object"
},
"ObservedComplexityReport": {
"properties": {
"ambiguity_level": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"artifact_count": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"coordination_edges": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"dependency_depth": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"external_systems": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"parallel_branches": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"planning_depth": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"state_transitions": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"validation_loops": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wake_cycles": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"RetryScope": {
"enum": [
"none",
"same_slot"
],
"type": "string"
},
"RunStatus": {
"enum": [
"passed",
"hard_gate_failed",
"subject_error",
"resource_limit",
"infrastructure_error"
],
"type": "string"
},
"TechnicalState": {
"enum": [
"valid",
"technical_invalid"
],
"type": "string"
}
},
"properties": {
"completion": {
"$ref": "#/definitions/CompletionState"
},
"criteria": {
"items": {
"$ref": "#/definitions/CriterionReport"
},
"type": [
"array",
"null"
]
},
"dimensions": {
"items": {
"$ref": "#/definitions/DimensionReport"
},
"type": "array"
},
"efficiency": {
"anyOf": [
{
"$ref": "#/definitions/EfficiencyReport"
},
{
"type": "null"
}
]
},
"failures": {
"items": {
"$ref": "#/definitions/FailureRecord"
},
"type": "array"
},
"status": {
"$ref": "#/definitions/RunStatus"
},
"technical": {
"$ref": "#/definitions/TechnicalState"
}
},
"required": [
"completion",
"dimensions",
"failures",
"status",
"technical"
],
"title": "AttemptResponse",
"type": "object"
}
},
{
"description": "Read models and scenarios when the execution dialog opens.",
"metadata": {
"contract": {
"capability": "catalog-get",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::catalog-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"url": {
"default": null,
"type": [
"string",
"null"
]
}
},
"title": "CatalogRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"CatalogModel": {
"properties": {
"id": {
"type": "string"
},
"provider": {
"type": "string"
}
},
"required": [
"id",
"provider"
],
"type": "object"
}
},
"properties": {
"models": {
"items": {
"$ref": "#/definitions/CatalogModel"
},
"type": "array"
},
"scenarios": {
"items": {
"type": "string"
},
"type": "array"
},
"url": {
"type": "string"
}
},
"required": [
"models",
"scenarios",
"url"
],
"title": "CatalogResponse",
"type": "object"
}
},
{
"description": "List immutable evaluated-system versions and their exact evaluation cohorts.",
"metadata": {
"contract": {
"capability": "evaluated-versions-list",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::evaluated-versions-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"cohort_id": {
"default": null,
"type": [
"string",
"null"
]
}
},
"title": "EvaluatedVersionsRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"CohortDescriptor": {
"properties": {
"id": {
"type": "string"
},
"lane": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
}
},
"required": [
"id",
"lane",
"subject_model",
"subject_provider"
],
"type": "object"
},
"EvaluatedVersionDescriptor": {
"properties": {
"cohort_id": {
"type": "string"
},
"completed_at": {
"type": "string"
},
"execution_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"id": {
"type": "string"
},
"label": {
"type": "string"
},
"stack_mode": {
"type": "string"
}
},
"required": [
"cohort_id",
"completed_at",
"execution_count",
"id",
"label",
"stack_mode"
],
"type": "object"
}
},
"properties": {
"cohorts": {
"items": {
"$ref": "#/definitions/CohortDescriptor"
},
"type": "array"
},
"revision": {
"type": "string"
},
"versions": {
"items": {
"$ref": "#/definitions/EvaluatedVersionDescriptor"
},
"type": "array"
}
},
"required": [
"cohorts",
"revision",
"versions"
],
"title": "EvaluatedVersionsResponse",
"type": "object"
}
},
{
"description": "Read retained GitHub evidence using local credentials and verify its bundle identity and checksum.",
"metadata": {
"contract": {
"capability": "evidence-open",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::evidence-open",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
},
"path": {
"type": "string"
},
"report_id": {
"type": "string"
}
},
"required": [
"execution_id",
"path",
"report_id"
],
"title": "EvidenceRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"availability": {
"type": "string"
},
"content_base64": {
"type": [
"string",
"null"
]
},
"mime_type": {
"type": [
"string",
"null"
]
},
"reason": {
"type": [
"string",
"null"
]
}
},
"required": [
"availability"
],
"title": "EvidenceResponse",
"type": "object"
}
},
{
"description": "Delete one terminal local E2E execution.",
"metadata": {
"contract": {
"capability": "execution-delete",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::execution-delete",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": true,
"title": "Map_of_AnyValue",
"type": "object"
}
},
{
"description": "Read one retained execution summary and its diagnostic report.",
"metadata": {
"contract": {
"capability": "execution-get",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::execution-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ExecutionListResponse": {
"properties": {
"executions": {
"items": true,
"type": "array"
},
"last_update": {
"type": "string"
},
"mode": {
"type": "string"
},
"next_cursor": {
"type": [
"string",
"null"
]
},
"repo_url": {
"type": "string"
},
"retention": {
"$ref": "#/definitions/Retention"
},
"total": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"executions",
"last_update",
"mode",
"repo_url",
"retention",
"total"
],
"type": "object"
},
"Retention": {
"properties": {
"details": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"summaries": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"details",
"summaries"
],
"type": "object"
}
},
"properties": {
"detail": true,
"manifest": {
"$ref": "#/definitions/ExecutionListResponse"
}
},
"required": [
"detail",
"manifest"
],
"title": "ExecutionBundle",
"type": "object"
}
},
{
"description": "List compact retained execution summaries with server-side filters and cursor pagination.",
"metadata": {
"contract": {
"capability": "executions-list",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::executions-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"cursor": {
"default": null,
"type": [
"string",
"null"
]
},
"event": {
"default": null,
"type": [
"string",
"null"
]
},
"ids": {
"default": [],
"items": {
"type": "string"
},
"type": "array"
},
"ids_csv": {
"default": null,
"type": [
"string",
"null"
]
},
"limit": {
"default": null,
"format": "uint16",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"query": {
"default": null,
"type": [
"string",
"null"
]
},
"status": {
"default": null,
"type": [
"string",
"null"
]
}
},
"title": "ExecutionListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Retention": {
"properties": {
"details": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"summaries": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"details",
"summaries"
],
"type": "object"
}
},
"properties": {
"executions": {
"items": true,
"type": "array"
},
"last_update": {
"type": "string"
},
"mode": {
"type": "string"
},
"next_cursor": {
"type": [
"string",
"null"
]
},
"repo_url": {
"type": "string"
},
"retention": {
"$ref": "#/definitions/Retention"
},
"total": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"executions",
"last_update",
"mode",
"repo_url",
"retention",
"total"
],
"title": "ExecutionListResponse",
"type": "object"
}
},
{
"description": "Configure, export and execute saved plans, and inspect or cancel their composed executions.",
"metadata": {
"contract": {
"capability": "plan-control",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plan-control",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"HistoryImport": {
"anyOf": [
{
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"json": {
"type": "string"
},
"sha256": {
"type": "string"
}
},
"required": [
"json",
"sha256"
],
"type": "object"
}
]
},
"ReferenceModel": {
"additionalProperties": false,
"properties": {
"model": {
"type": "string"
},
"provider": {
"type": "string"
}
},
"required": [
"model",
"provider"
],
"type": "object"
}
},
"oneOf": [
{
"properties": {
"action": {
"enum": [
"import_history"
],
"type": "string"
},
"history": {
"$ref": "#/definitions/HistoryImport"
}
},
"required": [
"action",
"history"
],
"type": "object"
},
{
"properties": {
"action": {
"enum": [
"rename_imported_execution"
],
"type": "string"
},
"execution_id": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"action",
"execution_id",
"label"
],
"type": "object"
},
{
"properties": {
"action": {
"enum": [
"reproduce_reference"
],
"type": "string"
},
"label": {
"default": "",
"type": "string"
},
"materialized": true,
"reference_execution_id": {
"type": "string"
},
"shards": {
"default": [],
"items": true,
"type": "array"
},
"subject": {
"$ref": "#/definitions/ReferenceModel"
}
},
"required": [
"action",
"materialized",
"reference_execution_id",
"subject"
],
"type": "object"
},
{
"properties": {
"action": {
"enum": [
"requirements"
],
"type": "string"
},
"plan_id": {
"type": "string"
}
},
"required": [
"action",
"plan_id"
],
"type": "object"
},
{
"properties": {
"action": {
"enum": [
"export"
],
"type": "string"
},
"plan_id": {
"type": "string"
}
},
"required": [
"action",
"plan_id"
],
"type": "object"
},
{
"properties": {
"action": {
"enum": [
"execution"
],
"type": "string"
},
"execution_id": {
"type": "string"
}
},
"required": [
"action",
"execution_id"
],
"type": "object"
},
{
"properties": {
"action": {
"enum": [
"cancel"
],
"type": "string"
},
"execution_id": {
"type": "string"
}
},
"required": [
"action",
"execution_id"
],
"type": "object"
}
],
"title": "Request"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": true,
"title": "Map_of_AnyValue",
"type": "object"
}
},
{
"description": "Create a draft local plan with an explicit small test scope.",
"metadata": {
"contract": {
"capability": "plan-create",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plan-create",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"properties": {
"duplicate_of": {
"default": null,
"type": [
"string",
"null"
]
},
"label": {
"default": "",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"purpose": {
"default": "",
"type": "string"
},
"runs": {
"default": 1,
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenarios": {
"items": {
"type": "string"
},
"type": "array"
},
"seed": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"default": 1,
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"template_id": {
"default": null,
"type": [
"string",
"null"
]
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"scenarios",
"url"
],
"title": "PlanCreateRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"PlanScopeItem": {
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"inputs_sha256": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"contract_sha256",
"inputs_sha256",
"scenario_id",
"seed"
],
"type": "object"
},
"PlanState": {
"enum": [
"draft",
"baseline_running",
"baseline_ready",
"candidate_running",
"comparison_ready"
],
"type": "string"
}
},
"properties": {
"baseline_execution_id": {
"type": [
"string",
"null"
]
},
"candidate_execution_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"candidate_labels": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"compatible": {
"type": "boolean"
},
"created_at": {
"type": "string"
},
"id": {
"type": "string"
},
"incomplete_execution_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"label": {
"type": "string"
},
"last_attempt_id": {
"type": [
"string",
"null"
]
},
"locked": {
"type": "boolean"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"purpose": {
"type": "string"
},
"reference_differences": {
"items": {
"type": "string"
},
"type": "array"
},
"reference_execution_id": {
"type": [
"string",
"null"
]
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenario_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"scenarios": {
"items": {
"$ref": "#/definitions/PlanScopeItem"
},
"type": "array"
},
"scope_hash": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"state": {
"$ref": "#/definitions/PlanState"
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"template_id": {
"type": [
"string",
"null"
]
},
"updated_at": {
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"candidate_execution_ids",
"candidate_labels",
"compatible",
"created_at",
"id",
"incomplete_execution_ids",
"label",
"locked",
"model",
"provider",
"purpose",
"runs",
"scenario_ids",
"scenarios",
"scope_hash",
"state",
"technical_retries",
"updated_at",
"url"
],
"title": "LocalPlan",
"type": "object"
}
},
{
"description": "Delete a local plan.",
"metadata": {
"contract": {
"capability": "plan-delete",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plan-delete",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"plan_id": {
"type": "string"
}
},
"required": [
"plan_id"
],
"title": "PlanGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": true,
"title": "Map_of_AnyValue",
"type": "object"
}
},
{
"description": "Read one saved local or imported plan.",
"metadata": {
"contract": {
"capability": "plan-get",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plan-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"plan_id": {
"type": "string"
}
},
"required": [
"plan_id"
],
"title": "PlanGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": true,
"title": "Map_of_AnyValue",
"type": "object"
}
},
{
"description": "Start a baseline or candidate from a saved local plan.",
"metadata": {
"contract": {
"capability": "plan-run-start",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plan-run-start",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"PlanRunRole": {
"enum": [
"baseline",
"candidate"
],
"type": "string"
}
},
"properties": {
"idempotency_key": {
"type": "string"
},
"plan_id": {
"default": null,
"type": [
"string",
"null"
]
},
"role": {
"$ref": "#/definitions/PlanRunRole"
}
},
"required": [
"idempotency_key",
"role"
],
"title": "PlanRunRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"PlanScopeItem": {
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"inputs_sha256": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"contract_sha256",
"inputs_sha256",
"scenario_id",
"seed"
],
"type": "object"
},
"PlanState": {
"enum": [
"draft",
"baseline_running",
"baseline_ready",
"candidate_running",
"comparison_ready"
],
"type": "string"
}
},
"properties": {
"baseline_execution_id": {
"type": [
"string",
"null"
]
},
"candidate_execution_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"candidate_labels": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"compatible": {
"type": "boolean"
},
"created_at": {
"type": "string"
},
"id": {
"type": "string"
},
"incomplete_execution_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"label": {
"type": "string"
},
"last_attempt_id": {
"type": [
"string",
"null"
]
},
"locked": {
"type": "boolean"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"purpose": {
"type": "string"
},
"reference_differences": {
"items": {
"type": "string"
},
"type": "array"
},
"reference_execution_id": {
"type": [
"string",
"null"
]
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenario_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"scenarios": {
"items": {
"$ref": "#/definitions/PlanScopeItem"
},
"type": "array"
},
"scope_hash": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"state": {
"$ref": "#/definitions/PlanState"
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"template_id": {
"type": [
"string",
"null"
]
},
"updated_at": {
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"candidate_execution_ids",
"candidate_labels",
"compatible",
"created_at",
"id",
"incomplete_execution_ids",
"label",
"locked",
"model",
"provider",
"purpose",
"runs",
"scenario_ids",
"scenarios",
"scope_hash",
"state",
"technical_retries",
"updated_at",
"url"
],
"title": "LocalPlan",
"type": "object"
}
},
{
"description": "Update a local plan.",
"metadata": {
"contract": {
"capability": "plan-update",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plan-update",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"properties": {
"candidate_labels": {
"additionalProperties": {
"type": "string"
},
"type": [
"object",
"null"
]
},
"label": {
"type": [
"string",
"null"
]
},
"model": {
"type": [
"string",
"null"
]
},
"plan_id": {
"default": null,
"type": [
"string",
"null"
]
},
"provider": {
"type": [
"string",
"null"
]
},
"purpose": {
"type": [
"string",
"null"
]
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"scenarios": {
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"url": {
"type": [
"string",
"null"
]
}
},
"title": "PlanUpdateRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"PlanScopeItem": {
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"inputs_sha256": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"contract_sha256",
"inputs_sha256",
"scenario_id",
"seed"
],
"type": "object"
},
"PlanState": {
"enum": [
"draft",
"baseline_running",
"baseline_ready",
"candidate_running",
"comparison_ready"
],
"type": "string"
}
},
"properties": {
"baseline_execution_id": {
"type": [
"string",
"null"
]
},
"candidate_execution_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"candidate_labels": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"compatible": {
"type": "boolean"
},
"created_at": {
"type": "string"
},
"id": {
"type": "string"
},
"incomplete_execution_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"label": {
"type": "string"
},
"last_attempt_id": {
"type": [
"string",
"null"
]
},
"locked": {
"type": "boolean"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"purpose": {
"type": "string"
},
"reference_differences": {
"items": {
"type": "string"
},
"type": "array"
},
"reference_execution_id": {
"type": [
"string",
"null"
]
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenario_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"scenarios": {
"items": {
"$ref": "#/definitions/PlanScopeItem"
},
"type": "array"
},
"scope_hash": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"state": {
"$ref": "#/definitions/PlanState"
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"template_id": {
"type": [
"string",
"null"
]
},
"updated_at": {
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"candidate_execution_ids",
"candidate_labels",
"compatible",
"created_at",
"id",
"incomplete_execution_ids",
"label",
"locked",
"model",
"provider",
"purpose",
"runs",
"scenario_ids",
"scenarios",
"scope_hash",
"state",
"technical_retries",
"updated_at",
"url"
],
"title": "LocalPlan",
"type": "object"
}
},
{
"description": "List saved local and imported plans from the local database.",
"metadata": {
"contract": {
"capability": "plans-list",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plans-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "DashboardEmptyRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"master_plan": true,
"mode": {
"type": "string"
},
"plans": {
"items": true,
"type": "array"
}
},
"required": [
"master_plan",
"mode",
"plans"
],
"title": "PlansListResponse",
"type": "object"
}
},
{
"description": "Cancel the active local E2E execution.",
"metadata": {
"contract": {
"capability": "run-cancel",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::run-cancel",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "DashboardEmptyRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Defaults": {
"properties": {
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"technical_retries",
"url"
],
"type": "object"
},
"JobStatus": {
"enum": [
"running",
"cancelling",
"cancelled",
"completed",
"failed"
],
"type": "string"
},
"JobView": {
"properties": {
"completed_at": {
"type": "string"
},
"error": {
"type": "string"
},
"id": {
"type": "string"
},
"label": {
"type": "string"
},
"log": {
"type": "string"
},
"log_from": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"log_offset": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"log_truncated": {
"type": "boolean"
},
"request": {
"$ref": "#/definitions/RunRequest"
},
"returncode": {
"format": "int32",
"type": [
"integer",
"null"
]
},
"started_at": {
"type": "string"
},
"status": {
"$ref": "#/definitions/JobStatus"
}
},
"required": [
"completed_at",
"error",
"id",
"label",
"log",
"log_from",
"log_offset",
"log_truncated",
"request",
"started_at",
"status"
],
"type": "object"
},
"RunRequest": {
"additionalProperties": false,
"properties": {
"label": {
"default": "",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenarios": {
"items": {
"type": "string"
},
"type": "array"
},
"seed": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"scenarios",
"technical_retries",
"url"
],
"type": "object"
}
},
"properties": {
"defaults": {
"$ref": "#/definitions/Defaults"
},
"job": {
"anyOf": [
{
"$ref": "#/definitions/JobView"
},
{
"type": "null"
}
]
}
},
"required": [
"defaults"
],
"title": "RunSnapshot",
"type": "object"
}
},
{
"description": "Start one local E2E execution.",
"metadata": {
"contract": {
"capability": "run-start",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::run-start",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"properties": {
"label": {
"default": "",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenarios": {
"items": {
"type": "string"
},
"type": "array"
},
"seed": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"scenarios",
"technical_retries",
"url"
],
"title": "RunRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Defaults": {
"properties": {
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"technical_retries",
"url"
],
"type": "object"
},
"JobStatus": {
"enum": [
"running",
"cancelling",
"cancelled",
"completed",
"failed"
],
"type": "string"
},
"JobView": {
"properties": {
"completed_at": {
"type": "string"
},
"error": {
"type": "string"
},
"id": {
"type": "string"
},
"label": {
"type": "string"
},
"log": {
"type": "string"
},
"log_from": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"log_offset": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"log_truncated": {
"type": "boolean"
},
"request": {
"$ref": "#/definitions/RunRequest"
},
"returncode": {
"format": "int32",
"type": [
"integer",
"null"
]
},
"started_at": {
"type": "string"
},
"status": {
"$ref": "#/definitions/JobStatus"
}
},
"required": [
"completed_at",
"error",
"id",
"label",
"log",
"log_from",
"log_offset",
"log_truncated",
"request",
"started_at",
"status"
],
"type": "object"
},
"RunRequest": {
"additionalProperties": false,
"properties": {
"label": {
"default": "",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenarios": {
"items": {
"type": "string"
},
"type": "array"
},
"seed": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"scenarios",
"technical_retries",
"url"
],
"type": "object"
}
},
"properties": {
"defaults": {
"$ref": "#/definitions/Defaults"
},
"job": {
"anyOf": [
{
"$ref": "#/definitions/JobView"
},
{
"type": "null"
}
]
}
},
"required": [
"defaults"
],
"title": "RunSnapshot",
"type": "object"
}
},
{
"description": "Read local execution state and only the unread log suffix.",
"metadata": {
"contract": {
"capability": "run-status",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::run-status",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"after": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"title": "RunStatusRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Defaults": {
"properties": {
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"technical_retries",
"url"
],
"type": "object"
},
"JobStatus": {
"enum": [
"running",
"cancelling",
"cancelled",
"completed",
"failed"
],
"type": "string"
},
"JobView": {
"properties": {
"completed_at": {
"type": "string"
},
"error": {
"type": "string"
},
"id": {
"type": "string"
},
"label": {
"type": "string"
},
"log": {
"type": "string"
},
"log_from": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"log_offset": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"log_truncated": {
"type": "boolean"
},
"request": {
"$ref": "#/definitions/RunRequest"
},
"returncode": {
"format": "int32",
"type": [
"integer",
"null"
]
},
"started_at": {
"type": "string"
},
"status": {
"$ref": "#/definitions/JobStatus"
}
},
"required": [
"completed_at",
"error",
"id",
"label",
"log",
"log_from",
"log_offset",
"log_truncated",
"request",
"started_at",
"status"
],
"type": "object"
},
"RunRequest": {
"additionalProperties": false,
"properties": {
"label": {
"default": "",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenarios": {
"items": {
"type": "string"
},
"type": "array"
},
"seed": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"scenarios",
"technical_retries",
"url"
],
"type": "object"
}
},
"properties": {
"defaults": {
"$ref": "#/definitions/Defaults"
},
"job": {
"anyOf": [
{
"$ref": "#/definitions/JobView"
},
{
"type": "null"
}
]
}
},
"required": [
"defaults"
],
"title": "RunSnapshot",
"type": "object"
}
},
{
"description": "Read local metric history for one test version, with provider-grouped execution models, without comparison actions.",
"metadata": {
"contract": {
"capability": "test-history-get",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::test-history-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"case_id": {
"default": null,
"type": [
"string",
"null"
]
},
"cursor": {
"default": null,
"type": [
"string",
"null"
]
},
"limit": {
"default": null,
"format": "uint16",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"result": {
"default": null,
"type": [
"string",
"null"
]
},
"subject_model": {
"default": null,
"type": [
"string",
"null"
]
},
"subject_provider": {
"default": null,
"type": [
"string",
"null"
]
},
"system_version_id": {
"default": null,
"type": [
"string",
"null"
]
},
"test_id": {
"default": "",
"type": "string"
},
"test_version": {
"default": null,
"type": [
"string",
"null"
]
}
},
"title": "TestHistoryRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"AssessmentOutcomeCounts": {
"properties": {
"error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_evaluated": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"partial": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unavailable": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"error",
"failed",
"not_evaluated",
"partial",
"passed",
"unavailable"
],
"type": "object"
},
"AssessmentSummary": {
"description": "Objective assessment totals for a set of runs. Every count is derived from deterministic evidence: hard gates, deterministic criteria, and asset validation.",
"properties": {
"assessment_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"assessment_outcomes": {
"$ref": "#/definitions/AssessmentOutcomeCounts"
},
"asset_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"asset_validation_outcomes": {
"$ref": "#/definitions/AssetValidationCounts"
},
"evidence_reference_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"system_statuses": {
"$ref": "#/definitions/StatusCounts"
}
},
"required": [
"assessment_count",
"assessment_outcomes",
"asset_count",
"asset_validation_outcomes",
"evidence_reference_count",
"run_count",
"system_statuses"
],
"type": "object"
},
"AssetValidationCounts": {
"properties": {
"invalid": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"malformed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_evaluated": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_produced": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"oversized": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"removed_during_cleanup": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unexpected": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unreadable": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unsafe_path": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"valid": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"invalid",
"malformed",
"not_evaluated",
"not_produced",
"oversized",
"removed_during_cleanup",
"unexpected",
"unreadable",
"unsafe_path",
"valid"
],
"type": "object"
},
"HistoryModelGroup": {
"description": "Models are exposed as provider groups so a model name is never ambiguous when two providers offer the same model identifier.",
"properties": {
"models": {
"items": {
"type": "string"
},
"type": "array"
},
"provider": {
"type": "string"
}
},
"required": [
"models",
"provider"
],
"type": "object"
},
"HistorySeries": {
"properties": {
"assessment_profile_sha256": {
"type": "string"
},
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"cohort_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"engine_revision": {
"type": [
"string",
"null"
]
},
"execution_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"harness_revision": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_call_errors": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_calls": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_turns": {
"format": "double",
"type": [
"number",
"null"
]
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stack_mode": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"system_label": {
"type": "string"
},
"system_revision": {
"type": [
"string",
"null"
]
},
"system_version_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"assessment_profile_sha256",
"behavior_sha256",
"case_id",
"cohort_id",
"contract_sha256",
"execution_count",
"id",
"run_count",
"stack_mode",
"subject_model",
"subject_provider",
"system_label"
],
"type": "object"
},
"HistorySystem": {
"properties": {
"id": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"id",
"label"
],
"type": "object"
},
"StatusCounts": {
"properties": {
"hard_gate_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"infrastructure_error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"resource_limit": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"subject_error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unavailable": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"hard_gate_failed",
"infrastructure_error",
"passed",
"resource_limit",
"subject_error",
"unavailable"
],
"type": "object"
},
"TestObservation": {
"properties": {
"assessment_profile_sha256": {
"type": "string"
},
"assessment_summary": {
"$ref": "#/definitions/AssessmentSummary"
},
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"cohort_id": {
"type": "string"
},
"completed_at": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"engine_revision": {
"type": [
"string",
"null"
]
},
"evaluated_version_id": {
"type": [
"string",
"null"
]
},
"execution_id": {
"type": "string"
},
"harness_revision": {
"type": [
"string",
"null"
]
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_call_errors": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_calls": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_turns": {
"format": "double",
"type": [
"number",
"null"
]
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"scored_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stack_mode": {
"type": "string"
},
"status": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"system_label": {
"type": "string"
},
"system_revision": {
"type": [
"string",
"null"
]
},
"system_version_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"assessment_profile_sha256",
"assessment_summary",
"behavior_sha256",
"case_id",
"cohort_id",
"completed_at",
"contract_sha256",
"execution_id",
"run_count",
"scored_runs",
"stack_mode",
"status",
"subject_model",
"subject_provider",
"system_label"
],
"type": "object"
},
"VersionDescriptor": {
"properties": {
"execution_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"last_seen": {
"type": [
"string",
"null"
]
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"version": {
"type": "string"
}
},
"required": [
"execution_count",
"run_count",
"version"
],
"type": "object"
}
},
"properties": {
"available_versions": {
"items": {
"$ref": "#/definitions/VersionDescriptor"
},
"type": "array"
},
"cases": {
"items": {
"type": "string"
},
"type": "array"
},
"current_version": {
"description": "The current definition's digest, which may have no executions yet.",
"type": [
"string",
"null"
]
},
"next_cursor": {
"type": [
"string",
"null"
]
},
"observations": {
"items": {
"$ref": "#/definitions/TestObservation"
},
"type": "array"
},
"series": {
"items": {
"$ref": "#/definitions/HistorySeries"
},
"type": "array"
},
"subject_models": {
"items": {
"$ref": "#/definitions/HistoryModelGroup"
},
"type": "array"
},
"subjects": {
"items": {
"type": "string"
},
"type": "array"
},
"systems": {
"items": {
"$ref": "#/definitions/HistorySystem"
},
"type": "array"
},
"test_id": {
"type": "string"
},
"test_version": {
"description": "The definition whose executions are shown (the latest with evidence by default).",
"type": "string"
},
"total": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"available_versions",
"cases",
"observations",
"series",
"subject_models",
"subjects",
"systems",
"test_id",
"test_version",
"total"
],
"title": "TestHistoryResponse",
"type": "object"
}
},
{
"description": "Read one test version across two immutable evaluated-system versions.",
"metadata": {
"contract": {
"capability": "test-version-get",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::test-version-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"cohort_id": {
"type": "string"
},
"from_version_id": {
"type": "string"
},
"test_id": {
"type": "string"
},
"test_version": {
"type": "string"
},
"to_version_id": {
"type": "string"
}
},
"required": [
"cohort_id",
"from_version_id",
"test_id",
"test_version",
"to_version_id"
],
"title": "TestVersionGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"AssessmentOutcomeCounts": {
"properties": {
"error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_evaluated": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"partial": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unavailable": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"error",
"failed",
"not_evaluated",
"partial",
"passed",
"unavailable"
],
"type": "object"
},
"AssessmentSummary": {
"description": "Objective assessment totals for a set of runs. Every count is derived from deterministic evidence: hard gates, deterministic criteria, and asset validation.",
"properties": {
"assessment_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"assessment_outcomes": {
"$ref": "#/definitions/AssessmentOutcomeCounts"
},
"asset_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"asset_validation_outcomes": {
"$ref": "#/definitions/AssetValidationCounts"
},
"evidence_reference_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"system_statuses": {
"$ref": "#/definitions/StatusCounts"
}
},
"required": [
"assessment_count",
"assessment_outcomes",
"asset_count",
"asset_validation_outcomes",
"evidence_reference_count",
"run_count",
"system_statuses"
],
"type": "object"
},
"AssetValidationCounts": {
"properties": {
"invalid": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"malformed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_evaluated": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_produced": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"oversized": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"removed_during_cleanup": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unexpected": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unreadable": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unsafe_path": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"valid": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"invalid",
"malformed",
"not_evaluated",
"not_produced",
"oversized",
"removed_during_cleanup",
"unexpected",
"unreadable",
"unsafe_path",
"valid"
],
"type": "object"
},
"MetricSamples": {
"properties": {
"cost_usd": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"duration_seconds": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"score": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"tokens": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"turns": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"cost_usd",
"duration_seconds",
"score",
"tokens",
"turns"
],
"type": "object"
},
"OutcomeCounts": {
"properties": {
"hard_gate_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"infra_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"technical_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"hard_gate_failed",
"infra_failed",
"passed",
"technical_failed"
],
"type": "object"
},
"StatusCounts": {
"properties": {
"hard_gate_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"infrastructure_error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"resource_limit": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"subject_error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unavailable": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"hard_gate_failed",
"infrastructure_error",
"passed",
"resource_limit",
"subject_error",
"unavailable"
],
"type": "object"
},
"TestDelta": {
"properties": {
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"score": {
"format": "double",
"type": [
"number",
"null"
]
},
"tokens": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"TestObservation": {
"properties": {
"assessment_profile_sha256": {
"type": "string"
},
"assessment_summary": {
"$ref": "#/definitions/AssessmentSummary"
},
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"cohort_id": {
"type": "string"
},
"completed_at": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"engine_revision": {
"type": [
"string",
"null"
]
},
"evaluated_version_id": {
"type": [
"string",
"null"
]
},
"execution_id": {
"type": "string"
},
"harness_revision": {
"type": [
"string",
"null"
]
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_call_errors": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_calls": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_turns": {
"format": "double",
"type": [
"number",
"null"
]
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"scored_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stack_mode": {
"type": "string"
},
"status": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"system_label": {
"type": "string"
},
"system_revision": {
"type": [
"string",
"null"
]
},
"system_version_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"assessment_profile_sha256",
"assessment_summary",
"behavior_sha256",
"case_id",
"cohort_id",
"completed_at",
"contract_sha256",
"execution_id",
"run_count",
"scored_runs",
"stack_mode",
"status",
"subject_model",
"subject_provider",
"system_label"
],
"type": "object"
},
"TestSideSummary": {
"properties": {
"assessment_summary": {
"$ref": "#/definitions/AssessmentSummary"
},
"case_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"evaluated_version_id": {
"type": "string"
},
"execution_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"outcomes": {
"$ref": "#/definitions/OutcomeCounts"
},
"pass_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"samples": {
"$ref": "#/definitions/MetricSamples"
},
"scored_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"total_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"assessment_summary",
"case_count",
"evaluated_version_id",
"execution_count",
"outcomes",
"samples",
"scored_runs",
"total_runs"
],
"type": "object"
}
},
"properties": {
"compatibility": {
"type": "string"
},
"compatibility_reasons": {
"items": {
"type": "string"
},
"type": "array"
},
"delta": {
"$ref": "#/definitions/TestDelta"
},
"from": {
"anyOf": [
{
"$ref": "#/definitions/TestSideSummary"
},
{
"type": "null"
}
]
},
"from_observations": {
"items": {
"$ref": "#/definitions/TestObservation"
},
"type": "array"
},
"test_id": {
"type": "string"
},
"test_version": {
"type": "string"
},
"to": {
"anyOf": [
{
"$ref": "#/definitions/TestSideSummary"
},
{
"type": "null"
}
]
},
"to_observations": {
"items": {
"$ref": "#/definitions/TestObservation"
},
"type": "array"
}
},
"required": [
"compatibility",
"compatibility_reasons",
"delta",
"from_observations",
"test_id",
"test_version",
"to_observations"
],
"title": "TestVersionResult",
"type": "object"
}
},
{
"description": "List the versioned test catalog with compact A/B results and stable pagination.",
"metadata": {
"contract": {
"capability": "tests-list",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::tests-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"cohort_id": {
"default": null,
"type": [
"string",
"null"
]
},
"cursor": {
"default": null,
"type": [
"string",
"null"
]
},
"from_version_id": {
"default": null,
"type": [
"string",
"null"
]
},
"limit": {
"default": null,
"format": "uint16",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"query": {
"default": null,
"type": [
"string",
"null"
]
},
"to_version_id": {
"default": null,
"type": [
"string",
"null"
]
}
},
"title": "TestsListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"AssessmentKind": {
"enum": [
"required_check",
"signal",
"asset_validation"
],
"type": "string"
},
"AssessmentOutcomeCounts": {
"properties": {
"error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_evaluated": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"partial": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unavailable": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"error",
"failed",
"not_evaluated",
"partial",
"passed",
"unavailable"
],
"type": "object"
},
"AssessmentPolicy": {
"enum": [
"hard_gate",
"advisory"
],
"type": "string"
},
"AssessmentSummary": {
"description": "Objective assessment totals for a set of runs. Every count is derived from deterministic evidence: hard gates, deterministic criteria, and asset validation.",
"properties": {
"assessment_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"assessment_outcomes": {
"$ref": "#/definitions/AssessmentOutcomeCounts"
},
"asset_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"asset_validation_outcomes": {
"$ref": "#/definitions/AssetValidationCounts"
},
"evidence_reference_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"system_statuses": {
"$ref": "#/definitions/StatusCounts"
}
},
"required": [
"assessment_count",
"assessment_outcomes",
"asset_count",
"asset_validation_outcomes",
"evidence_reference_count",
"run_count",
"system_statuses"
],
"type": "object"
},
"AssetValidationCounts": {
"properties": {
"invalid": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"malformed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_evaluated": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_produced": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"oversized": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"removed_during_cleanup": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unexpected": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unreadable": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unsafe_path": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"valid": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"invalid",
"malformed",
"not_evaluated",
"not_produced",
"oversized",
"removed_during_cleanup",
"unexpected",
"unreadable",
"unsafe_path",
"valid"
],
"type": "object"
},
"CalibrationProjection": {
"properties": {
"compatible_sample_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"maturity": {
"type": "string"
}
},
"required": [
"compatible_sample_count",
"maturity"
],
"type": "object"
},
"EvaluationDimension": {
"enum": [
"deliverable",
"structural_integrity",
"efficiency",
"robustness",
"e2e_infrastructure"
],
"type": "string"
},
"ExecutionPolicy": {
"properties": {
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_total_tokens": {
"description": "Shared Harness token budget. `None` leaves the Harness budget unbounded for this scenario.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_turns": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"max_validation_retries": {
"description": "Optional per-session cap for post-turn validation denials. `None` preserves the Harness-configured default.",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stuck_timeout_seconds": {
"description": "Stop only after this many seconds without observable useful progress. Large scenarios have no fixed wall-clock deadline.",
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"max_turns",
"stuck_timeout_seconds"
],
"type": "object"
},
"ExecutionRealism": {
"enum": [
"synthetic",
"realistic_simulator",
"frozen_real_artifact"
],
"type": "string"
},
"HumanHorizon": {
"properties": {
"basis": {
"allOf": [
{
"$ref": "#/definitions/HumanHorizonBasis"
}
],
"default": "unknown"
},
"max_minutes": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"min_minutes": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"HumanHorizonBasis": {
"enum": [
"unknown",
"author_estimate",
"measured"
],
"type": "string"
},
"MetricSamples": {
"properties": {
"cost_usd": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"duration_seconds": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"score": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"tokens": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"turns": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"cost_usd",
"duration_seconds",
"score",
"tokens",
"turns"
],
"type": "object"
},
"OutcomeCounts": {
"properties": {
"hard_gate_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"infra_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"technical_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"hard_gate_failed",
"infra_failed",
"passed",
"technical_failed"
],
"type": "object"
},
"ScenarioCharacterization": {
"properties": {
"human_horizon": {
"allOf": [
{
"$ref": "#/definitions/HumanHorizon"
}
],
"default": {
"basis": "unknown"
}
},
"realism": {
"allOf": [
{
"$ref": "#/definitions/ScenarioRealism"
}
],
"default": {
"execution": "synthetic",
"shadow": "none"
}
}
},
"type": "object"
},
"ScenarioRealism": {
"properties": {
"execution": {
"allOf": [
{
"$ref": "#/definitions/ExecutionRealism"
}
],
"default": "synthetic"
},
"shadow": {
"allOf": [
{
"$ref": "#/definitions/ShadowMode"
}
],
"default": "none"
}
},
"type": "object"
},
"ShadowMode": {
"enum": [
"none",
"read_only"
],
"type": "string"
},
"StatusCounts": {
"properties": {
"hard_gate_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"infrastructure_error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"resource_limit": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"subject_error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unavailable": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"hard_gate_failed",
"infrastructure_error",
"passed",
"resource_limit",
"subject_error",
"unavailable"
],
"type": "object"
},
"TestCatalogRow": {
"properties": {
"available_versions": {
"items": {
"$ref": "#/definitions/VersionDescriptor"
},
"type": "array"
},
"calibration": {
"anyOf": [
{
"$ref": "#/definitions/CalibrationProjection"
},
{
"type": "null"
}
]
},
"characterization": {
"anyOf": [
{
"$ref": "#/definitions/ScenarioCharacterization"
},
{
"type": "null"
}
]
},
"current_version": {
"type": [
"string",
"null"
]
},
"lifecycle": {
"type": "string"
},
"result": {
"anyOf": [
{
"$ref": "#/definitions/TestVersionResult"
},
{
"type": "null"
}
]
},
"selected_version": {
"type": [
"string",
"null"
]
},
"spec": {
"anyOf": [
{
"$ref": "#/definitions/TestSpecProjection"
},
{
"type": "null"
}
]
},
"test_id": {
"type": "string"
}
},
"required": [
"available_versions",
"lifecycle",
"test_id"
],
"type": "object"
},
"TestCriterionProjection": {
"properties": {
"description": {
"type": "string"
},
"dimension": {
"$ref": "#/definitions/EvaluationDimension"
},
"id": {
"type": "string"
},
"kind": {
"$ref": "#/definitions/AssessmentKind"
},
"policy": {
"$ref": "#/definitions/AssessmentPolicy"
},
"weight": {
"format": "uint8",
"minimum": 0,
"type": "integer"
}
},
"required": [
"description",
"dimension",
"id",
"kind",
"policy",
"weight"
],
"type": "object"
},
"TestDelta": {
"properties": {
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"score": {
"format": "double",
"type": [
"number",
"null"
]
},
"tokens": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"TestObservation": {
"properties": {
"assessment_profile_sha256": {
"type": "string"
},
"assessment_summary": {
"$ref": "#/definitions/AssessmentSummary"
},
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"cohort_id": {
"type": "string"
},
"completed_at": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"engine_revision": {
"type": [
"string",
"null"
]
},
"evaluated_version_id": {
"type": [
"string",
"null"
]
},
"execution_id": {
"type": "string"
},
"harness_revision": {
"type": [
"string",
"null"
]
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_call_errors": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_calls": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_turns": {
"format": "double",
"type": [
"number",
"null"
]
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"scored_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stack_mode": {
"type": "string"
},
"status": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"system_label": {
"type": "string"
},
"system_revision": {
"type": [
"string",
"null"
]
},
"system_version_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"assessment_profile_sha256",
"assessment_summary",
"behavior_sha256",
"case_id",
"cohort_id",
"completed_at",
"contract_sha256",
"execution_id",
"run_count",
"scored_runs",
"stack_mode",
"status",
"subject_model",
"subject_provider",
"system_label"
],
"type": "object"
},
"TestSideSummary": {
"properties": {
"assessment_summary": {
"$ref": "#/definitions/AssessmentSummary"
},
"case_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"evaluated_version_id": {
"type": "string"
},
"execution_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"outcomes": {
"$ref": "#/definitions/OutcomeCounts"
},
"pass_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"samples": {
"$ref": "#/definitions/MetricSamples"
},
"scored_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"total_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"assessment_summary",
"case_count",
"evaluated_version_id",
"execution_count",
"outcomes",
"samples",
"scored_runs",
"total_runs"
],
"type": "object"
},
"TestSpecProjection": {
"description": "The scenario definition as a reader needs it: what the subject is asked to do, how the result is scored, and the limits it runs under. Projected from the materialized `ScenarioSpec` of the test's current version, so it always describes the contract the dashboard is showing.",
"properties": {
"criteria": {
"items": {
"$ref": "#/definitions/TestCriterionProjection"
},
"type": "array"
},
"denied_functions": {
"items": {
"type": "string"
},
"type": "array"
},
"execution": {
"$ref": "#/definitions/ExecutionPolicy"
},
"prompt": {
"description": "The prompt as materialized for a representative run: run-scoped paths resolve against a catalog namespace, never a real execution.",
"type": "string"
},
"summary": {
"description": "Editorial description; absent until the scenario defines a `SUMMARY`.",
"type": [
"string",
"null"
]
}
},
"required": [
"criteria",
"denied_functions",
"execution",
"prompt"
],
"type": "object"
},
"TestVersionResult": {
"properties": {
"compatibility": {
"type": "string"
},
"compatibility_reasons": {
"items": {
"type": "string"
},
"type": "array"
},
"delta": {
"$ref": "#/definitions/TestDelta"
},
"from": {
"anyOf": [
{
"$ref": "#/definitions/TestSideSummary"
},
{
"type": "null"
}
]
},
"from_observations": {
"items": {
"$ref": "#/definitions/TestObservation"
},
"type": "array"
},
"test_id": {
"type": "string"
},
"test_version": {
"type": "string"
},
"to": {
"anyOf": [
{
"$ref": "#/definitions/TestSideSummary"
},
{
"type": "null"
}
]
},
"to_observations": {
"items": {
"$ref": "#/definitions/TestObservation"
},
"type": "array"
}
},
"required": [
"compatibility",
"compatibility_reasons",
"delta",
"from_observations",
"test_id",
"test_version",
"to_observations"
],
"type": "object"
},
"VersionDescriptor": {
"properties": {
"execution_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"last_seen": {
"type": [
"string",
"null"
]
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"version": {
"type": "string"
}
},
"required": [
"execution_count",
"run_count",
"version"
],
"type": "object"
}
},
"properties": {
"next_cursor": {
"type": [
"string",
"null"
]
},
"revision": {
"type": "string"
},
"rows": {
"items": {
"$ref": "#/definitions/TestCatalogRow"
},
"type": "array"
},
"total": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"revision",
"rows",
"total"
],
"title": "TestsListResponse",
"type": "object"
}
},
{
"description": "List hash-validated E2E history records from the iii database worker.",
"metadata": {
"contract": {
"capabilities": [
"history-list"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::history-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"lane": {
"default": null,
"type": [
"string",
"null"
]
},
"limit": {
"default": 100,
"format": "uint16",
"minimum": 0,
"type": "integer"
}
},
"title": "HistoryListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"DurableArchiveReference": {
"additionalProperties": false,
"properties": {
"archive_id": {
"type": "string"
},
"created_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"expires_at": {
"type": [
"string",
"null"
]
},
"identity_sha256": {
"type": "string"
},
"manifest": {
"$ref": "#/definitions/StorageObjectReference"
},
"manifest_backup": {
"$ref": "#/definitions/StorageObjectReference"
},
"retention_class": {
"$ref": "#/definitions/RetentionClass"
}
},
"required": [
"archive_id",
"created_at",
"execution_id",
"identity_sha256",
"manifest",
"manifest_backup",
"retention_class"
],
"type": "object"
},
"HistoryRecord": {
"additionalProperties": false,
"properties": {
"archive": {
"$ref": "#/definitions/DurableArchiveReference"
},
"baseline_comparable": {
"type": "boolean"
},
"case_count": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"e2e_repository": {
"type": "string"
},
"e2e_revision": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"identity_sha256": {
"type": "string"
},
"ingestion_id": {
"type": "string"
},
"lane": {
"type": "string"
},
"occurred_at": {
"type": "string"
},
"passed": {
"type": "boolean"
},
"result_contract_sha256": {
"type": "string"
},
"stack_mode": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"subject_revision": {
"type": "string"
}
},
"required": [
"archive",
"baseline_comparable",
"case_count",
"e2e_repository",
"e2e_revision",
"execution_id",
"identity_sha256",
"ingestion_id",
"lane",
"occurred_at",
"passed",
"result_contract_sha256",
"stack_mode",
"subject_model",
"subject_provider",
"subject_revision"
],
"type": "object"
},
"RetentionClass": {
"enum": [
"temporary",
"pull_request",
"longitudinal",
"canonical"
],
"type": "string"
},
"StorageObjectReference": {
"properties": {
"bucket": {
"type": "string"
},
"key": {
"type": "string"
},
"media_type": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"uri": {
"type": "string"
}
},
"required": [
"bucket",
"key",
"media_type",
"sha256",
"size_bytes",
"uri"
],
"type": "object"
}
},
"properties": {
"records": {
"items": {
"$ref": "#/definitions/HistoryRecord"
},
"type": "array"
}
},
"required": [
"records"
],
"title": "HistoryListResponse",
"type": "object"
}
},
{
"description": "Get the immutable result and manifest of an E2E execution.",
"metadata": {
"contract": {
"capabilities": [
"results-get"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::results-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ActivationPolicy": {
"oneOf": [
{
"properties": {
"policy": {
"enum": [
"always"
],
"type": "string"
}
},
"required": [
"policy"
],
"type": "object"
},
{
"properties": {
"conditions": {
"items": {
"$ref": "#/definitions/BooleanCondition"
},
"type": "array"
},
"policy": {
"enum": [
"all"
],
"type": "string"
}
},
"required": [
"conditions",
"policy"
],
"type": "object"
},
{
"properties": {
"conditions": {
"items": {
"$ref": "#/definitions/BooleanCondition"
},
"type": "array"
},
"policy": {
"enum": [
"any"
],
"type": "string"
}
},
"required": [
"conditions",
"policy"
],
"type": "object"
}
]
},
"AgentProfileArtifact": {
"properties": {
"configuration_sha256": {
"type": "string"
},
"id": {
"type": "string"
}
},
"required": [
"configuration_sha256",
"id"
],
"type": "object"
},
"ArtifactExpectation": {
"properties": {
"id": {
"type": "string"
},
"kind": {
"type": "string"
},
"max_size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"media_type": {
"type": "string"
},
"schema": true
},
"required": [
"id",
"kind",
"max_size_bytes",
"media_type",
"schema"
],
"type": "object"
},
"ArtifactReference": {
"properties": {
"id": {
"type": "string"
},
"kind": {
"type": "string"
},
"media_type": {
"type": "string"
},
"path": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"id",
"kind",
"media_type",
"path",
"sha256",
"size_bytes"
],
"type": "object"
},
"AssessmentContract": {
"additionalProperties": false,
"properties": {
"runs": {
"items": {
"$ref": "#/definitions/RunAssessmentContract"
},
"type": "array"
}
},
"required": [
"runs"
],
"type": "object"
},
"AssessmentKind": {
"enum": [
"required_check",
"signal",
"asset_validation"
],
"type": "string"
},
"AssessmentOutcome": {
"enum": [
"passed",
"failed",
"partial",
"not_evaluated",
"unavailable",
"error"
],
"type": "string"
},
"AssessmentPolicy": {
"enum": [
"hard_gate",
"advisory"
],
"type": "string"
},
"AssessmentResult": {
"properties": {
"criterion_id": {
"type": "string"
},
"dimension": {
"$ref": "#/definitions/EvaluationDimension"
},
"evidence": {
"items": {
"$ref": "#/definitions/EvidenceReference"
},
"type": "array"
},
"kind": {
"$ref": "#/definitions/AssessmentKind"
},
"outcome": {
"$ref": "#/definitions/AssessmentOutcome"
},
"policy": {
"$ref": "#/definitions/AssessmentPolicy"
},
"score": {
"anyOf": [
{
"$ref": "#/definitions/AssessmentScore"
},
{
"type": "null"
}
]
},
"summary": {
"type": "string"
},
"target": {
"$ref": "#/definitions/AssessmentTarget"
}
},
"required": [
"criterion_id",
"dimension",
"kind",
"outcome",
"policy",
"summary",
"target"
],
"type": "object"
},
"AssessmentScore": {
"properties": {
"awarded": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"possible": {
"format": "uint8",
"minimum": 0,
"type": "integer"
}
},
"required": [
"awarded",
"possible"
],
"type": "object"
},
"AssessmentTarget": {
"properties": {
"id": {
"type": "string"
},
"kind": {
"$ref": "#/definitions/AssessmentTargetKind"
}
},
"required": [
"id",
"kind"
],
"type": "object"
},
"AssessmentTargetKind": {
"enum": [
"criterion"
],
"type": "string"
},
"AssetValidationOutcome": {
"enum": [
"valid",
"invalid",
"malformed",
"oversized",
"not_produced",
"unreadable",
"unsafe_path",
"removed_during_cleanup",
"unexpected",
"not_evaluated"
],
"type": "string"
},
"AssetValidationResult": {
"properties": {
"asset_id": {
"type": "string"
},
"evidence": {
"items": {
"$ref": "#/definitions/EvidenceReference"
},
"type": "array"
},
"outcome": {
"$ref": "#/definitions/AssetValidationOutcome"
},
"summary": {
"type": "string"
}
},
"required": [
"asset_id",
"outcome",
"summary"
],
"type": "object"
},
"AuditEvidence": {
"properties": {
"detail": {
"description": "Sanitized, bounded description. Never contains secret material.",
"type": "string"
},
"function_id": {
"type": [
"string",
"null"
]
},
"message_index": {
"description": "Index of the transcript message the observation anchors to.",
"format": "uint",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"detail"
],
"type": "object"
},
"AuditFlag": {
"properties": {
"confidence": {
"format": "double",
"type": [
"number",
"null"
]
},
"evidence": {
"items": {
"$ref": "#/definitions/AuditEvidence"
},
"type": "array"
},
"kind": {
"$ref": "#/definitions/AuditFlagKind"
},
"severity": {
"$ref": "#/definitions/AuditSeverity"
},
"source": {
"$ref": "#/definitions/AuditFlagSource"
},
"summary": {
"type": "string"
}
},
"required": [
"kind",
"severity",
"source",
"summary"
],
"type": "object"
},
"AuditFlagKind": {
"oneOf": [
{
"description": "The subject invoked the E2E control plane or another function the scenario explicitly denies — the verification mechanism itself.",
"enum": [
"verifier_tampering"
],
"type": "string"
},
{
"description": "The subject addressed a session outside its own session tree.",
"enum": [
"out_of_scope_session_access"
],
"type": "string"
},
{
"description": "Live secret material from the runner environment appeared in the transcript.",
"enum": [
"secret_exposure"
],
"type": "string"
},
{
"description": "The subject invoked a destructive session or state operation.",
"enum": [
"destructive_action"
],
"type": "string"
}
]
},
"AuditFlagSource": {
"enum": [
"deterministic"
],
"type": "string"
},
"AuditReport": {
"properties": {
"flags": {
"items": {
"$ref": "#/definitions/AuditFlag"
},
"type": "array"
}
},
"required": [
"flags"
],
"type": "object"
},
"AuditSeverity": {
"enum": [
"info",
"warning",
"critical"
],
"type": "string"
},
"BooleanCondition": {
"additionalProperties": false,
"properties": {
"equals": {
"type": "boolean"
},
"node_id": {
"type": "string"
},
"port": {
"type": "string"
}
},
"required": [
"equals",
"node_id",
"port"
],
"type": "object"
},
"Capability": {
"description": "What a scenario needs from the engine, the runner host or a fixture to run. The closed list keeps the catalog honest: a new need is a new variant, and every id is spelled once.",
"enum": [
"browser::interactive",
"cross_repo_contract_simulator::v1",
"curl",
"docker",
"e2e::adaptive-flow-v1",
"e2e::control-plane-v1",
"e2e::filesystem",
"e2e::git",
"e2e::run-scoped-fixtures",
"e2e::shell",
"e2e::subagents",
"e2e::workflow-resume-v1",
"fixture::multi-origin-http",
"git",
"git::deterministic-fixture-v1",
"git::offline-bundle",
"github::trusted-handoff",
"harness::independent_session",
"harness::post-turn-validation",
"harness::scripted-dialogue-v1",
"iii::coder",
"iii::compose",
"iii::database",
"iii::functions",
"iii::registry",
"iii::shell",
"iii::state",
"iii::triggers",
"iii::workers",
"incident_fixture::v1",
"node",
"playwright",
"python3",
"release_shadow::read-only-v1",
"release_train_simulator::v1"
],
"type": "string"
},
"CapturedInvariant": {
"properties": {
"id": {
"type": "string"
},
"passed": {
"type": "boolean"
},
"reason": {
"type": "string"
}
},
"required": [
"id",
"passed",
"reason"
],
"type": "object"
},
"CompletionState": {
"enum": [
"completed",
"task_incomplete",
"undetermined"
],
"type": "string"
},
"ContaminationScope": {
"enum": [
"none",
"case",
"execution"
],
"type": "string"
},
"ControlPlaneEvidence": {
"properties": {
"functions": {
"items": {
"$ref": "#/definitions/FunctionContractEvidence"
},
"type": "array"
}
},
"required": [
"functions"
],
"type": "object"
},
"CostReport": {
"properties": {
"subject_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"total_usd": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"CriterionReport": {
"properties": {
"awarded": {
"format": "uint8",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"description": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"possible": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"reason": {
"type": "string"
}
},
"required": [
"id",
"possible",
"reason"
],
"type": "object"
},
"DeliverableContentFormat": {
"enum": [
"json",
"text_utf8"
],
"type": "string"
},
"DeliverableContract": {
"properties": {
"artifacts": {
"items": {
"$ref": "#/definitions/ArtifactExpectation"
},
"type": "array"
},
"capture_before_cleanup": {
"type": "boolean"
},
"invariants": {
"items": {
"$ref": "#/definitions/InvariantSpec"
},
"type": "array"
},
"provenance_required": {
"type": "boolean"
}
},
"required": [
"artifacts",
"capture_before_cleanup",
"invariants",
"provenance_required"
],
"type": "object"
},
"DeliverableReport": {
"properties": {
"artifact": {
"anyOf": [
{
"$ref": "#/definitions/ArtifactReference"
},
{
"type": "null"
}
]
},
"content_format": {
"$ref": "#/definitions/DeliverableContentFormat"
},
"content_sha256": {
"type": "string"
},
"content_size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"id": {
"type": "string"
},
"invariants": {
"items": {
"$ref": "#/definitions/CapturedInvariant"
},
"type": "array"
},
"kind": {
"type": "string"
},
"media_type": {
"type": "string"
},
"preview": true,
"provenance": {
"items": {
"$ref": "#/definitions/ProvenanceEvidence"
},
"type": "array"
},
"provenance_valid": {
"type": "boolean"
},
"schema_valid": {
"type": "boolean"
}
},
"required": [
"content_format",
"content_sha256",
"content_size_bytes",
"id",
"invariants",
"kind",
"media_type",
"preview",
"provenance",
"provenance_valid",
"schema_valid"
],
"type": "object"
},
"DependencyPolicy": {
"enum": [
"succeeded",
"terminal"
],
"type": "string"
},
"DimensionReport": {
"properties": {
"dimension": {
"$ref": "#/definitions/EvaluationDimension"
},
"passed": {
"type": [
"boolean",
"null"
]
},
"signals": true
},
"required": [
"dimension",
"signals"
],
"type": "object"
},
"DurableArchiveReference": {
"additionalProperties": false,
"properties": {
"archive_id": {
"type": "string"
},
"created_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"expires_at": {
"type": [
"string",
"null"
]
},
"identity_sha256": {
"type": "string"
},
"manifest": {
"$ref": "#/definitions/StorageObjectReference"
},
"manifest_backup": {
"$ref": "#/definitions/StorageObjectReference"
},
"retention_class": {
"$ref": "#/definitions/RetentionClass"
}
},
"required": [
"archive_id",
"created_at",
"execution_id",
"identity_sha256",
"manifest",
"manifest_backup",
"retention_class"
],
"type": "object"
},
"E2eManifest": {
"additionalProperties": false,
"properties": {
"control_plane": {
"$ref": "#/definitions/ControlPlaneEvidence"
},
"execution": {
"$ref": "#/definitions/ExecutionIdentity"
},
"observation_contract": {
"anyOf": [
{
"$ref": "#/definitions/ObservationRunContract"
},
{
"type": "null"
}
]
},
"subject": {
"$ref": "#/definitions/ModelArtifact"
},
"system_under_test": {
"$ref": "#/definitions/SystemUnderTestIdentity"
},
"worker_contracts": {
"items": {
"$ref": "#/definitions/ObservedWorkerContract"
},
"type": "array"
}
},
"required": [
"control_plane",
"execution",
"subject",
"system_under_test"
],
"type": "object"
},
"E2eObservationEnvelope": {
"additionalProperties": false,
"properties": {
"evidence": {
"$ref": "#/definitions/ObservationEvidence"
},
"identity": {
"$ref": "#/definitions/ObservationIdentity"
},
"mode": {
"$ref": "#/definitions/ObservationMode"
},
"outcome": {
"$ref": "#/definitions/ObservationOutcome"
},
"provenance": {
"$ref": "#/definitions/ObservationProvenance"
},
"samples": {
"items": {
"$ref": "#/definitions/ObservationSample"
},
"type": "array"
},
"schema": {
"type": "string"
}
},
"required": [
"evidence",
"identity",
"mode",
"outcome",
"provenance",
"samples",
"schema"
],
"type": "object"
},
"E2eReport": {
"additionalProperties": false,
"properties": {
"assessment_contract": {
"$ref": "#/definitions/AssessmentContract"
},
"engine_revision": {
"type": [
"string",
"null"
]
},
"execution": {
"$ref": "#/definitions/ExecutionIdentity"
},
"manifest": {
"anyOf": [
{
"$ref": "#/definitions/ArtifactReference"
},
{
"type": "null"
}
]
},
"objective_outcome": {
"$ref": "#/definitions/ObjectiveOutcome"
},
"observation_contract": {
"anyOf": [
{
"$ref": "#/definitions/ObservationRunContract"
},
{
"type": "null"
}
]
},
"passed": {
"type": "boolean"
},
"persistence_errors": {
"items": {
"type": "string"
},
"type": "array"
},
"redaction": {
"allOf": [
{
"$ref": "#/definitions/RedactionReport"
}
],
"default": {
"redacted_fields": 0,
"redacted_values": 0,
"rules": []
}
},
"report_state": {
"$ref": "#/definitions/ReportState"
},
"result_contract_sha256": {
"type": "string"
},
"scenarios": {
"items": {
"$ref": "#/definitions/E2eScenarioReport"
},
"type": "array"
},
"slot_start_deadline_seconds": {
"description": "Soft scheduling deadline: prevents starting slots after this interval.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"subject": {
"$ref": "#/definitions/ModelArtifact"
},
"system_under_test": {
"$ref": "#/definitions/SystemUnderTestIdentity"
}
},
"required": [
"assessment_contract",
"execution",
"objective_outcome",
"passed",
"report_state",
"result_contract_sha256",
"scenarios",
"subject",
"system_under_test"
],
"type": "object"
},
"E2eRunReport": {
"properties": {
"attempt_id": {
"type": "string"
},
"attempt_number": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"audit": {
"anyOf": [
{
"$ref": "#/definitions/AuditReport"
},
{
"type": "null"
}
],
"description": "Advisory behavioral audit over the captured transcript and metrics. Never contributes to score, status, gates, or longitudinal inputs."
},
"completion": {
"$ref": "#/definitions/CompletionState"
},
"cost": {
"$ref": "#/definitions/CostReport"
},
"criteria": {
"items": {
"$ref": "#/definitions/CriterionReport"
},
"type": "array"
},
"deliverables": {
"items": {
"$ref": "#/definitions/DeliverableReport"
},
"type": "array"
},
"dimensions": {
"items": {
"$ref": "#/definitions/DimensionReport"
},
"type": "array"
},
"efficiency": {
"anyOf": [
{
"$ref": "#/definitions/EfficiencyReport"
},
{
"type": "null"
}
]
},
"evaluators": {
"$ref": "#/definitions/EvaluatorStates"
},
"evidence": {
"items": {
"$ref": "#/definitions/ArtifactReference"
},
"type": "array"
},
"failures": {
"items": {
"$ref": "#/definitions/FailureRecord"
},
"type": "array"
},
"metrics": {
"anyOf": [
{
"$ref": "#/definitions/SessionMetricsPayload"
},
{
"type": "null"
}
]
},
"prompt": {
"type": "string"
},
"retry_attempts": {
"items": {
"$ref": "#/definitions/RetryAttemptReport"
},
"type": "array"
},
"run_id": {
"type": "string"
},
"scenario_flow": {
"anyOf": [
{
"$ref": "#/definitions/ScenarioFlowEvidence"
},
{
"type": "null"
}
],
"description": "Evidence-only flow identity emitted by Rust. This value is never accepted as runner input and cannot reconstruct executable configuration."
},
"scenario_measurements": {
"items": {
"$ref": "#/definitions/ScenarioMeasurement"
},
"type": "array"
},
"score": {
"format": "uint8",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"semantic_tests": {
"description": "Semantic tests executed inside a code-defined composite scenario. Product requests, polling and captures stay inside these meaningful test units.",
"items": {
"$ref": "#/definitions/WorkflowStepReport"
},
"type": "array"
},
"session_id": {
"type": "string"
},
"status": {
"$ref": "#/definitions/RunStatus"
},
"technical": {
"$ref": "#/definitions/TechnicalState"
},
"transcript": true,
"wall_time_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"worker_contracts": {
"description": "Exact request/response schemas observed after this scenario's setup. Run-scoped fixture functions cannot be observed in the suite preflight, so attempts retain them here and the suite folds them into the manifest.",
"items": {
"$ref": "#/definitions/ObservedWorkerContract"
},
"type": "array"
}
},
"required": [
"attempt_id",
"attempt_number",
"completion",
"cost",
"criteria",
"evaluators",
"prompt",
"run_id",
"session_id",
"status",
"technical",
"wall_time_ms"
],
"type": "object"
},
"E2eScenarioReport": {
"properties": {
"aggregate": {
"$ref": "#/definitions/ScenarioAggregate"
},
"behavior_sha256": {
"description": "Digest of the definition the case was materialized from. Absent only when no case could be materialized.",
"type": [
"string",
"null"
]
},
"case": {
"anyOf": [
{
"$ref": "#/definitions/ScenarioCase"
},
{
"type": "null"
}
]
},
"case_id": {
"default": "",
"type": "string"
},
"deferral_reason": {
"type": [
"string",
"null"
]
},
"execution_policy": {
"$ref": "#/definitions/ExecutionPolicy"
},
"passed": {
"type": "boolean"
},
"runs": {
"items": {
"$ref": "#/definitions/E2eRunReport"
},
"type": "array"
},
"scenario_id": {
"type": "string"
}
},
"required": [
"aggregate",
"execution_policy",
"passed",
"runs",
"scenario_id"
],
"type": "object"
},
"EfficiencyReport": {
"properties": {
"child_sessions": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"child_turns": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"critical_path_ms": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"effective_fan_out": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"function_call_errors": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"function_calls": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"input_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"observed_complexity": {
"$ref": "#/definitions/ObservedComplexityReport"
},
"observed_work": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"root_turns": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_attempts": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"total_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"transient_resumes": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"validation_retries": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wake_resumes": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wall_time_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"observed_complexity",
"technical_attempts",
"wall_time_ms"
],
"type": "object"
},
"EvaluationDimension": {
"enum": [
"deliverable",
"structural_integrity",
"efficiency",
"robustness",
"e2e_infrastructure"
],
"type": "string"
},
"EvaluatorAvailability": {
"enum": [
"not_required",
"pending",
"available",
"unavailable"
],
"type": "string"
},
"EvaluatorStates": {
"properties": {
"completion": {
"$ref": "#/definitions/EvaluatorAvailability"
}
},
"required": [
"completion"
],
"type": "object"
},
"EvidenceReference": {
"properties": {
"artifact_id": {
"type": "string"
},
"artifact_sha256": {
"type": "string"
},
"locator": {
"type": [
"string",
"null"
]
}
},
"required": [
"artifact_id",
"artifact_sha256"
],
"type": "object"
},
"ExecutionIdentity": {
"properties": {
"completed_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"lane": {
"type": "string"
},
"started_at": {
"type": "string"
}
},
"required": [
"completed_at",
"execution_id",
"lane",
"started_at"
],
"type": "object"
},
"ExecutionPhase": {
"enum": [
"requested",
"admitted",
"preflighting",
"materializing",
"setting_up",
"executing",
"collecting",
"evaluating",
"persisting",
"cleaning_up",
"finalizing",
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
},
"ExecutionPolicy": {
"properties": {
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_total_tokens": {
"description": "Shared Harness token budget. `None` leaves the Harness budget unbounded for this scenario.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_turns": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"max_validation_retries": {
"description": "Optional per-session cap for post-turn validation denials. `None` preserves the Harness-configured default.",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stuck_timeout_seconds": {
"description": "Stop only after this many seconds without observable useful progress. Large scenarios have no fixed wall-clock deadline.",
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"max_turns",
"stuck_timeout_seconds"
],
"type": "object"
},
"ExecutionRealism": {
"enum": [
"synthetic",
"realistic_simulator",
"frozen_real_artifact"
],
"type": "string"
},
"FailureDomain": {
"enum": [
"subject",
"resource",
"e2e_infrastructure"
],
"type": "string"
},
"FailurePhase": {
"enum": [
"setup",
"execute",
"collect",
"evaluate",
"cleanup"
],
"type": "string"
},
"FailureRecord": {
"properties": {
"code": {
"default": "",
"type": "string"
},
"contamination": {
"allOf": [
{
"$ref": "#/definitions/ContaminationScope"
}
],
"default": "none"
},
"domain": {
"$ref": "#/definitions/FailureDomain"
},
"message": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/FailurePhase"
},
"retry_scope": {
"allOf": [
{
"$ref": "#/definitions/RetryScope"
}
],
"default": "none"
},
"status": {
"allOf": [
{
"$ref": "#/definitions/RunStatus"
}
],
"description": "The failure's original classification, independent of the primary run status."
}
},
"required": [
"domain",
"message",
"phase",
"status"
],
"type": "object"
},
"FunctionContractEvidence": {
"properties": {
"function_id": {
"type": "string"
},
"request_schema": true,
"response_schema": true,
"sha256": {
"type": "string"
}
},
"required": [
"function_id",
"request_schema",
"response_schema",
"sha256"
],
"type": "object"
},
"HumanHorizon": {
"properties": {
"basis": {
"allOf": [
{
"$ref": "#/definitions/HumanHorizonBasis"
}
],
"default": "unknown"
},
"max_minutes": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"min_minutes": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"HumanHorizonBasis": {
"enum": [
"unknown",
"author_estimate",
"measured"
],
"type": "string"
},
"InvariantSpec": {
"properties": {
"description": {
"type": "string"
},
"id": {
"type": "string"
}
},
"required": [
"description",
"id"
],
"type": "object"
},
"ModelArtifact": {
"properties": {
"agent": {
"anyOf": [
{
"$ref": "#/definitions/AgentProfileArtifact"
},
{
"type": "null"
}
]
},
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
}
},
"required": [
"context_window",
"max_output_tokens",
"model",
"provider"
],
"type": "object"
},
"ObjectiveOutcome": {
"enum": [
"passed",
"failed",
"inconclusive"
],
"type": "string"
},
"ObservationCorrelation": {
"additionalProperties": false,
"properties": {
"deployment_id": {
"type": "string"
},
"operation_id": {
"type": "string"
},
"system": {
"type": "string"
}
},
"required": [
"deployment_id",
"operation_id",
"system"
],
"type": "object"
},
"ObservationDataAvailability": {
"enum": [
"complete",
"partial",
"unavailable"
],
"type": "string"
},
"ObservationDecision": {
"enum": [
"observe_only"
],
"type": "string"
},
"ObservationEnvironment": {
"enum": [
"demonstration"
],
"type": "string"
},
"ObservationEvidence": {
"additionalProperties": false,
"properties": {
"artifacts": {
"items": {
"$ref": "#/definitions/ArtifactReference"
},
"type": "array"
},
"manifest_sha256": {
"type": [
"string",
"null"
]
},
"results_sha256": {
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"ObservationExecutionIdentity": {
"additionalProperties": false,
"properties": {
"attempt": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"completed_at": {
"type": "string"
},
"id": {
"type": "string"
},
"request_sha256": {
"type": "string"
},
"run_contract_sha256": {
"type": "string"
},
"started_at": {
"type": "string"
}
},
"required": [
"attempt",
"completed_at",
"id",
"request_sha256",
"run_contract_sha256",
"started_at"
],
"type": "object"
},
"ObservationIdentity": {
"additionalProperties": false,
"properties": {
"execution": {
"$ref": "#/definitions/ObservationExecutionIdentity"
},
"plan": {
"$ref": "#/definitions/ObservationPlanIdentity"
},
"runner": {
"$ref": "#/definitions/RunnerIdentity"
},
"target": {
"$ref": "#/definitions/ObservationTargetIdentity"
}
},
"required": [
"execution",
"plan",
"runner",
"target"
],
"type": "object"
},
"ObservationMetric": {
"additionalProperties": false,
"description": "A metric is explicit about its unit and availability so a recorded zero cannot be mistaken for a missing measurement. The legacy `metrics` object remains alongside this additive projection for current consumers.",
"properties": {
"availability": {
"$ref": "#/definitions/ObservationDataAvailability"
},
"origin": {
"$ref": "#/definitions/ObservationMetricOrigin"
},
"unit": {
"type": "string"
},
"value": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"required": [
"availability",
"origin",
"unit"
],
"type": "object"
},
"ObservationMetricDerivation": {
"additionalProperties": false,
"properties": {
"formula": {
"type": "string"
},
"formula_version": {
"type": "string"
},
"metric": {
"type": "string"
},
"origin": {
"$ref": "#/definitions/ObservationMetricOrigin"
}
},
"required": [
"formula",
"formula_version",
"metric",
"origin"
],
"type": "object"
},
"ObservationMetricOrigin": {
"enum": [
"observed",
"derived_from_observed"
],
"type": "string"
},
"ObservationMode": {
"additionalProperties": false,
"properties": {
"decision": {
"$ref": "#/definitions/ObservationDecision"
},
"environment": {
"$ref": "#/definitions/ObservationEnvironment"
}
},
"required": [
"decision",
"environment"
],
"type": "object"
},
"ObservationObjective": {
"enum": [
"passed",
"hard_gate_failed",
"technical_failed",
"infrastructure_failed",
"cancelled",
"unsupported_plan"
],
"type": "string"
},
"ObservationOutcome": {
"additionalProperties": false,
"properties": {
"control_phase": {
"type": "string"
},
"data_availability": {
"$ref": "#/definitions/ObservationDataAvailability"
},
"error": {
"type": "string"
},
"objective": {
"$ref": "#/definitions/ObservationObjective"
},
"passed": {
"type": [
"boolean",
"null"
]
}
},
"required": [
"control_phase",
"data_availability",
"objective"
],
"type": "object"
},
"ObservationPlanIdentity": {
"additionalProperties": false,
"properties": {
"catalog_sha256": {
"type": "string"
},
"id": {
"type": "string"
},
"revision": {
"type": "string"
},
"sha256": {
"type": "string"
}
},
"required": [
"catalog_sha256",
"id",
"revision",
"sha256"
],
"type": "object"
},
"ObservationProvenance": {
"additionalProperties": false,
"properties": {
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"system_under_test": {
"anyOf": [
{
"$ref": "#/definitions/SystemUnderTestIdentity"
},
{
"type": "null"
}
]
}
},
"required": [
"subject_model",
"subject_provider"
],
"type": "object"
},
"ObservationRunContract": {
"additionalProperties": false,
"properties": {
"attempt": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"correlation": {
"$ref": "#/definitions/ObservationCorrelation"
},
"mode": {
"$ref": "#/definitions/ObservationMode"
},
"plan": {
"$ref": "#/definitions/ObservationPlanIdentity"
},
"runner": {
"$ref": "#/definitions/RunnerIdentity"
},
"selected_cases": {
"items": {
"$ref": "#/definitions/ObservationSelectedCase"
},
"type": "array"
},
"target": {
"$ref": "#/definitions/ObservationTargetIdentity"
}
},
"required": [
"attempt",
"correlation",
"mode",
"plan",
"runner",
"selected_cases",
"target"
],
"type": "object"
},
"ObservationSample": {
"additionalProperties": false,
"properties": {
"attempt_id": {
"type": "string"
},
"behavior_sha256": {
"type": [
"string",
"null"
]
},
"case_id": {
"type": "string"
},
"data_availability": {
"$ref": "#/definitions/ObservationDataAvailability"
},
"derivations": {
"items": {
"$ref": "#/definitions/ObservationMetricDerivation"
},
"type": "array"
},
"metric_values": {
"additionalProperties": {
"$ref": "#/definitions/ObservationMetric"
},
"default": {},
"type": "object"
},
"metrics": {
"anyOf": [
{
"$ref": "#/definitions/EfficiencyReport"
},
{
"type": "null"
}
]
},
"origin": {
"$ref": "#/definitions/ObservationMetricOrigin"
},
"run_id": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"status": {
"$ref": "#/definitions/RunStatus"
}
},
"required": [
"attempt_id",
"case_id",
"data_availability",
"origin",
"run_id",
"scenario_id",
"seed",
"status"
],
"type": "object"
},
"ObservationSelectedCase": {
"additionalProperties": false,
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"inputs_sha256": {
"type": "string"
},
"scenario_id": {
"$ref": "#/definitions/ScenarioId"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"contract_sha256",
"inputs_sha256",
"scenario_id",
"seed"
],
"type": "object"
},
"ObservationTargetIdentity": {
"additionalProperties": false,
"properties": {
"application": {
"type": "string"
},
"stack": {
"$ref": "#/definitions/StackIdentity"
},
"version": {
"type": "string"
}
},
"required": [
"application",
"stack",
"version"
],
"type": "object"
},
"ObservedComplexityReport": {
"properties": {
"ambiguity_level": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"artifact_count": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"coordination_edges": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"dependency_depth": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"external_systems": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"parallel_branches": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"planning_depth": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"state_transitions": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"validation_loops": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wake_cycles": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"ObservedWorkerContract": {
"additionalProperties": false,
"properties": {
"function_id": {
"type": "string"
},
"request_schema_sha256": {
"type": "string"
},
"response_schema_sha256": {
"type": "string"
}
},
"required": [
"function_id",
"request_schema_sha256",
"response_schema_sha256"
],
"type": "object"
},
"PortValueKind": {
"enum": [
"boolean",
"json",
"text_utf8",
"assessment"
],
"type": "string"
},
"ProvenanceEvidence": {
"properties": {
"kind": {
"type": "string"
},
"relation": {
"type": "string"
},
"source_id": {
"type": "string"
}
},
"required": [
"kind",
"relation",
"source_id"
],
"type": "object"
},
"RedactionReport": {
"properties": {
"redacted_fields": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"redacted_values": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"rules": {
"items": {
"type": "string"
},
"type": "array",
"uniqueItems": true
}
},
"required": [
"redacted_fields",
"redacted_values",
"rules"
],
"type": "object"
},
"ReportState": {
"enum": [
"complete",
"partial"
],
"type": "string"
},
"RetentionClass": {
"enum": [
"temporary",
"pull_request",
"longitudinal",
"canonical"
],
"type": "string"
},
"RetryAttemptReport": {
"properties": {
"attempt_id": {
"type": "string"
},
"attempt_number": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"completion": {
"$ref": "#/definitions/CompletionState"
},
"cost": {
"$ref": "#/definitions/CostReport"
},
"deliverables": {
"items": {
"$ref": "#/definitions/DeliverableReport"
},
"type": "array"
},
"dimensions": {
"items": {
"$ref": "#/definitions/DimensionReport"
},
"type": "array"
},
"efficiency": {
"anyOf": [
{
"$ref": "#/definitions/EfficiencyReport"
},
{
"type": "null"
}
]
},
"evaluators": {
"$ref": "#/definitions/EvaluatorStates"
},
"evidence": {
"items": {
"$ref": "#/definitions/ArtifactReference"
},
"type": "array"
},
"failures": {
"items": {
"$ref": "#/definitions/FailureRecord"
},
"type": "array"
},
"metrics": {
"anyOf": [
{
"$ref": "#/definitions/SessionMetricsPayload"
},
{
"type": "null"
}
]
},
"run_id": {
"type": "string"
},
"scenario_flow": {
"anyOf": [
{
"$ref": "#/definitions/ScenarioFlowEvidence"
},
{
"type": "null"
}
]
},
"score": {
"description": "The points the attempt's evaluated criteria awarded, as on the run.",
"format": "uint8",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"semantic_tests": {
"items": {
"$ref": "#/definitions/WorkflowStepReport"
},
"type": "array"
},
"session_id": {
"type": "string"
},
"status": {
"$ref": "#/definitions/RunStatus"
},
"technical": {
"$ref": "#/definitions/TechnicalState"
},
"transcript": true,
"wall_time_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"attempt_id",
"attempt_number",
"completion",
"cost",
"evaluators",
"failures",
"run_id",
"session_id",
"status",
"technical",
"wall_time_ms"
],
"type": "object"
},
"RetryScope": {
"enum": [
"none",
"same_slot"
],
"type": "string"
},
"RobustnessReport": {
"properties": {
"cost_per_successful_deliverable": {
"format": "double",
"type": [
"number",
"null"
]
},
"deliverable_success_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"eligible": {
"type": "boolean"
},
"flaky_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_wall_time_ms": {
"format": "double",
"type": [
"number",
"null"
]
},
"minimum_sample_size": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"p95_wall_time_ms": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"sample_size": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"structural_integrity_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"tail_minimum_sample_size": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"technical_failure_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"wall_time_variance": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"required": [
"eligible",
"minimum_sample_size",
"sample_size",
"tail_minimum_sample_size"
],
"type": "object"
},
"RunAssessmentContract": {
"properties": {
"assessments": {
"items": {
"$ref": "#/definitions/AssessmentResult"
},
"type": "array"
},
"assets": {
"items": {
"$ref": "#/definitions/AssetValidationResult"
},
"type": "array"
},
"attempt_id": {
"type": "string"
},
"run_id": {
"type": "string"
},
"system_status": {
"$ref": "#/definitions/SystemStatus"
}
},
"required": [
"attempt_id",
"run_id",
"system_status"
],
"type": "object"
},
"RunStatus": {
"enum": [
"passed",
"hard_gate_failed",
"subject_error",
"resource_limit",
"infrastructure_error"
],
"type": "string"
},
"RunnerIdentity": {
"additionalProperties": false,
"properties": {
"name": {
"type": "string"
},
"revision": {
"type": "string"
},
"version": {
"type": "string"
}
},
"required": [
"name",
"revision",
"version"
],
"type": "object"
},
"ScenarioAggregate": {
"properties": {
"completed_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"completion_evidence_coverage": {
"format": "double",
"type": [
"number",
"null"
]
},
"completion_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"cost": {
"$ref": "#/definitions/CostReport"
},
"deferred_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"execution_reliability": {
"format": "double",
"type": [
"number",
"null"
]
},
"failed_attempt_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"mean_score": {
"description": "Mean score of the technically valid runs that have one; null when none.",
"format": "double",
"type": [
"number",
"null"
]
},
"observed_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"pass_rate": {
"format": "double",
"type": "number"
},
"passed_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"planned_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"required_passes": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"robustness": {
"$ref": "#/definitions/RobustnessReport"
},
"scored_runs": {
"description": "Technically valid runs that have a score.",
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"task_incomplete_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"technical_failures": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"technical_invalid_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"technical_valid_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"tokens_completed_p50": {
"format": "double",
"type": [
"number",
"null"
]
},
"tokens_per_completion": {
"format": "double",
"type": [
"number",
"null"
]
},
"total_tokens_consumed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"undetermined_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
}
},
"required": [
"completed_runs",
"cost",
"deferred_runs",
"observed_runs",
"pass_rate",
"passed_runs",
"planned_runs",
"required_passes",
"robustness",
"scored_runs",
"task_incomplete_runs",
"technical_failures",
"technical_invalid_runs",
"technical_valid_runs",
"undetermined_runs"
],
"type": "object"
},
"ScenarioCase": {
"properties": {
"behavior_sha256": {
"description": "Digest of the scenario definition this case was materialized from: prompt, execution policy, criteria, and the case-independent contract. Two cases with the same digest were evaluated by the same definition. Sealed by `ScenarioId::materialize`; empty only before sealing.",
"type": "string"
},
"case_id": {
"type": "string"
},
"characterization": {
"allOf": [
{
"$ref": "#/definitions/ScenarioCharacterization"
}
],
"default": {
"human_horizon": {
"basis": "unknown"
},
"realism": {
"execution": "synthetic",
"shadow": "none"
}
}
},
"deliverable_contract": {
"$ref": "#/definitions/DeliverableContract"
},
"inputs": true,
"inputs_sha256": {
"type": "string"
},
"required_capabilities": {
"items": {
"$ref": "#/definitions/Capability"
},
"type": "array"
},
"scenario_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"deliverable_contract",
"inputs",
"inputs_sha256",
"required_capabilities",
"scenario_id",
"seed"
],
"type": "object"
},
"ScenarioCharacterization": {
"properties": {
"human_horizon": {
"allOf": [
{
"$ref": "#/definitions/HumanHorizon"
}
],
"default": {
"basis": "unknown"
}
},
"realism": {
"allOf": [
{
"$ref": "#/definitions/ScenarioRealism"
}
],
"default": {
"execution": "synthetic",
"shadow": "none"
}
}
},
"type": "object"
},
"ScenarioFlowEvidence": {
"additionalProperties": false,
"properties": {
"checkpoint": {
"$ref": "#/definitions/ArtifactReference"
},
"cleanup": {
"$ref": "#/definitions/WorkflowCleanupReport"
},
"definition_sha256": {
"type": "string"
},
"snapshot": true
},
"required": [
"checkpoint",
"cleanup",
"definition_sha256",
"snapshot"
],
"type": "object"
},
"ScenarioId": {
"enum": [
"registry_planning",
"registry_implementation",
"registry_environment",
"registry_verification",
"kanban_c1_foundation",
"kanban_c2_persistence",
"kanban_c3_board",
"kanban_c4_ticket_flow",
"kanban_c5_edit_move",
"kanban_c6_discussion",
"kanban_c7_live",
"linkly_tutorial",
"context_pressure",
"minimal_path",
"persistent_state",
"insert_record",
"sequential_pipeline",
"database_migration_recovery",
"shell_coder_sandbox",
"research_pipeline",
"fanout_ladder",
"incident_response",
"todo_worker_simple",
"todo_worker_planned",
"engineering_endurance_ladder",
"git_regression_forensics",
"mechanical_reaction",
"timer_wake",
"receiving_operation",
"validation_loop",
"subagent_validation",
"subagent_validation_failure",
"validation_self_repair",
"validation_scope_enforcement",
"validation_chain",
"secret_hygiene",
"prompt_injection_resilience",
"moving_target",
"poison_message",
"cleanup_under_failure",
"depth_ladder",
"quorum_fan_in",
"contention_ledger",
"wake_chain_soak",
"chess_engine_build",
"chess_play_ladder",
"trend_blog",
"trending_topics_build",
"typescript_chat_service",
"tool_contract_recovery",
"policy_bound_action",
"cross_app_transaction",
"performance_regression",
"browser_cross_site",
"release_train_recovery",
"cross_repo_contract_migration"
],
"type": "string"
},
"ScenarioMeasurement": {
"additionalProperties": false,
"properties": {
"id": {
"type": "string"
},
"origin": {
"$ref": "#/definitions/ObservationMetricOrigin"
},
"unit": {
"type": "string"
},
"value": {
"format": "double",
"type": "number"
}
},
"required": [
"id",
"origin",
"unit",
"value"
],
"type": "object"
},
"ScenarioRealism": {
"properties": {
"execution": {
"allOf": [
{
"$ref": "#/definitions/ExecutionRealism"
}
],
"default": "synthetic"
},
"shadow": {
"allOf": [
{
"$ref": "#/definitions/ShadowMode"
}
],
"default": "none"
}
},
"type": "object"
},
"SessionMetricsPayload": {
"properties": {
"by_session": {
"items": {
"$ref": "#/definitions/SessionUsage"
},
"type": "array"
},
"complete": {
"type": "boolean"
},
"root_session_id": {
"type": "string"
},
"totals": {
"$ref": "#/definitions/SessionUsageTotals"
},
"traces": {
"default": null
}
},
"required": [
"by_session",
"complete",
"root_session_id",
"totals"
],
"type": "object"
},
"SessionUsage": {
"properties": {
"cache_read_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"context": {
"default": null
},
"cost_usd": {
"default": null,
"format": "double",
"type": [
"number",
"null"
]
},
"depth": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"function_call_errors": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"function_calls": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"input_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"parent_session_id": {
"default": null,
"type": [
"string",
"null"
]
},
"reasoning_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"session_id": {
"type": "string"
},
"transient_resumes": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"turns": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"validation_retries": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wake_resumes": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"depth",
"function_call_errors",
"function_calls",
"session_id",
"turns"
],
"type": "object"
},
"SessionUsageTotals": {
"properties": {
"cache_read_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"default": null,
"format": "double",
"type": [
"number",
"null"
]
},
"function_call_errors": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"function_calls": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"input_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"reasoning_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"sessions": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"transient_resumes": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"turns": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"validation_retries": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wake_resumes": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"function_call_errors",
"function_calls",
"sessions",
"turns"
],
"type": "object"
},
"ShadowMode": {
"enum": [
"none",
"read_only"
],
"type": "string"
},
"StackIdentity": {
"oneOf": [
{
"properties": {
"mode": {
"enum": [
"source"
],
"type": "string"
},
"workers_repository": {
"type": "string"
},
"workers_revision": {
"type": "string"
}
},
"required": [
"mode",
"workers_repository",
"workers_revision"
],
"type": "object"
},
{
"properties": {
"mode": {
"enum": [
"registry"
],
"type": "string"
},
"stack_lock_digest": {
"type": "string"
},
"stack_versions": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"required": [
"mode",
"stack_lock_digest",
"stack_versions"
],
"type": "object"
}
]
},
"StorageObjectReference": {
"properties": {
"bucket": {
"type": "string"
},
"key": {
"type": "string"
},
"media_type": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"uri": {
"type": "string"
}
},
"required": [
"bucket",
"key",
"media_type",
"sha256",
"size_bytes",
"uri"
],
"type": "object"
},
"SystemStatus": {
"enum": [
"unavailable",
"passed",
"hard_gate_failed",
"subject_error",
"resource_limit",
"infrastructure_error"
],
"type": "string"
},
"SystemUnderTestIdentity": {
"properties": {
"contract_hashes": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"e2e_repository": {
"type": "string"
},
"e2e_revision": {
"type": "string"
},
"engine_revision": {
"type": [
"string",
"null"
]
},
"engine_version": {
"type": "string"
},
"harness_version": {
"type": "string"
},
"stack": {
"$ref": "#/definitions/StackIdentity"
}
},
"required": [
"contract_hashes",
"e2e_repository",
"e2e_revision",
"engine_version",
"harness_version",
"stack"
],
"type": "object"
},
"TechnicalState": {
"enum": [
"valid",
"technical_invalid"
],
"type": "string"
},
"TypedPortValue": {
"additionalProperties": false,
"properties": {
"kind": {
"$ref": "#/definitions/PortValueKind"
},
"value": true
},
"required": [
"kind",
"value"
],
"type": "object"
},
"WorkflowAssetReport": {
"additionalProperties": false,
"properties": {
"artifact": {
"$ref": "#/definitions/ArtifactReference"
},
"content_sha256": {
"type": "string"
},
"id": {
"type": "string"
},
"kind": {
"type": "string"
},
"media_type": {
"type": "string"
},
"namespaced_id": {
"type": "string"
},
"preview": true,
"preview_truncated": {
"type": "boolean"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"artifact",
"content_sha256",
"id",
"kind",
"media_type",
"namespaced_id",
"preview",
"preview_truncated",
"size_bytes"
],
"type": "object"
},
"WorkflowCleanupReport": {
"additionalProperties": false,
"properties": {
"duration_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"failure": {
"type": [
"string",
"null"
]
},
"status": {
"$ref": "#/definitions/WorkflowCleanupStatus"
}
},
"required": [
"duration_ms",
"status"
],
"type": "object"
},
"WorkflowCleanupStatus": {
"enum": [
"succeeded",
"failed"
],
"type": "string"
},
"WorkflowEvaluationOutcome": {
"enum": [
"passed",
"failed",
"advisory",
"not_evaluated"
],
"type": "string"
},
"WorkflowEvaluationResult": {
"additionalProperties": false,
"properties": {
"evidence_ids": {
"default": [],
"items": {
"type": "string"
},
"type": "array"
},
"id": {
"type": "string"
},
"outcome": {
"$ref": "#/definitions/WorkflowEvaluationOutcome"
},
"score": {
"format": "double",
"type": [
"number",
"null"
]
},
"summary": {
"type": "string"
}
},
"required": [
"id",
"outcome",
"summary"
],
"type": "object"
},
"WorkflowFailurePhase": {
"enum": [
"preflight",
"execute",
"capture",
"persist",
"evaluate",
"cleanup",
"cancel"
],
"type": "string"
},
"WorkflowGateResult": {
"additionalProperties": false,
"properties": {
"evidence_ids": {
"default": [],
"items": {
"type": "string"
},
"type": "array"
},
"id": {
"type": "string"
},
"passed": {
"type": "boolean"
},
"reason": {
"type": "string"
}
},
"required": [
"id",
"passed",
"reason"
],
"type": "object"
},
"WorkflowStepFailure": {
"additionalProperties": false,
"properties": {
"message": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/WorkflowFailurePhase"
},
"technical": {
"type": "boolean"
}
},
"required": [
"message",
"phase",
"technical"
],
"type": "object"
},
"WorkflowStepReport": {
"additionalProperties": false,
"properties": {
"activation": {
"$ref": "#/definitions/ActivationPolicy"
},
"assets": {
"items": {
"$ref": "#/definitions/WorkflowAssetReport"
},
"type": "array"
},
"completed_at": {
"type": [
"string",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"dependencies": {
"items": {
"type": "string"
},
"type": "array"
},
"dependency_policy": {
"$ref": "#/definitions/DependencyPolicy"
},
"duration_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"evaluations": {
"items": {
"$ref": "#/definitions/WorkflowEvaluationResult"
},
"type": "array"
},
"failures": {
"items": {
"$ref": "#/definitions/WorkflowStepFailure"
},
"type": "array"
},
"hard_gates": {
"items": {
"$ref": "#/definitions/WorkflowGateResult"
},
"type": "array"
},
"harness_session_id": {
"type": [
"string",
"null"
]
},
"metrics": true,
"node_id": {
"type": "string"
},
"outputs": {
"additionalProperties": {
"$ref": "#/definitions/TypedPortValue"
},
"type": "object"
},
"redaction": {
"allOf": [
{
"$ref": "#/definitions/RedactionReport"
}
],
"default": {
"redacted_fields": 0,
"redacted_values": 0,
"rules": []
}
},
"required": {
"type": "boolean"
},
"skip_reason": {
"type": [
"string",
"null"
]
},
"started_at": {
"type": [
"string",
"null"
]
},
"status": {
"$ref": "#/definitions/WorkflowStepStatus"
},
"step_type": {
"type": "string"
},
"transcript": true
},
"required": [
"activation",
"dependencies",
"dependency_policy",
"duration_ms",
"node_id",
"required",
"status",
"step_type"
],
"type": "object"
},
"WorkflowStepStatus": {
"enum": [
"pending",
"running",
"succeeded",
"hard_gate_failed",
"failed",
"skipped",
"cancelled"
],
"type": "string"
}
},
"properties": {
"archive": {
"anyOf": [
{
"$ref": "#/definitions/DurableArchiveReference"
},
{
"type": "null"
}
]
},
"execution_id": {
"type": "string"
},
"manifest": {
"anyOf": [
{
"$ref": "#/definitions/E2eManifest"
},
{
"type": "null"
}
]
},
"observation": {
"anyOf": [
{
"$ref": "#/definitions/E2eObservationEnvelope"
},
{
"type": "null"
}
]
},
"observation_artifact": {
"anyOf": [
{
"$ref": "#/definitions/ArtifactReference"
},
{
"type": "null"
}
]
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
},
"report": {
"anyOf": [
{
"$ref": "#/definitions/E2eReport"
},
{
"type": "null"
}
]
},
"request_sha256": {
"type": "string"
},
"result_path": {
"type": [
"string",
"null"
]
},
"run_contract_sha256": {
"type": [
"string",
"null"
]
}
},
"required": [
"execution_id",
"phase",
"request_sha256"
],
"title": "ResultsGetResponse",
"type": "object"
}
},
{
"description": "List durable E2E executions using comparable identity filters.",
"metadata": {
"contract": {
"capabilities": [
"results-list"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::results-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ScenarioId": {
"enum": [
"registry_planning",
"registry_implementation",
"registry_environment",
"registry_verification",
"kanban_c1_foundation",
"kanban_c2_persistence",
"kanban_c3_board",
"kanban_c4_ticket_flow",
"kanban_c5_edit_move",
"kanban_c6_discussion",
"kanban_c7_live",
"linkly_tutorial",
"context_pressure",
"minimal_path",
"persistent_state",
"insert_record",
"sequential_pipeline",
"database_migration_recovery",
"shell_coder_sandbox",
"research_pipeline",
"fanout_ladder",
"incident_response",
"todo_worker_simple",
"todo_worker_planned",
"engineering_endurance_ladder",
"git_regression_forensics",
"mechanical_reaction",
"timer_wake",
"receiving_operation",
"validation_loop",
"subagent_validation",
"subagent_validation_failure",
"validation_self_repair",
"validation_scope_enforcement",
"validation_chain",
"secret_hygiene",
"prompt_injection_resilience",
"moving_target",
"poison_message",
"cleanup_under_failure",
"depth_ladder",
"quorum_fan_in",
"contention_ledger",
"wake_chain_soak",
"chess_engine_build",
"chess_play_ladder",
"trend_blog",
"trending_topics_build",
"typescript_chat_service",
"tool_contract_recovery",
"policy_bound_action",
"cross_app_transaction",
"performance_regression",
"browser_cross_site",
"release_train_recovery",
"cross_repo_contract_migration"
],
"type": "string"
}
},
"properties": {
"lane": {
"default": null,
"type": [
"string",
"null"
]
},
"limit": {
"default": 50,
"format": "uint16",
"minimum": 0,
"type": "integer"
},
"scenario_id": {
"anyOf": [
{
"$ref": "#/definitions/ScenarioId"
},
{
"type": "null"
}
],
"default": null
}
},
"title": "ResultsListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ActiveAttempt": {
"properties": {
"attempt_id": {
"type": "string"
},
"resume_state_path": {
"type": [
"string",
"null"
]
},
"resume_state_sha256": {
"type": [
"string",
"null"
]
},
"run_id": {
"default": "",
"type": "string"
},
"scenario_id": {
"$ref": "#/definitions/ScenarioId"
},
"session_id": {
"type": "string"
}
},
"required": [
"attempt_id",
"scenario_id",
"session_id"
],
"type": "object"
},
"ExecutionPhase": {
"enum": [
"requested",
"admitted",
"preflighting",
"materializing",
"setting_up",
"executing",
"collecting",
"evaluating",
"persisting",
"cleaning_up",
"finalizing",
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
},
"JournalProgress": {
"properties": {
"active_attempt_id": {
"type": [
"string",
"null"
]
},
"attempts_finished": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"attempts_started": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"committed_events": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"last_event_sha256": {
"type": [
"string",
"null"
]
},
"phase": {
"type": [
"string",
"null"
]
},
"planned_slots": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"runs_committed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"slots_deferred": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"subject_observations_committed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"terminal": {
"type": "boolean"
},
"terminal_reason": {
"type": [
"string",
"null"
]
},
"terminal_state": {
"anyOf": [
{
"$ref": "#/definitions/JournalTerminalState"
},
{
"type": "null"
}
]
}
},
"required": [
"attempts_finished",
"attempts_started",
"committed_events",
"planned_slots",
"runs_committed",
"slots_deferred",
"subject_observations_committed",
"terminal"
],
"type": "object"
},
"JournalTerminalState": {
"enum": [
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
},
"PhaseTransition": {
"properties": {
"at": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
},
"reason": {
"type": "string"
}
},
"required": [
"at",
"phase",
"reason"
],
"type": "object"
},
"ScenarioId": {
"enum": [
"registry_planning",
"registry_implementation",
"registry_environment",
"registry_verification",
"kanban_c1_foundation",
"kanban_c2_persistence",
"kanban_c3_board",
"kanban_c4_ticket_flow",
"kanban_c5_edit_move",
"kanban_c6_discussion",
"kanban_c7_live",
"linkly_tutorial",
"context_pressure",
"minimal_path",
"persistent_state",
"insert_record",
"sequential_pipeline",
"database_migration_recovery",
"shell_coder_sandbox",
"research_pipeline",
"fanout_ladder",
"incident_response",
"todo_worker_simple",
"todo_worker_planned",
"engineering_endurance_ladder",
"git_regression_forensics",
"mechanical_reaction",
"timer_wake",
"receiving_operation",
"validation_loop",
"subagent_validation",
"subagent_validation_failure",
"validation_self_repair",
"validation_scope_enforcement",
"validation_chain",
"secret_hygiene",
"prompt_injection_resilience",
"moving_target",
"poison_message",
"cleanup_under_failure",
"depth_ladder",
"quorum_fan_in",
"contention_ledger",
"wake_chain_soak",
"chess_engine_build",
"chess_play_ladder",
"trend_blog",
"trending_topics_build",
"typescript_chat_service",
"tool_contract_recovery",
"policy_bound_action",
"cross_app_transaction",
"performance_regression",
"browser_cross_site",
"release_train_recovery",
"cross_repo_contract_migration"
],
"type": "string"
},
"StatusResponse": {
"properties": {
"active_attempt": {
"anyOf": [
{
"$ref": "#/definitions/ActiveAttempt"
},
{
"type": "null"
}
]
},
"cancel_requested": {
"type": "boolean"
},
"error": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"journal_progress": {
"$ref": "#/definitions/JournalProgress"
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
},
"terminal": {
"type": "boolean"
},
"transitions": {
"items": {
"$ref": "#/definitions/PhaseTransition"
},
"type": "array"
},
"updated_at": {
"type": "string"
}
},
"required": [
"cancel_requested",
"execution_id",
"journal_progress",
"phase",
"terminal",
"transitions",
"updated_at"
],
"type": "object"
}
},
"properties": {
"executions": {
"items": {
"$ref": "#/definitions/StatusResponse"
},
"type": "array"
}
},
"required": [
"executions"
],
"title": "ResultsListResponse",
"type": "object"
}
},
{
"description": "Delete expired E2E objects and tombstone their history records.",
"metadata": {
"contract": {
"capabilities": [
"retention-sweep"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::retention-sweep",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"before": {
"type": "string"
},
"dry_run": {
"default": false,
"type": "boolean"
},
"limit": {
"default": 100,
"format": "uint16",
"minimum": 0,
"type": "integer"
}
},
"required": [
"before"
],
"title": "RetentionSweepRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"deleted_archives": {
"items": {
"type": "string"
},
"type": "array"
},
"dry_run": {
"type": "boolean"
},
"inspected": {
"format": "uint32",
"minimum": 0,
"type": "integer"
}
},
"required": [
"deleted_archives",
"dry_run",
"inspected"
],
"title": "RetentionSweepResponse",
"type": "object"
}
},
{
"description": "Admit an asynchronous, idempotent E2E execution.",
"metadata": {
"contract": {
"capabilities": [
"run"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::run",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"definitions": {
"ObservationCorrelation": {
"additionalProperties": false,
"properties": {
"deployment_id": {
"type": "string"
},
"operation_id": {
"type": "string"
},
"system": {
"type": "string"
}
},
"required": [
"deployment_id",
"operation_id",
"system"
],
"type": "object"
},
"ObservationDecision": {
"enum": [
"observe_only"
],
"type": "string"
},
"ObservationEnvironment": {
"enum": [
"demonstration"
],
"type": "string"
},
"ObservationMode": {
"additionalProperties": false,
"properties": {
"decision": {
"$ref": "#/definitions/ObservationDecision"
},
"environment": {
"$ref": "#/definitions/ObservationEnvironment"
}
},
"required": [
"decision",
"environment"
],
"type": "object"
},
"ObservationPlanIdentity": {
"additionalProperties": false,
"properties": {
"catalog_sha256": {
"type": "string"
},
"id": {
"type": "string"
},
"revision": {
"type": "string"
},
"sha256": {
"type": "string"
}
},
"required": [
"catalog_sha256",
"id",
"revision",
"sha256"
],
"type": "object"
},
"ObservationRunContract": {
"additionalProperties": false,
"properties": {
"attempt": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"correlation": {
"$ref": "#/definitions/ObservationCorrelation"
},
"mode": {
"$ref": "#/definitions/ObservationMode"
},
"plan": {
"$ref": "#/definitions/ObservationPlanIdentity"
},
"runner": {
"$ref": "#/definitions/RunnerIdentity"
},
"selected_cases": {
"items": {
"$ref": "#/definitions/ObservationSelectedCase"
},
"type": "array"
},
"target": {
"$ref": "#/definitions/ObservationTargetIdentity"
}
},
"required": [
"attempt",
"correlation",
"mode",
"plan",
"runner",
"selected_cases",
"target"
],
"type": "object"
},
"ObservationSelectedCase": {
"additionalProperties": false,
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"inputs_sha256": {
"type": "string"
},
"scenario_id": {
"$ref": "#/definitions/ScenarioId"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"contract_sha256",
"inputs_sha256",
"scenario_id",
"seed"
],
"type": "object"
},
"ObservationTargetIdentity": {
"additionalProperties": false,
"properties": {
"application": {
"type": "string"
},
"stack": {
"$ref": "#/definitions/StackIdentity"
},
"version": {
"type": "string"
}
},
"required": [
"application",
"stack",
"version"
],
"type": "object"
},
"RunnerIdentity": {
"additionalProperties": false,
"properties": {
"name": {
"type": "string"
},
"revision": {
"type": "string"
},
"version": {
"type": "string"
}
},
"required": [
"name",
"revision",
"version"
],
"type": "object"
},
"ScenarioId": {
"enum": [
"registry_planning",
"registry_implementation",
"registry_environment",
"registry_verification",
"kanban_c1_foundation",
"kanban_c2_persistence",
"kanban_c3_board",
"kanban_c4_ticket_flow",
"kanban_c5_edit_move",
"kanban_c6_discussion",
"kanban_c7_live",
"linkly_tutorial",
"context_pressure",
"minimal_path",
"persistent_state",
"insert_record",
"sequential_pipeline",
"database_migration_recovery",
"shell_coder_sandbox",
"research_pipeline",
"fanout_ladder",
"incident_response",
"todo_worker_simple",
"todo_worker_planned",
"engineering_endurance_ladder",
"git_regression_forensics",
"mechanical_reaction",
"timer_wake",
"receiving_operation",
"validation_loop",
"subagent_validation",
"subagent_validation_failure",
"validation_self_repair",
"validation_scope_enforcement",
"validation_chain",
"secret_hygiene",
"prompt_injection_resilience",
"moving_target",
"poison_message",
"cleanup_under_failure",
"depth_ladder",
"quorum_fan_in",
"contention_ledger",
"wake_chain_soak",
"chess_engine_build",
"chess_play_ladder",
"trend_blog",
"trending_topics_build",
"typescript_chat_service",
"tool_contract_recovery",
"policy_bound_action",
"cross_app_transaction",
"performance_regression",
"browser_cross_site",
"release_train_recovery",
"cross_repo_contract_migration"
],
"type": "string"
},
"StackIdentity": {
"oneOf": [
{
"properties": {
"mode": {
"enum": [
"source"
],
"type": "string"
},
"workers_repository": {
"type": "string"
},
"workers_revision": {
"type": "string"
}
},
"required": [
"mode",
"workers_repository",
"workers_revision"
],
"type": "object"
},
{
"properties": {
"mode": {
"enum": [
"registry"
],
"type": "string"
},
"stack_lock_digest": {
"type": "string"
},
"stack_versions": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"required": [
"mode",
"stack_lock_digest",
"stack_versions"
],
"type": "object"
}
]
}
},
"properties": {
"agent": {
"type": [
"string",
"null"
]
},
"idempotency_key": {
"type": "string"
},
"label": {
"default": "",
"type": "string"
},
"lane": {
"default": "local",
"type": "string"
},
"model": {
"type": "string"
},
"progress_interval_seconds": {
"default": 15,
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"provider": {
"type": "string"
},
"rotating_seeds": {
"default": [],
"items": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": "array"
},
"run_contract": {
"anyOf": [
{
"$ref": "#/definitions/ObservationRunContract"
},
{
"type": "null"
}
]
},
"runs": {
"default": 1,
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenarios": {
"default": [],
"items": {
"$ref": "#/definitions/ScenarioId"
},
"type": "array"
},
"seed": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"slot_start_deadline_seconds": {
"description": "Soft scheduling budget, resolved before request hashing/admission.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"default": 1,
"format": "uint8",
"minimum": 0,
"type": "integer"
}
},
"required": [
"idempotency_key",
"model",
"provider"
],
"title": "RunRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ExecutionPhase": {
"enum": [
"requested",
"admitted",
"preflighting",
"materializing",
"setting_up",
"executing",
"collecting",
"evaluating",
"persisting",
"cleaning_up",
"finalizing",
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
}
},
"properties": {
"duplicate": {
"type": "boolean"
},
"execution_id": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
},
"request_sha256": {
"type": "string"
},
"run_contract_sha256": {
"type": [
"string",
"null"
]
}
},
"required": [
"duplicate",
"execution_id",
"phase",
"request_sha256"
],
"title": "RunAccepted",
"type": "object"
}
},
{
"description": "List every built-in E2E test with its immutable contract, capabilities, complexity, and materialized case.",
"metadata": {
"agent": {
"audience": "harness",
"category": "e2e_local_test_authoring",
"operation": "list_tests",
"starts_execution": false,
"writes_local_state": false
},
"contract": {
"capabilities": [
"scenarios-list"
],
"name": "e2e-control-plane"
},
"internal": false
},
"name": "e2e::scenarios-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"seed": {
"default": null,
"description": "Optional deterministic seed used to materialize each listed test case. Omit it to use each scenario's canonical seed.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"title": "ScenariosListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ArtifactExpectation": {
"properties": {
"id": {
"type": "string"
},
"kind": {
"type": "string"
},
"max_size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"media_type": {
"type": "string"
},
"schema": true
},
"required": [
"id",
"kind",
"max_size_bytes",
"media_type",
"schema"
],
"type": "object"
},
"Capability": {
"description": "What a scenario needs from the engine, the runner host or a fixture to run. The closed list keeps the catalog honest: a new need is a new variant, and every id is spelled once.",
"enum": [
"browser::interactive",
"cross_repo_contract_simulator::v1",
"curl",
"docker",
"e2e::adaptive-flow-v1",
"e2e::control-plane-v1",
"e2e::filesystem",
"e2e::git",
"e2e::run-scoped-fixtures",
"e2e::shell",
"e2e::subagents",
"e2e::workflow-resume-v1",
"fixture::multi-origin-http",
"git",
"git::deterministic-fixture-v1",
"git::offline-bundle",
"github::trusted-handoff",
"harness::independent_session",
"harness::post-turn-validation",
"harness::scripted-dialogue-v1",
"iii::coder",
"iii::compose",
"iii::database",
"iii::functions",
"iii::registry",
"iii::shell",
"iii::state",
"iii::triggers",
"iii::workers",
"incident_fixture::v1",
"node",
"playwright",
"python3",
"release_shadow::read-only-v1",
"release_train_simulator::v1"
],
"type": "string"
},
"DeliverableContract": {
"properties": {
"artifacts": {
"items": {
"$ref": "#/definitions/ArtifactExpectation"
},
"type": "array"
},
"capture_before_cleanup": {
"type": "boolean"
},
"invariants": {
"items": {
"$ref": "#/definitions/InvariantSpec"
},
"type": "array"
},
"provenance_required": {
"type": "boolean"
}
},
"required": [
"artifacts",
"capture_before_cleanup",
"invariants",
"provenance_required"
],
"type": "object"
},
"ExecutionPolicy": {
"properties": {
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_total_tokens": {
"description": "Shared Harness token budget. `None` leaves the Harness budget unbounded for this scenario.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_turns": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"max_validation_retries": {
"description": "Optional per-session cap for post-turn validation denials. `None` preserves the Harness-configured default.",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stuck_timeout_seconds": {
"description": "Stop only after this many seconds without observable useful progress. Large scenarios have no fixed wall-clock deadline.",
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"max_turns",
"stuck_timeout_seconds"
],
"type": "object"
},
"ExecutionRealism": {
"enum": [
"synthetic",
"realistic_simulator",
"frozen_real_artifact"
],
"type": "string"
},
"HumanHorizon": {
"properties": {
"basis": {
"allOf": [
{
"$ref": "#/definitions/HumanHorizonBasis"
}
],
"default": "unknown"
},
"max_minutes": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"min_minutes": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"HumanHorizonBasis": {
"enum": [
"unknown",
"author_estimate",
"measured"
],
"type": "string"
},
"InvariantSpec": {
"properties": {
"description": {
"type": "string"
},
"id": {
"type": "string"
}
},
"required": [
"description",
"id"
],
"type": "object"
},
"RunnerIdentity": {
"additionalProperties": false,
"properties": {
"name": {
"type": "string"
},
"revision": {
"type": "string"
},
"version": {
"type": "string"
}
},
"required": [
"name",
"revision",
"version"
],
"type": "object"
},
"ScenarioCharacterization": {
"properties": {
"human_horizon": {
"allOf": [
{
"$ref": "#/definitions/HumanHorizon"
}
],
"default": {
"basis": "unknown"
}
},
"realism": {
"allOf": [
{
"$ref": "#/definitions/ScenarioRealism"
}
],
"default": {
"execution": "synthetic",
"shadow": "none"
}
}
},
"type": "object"
},
"ScenarioDescriptor": {
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"characterization": {
"$ref": "#/definitions/ScenarioCharacterization"
},
"contract_sha256": {
"type": "string"
},
"deliverable_contract": {
"$ref": "#/definitions/DeliverableContract"
},
"inputs_sha256": {
"type": "string"
},
"required_capabilities": {
"items": {
"$ref": "#/definitions/Capability"
},
"type": "array"
},
"resource_envelope": {
"$ref": "#/definitions/ScenarioResourceEnvelope"
},
"scenario_id": {
"$ref": "#/definitions/ScenarioId"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"characterization",
"contract_sha256",
"deliverable_contract",
"inputs_sha256",
"required_capabilities",
"resource_envelope",
"scenario_id",
"seed"
],
"type": "object"
},
"ScenarioId": {
"enum": [
"registry_planning",
"registry_implementation",
"registry_environment",
"registry_verification",
"kanban_c1_foundation",
"kanban_c2_persistence",
"kanban_c3_board",
"kanban_c4_ticket_flow",
"kanban_c5_edit_move",
"kanban_c6_discussion",
"kanban_c7_live",
"linkly_tutorial",
"context_pressure",
"minimal_path",
"persistent_state",
"insert_record",
"sequential_pipeline",
"database_migration_recovery",
"shell_coder_sandbox",
"research_pipeline",
"fanout_ladder",
"incident_response",
"todo_worker_simple",
"todo_worker_planned",
"engineering_endurance_ladder",
"git_regression_forensics",
"mechanical_reaction",
"timer_wake",
"receiving_operation",
"validation_loop",
"subagent_validation",
"subagent_validation_failure",
"validation_self_repair",
"validation_scope_enforcement",
"validation_chain",
"secret_hygiene",
"prompt_injection_resilience",
"moving_target",
"poison_message",
"cleanup_under_failure",
"depth_ladder",
"quorum_fan_in",
"contention_ledger",
"wake_chain_soak",
"chess_engine_build",
"chess_play_ladder",
"trend_blog",
"trending_topics_build",
"typescript_chat_service",
"tool_contract_recovery",
"policy_bound_action",
"cross_app_transaction",
"performance_regression",
"browser_cross_site",
"release_train_recovery",
"cross_repo_contract_migration"
],
"type": "string"
},
"ScenarioRealism": {
"properties": {
"execution": {
"allOf": [
{
"$ref": "#/definitions/ExecutionRealism"
}
],
"default": "synthetic"
},
"shadow": {
"allOf": [
{
"$ref": "#/definitions/ShadowMode"
}
],
"default": "none"
}
},
"type": "object"
},
"ScenarioResourceEnvelope": {
"properties": {
"execution": {
"$ref": "#/definitions/ExecutionPolicy"
},
"workflow": {
"anyOf": [
{
"$ref": "#/definitions/WorkflowResourceEnvelope"
},
{
"type": "null"
}
]
}
},
"required": [
"execution"
],
"type": "object"
},
"ShadowMode": {
"enum": [
"none",
"read_only"
],
"type": "string"
},
"WorkflowResourceEnvelope": {
"properties": {
"max_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"max_nodes": {
"format": "uint16",
"minimum": 0,
"type": "integer"
},
"max_parallel": {
"format": "uint16",
"minimum": 0,
"type": "integer"
},
"max_total_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"step_timeout_seconds": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"workflow_timeout_seconds": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"max_nodes",
"max_parallel",
"step_timeout_seconds",
"technical_retries",
"workflow_timeout_seconds"
],
"type": "object"
}
},
"properties": {
"catalog_sha256": {
"type": "string"
},
"runner": {
"$ref": "#/definitions/RunnerIdentity"
},
"scenarios": {
"items": {
"$ref": "#/definitions/ScenarioDescriptor"
},
"type": "array"
},
"schema": {
"type": "string"
}
},
"required": [
"catalog_sha256",
"runner",
"scenarios",
"schema"
],
"title": "ScenariosListResponse",
"type": "object"
}
},
{
"description": "Read the durable phase and progress of an E2E execution.",
"metadata": {
"contract": {
"capabilities": [
"status"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::status",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ActiveAttempt": {
"properties": {
"attempt_id": {
"type": "string"
},
"resume_state_path": {
"type": [
"string",
"null"
]
},
"resume_state_sha256": {
"type": [
"string",
"null"
]
},
"run_id": {
"default": "",
"type": "string"
},
"scenario_id": {
"$ref": "#/definitions/ScenarioId"
},
"session_id": {
"type": "string"
}
},
"required": [
"attempt_id",
"scenario_id",
"session_id"
],
"type": "object"
},
"ExecutionPhase": {
"enum": [
"requested",
"admitted",
"preflighting",
"materializing",
"setting_up",
"executing",
"collecting",
"evaluating",
"persisting",
"cleaning_up",
"finalizing",
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
},
"JournalProgress": {
"properties": {
"active_attempt_id": {
"type": [
"string",
"null"
]
},
"attempts_finished": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"attempts_started": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"committed_events": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"last_event_sha256": {
"type": [
"string",
"null"
]
},
"phase": {
"type": [
"string",
"null"
]
},
"planned_slots": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"runs_committed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"slots_deferred": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"subject_observations_committed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"terminal": {
"type": "boolean"
},
"terminal_reason": {
"type": [
"string",
"null"
]
},
"terminal_state": {
"anyOf": [
{
"$ref": "#/definitions/JournalTerminalState"
},
{
"type": "null"
}
]
}
},
"required": [
"attempts_finished",
"attempts_started",
"committed_events",
"planned_slots",
"runs_committed",
"slots_deferred",
"subject_observations_committed",
"terminal"
],
"type": "object"
},
"JournalTerminalState": {
"enum": [
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
},
"PhaseTransition": {
"properties": {
"at": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
},
"reason": {
"type": "string"
}
},
"required": [
"at",
"phase",
"reason"
],
"type": "object"
},
"ScenarioId": {
"enum": [
"registry_planning",
"registry_implementation",
"registry_environment",
"registry_verification",
"kanban_c1_foundation",
"kanban_c2_persistence",
"kanban_c3_board",
"kanban_c4_ticket_flow",
"kanban_c5_edit_move",
"kanban_c6_discussion",
"kanban_c7_live",
"linkly_tutorial",
"context_pressure",
"minimal_path",
"persistent_state",
"insert_record",
"sequential_pipeline",
"database_migration_recovery",
"shell_coder_sandbox",
"research_pipeline",
"fanout_ladder",
"incident_response",
"todo_worker_simple",
"todo_worker_planned",
"engineering_endurance_ladder",
"git_regression_forensics",
"mechanical_reaction",
"timer_wake",
"receiving_operation",
"validation_loop",
"subagent_validation",
"subagent_validation_failure",
"validation_self_repair",
"validation_scope_enforcement",
"validation_chain",
"secret_hygiene",
"prompt_injection_resilience",
"moving_target",
"poison_message",
"cleanup_under_failure",
"depth_ladder",
"quorum_fan_in",
"contention_ledger",
"wake_chain_soak",
"chess_engine_build",
"chess_play_ladder",
"trend_blog",
"trending_topics_build",
"typescript_chat_service",
"tool_contract_recovery",
"policy_bound_action",
"cross_app_transaction",
"performance_regression",
"browser_cross_site",
"release_train_recovery",
"cross_repo_contract_migration"
],
"type": "string"
}
},
"properties": {
"active_attempt": {
"anyOf": [
{
"$ref": "#/definitions/ActiveAttempt"
},
{
"type": "null"
}
]
},
"cancel_requested": {
"type": "boolean"
},
"error": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"journal_progress": {
"$ref": "#/definitions/JournalProgress"
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
},
"terminal": {
"type": "boolean"
},
"transitions": {
"items": {
"$ref": "#/definitions/PhaseTransition"
},
"type": "array"
},
"updated_at": {
"type": "string"
}
},
"required": [
"cancel_requested",
"execution_id",
"journal_progress",
"phase",
"terminal",
"transitions",
"updated_at"
],
"title": "StatusResponse",
"type": "object"
}
},
{
"description": "Serve the Harness E2E injectable Console assets.",
"metadata": {
"internal": true
},
"name": "e2e::ui-content",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"path": {
"type": "string"
}
},
"required": [
"path"
],
"title": "UiContentRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"content": {
"type": "string"
},
"content_type": {
"type": "string"
}
},
"required": [
"content",
"content_type"
],
"title": "UiContentResponse",
"type": "object"
}
}
],
"triggers": [
{
"description": "A local E2E execution started, produced progress, or reached a terminal state.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"title": "EventBindingConfig",
"type": "object"
},
"metadata": {},
"name": "e2e::dashboard::changed",
"return_schema": {}
}
]
}