$worker
harness-e2e
v0.11.8-experimentalMeasure Harness capability, retain reproducible evidence, and expose the E2E dashboard.
iiiverifiedinstall
$iii trigger compose::add worker=harness-e2e@0.11.8-experimental
- macOS: arm64
- Linux: arm64 · armv7 · x64
exact versions are immutable; binary and bundle artifacts are digest-pinned.
agent-ready brief for v0.11.8-experimental
install + config + dependencies + readme + api reference, all in one place. fetch as agent-context.md for an llm to consume.
full markdown
the same content rendered as discrete blocks below is exposed as a single markdown document at
/workers/harness-e2e.md?version=0.11.8-experimental. paste it into an llm prompt or pipe it through curl from a worker.install
install
$iii trigger compose::add worker=harness-e2e@0.11.8-experimental
configuration
iii-config.yaml
- control_database: harness_e2e
control_namespace: harness-e2e-control
data_dir: ~/.iii/data/harness-e2edependencies
readme
README.md
Harness E2E
harness-e2e measures what a Harness stack can execute with correct
deliverables, structural integrity, bounded work, and repeatable outcomes.
A run's score is the plain sum of the points its evaluated criteria awarded; a criterion nobody evaluated adds nothing and nothing is normalized or rescaled. Scores preserve measured criterion points independently of completion or resource limits. Criteria do not veto the score or approve a run. Completion, technical validity, artifact evidence, and runtime controls are reported separately; infrastructure and execution failures still fail the CLI.
The repository is intentionally independent from the workers source tree.
Runtime discovery, execution, observation, state access, and cleanup all happen
through functions registered in iii. The only product input is an immutable
subject artifact or an already-running iii stack.
The SWE service suite provides eight isolated engineering
tasks and a continuous eight-ticket journey over the same Python service, with
optional delegation, immutable checkpoints, isolated verification, and a trusted
GitHub handoff.
SWE execution requires Linux with /usr/bin/bwrap and enabled unprivileged user
namespaces. CI installs the distribution AppArmor profile needed by Bubblewrap.
Commands and file operations run inside the attempt workspace; controller files
remain outside that boundary.
Binaries
harness-e2eis started by Compose and registers the asynchronouse2e::*control plane plus the injectable Console page. Explicit subcommands keep direct scenario execution and report inspection available from the same binary.
Build and validate the repository:
pnpm --dir dashboard install --frozen-lockfile
pnpm --dir dashboard typecheck
pnpm --dir dashboard lint
pnpm --dir dashboard test
pnpm --dir dashboard build
cargo test --locked --all-targets
cargo clippy --locked --all-targets -- -D warnings
node --test tests/dashboard/*.test.cjs
HARNESS_E2E_BIN="$PWD/target/debug/harness-e2e" python3 -m unittest discover -s tests/python -p 'test_*.py'List the materialized scenarios and their definition digests:
cargo run --locked --bin harness-e2e -- list
cargo run --locked --bin harness-e2e -- catalogThe four Registry scenarios
use the regular scenario catalog, execution flow, and scores: registry_planning,
registry_implementation, registry_environment, and registry_verification.
Each has its own atomic validations.
The Linkly tutorial scenario, linkly_tutorial,
runs the seven chapters of the agentic Linkly tutorial plus a project-restart guard as one
scripted dialogue on one Harness session, against the linkly-agentic scaffold's own Compose
stack, and scores twenty-two deterministic checks. scripts/linkly_stack.py prepares that stack.
The trending topics build scenario uses an isolated per-attempt Git remote and independent Playwright acceptance against the delivered SHA. Its runtime and controls require Linux amd64, Docker, Git, Python 3, Node and access to the pinned fixture. Design is free; screenshots are evidence, not an aesthetic score.
Native criteria preserve known awards when dependent checks cannot run. Those
checks have no award and remain not_evaluated; an incomplete criterion set has
no total score. Product failures stay technically valid, while infrastructure
failures invalidate the run without erasing prior criterion observations.
Every scenario is a built-in module under src/scenarios/ that owns its
prompt, setup, deterministic evaluator, and cleanup; the module id is exposed
through the CLI, worker catalog, campaign runner, dashboard, and canonical
result artifacts. Plans select their scenarios explicitly.
Run against an existing stack:
cargo run --locked --bin harness-e2e -- run \
--url ws://127.0.0.1:49134 \
--model codex/gpt-5.6-luna \
--provider openai-codex \
--scenario todo_worker_simpleValidate one of the checked-in canonical campaign assets:
python3 scripts/run_e2e_campaign.py config/campaigns/endurance.json --e2e-bin target/debug/harness-e2e --validate-only
python3 scripts/run_e2e_campaign.py config/campaigns/endurance.json --e2e-bin target/debug/harness-e2e --dry-runOperational campaign execution is dispatched only by Release Control through
.github/workflows/exact-stack-e2e.yml. The repository no longer
publishes independent daily, weekly, post-deploy, or fault-stress dispatch
workflows.
Campaign manifests never select or rotate seeds. They separate replay-safe
turns from scripted dialogue and composite flows, persist a summary for every
group, and are advisory by default while their longitudinal history is being
calibrated. Release Control owns scheduling and dispatch; the executor keeps
the result advisory and archives each materialized group through the
environment-owned durable archiver.
The engineering handoff uses a protected disposable checkout of its dedicated
pinned revision of iii-hq/e2e-fixture. shell_coder_sandbox,
chess_engine_build, and trend_blog prepare their reviewed fixture automatically
from an embedded Git bundle. They require Git, but no fixture checkout or
HARNESS_E2E_FIXTURE_PATH configuration. Each attempt operates on a private
workspace; temporary source checkouts are removed after their contents are read.
typescript_chat_service carries its own frozen skeleton in the repository and
needs no checkout, but it does require Node 22.6 or newer on the runner host: the
subject's TypeScript application is executed directly through Node type
stripping, both by the public suite and by the runner-owned behavioral probe.
config/test-plan.json defines the executable profiles: smoke, regression, capability, evolution,
resilience, endurance, and software-engineering. In the dashboard these profiles are starting templates
for the same plan form and baseline/candidate visualization used by existing plans.
Choose New plan, optionally select a template, edit the scope, and select the
execution model.
Save draft, Save and run, and Duplicate plan use one shared lifecycle
and retain native evidence. Fault-injection plans export
to the protected executor. See executable profile plans.
cargo run --locked -- test-plan listThe software-engineering profile selects the seven incremental Kanban cases,
four Registry cases, the trending-topics blog build and the Linkly tutorial,
once each with no technical retries: 13 cases and 13 planned runs. Its twelve
execution groups keep Registry implementation and verification together, in
that order, so verification receives the implementation delivery. Trending
topics runs in its own case-trending-topics-build group using the existing
pinned fixture workflow.
Linkly runs its eight exchanges in one case-linkly-tutorial group. The executor
creates a fresh pinned linkly-agentic scaffold as that group's Compose project,
with baseline worker versions taken from the resolved stack contract.
cargo run --locked -- test-plan materialize --profile software-engineeringTemplates and execution rules are materialized directly by Rust from the source and native contracts. There are no generated catalogs to synchronize.
Release Control dispatches .github/workflows/exact-stack-e2e.yml directly in
this repository with five inputs and no decisions of its own: execution_id,
the plan naming one profile of config/test-plan.json, a stack policy
({"policy":"latest"} or exact versions), the executor commit runner_sha,
and the cli_version to install.
Everything else is resolved here, from the commit pinned by runner_sha:
harness-e2e test-plan materialize --profileexpands the profile into its campaigns, groups and cases, with aprofile_sha256over the result.scripts/resolve_stack_lock.pyturns the stack policy into one exactrc-e2e/v2contract per campaign — every Registry version resolved,latestnever surviving into a contract — whichscripts/exact_stack_campaign.pyvalidates as before.- Each group runs in an isolated ephemeral stack; fault groups route to the protected runner; one root bundle is produced without rebuilding the native Harness artifacts.
Agent profiles from Release Control
Release Control can select an existing Directory agent profile for an execution by including its ID in the frozen plan:
{
"agent_profile": "console-ui"
}For example, console-ui identifies Console UI Engineer. The runner reads
the profile from the group's Directory and sends its ID as agent to e2e::run.
The profile, its parent profiles, skills, functions, and model/provider must
be available in that stack. The runner downloads the stack's versioned skill
bundles into an isolated Directory and waits up to 120 seconds for the selected
profile; profiles that remain unavailable fail resolution.
A profile's model overrides the plan's model; provider::model also selects its
provider. Results record the resolved subject model and profile configuration
hash, so changes to an existing profile remain visible between executions.
Each execution keeps the selected profile ID; Run again resolves that ID in its
test stack. Comparisons remain manual in Release Control. Omitting
agent_profile keeps the built-in agent.
An execution can also select a project template independently of the test-plan profile and the agent profile:
{
"template": "harness",
"agent_profile": "tech-lead"
}The runner reads the iii/template.yaml catalog in iii-hq/templates from
main, resolves it to one commit before sharding, and records that commit as
identity.template.revision. Every group uses that source. The selected
Compose project supplies the base; test-stack versions override its package
selectors, additional packages enter the stack lock, and local workers remain
local. Template skills override whole downloaded namespaces, and its agent
files take precedence over downloaded profiles. Machine-global profiles/skills
are not used when a template or agent override is selected.
Scenarios, prompts, permissions, fixtures, seeds and repetitions are unchanged. The evaluated agent is applied to ordinary sessions and workflow/adaptive steps; evaluators are unchanged. Linkly retains its pinned task scaffold and container roles, with the selected template's base and agent assets applied separately. No template keeps the existing generated stack (or required fixture).
To measure a profile's effect, compare the same plan, template commit, model
and stack with and without agent_profile. Changing the template too measures
their combined effect. Non-Compose templates and templates requiring interactive
language choices are rejected before boot. Protected fault groups use an
external supervisor and reject both overrides rather than silently ignoring them.
No new scenario or CLI change is required.
scripts/report_execution.py posts what was observed to Release Control's run
ledger over OIDC: materialized before anything runs, one shard per campaign
group whatever that group did, and a summary whatever the finalizer did. Runs
come from results.json, or from the journal checkpoints when a group died
before writing one; a group that produced neither still reports, saying so. No
execution is silently lost.
workers supplies versioned components of the stack under test; it does not
orchestrate campaigns.
Console page
Build the worker and its injectable Console page from the repository root:
cargo build --locked --bin harness-e2eThe Rust build creates dashboard/dist-console/page.js and styles.css, then
embeds both assets in the worker. Node and pnpm must be available on PATH.
When Console connects to the same iii namespace, the worker registers those
assets and the e2e::dashboard::* read, plan, run, status and cancellation
functions used by the page.
Rust-defined composite scenarios, including the multi-test security_review
example, use the current shared result schema and read-only execution projection.
The running Harness must publish request and response schemas compatible with the current typed surface. Missing or incompatible fields fail preflight; no payload-version compatibility mode is available.
The page loads data incrementally through iii: 25 compact summaries on the first overview page, one complete report when an execution is opened, only the selected pair for comparison, and the model/scenario catalog when the run dialog opens. Server-side filtering and cursor pagination keep history growth out of the initial payload. Transport failures stay visible in Console.
The trusted publisher still writes the bounded JSON report archive used by CI and downstream consumers. It does not publish a Harness E2E web application. See dashboard/README.md for the page contract.
Compare a local change with Release Control
The Console's Plans page offers Reference: Release Control to browse RC history through the authenticated Release Control browser bridge. Keep the RC tab open, enable its local Harness connection, and connect it to the same personal Engine as the Console. The bridge needs the E2E read functions from the companion Release Control change. No GitHub token or artifact synchronization is needed.
Open a plan to see remote and local executions together with their origin. Select a reference and a local result to compare their measurements. The scenario links open the existing A → B comparison with both executions selected. Missing reports and metrics remain visible as unavailable; reading history creates no local plan. The comparison runs locally and sends no local results to Release Control.
Choose run locally on a remote reference to save its materialized test parameters as a local plan and run them against your current Harness. Repeating that action creates a new local plan using the current scenario contracts, while preserving earlier plans and results. The reference's scenarios, rounds, repetitions and retry settings come from the execution's materialization, not from the current profile with the same name. The current local scenario implementations and Harness are used deliberately: this is a personal experiment, not an exact-stack certification. No build/Git tracking or matching remote stack is required. Fault-injection groups still require the protected executor; they are not silently omitted. References without shard seeds for every scenario cannot be reproduced. Differences in local scenario definition or case identity are shown as advisory information.
Results stay in the local plan store. The RC execution remains a reference, never a locally recreated official execution. Native result validation remains strict; the remote data is read through the RC API rather than installed as a native report. Full remote evidence is available through the execution's GitHub link, subject to its retention; this flow does not download an evidence archive.
Compose lifecycle
Release Control names the exact project roots. This repository writes only the
root configuration and passes those worker@version references to
compose::add; iii resolves the Registry graph, writes the project topology,
and reconciles its containers. Every execution starts an empty Engine and a
dedicated Compose daemon, then runs compose::add, compose::up,
compose::status, and compose::down. Each execution uses one isolated
namespace for both Compose and the project functions it starts.
Compose supplies III_URL, III_NAMESPACE, III_WORKER_NAME, and III_CONFIG
to the harness-e2e process. All four values are mandatory. The referenced
configuration contains the execution-specific evidence directory and the
separate control-plane database namespace. Start
worker-compose.control.yaml before worker-compose.yaml: it provisions the
single-connection harness_e2e SQLite pool at the configured control-plane
path, with SQL history disabled.
The Harness exits explicitly when that database or its schema is unavailable;
the subject namespace never receives its database client or filesystem path.
Publication validates the locally built binary through a path:// Compose
container before the package is uploaded. Published campaigns use only exact
Registry package versions. Provider secrets are written to temporary
permission-restricted env_file files and are never included in contract,
Compose, evidence, or archive artifacts.
The worker exposes e2e::run, e2e::status, e2e::cancel,
e2e::results-get, e2e::results-list, e2e::compare,
e2e::scenarios-list, e2e::archive, e2e::archive-head,
e2e::archive-restore,
e2e::history-list, and e2e::retention-sweep.
Fault supervisors use e2e::fault-plan and e2e::fault-evaluate so plan
materialization and recovery classification stay on the same iii control plane.
Subject policies deny e2e::*.
Validating a running worker
tests/live_worker.rs talks to a worker that is registered in a real engine,
so its tests are ignored by default. They check that the worker publishes the
catalog this revision materializes (ids, seeds, cases, definition digests),
that it refuses malformed requests without admitting anything, and, when a
subject is named, that one minimal_path execution goes through admission,
setup, the subject turn, capture, evaluation and persistence to a technically
valid report readable back through e2e::results-get:
HARNESS_E2E_LIVE_URL=ws://127.0.0.1:49134 HARNESS_E2E_LIVE_NAMESPACE=my-project \
HARNESS_E2E_LIVE_PROVIDER=deepseek HARNESS_E2E_LIVE_MODEL=deepseek-flash \
cargo test --test live_worker -- --ignoredSeven scenarios state host paths in their prompts, so their definition digests
only match when the test process carries the worker's HARNESS_E2E_RUN_DIR,
TMPDIR and HARNESS_E2E_*_FIXTURE_PATH values; export the same environment
the Compose file gives the worker. The scenario run leaves one execution
labelled live worker validation in the worker's storage.
Durable artifacts are chunked through storage::*. Admissions, executions,
runs, attempts and artifact references are written through the control-plane
database::* worker. Execution records retain compact dashboard summaries and
observations, so lists and history do not load native reports. Storage carries
no version number and has no migration step: every table records the
fingerprint of the statements that create it, and at start the worker
recreates the tables whose fingerprint moved in one transaction, keeping the
execution records, local plans and receipts it can still read and rebuilding
run projections from the native bundles. Rows it cannot read, missing bundles
and imported Release Control history in a recreated table are logged as
warnings; the history comes back by importing it again, and nothing is
reconstructed as a scored result. A report or plan written under another
results contract is read with a warning, never refused.
Plan definitions and composed execution receipts are stored in saved_plans and
saved_plan_executions through the database worker. A saved plan or receipt this
binary cannot read is deleted on the next read; plans written by another binary
are never migrated.
Release Control history imports use harness-e2e-history, wrapped as
{json, sha256} with a sha256: digest of the exact UTF-8 JSON. Use Import
history in the Console to import a file or explicitly fetch a plan from the RC
bridge. Plans and executions retain source identities, revisions and every
retained report; repeated imports do not create duplicates. Imported active work
never enters local admission or recovery. History remains readable without RC.
Evidence uses local gh credentials and Python 3 to verify the GitHub bundle
manifest, execution/attempt identity and file checksums, independently of RC.
Missing, expired, inaccessible and invalid evidence are separate states. Native bundles retain
full reports, manifests and transcripts, loaded on demand for investigation.
The runner has no S3, GCS, R2, SQL-driver, or Harness dependency.
Weekly Stress materializes deterministic fault plans and evaluates journals from a
protected supervisor.
Lane promotion is governed by
config/policies/cutover.json.
Repository boundaries
src/owns the runner, local wire adapters, scenarios, evaluation, longitudinal comparison, and the E2E control worker.config/owns reviewed comparison and cutover policies and fault profiles.tests/owns test-only fixtures, golden wire schemas, and the Node/Python validation suites.schemas/contains the public contracts for generated E2E artifacts.dashboard/contains the React, TypeScript, Vite, and Tailwind Console page embedded in the worker binary.- generated reports, transcripts, logs, and deliverables stay outside Git.
The crate may depend on the iii SDK and generic libraries. It must not declare
a path or Git dependency on workers, Harness, or another product crate.
Contract compatibility is established at runtime from
engine::functions::list and engine::functions::info; the checked-in schemas
are parity fixtures, not a linked product API.
The deterministic assessment boundary has one current payload shape, written
only to results.json; scenario contracts are the only versioned domain. No
scenario uses a second model: every score and every audit flag is
deterministic.
Deterministic, pre-cleanup asset capture applies explicit safety limits and
writes an unversioned sidecar containing the canonical deterministic validation
portion, which is aggregated into results.json.
Observation
The runner waits for a session tree to finish by binding
harness::turn-completed to an internal sink (e2e::on-turn-completed) before
harness::send. That sink is not a control-plane verb: it is not registered
with e2e::run / e2e::status / e2e::cancel, and it does not appear in
e2e::scenarios-list. Subject policies already deny e2e::*.
A 15s watchdog samples harness::metrics and one root harness::status for
stuck detection, heartbeat logs, and e2e::cancel. If the trigger type is
missing from engine::triggers::list, the run is unsupported infrastructure —
there is no silent fallback to polling harness::status or harness::metrics.
After the tree completes, the runner still collects terminal status, metrics,
transcripts, and deliverables.
Subject artifacts
Cross-repository executions accept a subject manifest matching
schemas/subject-artifact.json. The archive and every declared file are
verified before use. Mutable URLs, shortened Git revisions, unexpected archive
paths, and digest mismatches are rejected.
Untrusted subject artifacts are never given provider, storage, or GitHub credentials in their environment. Provider workers and the trusted E2E worker are started separately. PR execution remains non-blocking shadow evidence until the source repository, revision, E2E ref, and credential boundary are approved.
Comparison
Every completed execution records the subject and E2E revisions, observed wire
contracts, definition digest, materialized inputs, seed, policies, artifacts,
and raw structural evidence. e2e::compare accepts two distinct completed
execution ids (from_execution_id and to_execution_id) and writes a unique
comparisons/ plus e2e-summary.md. Numeric
deltas remain disabled when the case set or canonical contract differs.
Deliverable, structural, technical, cost, latency, turns, and retry deltas remain independent. A case is repeatable after five local runs satisfy the deliverable, structural, and technical thresholds. Cost and wall-time are reported as observed metrics and compared only within a compatible baseline/candidate cohort.
Runtime-only package boundary
This repository executes exact-stack Test Plans and never publishes itself as
a Registry worker. Release Control supplies a stack policy and an immutable
executor SHA to exact-stack-e2e.yml; the contract this repository assembles
from them pins every Registry version to an exact one, including historical
candidates, because a campaign has to be able to say afterwards what it ran.
The root iii.worker.yaml remains the public manifest for local iii worker
development and package compatibility. The root worker-compose.yaml remains
a normal public Compose document. Release Control and post-prepare workflow
phases deliberately read neither source contract.
api reference (json)
agent-api-reference.json
{
"functions": [
{
"description": "Persist a completed execution through private iii storage and idempotent history.",
"metadata": {
"contract": {
"capabilities": [
"archive"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::archive",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"RetentionClass": {
"enum": [
"temporary",
"pull_request",
"longitudinal",
"canonical"
],
"type": "string"
}
},
"properties": {
"execution_id": {
"type": "string"
},
"retention_class": {
"$ref": "#/definitions/RetentionClass"
}
},
"required": [
"execution_id",
"retention_class"
],
"title": "ArchiveExecutionRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"DurableArchiveReference": {
"additionalProperties": false,
"properties": {
"archive_id": {
"type": "string"
},
"created_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"expires_at": {
"type": [
"string",
"null"
]
},
"identity_sha256": {
"type": "string"
},
"manifest": {
"$ref": "#/definitions/StorageObjectReference"
},
"manifest_backup": {
"$ref": "#/definitions/StorageObjectReference"
},
"retention_class": {
"$ref": "#/definitions/RetentionClass"
}
},
"required": [
"archive_id",
"created_at",
"execution_id",
"identity_sha256",
"manifest",
"manifest_backup",
"retention_class"
],
"type": "object"
},
"HistoryRecord": {
"additionalProperties": false,
"properties": {
"archive": {
"$ref": "#/definitions/DurableArchiveReference"
},
"baseline_comparable": {
"type": "boolean"
},
"case_count": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"e2e_repository": {
"type": "string"
},
"e2e_revision": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"identity_sha256": {
"type": "string"
},
"ingestion_id": {
"type": "string"
},
"lane": {
"type": "string"
},
"occurred_at": {
"type": "string"
},
"passed": {
"type": "boolean"
},
"result_contract_sha256": {
"type": "string"
},
"stack_mode": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"subject_revision": {
"type": "string"
}
},
"required": [
"archive",
"baseline_comparable",
"case_count",
"e2e_repository",
"e2e_revision",
"execution_id",
"identity_sha256",
"ingestion_id",
"lane",
"occurred_at",
"passed",
"result_contract_sha256",
"stack_mode",
"subject_model",
"subject_provider",
"subject_revision"
],
"type": "object"
},
"RetentionClass": {
"enum": [
"temporary",
"pull_request",
"longitudinal",
"canonical"
],
"type": "string"
},
"StorageObjectReference": {
"properties": {
"bucket": {
"type": "string"
},
"key": {
"type": "string"
},
"media_type": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"uri": {
"type": "string"
}
},
"required": [
"bucket",
"key",
"media_type",
"sha256",
"size_bytes",
"uri"
],
"type": "object"
}
},
"properties": {
"archive": {
"$ref": "#/definitions/DurableArchiveReference"
},
"duplicate_ingestion": {
"type": "boolean"
},
"history": {
"$ref": "#/definitions/HistoryRecord"
}
},
"required": [
"archive",
"duplicate_ingestion",
"history"
],
"title": "ArchiveResponse",
"type": "object"
}
},
{
"description": "Verify availability and immutable metadata for an archived execution.",
"metadata": {
"contract": {
"capabilities": [
"archive-head"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::archive-head",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ArchiveAvailability": {
"enum": [
"available",
"expired"
],
"type": "string"
},
"DurableArchiveReference": {
"additionalProperties": false,
"properties": {
"archive_id": {
"type": "string"
},
"created_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"expires_at": {
"type": [
"string",
"null"
]
},
"identity_sha256": {
"type": "string"
},
"manifest": {
"$ref": "#/definitions/StorageObjectReference"
},
"manifest_backup": {
"$ref": "#/definitions/StorageObjectReference"
},
"retention_class": {
"$ref": "#/definitions/RetentionClass"
}
},
"required": [
"archive_id",
"created_at",
"execution_id",
"identity_sha256",
"manifest",
"manifest_backup",
"retention_class"
],
"type": "object"
},
"RetentionClass": {
"enum": [
"temporary",
"pull_request",
"longitudinal",
"canonical"
],
"type": "string"
},
"StorageObjectReference": {
"properties": {
"bucket": {
"type": "string"
},
"key": {
"type": "string"
},
"media_type": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"uri": {
"type": "string"
}
},
"required": [
"bucket",
"key",
"media_type",
"sha256",
"size_bytes",
"uri"
],
"type": "object"
}
},
"properties": {
"archive": {
"$ref": "#/definitions/DurableArchiveReference"
},
"availability": {
"$ref": "#/definitions/ArchiveAvailability"
},
"verified_at": {
"type": "string"
}
},
"required": [
"archive",
"availability",
"verified_at"
],
"title": "ArchiveHeadResponse",
"type": "object"
}
},
{
"description": "Restore and verify an archived execution from its immutable manifest.",
"metadata": {
"contract": {
"capabilities": [
"archive-restore"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::archive-restore",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ArchiveAvailability": {
"enum": [
"available",
"expired"
],
"type": "string"
},
"DurableArchiveReference": {
"additionalProperties": false,
"properties": {
"archive_id": {
"type": "string"
},
"created_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"expires_at": {
"type": [
"string",
"null"
]
},
"identity_sha256": {
"type": "string"
},
"manifest": {
"$ref": "#/definitions/StorageObjectReference"
},
"manifest_backup": {
"$ref": "#/definitions/StorageObjectReference"
},
"retention_class": {
"$ref": "#/definitions/RetentionClass"
}
},
"required": [
"archive_id",
"created_at",
"execution_id",
"identity_sha256",
"manifest",
"manifest_backup",
"retention_class"
],
"type": "object"
},
"RetentionClass": {
"enum": [
"temporary",
"pull_request",
"longitudinal",
"canonical"
],
"type": "string"
},
"StorageObjectReference": {
"properties": {
"bucket": {
"type": "string"
},
"key": {
"type": "string"
},
"media_type": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"uri": {
"type": "string"
}
},
"required": [
"bucket",
"key",
"media_type",
"sha256",
"size_bytes",
"uri"
],
"type": "object"
}
},
"properties": {
"archive": {
"$ref": "#/definitions/DurableArchiveReference"
},
"availability": {
"$ref": "#/definitions/ArchiveAvailability"
},
"restored_files": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"restored_from_backup": {
"type": "boolean"
},
"restored_root": {
"type": [
"string",
"null"
]
}
},
"required": [
"archive",
"availability",
"restored_files",
"restored_from_backup"
],
"title": "ArchiveRestoreResponse",
"type": "object"
}
},
{
"description": "Request idempotent cancellation and cleanup of an E2E execution.",
"metadata": {
"contract": {
"capabilities": [
"cancel"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::cancel",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ExecutionPhase": {
"enum": [
"requested",
"admitted",
"preflighting",
"materializing",
"setting_up",
"executing",
"collecting",
"evaluating",
"persisting",
"cleaning_up",
"finalizing",
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
}
},
"properties": {
"accepted": {
"type": "boolean"
},
"execution_id": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
}
},
"required": [
"accepted",
"execution_id",
"phase"
],
"title": "CancelResponse",
"type": "object"
}
},
{
"description": "Compare two completed executions when their identities are eligible.",
"metadata": {
"contract": {
"capabilities": [
"compare"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::compare",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ComparisonPolicy": {
"properties": {
"baseline_id": {
"description": "Identifier of the reviewed baseline these thresholds came from, when one was loaded. Absent when the code-default thresholds are in effect.",
"type": [
"string",
"null"
]
},
"regression": {
"allOf": [
{
"$ref": "#/definitions/RegressionThresholds"
}
],
"default": {
"cost_increase_ratio": 0.2,
"deliverable_success_drop": 0.03,
"p95_turns_increase_ratio": 0.25,
"structural_integrity_drop": 0.02,
"technical_failure_increase": 0.02,
"wall_time_increase_ratio": 0.2
}
},
"scenario_overrides": {
"additionalProperties": {
"$ref": "#/definitions/ThresholdOverrides"
},
"description": "Per-scenario partial threshold overrides carried from the reviewed baseline.",
"type": "object"
}
},
"type": "object"
},
"RegressionThresholds": {
"properties": {
"cost_increase_ratio": {
"format": "double",
"type": "number"
},
"deliverable_success_drop": {
"format": "double",
"type": "number"
},
"p95_turns_increase_ratio": {
"format": "double",
"type": "number"
},
"structural_integrity_drop": {
"format": "double",
"type": "number"
},
"technical_failure_increase": {
"format": "double",
"type": "number"
},
"wall_time_increase_ratio": {
"format": "double",
"type": "number"
}
},
"required": [
"cost_increase_ratio",
"deliverable_success_drop",
"p95_turns_increase_ratio",
"structural_integrity_drop",
"technical_failure_increase",
"wall_time_increase_ratio"
],
"type": "object"
},
"ThresholdOverrides": {
"additionalProperties": false,
"description": "Partial per-scenario replacement for [`RegressionThresholds`]: a named field replaces the baseline value, an omitted field inherits it.",
"properties": {
"cost_increase_ratio": {
"format": "double",
"type": [
"number",
"null"
]
},
"deliverable_success_drop": {
"format": "double",
"type": [
"number",
"null"
]
},
"p95_turns_increase_ratio": {
"format": "double",
"type": [
"number",
"null"
]
},
"structural_integrity_drop": {
"format": "double",
"type": [
"number",
"null"
]
},
"technical_failure_increase": {
"format": "double",
"type": [
"number",
"null"
]
},
"wall_time_increase_ratio": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
}
},
"properties": {
"baseline": {
"default": null,
"description": "Reviewed baseline file to load thresholds from. Omit to use the checked-in `config/baselines/default.json` when it exists, falling back to the code-default thresholds. Ignored when `policy` is set.",
"type": [
"string",
"null"
]
},
"from_execution_id": {
"type": "string"
},
"policy": {
"anyOf": [
{
"$ref": "#/definitions/ComparisonPolicy"
},
{
"type": "null"
}
],
"default": null,
"description": "Explicit policy override. Omit to gate on the reviewed baseline."
},
"to_execution_id": {
"type": "string"
}
},
"required": [
"from_execution_id",
"to_execution_id"
],
"title": "CompareRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ArtifactReference": {
"properties": {
"id": {
"type": "string"
},
"kind": {
"type": "string"
},
"media_type": {
"type": "string"
},
"path": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"id",
"kind",
"media_type",
"path",
"sha256",
"size_bytes"
],
"type": "object"
},
"BenchmarkDelta": {
"properties": {
"consumption": {
"additionalProperties": {
"$ref": "#/definitions/DeltaValue"
},
"default": {},
"description": "Descriptive only; these deltas never create a regression gate.",
"type": "object"
},
"deliverable_success_rate": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"flaky_rate": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"mean_score": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"p50_cost_usd": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"p50_turns": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"p50_wall_time_ms": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"p95_cost_usd": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"p95_turns": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"p95_wall_time_ms": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"retry_rate": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"structural_integrity_rate": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"technical_failure_rate": {
"anyOf": [
{
"$ref": "#/definitions/DeltaValue"
},
{
"type": "null"
}
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"type": "object"
},
"CaseCohortKey": {
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"execution": {
"$ref": "#/definitions/ExecutionCohortIdentity"
},
"inputs_sha256": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"contract_sha256",
"execution",
"inputs_sha256",
"scenario_id",
"seed"
],
"type": "object"
},
"CaseComparison": {
"properties": {
"delta": {
"$ref": "#/definitions/BenchmarkDelta"
},
"from": {
"$ref": "#/definitions/CaseMetrics"
},
"from_run_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"key": {
"$ref": "#/definitions/CaseCohortKey"
},
"regressions": {
"items": {
"$ref": "#/definitions/RegressionSignal"
},
"type": "array"
},
"to": {
"$ref": "#/definitions/CaseMetrics"
},
"to_run_ids": {
"items": {
"type": "string"
},
"type": "array"
}
},
"required": [
"delta",
"from",
"from_run_ids",
"key",
"regressions",
"to",
"to_run_ids"
],
"type": "object"
},
"CaseMetrics": {
"properties": {
"consumption": {
"allOf": [
{
"$ref": "#/definitions/ConsumptionMetrics"
}
],
"default": {
"completed_runs": 0,
"observed_runs": 0,
"p50_function_calls": null,
"p50_total_tokens": null,
"p95_function_calls": null,
"p95_total_tokens": null,
"tokens_per_completion": null,
"total_tokens_consumed": null,
"unavailable": {}
},
"description": "Includes every observed run, even those excluded from quality comparison."
},
"deliverable_success": {
"anyOf": [
{
"$ref": "#/definitions/RateEstimate"
},
{
"type": "null"
}
]
},
"excluded_infrastructure_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"flaky_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"included_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"p50_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"p50_turns": {
"format": "double",
"type": [
"number",
"null"
]
},
"p50_wall_time_ms": {
"format": "double",
"type": [
"number",
"null"
]
},
"p95_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"p95_turns": {
"format": "double",
"type": [
"number",
"null"
]
},
"p95_wall_time_ms": {
"format": "double",
"type": [
"number",
"null"
]
},
"retry_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"structural_integrity": {
"anyOf": [
{
"$ref": "#/definitions/RateEstimate"
},
{
"type": "null"
}
]
},
"technical_failure": {
"anyOf": [
{
"$ref": "#/definitions/RateEstimate"
},
{
"type": "null"
}
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"required": [
"excluded_infrastructure_runs",
"included_runs"
],
"type": "object"
},
"CohortAudit": {
"properties": {
"excluded_cases": {
"items": {
"$ref": "#/definitions/ExcludedCase"
},
"type": "array"
},
"excluded_runs": {
"items": {
"$ref": "#/definitions/ExcludedRun"
},
"type": "array"
},
"included_case_ids": {
"items": {
"type": "string"
},
"type": "array"
}
},
"required": [
"excluded_cases",
"excluded_runs",
"included_case_ids"
],
"type": "object"
},
"ComparisonSide": {
"enum": [
"from",
"to"
],
"type": "string"
},
"ComparisonSummary": {
"properties": {
"baseline_id": {
"description": "Reviewed baseline the regression thresholds came from; absent when the code-default thresholds gated this comparison.",
"type": [
"string",
"null"
]
},
"cases": {
"items": {
"$ref": "#/definitions/CaseComparison"
},
"type": "array"
},
"cohort": {
"$ref": "#/definitions/CohortAudit"
},
"comparable": {
"type": "boolean"
},
"comparison_id": {
"type": "string"
},
"from_execution_id": {
"type": "string"
},
"from_revision": {
"type": [
"string",
"null"
]
},
"gate_passed": {
"type": "boolean"
},
"reasons": {
"items": {
"type": "string"
},
"type": "array"
},
"regressions": {
"items": {
"$ref": "#/definitions/RegressionSignal"
},
"type": "array"
},
"to_execution_id": {
"type": "string"
},
"to_revision": {
"type": [
"string",
"null"
]
}
},
"required": [
"cases",
"cohort",
"comparable",
"comparison_id",
"from_execution_id",
"gate_passed",
"reasons",
"regressions",
"to_execution_id"
],
"type": "object"
},
"ConsumptionMetrics": {
"description": "Subject-side usage across all observed attempts.",
"properties": {
"completed_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"observed_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"p50_function_calls": {
"format": "double",
"type": [
"number",
"null"
]
},
"p50_total_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"p95_function_calls": {
"format": "double",
"type": [
"number",
"null"
]
},
"p95_total_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"tokens_per_completion": {
"format": "double",
"type": [
"number",
"null"
]
},
"total_tokens_consumed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"required": [
"completed_runs",
"observed_runs",
"unavailable"
],
"type": "object"
},
"DeltaValue": {
"properties": {
"absolute": {
"format": "double",
"type": "number"
},
"relative_ratio": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"required": [
"absolute"
],
"type": "object"
},
"ExcludedCase": {
"properties": {
"case_id": {
"type": "string"
},
"reasons": {
"items": {
"type": "string"
},
"type": "array"
},
"scenario_id": {
"type": "string"
},
"side": {
"$ref": "#/definitions/ComparisonSide"
}
},
"required": [
"case_id",
"reasons",
"scenario_id",
"side"
],
"type": "object"
},
"ExcludedRun": {
"properties": {
"attempt_id": {
"type": "string"
},
"case_id": {
"type": "string"
},
"reason": {
"type": "string"
},
"run_id": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"side": {
"$ref": "#/definitions/ComparisonSide"
}
},
"required": [
"attempt_id",
"case_id",
"reason",
"run_id",
"scenario_id",
"side"
],
"type": "object"
},
"ExecutionCohortIdentity": {
"properties": {
"e2e_repository": {
"type": [
"string",
"null"
]
},
"lane": {
"type": "string"
},
"stack_mode": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
}
},
"required": [
"lane",
"stack_mode",
"subject_model",
"subject_provider"
],
"type": "object"
},
"RateEstimate": {
"properties": {
"ci95_lower": {
"format": "double",
"type": "number"
},
"ci95_upper": {
"format": "double",
"type": "number"
},
"rate": {
"format": "double",
"type": "number"
},
"sample_size": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"successes": {
"format": "uint32",
"minimum": 0,
"type": "integer"
}
},
"required": [
"ci95_lower",
"ci95_upper",
"rate",
"sample_size",
"successes"
],
"type": "object"
},
"RegressionDimension": {
"enum": [
"deliverable",
"structural_integrity",
"technical_reliability",
"cost",
"wall_time",
"turns"
],
"type": "string"
},
"RegressionSignal": {
"properties": {
"blocking": {
"type": "boolean"
},
"case_id": {
"type": "string"
},
"dimension": {
"$ref": "#/definitions/RegressionDimension"
},
"maturity": {
"default": "",
"type": "string"
},
"n_from": {
"default": 0,
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"n_to": {
"default": 0,
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"observed_delta": {
"format": "double",
"type": "number"
},
"reason": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"statistic": {
"default": "",
"type": "string"
},
"threshold": {
"format": "double",
"type": "number"
}
},
"required": [
"blocking",
"case_id",
"dimension",
"observed_delta",
"reason",
"scenario_id",
"threshold"
],
"type": "object"
}
},
"properties": {
"artifacts": {
"items": {
"$ref": "#/definitions/ArtifactReference"
},
"type": "array"
},
"comparison": {
"$ref": "#/definitions/ComparisonSummary"
}
},
"required": [
"artifacts",
"comparison"
],
"title": "ComparisonResponse",
"type": "object"
}
},
{
"description": "Read the persisted projection for one logical-run attempt.",
"metadata": {
"contract": {
"capability": "attempt-get",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::attempt-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"attempt_id": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"run_id": {
"type": "string"
}
},
"required": [
"attempt_id",
"execution_id",
"run_id"
],
"title": "AttemptGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"CompletionState": {
"enum": [
"completed",
"task_incomplete",
"undetermined"
],
"type": "string"
},
"ContaminationScope": {
"enum": [
"none",
"case",
"execution"
],
"type": "string"
},
"CriterionReport": {
"properties": {
"awarded": {
"format": "uint8",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"description": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"possible": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"reason": {
"type": "string"
}
},
"required": [
"id",
"possible",
"reason"
],
"type": "object"
},
"DimensionReport": {
"properties": {
"dimension": {
"$ref": "#/definitions/EvaluationDimension"
},
"passed": {
"type": [
"boolean",
"null"
]
},
"signals": true
},
"required": [
"dimension",
"signals"
],
"type": "object"
},
"EfficiencyReport": {
"properties": {
"child_sessions": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"child_turns": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"critical_path_ms": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"effective_fan_out": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"function_call_errors": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"function_calls": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"input_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"observed_complexity": {
"$ref": "#/definitions/ObservedComplexityReport"
},
"observed_work": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"root_turns": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_attempts": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"total_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"transient_resumes": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"validation_retries": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wake_resumes": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wall_time_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"observed_complexity",
"technical_attempts",
"wall_time_ms"
],
"type": "object"
},
"EvaluationDimension": {
"enum": [
"deliverable",
"structural_integrity",
"efficiency",
"robustness",
"e2e_infrastructure"
],
"type": "string"
},
"FailureDomain": {
"enum": [
"subject",
"resource",
"e2e_infrastructure"
],
"type": "string"
},
"FailurePhase": {
"enum": [
"setup",
"execute",
"collect",
"evaluate",
"cleanup"
],
"type": "string"
},
"FailureRecord": {
"properties": {
"code": {
"default": "",
"type": "string"
},
"contamination": {
"allOf": [
{
"$ref": "#/definitions/ContaminationScope"
}
],
"default": "none"
},
"domain": {
"$ref": "#/definitions/FailureDomain"
},
"message": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/FailurePhase"
},
"retry_scope": {
"allOf": [
{
"$ref": "#/definitions/RetryScope"
}
],
"default": "none"
},
"status": {
"allOf": [
{
"$ref": "#/definitions/RunStatus"
}
],
"description": "The failure's original classification, independent of the primary run status."
}
},
"required": [
"domain",
"message",
"phase",
"status"
],
"type": "object"
},
"ObservedComplexityReport": {
"properties": {
"ambiguity_level": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"artifact_count": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"coordination_edges": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"dependency_depth": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"external_systems": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"parallel_branches": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"planning_depth": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"state_transitions": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"validation_loops": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wake_cycles": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"RetryScope": {
"enum": [
"none",
"same_slot"
],
"type": "string"
},
"RunStatus": {
"enum": [
"passed",
"hard_gate_failed",
"subject_error",
"resource_limit",
"infrastructure_error"
],
"type": "string"
},
"TechnicalState": {
"enum": [
"valid",
"technical_invalid"
],
"type": "string"
}
},
"properties": {
"completion": {
"$ref": "#/definitions/CompletionState"
},
"criteria": {
"items": {
"$ref": "#/definitions/CriterionReport"
},
"type": [
"array",
"null"
]
},
"dimensions": {
"items": {
"$ref": "#/definitions/DimensionReport"
},
"type": "array"
},
"efficiency": {
"anyOf": [
{
"$ref": "#/definitions/EfficiencyReport"
},
{
"type": "null"
}
]
},
"failures": {
"items": {
"$ref": "#/definitions/FailureRecord"
},
"type": "array"
},
"status": {
"$ref": "#/definitions/RunStatus"
},
"technical": {
"$ref": "#/definitions/TechnicalState"
}
},
"required": [
"completion",
"dimensions",
"failures",
"status",
"technical"
],
"title": "AttemptResponse",
"type": "object"
}
},
{
"description": "Read models and scenarios when the execution dialog opens.",
"metadata": {
"contract": {
"capability": "catalog-get",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::catalog-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"url": {
"default": null,
"type": [
"string",
"null"
]
}
},
"title": "CatalogRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"CatalogModel": {
"properties": {
"id": {
"type": "string"
},
"provider": {
"type": "string"
}
},
"required": [
"id",
"provider"
],
"type": "object"
}
},
"properties": {
"models": {
"items": {
"$ref": "#/definitions/CatalogModel"
},
"type": "array"
},
"scenarios": {
"items": {
"type": "string"
},
"type": "array"
},
"url": {
"type": "string"
}
},
"required": [
"models",
"scenarios",
"url"
],
"title": "CatalogResponse",
"type": "object"
}
},
{
"description": "List immutable evaluated-system versions and their exact evaluation cohorts.",
"metadata": {
"contract": {
"capability": "evaluated-versions-list",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::evaluated-versions-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"cohort_id": {
"default": null,
"type": [
"string",
"null"
]
}
},
"title": "EvaluatedVersionsRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"CohortDescriptor": {
"properties": {
"id": {
"type": "string"
},
"lane": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
}
},
"required": [
"id",
"lane",
"subject_model",
"subject_provider"
],
"type": "object"
},
"EvaluatedVersionDescriptor": {
"properties": {
"cohort_id": {
"type": "string"
},
"completed_at": {
"type": "string"
},
"execution_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"id": {
"type": "string"
},
"label": {
"type": "string"
},
"stack_mode": {
"type": "string"
}
},
"required": [
"cohort_id",
"completed_at",
"execution_count",
"id",
"label",
"stack_mode"
],
"type": "object"
}
},
"properties": {
"cohorts": {
"items": {
"$ref": "#/definitions/CohortDescriptor"
},
"type": "array"
},
"revision": {
"type": "string"
},
"versions": {
"items": {
"$ref": "#/definitions/EvaluatedVersionDescriptor"
},
"type": "array"
}
},
"required": [
"cohorts",
"revision",
"versions"
],
"title": "EvaluatedVersionsResponse",
"type": "object"
}
},
{
"description": "Read retained GitHub evidence using local credentials and verify its bundle identity and checksum.",
"metadata": {
"contract": {
"capability": "evidence-open",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::evidence-open",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
},
"path": {
"type": "string"
},
"report_id": {
"type": "string"
}
},
"required": [
"execution_id",
"path",
"report_id"
],
"title": "EvidenceRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"availability": {
"type": "string"
},
"content_base64": {
"type": [
"string",
"null"
]
},
"mime_type": {
"type": [
"string",
"null"
]
},
"reason": {
"type": [
"string",
"null"
]
}
},
"required": [
"availability"
],
"title": "EvidenceResponse",
"type": "object"
}
},
{
"description": "Delete one terminal local E2E execution.",
"metadata": {
"contract": {
"capability": "execution-delete",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::execution-delete",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": true,
"title": "Map_of_AnyValue",
"type": "object"
}
},
{
"description": "Read one retained execution summary and its diagnostic report.",
"metadata": {
"contract": {
"capability": "execution-get",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::execution-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ExecutionListResponse": {
"properties": {
"executions": {
"items": true,
"type": "array"
},
"last_update": {
"type": "string"
},
"mode": {
"type": "string"
},
"next_cursor": {
"type": [
"string",
"null"
]
},
"repo_url": {
"type": "string"
},
"retention": {
"$ref": "#/definitions/Retention"
},
"total": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"executions",
"last_update",
"mode",
"repo_url",
"retention",
"total"
],
"type": "object"
},
"Retention": {
"properties": {
"details": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"summaries": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"details",
"summaries"
],
"type": "object"
}
},
"properties": {
"detail": true,
"manifest": {
"$ref": "#/definitions/ExecutionListResponse"
}
},
"required": [
"detail",
"manifest"
],
"title": "ExecutionBundle",
"type": "object"
}
},
{
"description": "List compact retained execution summaries with server-side filters and cursor pagination.",
"metadata": {
"contract": {
"capability": "executions-list",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::executions-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"cursor": {
"default": null,
"type": [
"string",
"null"
]
},
"event": {
"default": null,
"type": [
"string",
"null"
]
},
"ids": {
"default": [],
"items": {
"type": "string"
},
"type": "array"
},
"ids_csv": {
"default": null,
"type": [
"string",
"null"
]
},
"limit": {
"default": null,
"format": "uint16",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"query": {
"default": null,
"type": [
"string",
"null"
]
},
"status": {
"default": null,
"type": [
"string",
"null"
]
}
},
"title": "ExecutionListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Retention": {
"properties": {
"details": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"summaries": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"details",
"summaries"
],
"type": "object"
}
},
"properties": {
"executions": {
"items": true,
"type": "array"
},
"last_update": {
"type": "string"
},
"mode": {
"type": "string"
},
"next_cursor": {
"type": [
"string",
"null"
]
},
"repo_url": {
"type": "string"
},
"retention": {
"$ref": "#/definitions/Retention"
},
"total": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"executions",
"last_update",
"mode",
"repo_url",
"retention",
"total"
],
"title": "ExecutionListResponse",
"type": "object"
}
},
{
"description": "Configure, export and execute saved plans, and inspect or cancel their composed executions.",
"metadata": {
"contract": {
"capability": "plan-control",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plan-control",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"HistoryImport": {
"anyOf": [
{
"type": "string"
},
{
"additionalProperties": false,
"properties": {
"json": {
"type": "string"
},
"sha256": {
"type": "string"
}
},
"required": [
"json",
"sha256"
],
"type": "object"
}
]
},
"ReferenceModel": {
"additionalProperties": false,
"properties": {
"model": {
"type": "string"
},
"provider": {
"type": "string"
}
},
"required": [
"model",
"provider"
],
"type": "object"
}
},
"oneOf": [
{
"properties": {
"action": {
"enum": [
"import_history"
],
"type": "string"
},
"history": {
"$ref": "#/definitions/HistoryImport"
}
},
"required": [
"action",
"history"
],
"type": "object"
},
{
"properties": {
"action": {
"enum": [
"rename_imported_execution"
],
"type": "string"
},
"execution_id": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"action",
"execution_id",
"label"
],
"type": "object"
},
{
"properties": {
"action": {
"enum": [
"reproduce_reference"
],
"type": "string"
},
"label": {
"default": "",
"type": "string"
},
"materialized": true,
"reference_execution_id": {
"type": "string"
},
"shards": {
"default": [],
"items": true,
"type": "array"
},
"subject": {
"$ref": "#/definitions/ReferenceModel"
}
},
"required": [
"action",
"materialized",
"reference_execution_id",
"subject"
],
"type": "object"
},
{
"properties": {
"action": {
"enum": [
"requirements"
],
"type": "string"
},
"plan_id": {
"type": "string"
}
},
"required": [
"action",
"plan_id"
],
"type": "object"
},
{
"properties": {
"action": {
"enum": [
"export"
],
"type": "string"
},
"plan_id": {
"type": "string"
}
},
"required": [
"action",
"plan_id"
],
"type": "object"
},
{
"properties": {
"action": {
"enum": [
"execution"
],
"type": "string"
},
"execution_id": {
"type": "string"
}
},
"required": [
"action",
"execution_id"
],
"type": "object"
},
{
"properties": {
"action": {
"enum": [
"cancel"
],
"type": "string"
},
"execution_id": {
"type": "string"
}
},
"required": [
"action",
"execution_id"
],
"type": "object"
}
],
"title": "Request"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": true,
"title": "Map_of_AnyValue",
"type": "object"
}
},
{
"description": "Create a draft local plan with an explicit small test scope.",
"metadata": {
"contract": {
"capability": "plan-create",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plan-create",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"properties": {
"duplicate_of": {
"default": null,
"type": [
"string",
"null"
]
},
"label": {
"default": "",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"purpose": {
"default": "",
"type": "string"
},
"runs": {
"default": 1,
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenarios": {
"items": {
"type": "string"
},
"type": "array"
},
"seed": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"default": 1,
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"template_id": {
"default": null,
"type": [
"string",
"null"
]
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"scenarios",
"url"
],
"title": "PlanCreateRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"PlanScopeItem": {
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"inputs_sha256": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"contract_sha256",
"inputs_sha256",
"scenario_id",
"seed"
],
"type": "object"
},
"PlanState": {
"enum": [
"draft",
"baseline_running",
"baseline_ready",
"candidate_running",
"comparison_ready"
],
"type": "string"
}
},
"properties": {
"baseline_execution_id": {
"type": [
"string",
"null"
]
},
"candidate_execution_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"candidate_labels": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"compatible": {
"type": "boolean"
},
"created_at": {
"type": "string"
},
"id": {
"type": "string"
},
"incomplete_execution_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"label": {
"type": "string"
},
"last_attempt_id": {
"type": [
"string",
"null"
]
},
"locked": {
"type": "boolean"
},
"model": {
"type": "string"
},
"protected_executor_required": {
"type": "boolean"
},
"provider": {
"type": "string"
},
"purpose": {
"type": "string"
},
"reference_differences": {
"items": {
"type": "string"
},
"type": "array"
},
"reference_execution_id": {
"type": [
"string",
"null"
]
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenario_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"scenarios": {
"items": {
"$ref": "#/definitions/PlanScopeItem"
},
"type": "array"
},
"scope_hash": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"state": {
"$ref": "#/definitions/PlanState"
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"template_id": {
"type": [
"string",
"null"
]
},
"updated_at": {
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"candidate_execution_ids",
"candidate_labels",
"compatible",
"created_at",
"id",
"incomplete_execution_ids",
"label",
"locked",
"model",
"protected_executor_required",
"provider",
"purpose",
"runs",
"scenario_ids",
"scenarios",
"scope_hash",
"state",
"technical_retries",
"updated_at",
"url"
],
"title": "LocalPlan",
"type": "object"
}
},
{
"description": "Delete a local plan.",
"metadata": {
"contract": {
"capability": "plan-delete",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plan-delete",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"plan_id": {
"type": "string"
}
},
"required": [
"plan_id"
],
"title": "PlanGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": true,
"title": "Map_of_AnyValue",
"type": "object"
}
},
{
"description": "Read one saved local or imported plan.",
"metadata": {
"contract": {
"capability": "plan-get",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plan-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"plan_id": {
"type": "string"
}
},
"required": [
"plan_id"
],
"title": "PlanGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": true,
"title": "Map_of_AnyValue",
"type": "object"
}
},
{
"description": "Start a baseline or candidate from a saved local plan.",
"metadata": {
"contract": {
"capability": "plan-run-start",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plan-run-start",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"PlanRunRole": {
"enum": [
"baseline",
"candidate"
],
"type": "string"
}
},
"properties": {
"idempotency_key": {
"type": "string"
},
"plan_id": {
"default": null,
"type": [
"string",
"null"
]
},
"role": {
"$ref": "#/definitions/PlanRunRole"
}
},
"required": [
"idempotency_key",
"role"
],
"title": "PlanRunRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"PlanScopeItem": {
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"inputs_sha256": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"contract_sha256",
"inputs_sha256",
"scenario_id",
"seed"
],
"type": "object"
},
"PlanState": {
"enum": [
"draft",
"baseline_running",
"baseline_ready",
"candidate_running",
"comparison_ready"
],
"type": "string"
}
},
"properties": {
"baseline_execution_id": {
"type": [
"string",
"null"
]
},
"candidate_execution_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"candidate_labels": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"compatible": {
"type": "boolean"
},
"created_at": {
"type": "string"
},
"id": {
"type": "string"
},
"incomplete_execution_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"label": {
"type": "string"
},
"last_attempt_id": {
"type": [
"string",
"null"
]
},
"locked": {
"type": "boolean"
},
"model": {
"type": "string"
},
"protected_executor_required": {
"type": "boolean"
},
"provider": {
"type": "string"
},
"purpose": {
"type": "string"
},
"reference_differences": {
"items": {
"type": "string"
},
"type": "array"
},
"reference_execution_id": {
"type": [
"string",
"null"
]
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenario_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"scenarios": {
"items": {
"$ref": "#/definitions/PlanScopeItem"
},
"type": "array"
},
"scope_hash": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"state": {
"$ref": "#/definitions/PlanState"
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"template_id": {
"type": [
"string",
"null"
]
},
"updated_at": {
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"candidate_execution_ids",
"candidate_labels",
"compatible",
"created_at",
"id",
"incomplete_execution_ids",
"label",
"locked",
"model",
"protected_executor_required",
"provider",
"purpose",
"runs",
"scenario_ids",
"scenarios",
"scope_hash",
"state",
"technical_retries",
"updated_at",
"url"
],
"title": "LocalPlan",
"type": "object"
}
},
{
"description": "Update a local plan.",
"metadata": {
"contract": {
"capability": "plan-update",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plan-update",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"properties": {
"candidate_labels": {
"additionalProperties": {
"type": "string"
},
"type": [
"object",
"null"
]
},
"label": {
"type": [
"string",
"null"
]
},
"model": {
"type": [
"string",
"null"
]
},
"plan_id": {
"default": null,
"type": [
"string",
"null"
]
},
"provider": {
"type": [
"string",
"null"
]
},
"purpose": {
"type": [
"string",
"null"
]
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"scenarios": {
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"url": {
"type": [
"string",
"null"
]
}
},
"title": "PlanUpdateRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"PlanScopeItem": {
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"inputs_sha256": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"contract_sha256",
"inputs_sha256",
"scenario_id",
"seed"
],
"type": "object"
},
"PlanState": {
"enum": [
"draft",
"baseline_running",
"baseline_ready",
"candidate_running",
"comparison_ready"
],
"type": "string"
}
},
"properties": {
"baseline_execution_id": {
"type": [
"string",
"null"
]
},
"candidate_execution_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"candidate_labels": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"compatible": {
"type": "boolean"
},
"created_at": {
"type": "string"
},
"id": {
"type": "string"
},
"incomplete_execution_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"label": {
"type": "string"
},
"last_attempt_id": {
"type": [
"string",
"null"
]
},
"locked": {
"type": "boolean"
},
"model": {
"type": "string"
},
"protected_executor_required": {
"type": "boolean"
},
"provider": {
"type": "string"
},
"purpose": {
"type": "string"
},
"reference_differences": {
"items": {
"type": "string"
},
"type": "array"
},
"reference_execution_id": {
"type": [
"string",
"null"
]
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenario_ids": {
"items": {
"type": "string"
},
"type": "array"
},
"scenarios": {
"items": {
"$ref": "#/definitions/PlanScopeItem"
},
"type": "array"
},
"scope_hash": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"state": {
"$ref": "#/definitions/PlanState"
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"template_id": {
"type": [
"string",
"null"
]
},
"updated_at": {
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"candidate_execution_ids",
"candidate_labels",
"compatible",
"created_at",
"id",
"incomplete_execution_ids",
"label",
"locked",
"model",
"protected_executor_required",
"provider",
"purpose",
"runs",
"scenario_ids",
"scenarios",
"scope_hash",
"state",
"technical_retries",
"updated_at",
"url"
],
"title": "LocalPlan",
"type": "object"
}
},
{
"description": "List saved local and imported plans from the local database.",
"metadata": {
"contract": {
"capability": "plans-list",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::plans-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "DashboardEmptyRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"master_plan": true,
"mode": {
"type": "string"
},
"plans": {
"items": true,
"type": "array"
}
},
"required": [
"master_plan",
"mode",
"plans"
],
"title": "PlansListResponse",
"type": "object"
}
},
{
"description": "Cancel the active local E2E execution.",
"metadata": {
"contract": {
"capability": "run-cancel",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::run-cancel",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "DashboardEmptyRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Defaults": {
"properties": {
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"technical_retries",
"url"
],
"type": "object"
},
"JobStatus": {
"enum": [
"running",
"cancelling",
"cancelled",
"completed",
"failed"
],
"type": "string"
},
"JobView": {
"properties": {
"completed_at": {
"type": "string"
},
"error": {
"type": "string"
},
"id": {
"type": "string"
},
"label": {
"type": "string"
},
"log": {
"type": "string"
},
"log_from": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"log_offset": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"log_truncated": {
"type": "boolean"
},
"request": {
"$ref": "#/definitions/RunRequest"
},
"returncode": {
"format": "int32",
"type": [
"integer",
"null"
]
},
"started_at": {
"type": "string"
},
"status": {
"$ref": "#/definitions/JobStatus"
}
},
"required": [
"completed_at",
"error",
"id",
"label",
"log",
"log_from",
"log_offset",
"log_truncated",
"request",
"started_at",
"status"
],
"type": "object"
},
"RunRequest": {
"additionalProperties": false,
"properties": {
"label": {
"default": "",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenarios": {
"items": {
"type": "string"
},
"type": "array"
},
"seed": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"scenarios",
"technical_retries",
"url"
],
"type": "object"
}
},
"properties": {
"defaults": {
"$ref": "#/definitions/Defaults"
},
"job": {
"anyOf": [
{
"$ref": "#/definitions/JobView"
},
{
"type": "null"
}
]
}
},
"required": [
"defaults"
],
"title": "RunSnapshot",
"type": "object"
}
},
{
"description": "Start one local E2E execution.",
"metadata": {
"contract": {
"capability": "run-start",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::run-start",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"properties": {
"label": {
"default": "",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenarios": {
"items": {
"type": "string"
},
"type": "array"
},
"seed": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"scenarios",
"technical_retries",
"url"
],
"title": "RunRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Defaults": {
"properties": {
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"technical_retries",
"url"
],
"type": "object"
},
"JobStatus": {
"enum": [
"running",
"cancelling",
"cancelled",
"completed",
"failed"
],
"type": "string"
},
"JobView": {
"properties": {
"completed_at": {
"type": "string"
},
"error": {
"type": "string"
},
"id": {
"type": "string"
},
"label": {
"type": "string"
},
"log": {
"type": "string"
},
"log_from": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"log_offset": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"log_truncated": {
"type": "boolean"
},
"request": {
"$ref": "#/definitions/RunRequest"
},
"returncode": {
"format": "int32",
"type": [
"integer",
"null"
]
},
"started_at": {
"type": "string"
},
"status": {
"$ref": "#/definitions/JobStatus"
}
},
"required": [
"completed_at",
"error",
"id",
"label",
"log",
"log_from",
"log_offset",
"log_truncated",
"request",
"started_at",
"status"
],
"type": "object"
},
"RunRequest": {
"additionalProperties": false,
"properties": {
"label": {
"default": "",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenarios": {
"items": {
"type": "string"
},
"type": "array"
},
"seed": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"scenarios",
"technical_retries",
"url"
],
"type": "object"
}
},
"properties": {
"defaults": {
"$ref": "#/definitions/Defaults"
},
"job": {
"anyOf": [
{
"$ref": "#/definitions/JobView"
},
{
"type": "null"
}
]
}
},
"required": [
"defaults"
],
"title": "RunSnapshot",
"type": "object"
}
},
{
"description": "Read local execution state and only the unread log suffix.",
"metadata": {
"contract": {
"capability": "run-status",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::run-status",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"after": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"title": "RunStatusRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"Defaults": {
"properties": {
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"technical_retries",
"url"
],
"type": "object"
},
"JobStatus": {
"enum": [
"running",
"cancelling",
"cancelled",
"completed",
"failed"
],
"type": "string"
},
"JobView": {
"properties": {
"completed_at": {
"type": "string"
},
"error": {
"type": "string"
},
"id": {
"type": "string"
},
"label": {
"type": "string"
},
"log": {
"type": "string"
},
"log_from": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"log_offset": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"log_truncated": {
"type": "boolean"
},
"request": {
"$ref": "#/definitions/RunRequest"
},
"returncode": {
"format": "int32",
"type": [
"integer",
"null"
]
},
"started_at": {
"type": "string"
},
"status": {
"$ref": "#/definitions/JobStatus"
}
},
"required": [
"completed_at",
"error",
"id",
"label",
"log",
"log_from",
"log_offset",
"log_truncated",
"request",
"started_at",
"status"
],
"type": "object"
},
"RunRequest": {
"additionalProperties": false,
"properties": {
"label": {
"default": "",
"type": "string"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenarios": {
"items": {
"type": "string"
},
"type": "array"
},
"seed": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"url": {
"type": "string"
}
},
"required": [
"model",
"provider",
"runs",
"scenarios",
"technical_retries",
"url"
],
"type": "object"
}
},
"properties": {
"defaults": {
"$ref": "#/definitions/Defaults"
},
"job": {
"anyOf": [
{
"$ref": "#/definitions/JobView"
},
{
"type": "null"
}
]
}
},
"required": [
"defaults"
],
"title": "RunSnapshot",
"type": "object"
}
},
{
"description": "Read local metric history for one test version, with provider-grouped execution models, without comparison actions.",
"metadata": {
"contract": {
"capability": "test-history-get",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::test-history-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"case_id": {
"default": null,
"type": [
"string",
"null"
]
},
"cursor": {
"default": null,
"type": [
"string",
"null"
]
},
"limit": {
"default": null,
"format": "uint16",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"result": {
"default": null,
"type": [
"string",
"null"
]
},
"subject_model": {
"default": null,
"type": [
"string",
"null"
]
},
"subject_provider": {
"default": null,
"type": [
"string",
"null"
]
},
"system_version_id": {
"default": null,
"type": [
"string",
"null"
]
},
"test_id": {
"default": "",
"type": "string"
},
"test_version": {
"default": null,
"type": [
"string",
"null"
]
}
},
"title": "TestHistoryRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"AssessmentOutcomeCounts": {
"properties": {
"error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_evaluated": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"partial": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unavailable": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"error",
"failed",
"not_evaluated",
"partial",
"passed",
"unavailable"
],
"type": "object"
},
"AssessmentSummary": {
"description": "Objective assessment totals for a set of runs. Every count is derived from deterministic evidence: hard gates, deterministic criteria, and asset validation.",
"properties": {
"assessment_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"assessment_outcomes": {
"$ref": "#/definitions/AssessmentOutcomeCounts"
},
"asset_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"asset_validation_outcomes": {
"$ref": "#/definitions/AssetValidationCounts"
},
"evidence_reference_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"system_statuses": {
"$ref": "#/definitions/StatusCounts"
}
},
"required": [
"assessment_count",
"assessment_outcomes",
"asset_count",
"asset_validation_outcomes",
"evidence_reference_count",
"run_count",
"system_statuses"
],
"type": "object"
},
"AssetValidationCounts": {
"properties": {
"invalid": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"malformed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_evaluated": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_produced": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"oversized": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"removed_during_cleanup": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unexpected": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unreadable": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unsafe_path": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"valid": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"invalid",
"malformed",
"not_evaluated",
"not_produced",
"oversized",
"removed_during_cleanup",
"unexpected",
"unreadable",
"unsafe_path",
"valid"
],
"type": "object"
},
"HistoryModelGroup": {
"description": "Models are exposed as provider groups so a model name is never ambiguous when two providers offer the same model identifier.",
"properties": {
"models": {
"items": {
"type": "string"
},
"type": "array"
},
"provider": {
"type": "string"
}
},
"required": [
"models",
"provider"
],
"type": "object"
},
"HistorySeries": {
"properties": {
"assessment_profile_sha256": {
"type": "string"
},
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"cohort_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"engine_revision": {
"type": [
"string",
"null"
]
},
"execution_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"harness_revision": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_call_errors": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_calls": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_turns": {
"format": "double",
"type": [
"number",
"null"
]
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stack_mode": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"system_label": {
"type": "string"
},
"system_revision": {
"type": [
"string",
"null"
]
},
"system_version_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"assessment_profile_sha256",
"behavior_sha256",
"case_id",
"cohort_id",
"contract_sha256",
"execution_count",
"id",
"run_count",
"stack_mode",
"subject_model",
"subject_provider",
"system_label"
],
"type": "object"
},
"HistorySystem": {
"properties": {
"id": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"id",
"label"
],
"type": "object"
},
"StatusCounts": {
"properties": {
"hard_gate_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"infrastructure_error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"resource_limit": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"subject_error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unavailable": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"hard_gate_failed",
"infrastructure_error",
"passed",
"resource_limit",
"subject_error",
"unavailable"
],
"type": "object"
},
"TestObservation": {
"properties": {
"assessment_profile_sha256": {
"type": "string"
},
"assessment_summary": {
"$ref": "#/definitions/AssessmentSummary"
},
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"cohort_id": {
"type": "string"
},
"completed_at": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"engine_revision": {
"type": [
"string",
"null"
]
},
"evaluated_version_id": {
"type": [
"string",
"null"
]
},
"execution_id": {
"type": "string"
},
"harness_revision": {
"type": [
"string",
"null"
]
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_call_errors": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_calls": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_turns": {
"format": "double",
"type": [
"number",
"null"
]
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"scored_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stack_mode": {
"type": "string"
},
"status": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"system_label": {
"type": "string"
},
"system_revision": {
"type": [
"string",
"null"
]
},
"system_version_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"assessment_profile_sha256",
"assessment_summary",
"behavior_sha256",
"case_id",
"cohort_id",
"completed_at",
"contract_sha256",
"execution_id",
"run_count",
"scored_runs",
"stack_mode",
"status",
"subject_model",
"subject_provider",
"system_label"
],
"type": "object"
},
"VersionDescriptor": {
"properties": {
"execution_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"last_seen": {
"type": [
"string",
"null"
]
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"version": {
"type": "string"
}
},
"required": [
"execution_count",
"run_count",
"version"
],
"type": "object"
}
},
"properties": {
"available_versions": {
"items": {
"$ref": "#/definitions/VersionDescriptor"
},
"type": "array"
},
"cases": {
"items": {
"type": "string"
},
"type": "array"
},
"current_version": {
"description": "The current definition's digest, which may have no executions yet.",
"type": [
"string",
"null"
]
},
"next_cursor": {
"type": [
"string",
"null"
]
},
"observations": {
"items": {
"$ref": "#/definitions/TestObservation"
},
"type": "array"
},
"series": {
"items": {
"$ref": "#/definitions/HistorySeries"
},
"type": "array"
},
"subject_models": {
"items": {
"$ref": "#/definitions/HistoryModelGroup"
},
"type": "array"
},
"subjects": {
"items": {
"type": "string"
},
"type": "array"
},
"systems": {
"items": {
"$ref": "#/definitions/HistorySystem"
},
"type": "array"
},
"test_id": {
"type": "string"
},
"test_version": {
"description": "The definition whose executions are shown (the latest with evidence by default).",
"type": "string"
},
"total": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"available_versions",
"cases",
"observations",
"series",
"subject_models",
"subjects",
"systems",
"test_id",
"test_version",
"total"
],
"title": "TestHistoryResponse",
"type": "object"
}
},
{
"description": "Read one test version across two immutable evaluated-system versions.",
"metadata": {
"contract": {
"capability": "test-version-get",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::test-version-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"cohort_id": {
"type": "string"
},
"from_version_id": {
"type": "string"
},
"test_id": {
"type": "string"
},
"test_version": {
"type": "string"
},
"to_version_id": {
"type": "string"
}
},
"required": [
"cohort_id",
"from_version_id",
"test_id",
"test_version",
"to_version_id"
],
"title": "TestVersionGetRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"AssessmentOutcomeCounts": {
"properties": {
"error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_evaluated": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"partial": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unavailable": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"error",
"failed",
"not_evaluated",
"partial",
"passed",
"unavailable"
],
"type": "object"
},
"AssessmentSummary": {
"description": "Objective assessment totals for a set of runs. Every count is derived from deterministic evidence: hard gates, deterministic criteria, and asset validation.",
"properties": {
"assessment_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"assessment_outcomes": {
"$ref": "#/definitions/AssessmentOutcomeCounts"
},
"asset_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"asset_validation_outcomes": {
"$ref": "#/definitions/AssetValidationCounts"
},
"evidence_reference_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"system_statuses": {
"$ref": "#/definitions/StatusCounts"
}
},
"required": [
"assessment_count",
"assessment_outcomes",
"asset_count",
"asset_validation_outcomes",
"evidence_reference_count",
"run_count",
"system_statuses"
],
"type": "object"
},
"AssetValidationCounts": {
"properties": {
"invalid": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"malformed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_evaluated": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_produced": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"oversized": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"removed_during_cleanup": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unexpected": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unreadable": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unsafe_path": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"valid": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"invalid",
"malformed",
"not_evaluated",
"not_produced",
"oversized",
"removed_during_cleanup",
"unexpected",
"unreadable",
"unsafe_path",
"valid"
],
"type": "object"
},
"MetricSamples": {
"properties": {
"cost_usd": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"duration_seconds": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"score": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"tokens": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"turns": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"cost_usd",
"duration_seconds",
"score",
"tokens",
"turns"
],
"type": "object"
},
"OutcomeCounts": {
"properties": {
"hard_gate_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"infra_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"technical_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"hard_gate_failed",
"infra_failed",
"passed",
"technical_failed"
],
"type": "object"
},
"StatusCounts": {
"properties": {
"hard_gate_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"infrastructure_error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"resource_limit": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"subject_error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unavailable": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"hard_gate_failed",
"infrastructure_error",
"passed",
"resource_limit",
"subject_error",
"unavailable"
],
"type": "object"
},
"TestDelta": {
"properties": {
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"score": {
"format": "double",
"type": [
"number",
"null"
]
},
"tokens": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"TestObservation": {
"properties": {
"assessment_profile_sha256": {
"type": "string"
},
"assessment_summary": {
"$ref": "#/definitions/AssessmentSummary"
},
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"cohort_id": {
"type": "string"
},
"completed_at": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"engine_revision": {
"type": [
"string",
"null"
]
},
"evaluated_version_id": {
"type": [
"string",
"null"
]
},
"execution_id": {
"type": "string"
},
"harness_revision": {
"type": [
"string",
"null"
]
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_call_errors": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_calls": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_turns": {
"format": "double",
"type": [
"number",
"null"
]
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"scored_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stack_mode": {
"type": "string"
},
"status": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"system_label": {
"type": "string"
},
"system_revision": {
"type": [
"string",
"null"
]
},
"system_version_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"assessment_profile_sha256",
"assessment_summary",
"behavior_sha256",
"case_id",
"cohort_id",
"completed_at",
"contract_sha256",
"execution_id",
"run_count",
"scored_runs",
"stack_mode",
"status",
"subject_model",
"subject_provider",
"system_label"
],
"type": "object"
},
"TestSideSummary": {
"properties": {
"assessment_summary": {
"$ref": "#/definitions/AssessmentSummary"
},
"case_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"evaluated_version_id": {
"type": "string"
},
"execution_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"outcomes": {
"$ref": "#/definitions/OutcomeCounts"
},
"pass_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"samples": {
"$ref": "#/definitions/MetricSamples"
},
"scored_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"total_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"assessment_summary",
"case_count",
"evaluated_version_id",
"execution_count",
"outcomes",
"samples",
"scored_runs",
"total_runs"
],
"type": "object"
}
},
"properties": {
"compatibility": {
"type": "string"
},
"compatibility_reasons": {
"items": {
"type": "string"
},
"type": "array"
},
"delta": {
"$ref": "#/definitions/TestDelta"
},
"from": {
"anyOf": [
{
"$ref": "#/definitions/TestSideSummary"
},
{
"type": "null"
}
]
},
"from_observations": {
"items": {
"$ref": "#/definitions/TestObservation"
},
"type": "array"
},
"test_id": {
"type": "string"
},
"test_version": {
"type": "string"
},
"to": {
"anyOf": [
{
"$ref": "#/definitions/TestSideSummary"
},
{
"type": "null"
}
]
},
"to_observations": {
"items": {
"$ref": "#/definitions/TestObservation"
},
"type": "array"
}
},
"required": [
"compatibility",
"compatibility_reasons",
"delta",
"from_observations",
"test_id",
"test_version",
"to_observations"
],
"title": "TestVersionResult",
"type": "object"
}
},
{
"description": "List the versioned test catalog with compact A/B results and stable pagination.",
"metadata": {
"contract": {
"capability": "tests-list",
"name": "harness-e2e-dashboard"
},
"internal": true
},
"name": "e2e::dashboard::tests-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"cohort_id": {
"default": null,
"type": [
"string",
"null"
]
},
"cursor": {
"default": null,
"type": [
"string",
"null"
]
},
"from_version_id": {
"default": null,
"type": [
"string",
"null"
]
},
"limit": {
"default": null,
"format": "uint16",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"query": {
"default": null,
"type": [
"string",
"null"
]
},
"to_version_id": {
"default": null,
"type": [
"string",
"null"
]
}
},
"title": "TestsListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"AssessmentKind": {
"enum": [
"required_check",
"signal",
"asset_validation"
],
"type": "string"
},
"AssessmentOutcomeCounts": {
"properties": {
"error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_evaluated": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"partial": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unavailable": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"error",
"failed",
"not_evaluated",
"partial",
"passed",
"unavailable"
],
"type": "object"
},
"AssessmentPolicy": {
"enum": [
"hard_gate",
"advisory"
],
"type": "string"
},
"AssessmentSummary": {
"description": "Objective assessment totals for a set of runs. Every count is derived from deterministic evidence: hard gates, deterministic criteria, and asset validation.",
"properties": {
"assessment_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"assessment_outcomes": {
"$ref": "#/definitions/AssessmentOutcomeCounts"
},
"asset_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"asset_validation_outcomes": {
"$ref": "#/definitions/AssetValidationCounts"
},
"evidence_reference_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"system_statuses": {
"$ref": "#/definitions/StatusCounts"
}
},
"required": [
"assessment_count",
"assessment_outcomes",
"asset_count",
"asset_validation_outcomes",
"evidence_reference_count",
"run_count",
"system_statuses"
],
"type": "object"
},
"AssetValidationCounts": {
"properties": {
"invalid": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"malformed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_evaluated": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not_produced": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"oversized": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"removed_during_cleanup": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unexpected": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unreadable": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unsafe_path": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"valid": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"invalid",
"malformed",
"not_evaluated",
"not_produced",
"oversized",
"removed_during_cleanup",
"unexpected",
"unreadable",
"unsafe_path",
"valid"
],
"type": "object"
},
"CalibrationProjection": {
"properties": {
"compatible_sample_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"maturity": {
"type": "string"
}
},
"required": [
"compatible_sample_count",
"maturity"
],
"type": "object"
},
"EvaluationDimension": {
"enum": [
"deliverable",
"structural_integrity",
"efficiency",
"robustness",
"e2e_infrastructure"
],
"type": "string"
},
"ExecutionPolicy": {
"properties": {
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_total_tokens": {
"description": "Shared Harness token budget. `None` leaves the Harness budget unbounded for this scenario.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_turns": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"max_validation_retries": {
"description": "Optional per-session cap for post-turn validation denials. `None` preserves the Harness-configured default.",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stuck_timeout_seconds": {
"description": "Stop only after this many seconds without observable useful progress. Large scenarios have no fixed wall-clock deadline.",
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"max_turns",
"stuck_timeout_seconds"
],
"type": "object"
},
"ExecutionRealism": {
"enum": [
"synthetic",
"realistic_simulator",
"frozen_real_artifact"
],
"type": "string"
},
"HumanHorizon": {
"properties": {
"basis": {
"allOf": [
{
"$ref": "#/definitions/HumanHorizonBasis"
}
],
"default": "unknown"
},
"max_minutes": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"min_minutes": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"HumanHorizonBasis": {
"enum": [
"unknown",
"author_estimate",
"measured"
],
"type": "string"
},
"MetricSamples": {
"properties": {
"cost_usd": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"duration_seconds": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"score": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"tokens": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"turns": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"cost_usd",
"duration_seconds",
"score",
"tokens",
"turns"
],
"type": "object"
},
"OutcomeCounts": {
"properties": {
"hard_gate_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"infra_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"technical_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"hard_gate_failed",
"infra_failed",
"passed",
"technical_failed"
],
"type": "object"
},
"ScenarioCharacterization": {
"properties": {
"human_horizon": {
"allOf": [
{
"$ref": "#/definitions/HumanHorizon"
}
],
"default": {
"basis": "unknown"
}
},
"realism": {
"allOf": [
{
"$ref": "#/definitions/ScenarioRealism"
}
],
"default": {
"execution": "synthetic",
"shadow": "none"
}
}
},
"type": "object"
},
"ScenarioRealism": {
"properties": {
"execution": {
"allOf": [
{
"$ref": "#/definitions/ExecutionRealism"
}
],
"default": "synthetic"
},
"shadow": {
"allOf": [
{
"$ref": "#/definitions/ShadowMode"
}
],
"default": "none"
}
},
"type": "object"
},
"ShadowMode": {
"enum": [
"none",
"read_only"
],
"type": "string"
},
"StatusCounts": {
"properties": {
"hard_gate_failed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"infrastructure_error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"passed": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"resource_limit": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"subject_error": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"unavailable": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"hard_gate_failed",
"infrastructure_error",
"passed",
"resource_limit",
"subject_error",
"unavailable"
],
"type": "object"
},
"TestCatalogRow": {
"properties": {
"available_versions": {
"items": {
"$ref": "#/definitions/VersionDescriptor"
},
"type": "array"
},
"calibration": {
"anyOf": [
{
"$ref": "#/definitions/CalibrationProjection"
},
{
"type": "null"
}
]
},
"characterization": {
"anyOf": [
{
"$ref": "#/definitions/ScenarioCharacterization"
},
{
"type": "null"
}
]
},
"current_version": {
"type": [
"string",
"null"
]
},
"lifecycle": {
"type": "string"
},
"result": {
"anyOf": [
{
"$ref": "#/definitions/TestVersionResult"
},
{
"type": "null"
}
]
},
"selected_version": {
"type": [
"string",
"null"
]
},
"spec": {
"anyOf": [
{
"$ref": "#/definitions/TestSpecProjection"
},
{
"type": "null"
}
]
},
"test_id": {
"type": "string"
}
},
"required": [
"available_versions",
"lifecycle",
"test_id"
],
"type": "object"
},
"TestCriterionProjection": {
"properties": {
"description": {
"type": "string"
},
"dimension": {
"$ref": "#/definitions/EvaluationDimension"
},
"id": {
"type": "string"
},
"kind": {
"$ref": "#/definitions/AssessmentKind"
},
"policy": {
"$ref": "#/definitions/AssessmentPolicy"
},
"weight": {
"format": "uint8",
"minimum": 0,
"type": "integer"
}
},
"required": [
"description",
"dimension",
"id",
"kind",
"policy",
"weight"
],
"type": "object"
},
"TestDelta": {
"properties": {
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"score": {
"format": "double",
"type": [
"number",
"null"
]
},
"tokens": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"TestObservation": {
"properties": {
"assessment_profile_sha256": {
"type": "string"
},
"assessment_summary": {
"$ref": "#/definitions/AssessmentSummary"
},
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"cohort_id": {
"type": "string"
},
"completed_at": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"engine_revision": {
"type": [
"string",
"null"
]
},
"evaluated_version_id": {
"type": [
"string",
"null"
]
},
"execution_id": {
"type": "string"
},
"harness_revision": {
"type": [
"string",
"null"
]
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_call_errors": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_function_calls": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_turns": {
"format": "double",
"type": [
"number",
"null"
]
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"scored_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stack_mode": {
"type": "string"
},
"status": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"system_label": {
"type": "string"
},
"system_revision": {
"type": [
"string",
"null"
]
},
"system_version_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"assessment_profile_sha256",
"assessment_summary",
"behavior_sha256",
"case_id",
"cohort_id",
"completed_at",
"contract_sha256",
"execution_id",
"run_count",
"scored_runs",
"stack_mode",
"status",
"subject_model",
"subject_provider",
"system_label"
],
"type": "object"
},
"TestSideSummary": {
"properties": {
"assessment_summary": {
"$ref": "#/definitions/AssessmentSummary"
},
"case_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"evaluated_version_id": {
"type": "string"
},
"execution_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"mean_score": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_duration_seconds": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_tokens": {
"format": "double",
"type": [
"number",
"null"
]
},
"outcomes": {
"$ref": "#/definitions/OutcomeCounts"
},
"pass_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"samples": {
"$ref": "#/definitions/MetricSamples"
},
"scored_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"total_runs": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"assessment_summary",
"case_count",
"evaluated_version_id",
"execution_count",
"outcomes",
"samples",
"scored_runs",
"total_runs"
],
"type": "object"
},
"TestSpecProjection": {
"description": "The scenario definition as a reader needs it: what the subject is asked to do, how the result is scored, and the limits it runs under. Projected from the materialized `ScenarioSpec` of the test's current version, so it always describes the contract the dashboard is showing.",
"properties": {
"criteria": {
"items": {
"$ref": "#/definitions/TestCriterionProjection"
},
"type": "array"
},
"denied_functions": {
"items": {
"type": "string"
},
"type": "array"
},
"execution": {
"$ref": "#/definitions/ExecutionPolicy"
},
"prompt": {
"description": "The prompt as materialized for a representative run: run-scoped paths resolve against a catalog namespace, never a real execution.",
"type": "string"
},
"summary": {
"description": "Editorial description; absent until the scenario defines a `SUMMARY`.",
"type": [
"string",
"null"
]
}
},
"required": [
"criteria",
"denied_functions",
"execution",
"prompt"
],
"type": "object"
},
"TestVersionResult": {
"properties": {
"compatibility": {
"type": "string"
},
"compatibility_reasons": {
"items": {
"type": "string"
},
"type": "array"
},
"delta": {
"$ref": "#/definitions/TestDelta"
},
"from": {
"anyOf": [
{
"$ref": "#/definitions/TestSideSummary"
},
{
"type": "null"
}
]
},
"from_observations": {
"items": {
"$ref": "#/definitions/TestObservation"
},
"type": "array"
},
"test_id": {
"type": "string"
},
"test_version": {
"type": "string"
},
"to": {
"anyOf": [
{
"$ref": "#/definitions/TestSideSummary"
},
{
"type": "null"
}
]
},
"to_observations": {
"items": {
"$ref": "#/definitions/TestObservation"
},
"type": "array"
}
},
"required": [
"compatibility",
"compatibility_reasons",
"delta",
"from_observations",
"test_id",
"test_version",
"to_observations"
],
"type": "object"
},
"VersionDescriptor": {
"properties": {
"execution_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"last_seen": {
"type": [
"string",
"null"
]
},
"run_count": {
"format": "uint",
"minimum": 0,
"type": "integer"
},
"version": {
"type": "string"
}
},
"required": [
"execution_count",
"run_count",
"version"
],
"type": "object"
}
},
"properties": {
"next_cursor": {
"type": [
"string",
"null"
]
},
"revision": {
"type": "string"
},
"rows": {
"items": {
"$ref": "#/definitions/TestCatalogRow"
},
"type": "array"
},
"total": {
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"revision",
"rows",
"total"
],
"title": "TestsListResponse",
"type": "object"
}
},
{
"description": "Classify a protected supervisor's fault journal against canonical execution evidence.",
"metadata": {
"contract": {
"capabilities": [
"fault-evaluate"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::fault-evaluate",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"CancellationRule": {
"additionalProperties": false,
"properties": {
"after_action": {
"type": "string"
},
"grace_period_ms": {
"default": 30000,
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"after_action"
],
"type": "object"
},
"DelayRule": {
"additionalProperties": false,
"properties": {
"delay_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"occurrences": {
"default": 1,
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"target": {
"type": "string"
}
},
"required": [
"delay_ms",
"target"
],
"type": "object"
},
"DisconnectRule": {
"additionalProperties": false,
"properties": {
"after_action": {
"type": "string"
},
"duration_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"target": {
"type": "string"
}
},
"required": [
"after_action",
"duration_ms",
"target"
],
"type": "object"
},
"ExpectedTerminalOutcome": {
"oneOf": [
{
"enum": [
"recovered",
"cancelled"
],
"type": "string"
},
{
"description": "The subject is expected to finish WITHOUT a correct deliverable: a hard-gate-failed but structurally clean, fully cleaned-up, bounded run. A wrong-but-structural result is the expectation, so it classifies as `correct_recovery`; a passed deliverable means the perturbation failed to bite and is a benchmark infrastructure failure. Degraded profiles must not declare a cancellation rule and require canonical results.",
"enum": [
"degraded"
],
"type": "string"
}
]
},
"FaultAction": {
"additionalProperties": false,
"properties": {
"action_id": {
"type": "string"
},
"duration_ms": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"kind": {
"$ref": "#/definitions/FaultActionKind"
},
"sequence": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"target": {
"type": "string"
},
"trigger_attempt": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"action_id",
"kind",
"sequence",
"target"
],
"type": "object"
},
"FaultActionKind": {
"enum": [
"delay_call",
"fail_first_attempt",
"malformed_result",
"throttle_burst",
"duplicate_event",
"child_timeout",
"transient_disconnect",
"out_of_order_result",
"cancel_execution"
],
"type": "string"
},
"FaultJournal": {
"additionalProperties": false,
"properties": {
"actions": {
"items": {
"$ref": "#/definitions/FaultObservation"
},
"type": "array"
},
"cancellation_observed": {
"type": "boolean"
},
"cleanup_completed": {
"type": "boolean"
},
"cleanup_evidence": {
"additionalProperties": {
"type": "string"
},
"default": {},
"type": "object"
},
"completed_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"infrastructure_failures": {
"default": [],
"items": {
"type": "string"
},
"type": "array"
},
"lane": {
"type": "string"
},
"profile_sha256": {
"type": "string"
},
"results_sha256": {
"type": [
"string",
"null"
]
},
"started_at": {
"type": "string"
}
},
"required": [
"actions",
"cancellation_observed",
"cleanup_completed",
"completed_at",
"execution_id",
"lane",
"profile_sha256",
"started_at"
],
"type": "object"
},
"FaultObservation": {
"additionalProperties": false,
"properties": {
"action_id": {
"type": "string"
},
"applied_at": {
"type": [
"string",
"null"
]
},
"detail": {
"default": "",
"type": "string"
},
"kind": {
"$ref": "#/definitions/FaultActionKind"
},
"recovered_at": {
"type": [
"string",
"null"
]
},
"status": {
"$ref": "#/definitions/FaultObservationStatus"
}
},
"required": [
"action_id",
"kind",
"status"
],
"type": "object"
},
"FaultObservationStatus": {
"enum": [
"recovered",
"unrecovered",
"not_triggered",
"injector_failed"
],
"type": "string"
},
"FaultPlan": {
"additionalProperties": false,
"properties": {
"actions": {
"items": {
"$ref": "#/definitions/FaultAction"
},
"type": "array"
},
"expected_outcome": {
"$ref": "#/definitions/ExpectedTerminalOutcome"
},
"profile_id": {
"type": "string"
},
"profile_sha256": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"actions",
"expected_outcome",
"profile_id",
"profile_sha256",
"seed"
],
"type": "object"
},
"FaultProfile": {
"additionalProperties": false,
"properties": {
"cancellation": {
"anyOf": [
{
"$ref": "#/definitions/CancellationRule"
},
{
"type": "null"
}
],
"default": null
},
"child_timeout_ms": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"delayed_calls": {
"default": [],
"items": {
"$ref": "#/definitions/DelayRule"
},
"type": "array"
},
"duplicate_events": {
"default": false,
"type": "boolean"
},
"expected_outcome": {
"allOf": [
{
"$ref": "#/definitions/ExpectedTerminalOutcome"
}
],
"default": "recovered"
},
"fail_first_attempt": {
"default": [],
"items": {
"type": "string"
},
"type": "array"
},
"malformed_results": {
"default": [],
"description": "The supervisor corrupts the result payload of the first call to each named target: the subject receives undecodable, contract-breaking bytes exactly once per target.",
"items": {
"type": "string"
},
"type": "array"
},
"out_of_order_results": {
"default": false,
"type": "boolean"
},
"profile_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"throttle_burst": {
"anyOf": [
{
"$ref": "#/definitions/ThrottleRule"
},
{
"type": "null"
}
],
"default": null
},
"transient_disconnect": {
"anyOf": [
{
"$ref": "#/definitions/DisconnectRule"
},
{
"type": "null"
}
],
"default": null
}
},
"required": [
"profile_id",
"seed"
],
"type": "object"
},
"ThrottleRule": {
"additionalProperties": false,
"description": "The supervisor answers the target's next `responses` calls with a provider-throttle (429-style) rejection, spaced `spacing_ms` apart.",
"properties": {
"responses": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"spacing_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"target": {
"type": "string"
}
},
"required": [
"responses",
"spacing_ms",
"target"
],
"type": "object"
}
},
"properties": {
"execution_id": {
"type": "string"
},
"journal": {
"$ref": "#/definitions/FaultJournal"
},
"plan": {
"$ref": "#/definitions/FaultPlan"
},
"profile": {
"$ref": "#/definitions/FaultProfile"
}
},
"required": [
"execution_id",
"journal",
"plan",
"profile"
],
"title": "FaultEvaluateRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"definitions": {
"RecoveryClassification": {
"enum": [
"correct_recovery",
"incorrect_result",
"structural_failure",
"infrastructure_failure"
],
"type": "string"
}
},
"properties": {
"cancellation_observed": {
"type": "boolean"
},
"classification": {
"$ref": "#/definitions/RecoveryClassification"
},
"cleanup_completed": {
"type": "boolean"
},
"deliverable_correct": {
"type": [
"boolean",
"null"
]
},
"execution_id": {
"type": "string"
},
"planned_actions": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"profile_id": {
"type": "string"
},
"profile_sha256": {
"type": "string"
},
"reasons": {
"items": {
"type": "string"
},
"type": "array"
},
"recovered_actions": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"results_sha256": {
"type": [
"string",
"null"
]
},
"structural_integrity": {
"type": [
"boolean",
"null"
]
}
},
"required": [
"cancellation_observed",
"classification",
"cleanup_completed",
"execution_id",
"planned_actions",
"profile_id",
"profile_sha256",
"reasons",
"recovered_actions"
],
"title": "FaultEvaluation",
"type": "object"
}
},
{
"description": "Materialize a deterministic, versioned fault plan for a protected supervisor.",
"metadata": {
"contract": {
"capabilities": [
"fault-plan"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::fault-plan",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"CancellationRule": {
"additionalProperties": false,
"properties": {
"after_action": {
"type": "string"
},
"grace_period_ms": {
"default": 30000,
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"after_action"
],
"type": "object"
},
"DelayRule": {
"additionalProperties": false,
"properties": {
"delay_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"occurrences": {
"default": 1,
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"target": {
"type": "string"
}
},
"required": [
"delay_ms",
"target"
],
"type": "object"
},
"DisconnectRule": {
"additionalProperties": false,
"properties": {
"after_action": {
"type": "string"
},
"duration_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"target": {
"type": "string"
}
},
"required": [
"after_action",
"duration_ms",
"target"
],
"type": "object"
},
"ExpectedTerminalOutcome": {
"oneOf": [
{
"enum": [
"recovered",
"cancelled"
],
"type": "string"
},
{
"description": "The subject is expected to finish WITHOUT a correct deliverable: a hard-gate-failed but structurally clean, fully cleaned-up, bounded run. A wrong-but-structural result is the expectation, so it classifies as `correct_recovery`; a passed deliverable means the perturbation failed to bite and is a benchmark infrastructure failure. Degraded profiles must not declare a cancellation rule and require canonical results.",
"enum": [
"degraded"
],
"type": "string"
}
]
},
"FaultProfile": {
"additionalProperties": false,
"properties": {
"cancellation": {
"anyOf": [
{
"$ref": "#/definitions/CancellationRule"
},
{
"type": "null"
}
],
"default": null
},
"child_timeout_ms": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"delayed_calls": {
"default": [],
"items": {
"$ref": "#/definitions/DelayRule"
},
"type": "array"
},
"duplicate_events": {
"default": false,
"type": "boolean"
},
"expected_outcome": {
"allOf": [
{
"$ref": "#/definitions/ExpectedTerminalOutcome"
}
],
"default": "recovered"
},
"fail_first_attempt": {
"default": [],
"items": {
"type": "string"
},
"type": "array"
},
"malformed_results": {
"default": [],
"description": "The supervisor corrupts the result payload of the first call to each named target: the subject receives undecodable, contract-breaking bytes exactly once per target.",
"items": {
"type": "string"
},
"type": "array"
},
"out_of_order_results": {
"default": false,
"type": "boolean"
},
"profile_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"throttle_burst": {
"anyOf": [
{
"$ref": "#/definitions/ThrottleRule"
},
{
"type": "null"
}
],
"default": null
},
"transient_disconnect": {
"anyOf": [
{
"$ref": "#/definitions/DisconnectRule"
},
{
"type": "null"
}
],
"default": null
}
},
"required": [
"profile_id",
"seed"
],
"type": "object"
},
"ThrottleRule": {
"additionalProperties": false,
"description": "The supervisor answers the target's next `responses` calls with a provider-throttle (429-style) rejection, spaced `spacing_ms` apart.",
"properties": {
"responses": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"spacing_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"target": {
"type": "string"
}
},
"required": [
"responses",
"spacing_ms",
"target"
],
"type": "object"
}
},
"properties": {
"profile": {
"$ref": "#/definitions/FaultProfile"
}
},
"required": [
"profile"
],
"title": "FaultPlanRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"definitions": {
"ExpectedTerminalOutcome": {
"oneOf": [
{
"enum": [
"recovered",
"cancelled"
],
"type": "string"
},
{
"description": "The subject is expected to finish WITHOUT a correct deliverable: a hard-gate-failed but structurally clean, fully cleaned-up, bounded run. A wrong-but-structural result is the expectation, so it classifies as `correct_recovery`; a passed deliverable means the perturbation failed to bite and is a benchmark infrastructure failure. Degraded profiles must not declare a cancellation rule and require canonical results.",
"enum": [
"degraded"
],
"type": "string"
}
]
},
"FaultAction": {
"additionalProperties": false,
"properties": {
"action_id": {
"type": "string"
},
"duration_ms": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"kind": {
"$ref": "#/definitions/FaultActionKind"
},
"sequence": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"target": {
"type": "string"
},
"trigger_attempt": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"action_id",
"kind",
"sequence",
"target"
],
"type": "object"
},
"FaultActionKind": {
"enum": [
"delay_call",
"fail_first_attempt",
"malformed_result",
"throttle_burst",
"duplicate_event",
"child_timeout",
"transient_disconnect",
"out_of_order_result",
"cancel_execution"
],
"type": "string"
}
},
"properties": {
"actions": {
"items": {
"$ref": "#/definitions/FaultAction"
},
"type": "array"
},
"expected_outcome": {
"$ref": "#/definitions/ExpectedTerminalOutcome"
},
"profile_id": {
"type": "string"
},
"profile_sha256": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"actions",
"expected_outcome",
"profile_id",
"profile_sha256",
"seed"
],
"title": "FaultPlan",
"type": "object"
}
},
{
"description": "List hash-validated E2E history records from the iii database worker.",
"metadata": {
"contract": {
"capabilities": [
"history-list"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::history-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"lane": {
"default": null,
"type": [
"string",
"null"
]
},
"limit": {
"default": 100,
"format": "uint16",
"minimum": 0,
"type": "integer"
}
},
"title": "HistoryListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"DurableArchiveReference": {
"additionalProperties": false,
"properties": {
"archive_id": {
"type": "string"
},
"created_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"expires_at": {
"type": [
"string",
"null"
]
},
"identity_sha256": {
"type": "string"
},
"manifest": {
"$ref": "#/definitions/StorageObjectReference"
},
"manifest_backup": {
"$ref": "#/definitions/StorageObjectReference"
},
"retention_class": {
"$ref": "#/definitions/RetentionClass"
}
},
"required": [
"archive_id",
"created_at",
"execution_id",
"identity_sha256",
"manifest",
"manifest_backup",
"retention_class"
],
"type": "object"
},
"HistoryRecord": {
"additionalProperties": false,
"properties": {
"archive": {
"$ref": "#/definitions/DurableArchiveReference"
},
"baseline_comparable": {
"type": "boolean"
},
"case_count": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"e2e_repository": {
"type": "string"
},
"e2e_revision": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"identity_sha256": {
"type": "string"
},
"ingestion_id": {
"type": "string"
},
"lane": {
"type": "string"
},
"occurred_at": {
"type": "string"
},
"passed": {
"type": "boolean"
},
"result_contract_sha256": {
"type": "string"
},
"stack_mode": {
"type": "string"
},
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"subject_revision": {
"type": "string"
}
},
"required": [
"archive",
"baseline_comparable",
"case_count",
"e2e_repository",
"e2e_revision",
"execution_id",
"identity_sha256",
"ingestion_id",
"lane",
"occurred_at",
"passed",
"result_contract_sha256",
"stack_mode",
"subject_model",
"subject_provider",
"subject_revision"
],
"type": "object"
},
"RetentionClass": {
"enum": [
"temporary",
"pull_request",
"longitudinal",
"canonical"
],
"type": "string"
},
"StorageObjectReference": {
"properties": {
"bucket": {
"type": "string"
},
"key": {
"type": "string"
},
"media_type": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"uri": {
"type": "string"
}
},
"required": [
"bucket",
"key",
"media_type",
"sha256",
"size_bytes",
"uri"
],
"type": "object"
}
},
"properties": {
"records": {
"items": {
"$ref": "#/definitions/HistoryRecord"
},
"type": "array"
}
},
"required": [
"records"
],
"title": "HistoryListResponse",
"type": "object"
}
},
{
"description": "Get the immutable result and manifest of an E2E execution.",
"metadata": {
"contract": {
"capabilities": [
"results-get"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::results-get",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ActivationPolicy": {
"oneOf": [
{
"properties": {
"policy": {
"enum": [
"always"
],
"type": "string"
}
},
"required": [
"policy"
],
"type": "object"
},
{
"properties": {
"conditions": {
"items": {
"$ref": "#/definitions/BooleanCondition"
},
"type": "array"
},
"policy": {
"enum": [
"all"
],
"type": "string"
}
},
"required": [
"conditions",
"policy"
],
"type": "object"
},
{
"properties": {
"conditions": {
"items": {
"$ref": "#/definitions/BooleanCondition"
},
"type": "array"
},
"policy": {
"enum": [
"any"
],
"type": "string"
}
},
"required": [
"conditions",
"policy"
],
"type": "object"
}
]
},
"AgentProfileArtifact": {
"properties": {
"configuration_sha256": {
"type": "string"
},
"id": {
"type": "string"
}
},
"required": [
"configuration_sha256",
"id"
],
"type": "object"
},
"ArtifactExpectation": {
"properties": {
"id": {
"type": "string"
},
"kind": {
"type": "string"
},
"max_size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"media_type": {
"type": "string"
},
"schema": true
},
"required": [
"id",
"kind",
"max_size_bytes",
"media_type",
"schema"
],
"type": "object"
},
"ArtifactReference": {
"properties": {
"id": {
"type": "string"
},
"kind": {
"type": "string"
},
"media_type": {
"type": "string"
},
"path": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"id",
"kind",
"media_type",
"path",
"sha256",
"size_bytes"
],
"type": "object"
},
"AssessmentContract": {
"additionalProperties": false,
"properties": {
"runs": {
"items": {
"$ref": "#/definitions/RunAssessmentContract"
},
"type": "array"
}
},
"required": [
"runs"
],
"type": "object"
},
"AssessmentKind": {
"enum": [
"required_check",
"signal",
"asset_validation"
],
"type": "string"
},
"AssessmentOutcome": {
"enum": [
"passed",
"failed",
"partial",
"not_evaluated",
"unavailable",
"error"
],
"type": "string"
},
"AssessmentPolicy": {
"enum": [
"hard_gate",
"advisory"
],
"type": "string"
},
"AssessmentResult": {
"properties": {
"criterion_id": {
"type": "string"
},
"dimension": {
"$ref": "#/definitions/EvaluationDimension"
},
"evidence": {
"items": {
"$ref": "#/definitions/EvidenceReference"
},
"type": "array"
},
"kind": {
"$ref": "#/definitions/AssessmentKind"
},
"outcome": {
"$ref": "#/definitions/AssessmentOutcome"
},
"policy": {
"$ref": "#/definitions/AssessmentPolicy"
},
"score": {
"anyOf": [
{
"$ref": "#/definitions/AssessmentScore"
},
{
"type": "null"
}
]
},
"summary": {
"type": "string"
},
"target": {
"$ref": "#/definitions/AssessmentTarget"
}
},
"required": [
"criterion_id",
"dimension",
"kind",
"outcome",
"policy",
"summary",
"target"
],
"type": "object"
},
"AssessmentScore": {
"properties": {
"awarded": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"possible": {
"format": "uint8",
"minimum": 0,
"type": "integer"
}
},
"required": [
"awarded",
"possible"
],
"type": "object"
},
"AssessmentTarget": {
"properties": {
"id": {
"type": "string"
},
"kind": {
"$ref": "#/definitions/AssessmentTargetKind"
}
},
"required": [
"id",
"kind"
],
"type": "object"
},
"AssessmentTargetKind": {
"enum": [
"criterion"
],
"type": "string"
},
"AssetValidationOutcome": {
"enum": [
"valid",
"invalid",
"malformed",
"oversized",
"not_produced",
"unreadable",
"unsafe_path",
"removed_during_cleanup",
"unexpected",
"not_evaluated"
],
"type": "string"
},
"AssetValidationResult": {
"properties": {
"asset_id": {
"type": "string"
},
"evidence": {
"items": {
"$ref": "#/definitions/EvidenceReference"
},
"type": "array"
},
"outcome": {
"$ref": "#/definitions/AssetValidationOutcome"
},
"summary": {
"type": "string"
}
},
"required": [
"asset_id",
"outcome",
"summary"
],
"type": "object"
},
"AuditEvidence": {
"properties": {
"detail": {
"description": "Sanitized, bounded description. Never contains secret material.",
"type": "string"
},
"function_id": {
"type": [
"string",
"null"
]
},
"message_index": {
"description": "Index of the transcript message the observation anchors to.",
"format": "uint",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"detail"
],
"type": "object"
},
"AuditFlag": {
"properties": {
"confidence": {
"format": "double",
"type": [
"number",
"null"
]
},
"evidence": {
"items": {
"$ref": "#/definitions/AuditEvidence"
},
"type": "array"
},
"kind": {
"$ref": "#/definitions/AuditFlagKind"
},
"severity": {
"$ref": "#/definitions/AuditSeverity"
},
"source": {
"$ref": "#/definitions/AuditFlagSource"
},
"summary": {
"type": "string"
}
},
"required": [
"kind",
"severity",
"source",
"summary"
],
"type": "object"
},
"AuditFlagKind": {
"oneOf": [
{
"description": "The subject invoked the E2E control plane or another function the scenario explicitly denies — the verification mechanism itself.",
"enum": [
"verifier_tampering"
],
"type": "string"
},
{
"description": "The subject addressed a session outside its own session tree.",
"enum": [
"out_of_scope_session_access"
],
"type": "string"
},
{
"description": "Live secret material from the runner environment appeared in the transcript.",
"enum": [
"secret_exposure"
],
"type": "string"
},
{
"description": "The subject invoked a destructive session or state operation.",
"enum": [
"destructive_action"
],
"type": "string"
}
]
},
"AuditFlagSource": {
"enum": [
"deterministic"
],
"type": "string"
},
"AuditReport": {
"properties": {
"flags": {
"items": {
"$ref": "#/definitions/AuditFlag"
},
"type": "array"
}
},
"required": [
"flags"
],
"type": "object"
},
"AuditSeverity": {
"enum": [
"info",
"warning",
"critical"
],
"type": "string"
},
"BooleanCondition": {
"additionalProperties": false,
"properties": {
"equals": {
"type": "boolean"
},
"node_id": {
"type": "string"
},
"port": {
"type": "string"
}
},
"required": [
"equals",
"node_id",
"port"
],
"type": "object"
},
"Capability": {
"description": "What a scenario needs from the engine, the runner host or a fixture to run. The closed list keeps the catalog honest: a new need is a new variant, and every id is spelled once.",
"enum": [
"browser::interactive",
"cross_repo_contract_simulator::v1",
"curl",
"docker",
"e2e::adaptive-flow-v1",
"e2e::control-plane-v1",
"e2e::filesystem",
"e2e::git",
"e2e::run-scoped-fixtures",
"e2e::shell",
"e2e::subagents",
"e2e::workflow-resume-v1",
"fixture::multi-origin-http",
"git",
"git::deterministic-fixture-v1",
"git::offline-bundle",
"github::security-read",
"github::trusted-handoff",
"harness::independent_session",
"harness::post-turn-validation",
"harness::scripted-dialogue-v1",
"iii::coder",
"iii::compose",
"iii::database",
"iii::functions",
"iii::registry",
"iii::shell",
"iii::state",
"iii::triggers",
"iii::workers",
"incident_fixture::v1",
"node",
"playwright",
"python3",
"release_shadow::read-only-v1",
"release_train_simulator::v1",
"security_scan::on-demand",
"security_scan::v1",
"swe::isolated-python-workspace"
],
"type": "string"
},
"CapturedInvariant": {
"properties": {
"id": {
"type": "string"
},
"passed": {
"type": "boolean"
},
"reason": {
"type": "string"
}
},
"required": [
"id",
"passed",
"reason"
],
"type": "object"
},
"CompletionState": {
"enum": [
"completed",
"task_incomplete",
"undetermined"
],
"type": "string"
},
"ContaminationScope": {
"enum": [
"none",
"case",
"execution"
],
"type": "string"
},
"ControlPlaneEvidence": {
"properties": {
"functions": {
"items": {
"$ref": "#/definitions/FunctionContractEvidence"
},
"type": "array"
}
},
"required": [
"functions"
],
"type": "object"
},
"CostReport": {
"properties": {
"subject_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"total_usd": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"type": "object"
},
"CriterionReport": {
"properties": {
"awarded": {
"format": "uint8",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"description": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"possible": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"reason": {
"type": "string"
}
},
"required": [
"id",
"possible",
"reason"
],
"type": "object"
},
"DeliverableContentFormat": {
"enum": [
"json",
"text_utf8"
],
"type": "string"
},
"DeliverableContract": {
"properties": {
"artifacts": {
"items": {
"$ref": "#/definitions/ArtifactExpectation"
},
"type": "array"
},
"capture_before_cleanup": {
"type": "boolean"
},
"invariants": {
"items": {
"$ref": "#/definitions/InvariantSpec"
},
"type": "array"
},
"provenance_required": {
"type": "boolean"
}
},
"required": [
"artifacts",
"capture_before_cleanup",
"invariants",
"provenance_required"
],
"type": "object"
},
"DeliverableReport": {
"properties": {
"artifact": {
"anyOf": [
{
"$ref": "#/definitions/ArtifactReference"
},
{
"type": "null"
}
]
},
"content_format": {
"$ref": "#/definitions/DeliverableContentFormat"
},
"content_sha256": {
"type": "string"
},
"content_size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"id": {
"type": "string"
},
"invariants": {
"items": {
"$ref": "#/definitions/CapturedInvariant"
},
"type": "array"
},
"kind": {
"type": "string"
},
"media_type": {
"type": "string"
},
"preview": true,
"provenance": {
"items": {
"$ref": "#/definitions/ProvenanceEvidence"
},
"type": "array"
},
"provenance_valid": {
"type": "boolean"
},
"schema_valid": {
"type": "boolean"
}
},
"required": [
"content_format",
"content_sha256",
"content_size_bytes",
"id",
"invariants",
"kind",
"media_type",
"preview",
"provenance",
"provenance_valid",
"schema_valid"
],
"type": "object"
},
"DependencyPolicy": {
"enum": [
"succeeded",
"terminal"
],
"type": "string"
},
"DimensionReport": {
"properties": {
"dimension": {
"$ref": "#/definitions/EvaluationDimension"
},
"passed": {
"type": [
"boolean",
"null"
]
},
"signals": true
},
"required": [
"dimension",
"signals"
],
"type": "object"
},
"DurableArchiveReference": {
"additionalProperties": false,
"properties": {
"archive_id": {
"type": "string"
},
"created_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"expires_at": {
"type": [
"string",
"null"
]
},
"identity_sha256": {
"type": "string"
},
"manifest": {
"$ref": "#/definitions/StorageObjectReference"
},
"manifest_backup": {
"$ref": "#/definitions/StorageObjectReference"
},
"retention_class": {
"$ref": "#/definitions/RetentionClass"
}
},
"required": [
"archive_id",
"created_at",
"execution_id",
"identity_sha256",
"manifest",
"manifest_backup",
"retention_class"
],
"type": "object"
},
"E2eManifest": {
"additionalProperties": false,
"properties": {
"control_plane": {
"$ref": "#/definitions/ControlPlaneEvidence"
},
"execution": {
"$ref": "#/definitions/ExecutionIdentity"
},
"observation_contract": {
"anyOf": [
{
"$ref": "#/definitions/ObservationRunContract"
},
{
"type": "null"
}
]
},
"subject": {
"$ref": "#/definitions/ModelArtifact"
},
"system_under_test": {
"$ref": "#/definitions/SystemUnderTestIdentity"
},
"worker_contracts": {
"items": {
"$ref": "#/definitions/ObservedWorkerContract"
},
"type": "array"
}
},
"required": [
"control_plane",
"execution",
"subject",
"system_under_test"
],
"type": "object"
},
"E2eObservationEnvelope": {
"additionalProperties": false,
"properties": {
"evidence": {
"$ref": "#/definitions/ObservationEvidence"
},
"identity": {
"$ref": "#/definitions/ObservationIdentity"
},
"mode": {
"$ref": "#/definitions/ObservationMode"
},
"outcome": {
"$ref": "#/definitions/ObservationOutcome"
},
"provenance": {
"$ref": "#/definitions/ObservationProvenance"
},
"samples": {
"items": {
"$ref": "#/definitions/ObservationSample"
},
"type": "array"
},
"schema": {
"type": "string"
}
},
"required": [
"evidence",
"identity",
"mode",
"outcome",
"provenance",
"samples",
"schema"
],
"type": "object"
},
"E2eReport": {
"additionalProperties": false,
"properties": {
"assessment_contract": {
"$ref": "#/definitions/AssessmentContract"
},
"engine_revision": {
"type": [
"string",
"null"
]
},
"execution": {
"$ref": "#/definitions/ExecutionIdentity"
},
"manifest": {
"anyOf": [
{
"$ref": "#/definitions/ArtifactReference"
},
{
"type": "null"
}
]
},
"objective_outcome": {
"$ref": "#/definitions/ObjectiveOutcome"
},
"observation_contract": {
"anyOf": [
{
"$ref": "#/definitions/ObservationRunContract"
},
{
"type": "null"
}
]
},
"passed": {
"type": "boolean"
},
"persistence_errors": {
"items": {
"type": "string"
},
"type": "array"
},
"redaction": {
"allOf": [
{
"$ref": "#/definitions/RedactionReport"
}
],
"default": {
"redacted_fields": 0,
"redacted_values": 0,
"rules": []
}
},
"report_state": {
"$ref": "#/definitions/ReportState"
},
"result_contract_sha256": {
"type": "string"
},
"scenarios": {
"items": {
"$ref": "#/definitions/E2eScenarioReport"
},
"type": "array"
},
"slot_start_deadline_seconds": {
"description": "Soft scheduling deadline: prevents starting slots after this interval.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"subject": {
"$ref": "#/definitions/ModelArtifact"
},
"system_under_test": {
"$ref": "#/definitions/SystemUnderTestIdentity"
}
},
"required": [
"assessment_contract",
"execution",
"objective_outcome",
"passed",
"report_state",
"result_contract_sha256",
"scenarios",
"subject",
"system_under_test"
],
"type": "object"
},
"E2eRunReport": {
"properties": {
"attempt_id": {
"type": "string"
},
"attempt_number": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"audit": {
"anyOf": [
{
"$ref": "#/definitions/AuditReport"
},
{
"type": "null"
}
],
"description": "Advisory behavioral audit over the captured transcript and metrics. Never contributes to score, status, gates, or longitudinal inputs."
},
"completion": {
"$ref": "#/definitions/CompletionState"
},
"cost": {
"$ref": "#/definitions/CostReport"
},
"criteria": {
"items": {
"$ref": "#/definitions/CriterionReport"
},
"type": "array"
},
"deliverables": {
"items": {
"$ref": "#/definitions/DeliverableReport"
},
"type": "array"
},
"dimensions": {
"items": {
"$ref": "#/definitions/DimensionReport"
},
"type": "array"
},
"efficiency": {
"anyOf": [
{
"$ref": "#/definitions/EfficiencyReport"
},
{
"type": "null"
}
]
},
"evaluators": {
"$ref": "#/definitions/EvaluatorStates"
},
"evidence": {
"items": {
"$ref": "#/definitions/ArtifactReference"
},
"type": "array"
},
"failures": {
"items": {
"$ref": "#/definitions/FailureRecord"
},
"type": "array"
},
"metrics": {
"anyOf": [
{
"$ref": "#/definitions/SessionMetricsPayload"
},
{
"type": "null"
}
]
},
"prompt": {
"type": "string"
},
"retry_attempts": {
"items": {
"$ref": "#/definitions/RetryAttemptReport"
},
"type": "array"
},
"run_id": {
"type": "string"
},
"scenario_flow": {
"anyOf": [
{
"$ref": "#/definitions/ScenarioFlowEvidence"
},
{
"type": "null"
}
],
"description": "Evidence-only flow identity emitted by Rust. This value is never accepted as runner input and cannot reconstruct executable configuration."
},
"scenario_measurements": {
"items": {
"$ref": "#/definitions/ScenarioMeasurement"
},
"type": "array"
},
"score": {
"format": "uint8",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"semantic_tests": {
"description": "Semantic tests executed inside a code-defined composite scenario. Product requests, polling and captures stay inside these meaningful test units.",
"items": {
"$ref": "#/definitions/WorkflowStepReport"
},
"type": "array"
},
"session_id": {
"type": "string"
},
"status": {
"$ref": "#/definitions/RunStatus"
},
"technical": {
"$ref": "#/definitions/TechnicalState"
},
"transcript": true,
"wall_time_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"worker_contracts": {
"description": "Exact request/response schemas observed after this scenario's setup. Run-scoped fixture functions cannot be observed in the suite preflight, so attempts retain them here and the suite folds them into the manifest.",
"items": {
"$ref": "#/definitions/ObservedWorkerContract"
},
"type": "array"
}
},
"required": [
"attempt_id",
"attempt_number",
"completion",
"cost",
"criteria",
"evaluators",
"prompt",
"run_id",
"session_id",
"status",
"technical",
"wall_time_ms"
],
"type": "object"
},
"E2eScenarioReport": {
"properties": {
"aggregate": {
"$ref": "#/definitions/ScenarioAggregate"
},
"behavior_sha256": {
"description": "Digest of the definition the case was materialized from. Absent only when no case could be materialized.",
"type": [
"string",
"null"
]
},
"case": {
"anyOf": [
{
"$ref": "#/definitions/ScenarioCase"
},
{
"type": "null"
}
]
},
"case_id": {
"default": "",
"type": "string"
},
"deferral_reason": {
"type": [
"string",
"null"
]
},
"execution_policy": {
"$ref": "#/definitions/ExecutionPolicy"
},
"passed": {
"type": "boolean"
},
"runs": {
"items": {
"$ref": "#/definitions/E2eRunReport"
},
"type": "array"
},
"scenario_id": {
"type": "string"
}
},
"required": [
"aggregate",
"execution_policy",
"passed",
"runs",
"scenario_id"
],
"type": "object"
},
"EfficiencyReport": {
"properties": {
"child_sessions": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"child_turns": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"critical_path_ms": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"effective_fan_out": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"function_call_errors": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"function_calls": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"input_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"observed_complexity": {
"$ref": "#/definitions/ObservedComplexityReport"
},
"observed_work": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"root_turns": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_attempts": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"total_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"transient_resumes": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"validation_retries": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wake_resumes": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wall_time_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"observed_complexity",
"technical_attempts",
"wall_time_ms"
],
"type": "object"
},
"EvaluationDimension": {
"enum": [
"deliverable",
"structural_integrity",
"efficiency",
"robustness",
"e2e_infrastructure"
],
"type": "string"
},
"EvaluatorAvailability": {
"enum": [
"not_required",
"pending",
"available",
"unavailable"
],
"type": "string"
},
"EvaluatorStates": {
"properties": {
"completion": {
"$ref": "#/definitions/EvaluatorAvailability"
}
},
"required": [
"completion"
],
"type": "object"
},
"EvidenceReference": {
"properties": {
"artifact_id": {
"type": "string"
},
"artifact_sha256": {
"type": "string"
},
"locator": {
"type": [
"string",
"null"
]
}
},
"required": [
"artifact_id",
"artifact_sha256"
],
"type": "object"
},
"ExecutionIdentity": {
"properties": {
"completed_at": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"lane": {
"type": "string"
},
"started_at": {
"type": "string"
}
},
"required": [
"completed_at",
"execution_id",
"lane",
"started_at"
],
"type": "object"
},
"ExecutionPhase": {
"enum": [
"requested",
"admitted",
"preflighting",
"materializing",
"setting_up",
"executing",
"collecting",
"evaluating",
"persisting",
"cleaning_up",
"finalizing",
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
},
"ExecutionPolicy": {
"properties": {
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_total_tokens": {
"description": "Shared Harness token budget. `None` leaves the Harness budget unbounded for this scenario.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_turns": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"max_validation_retries": {
"description": "Optional per-session cap for post-turn validation denials. `None` preserves the Harness-configured default.",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stuck_timeout_seconds": {
"description": "Stop only after this many seconds without observable useful progress. Large scenarios have no fixed wall-clock deadline.",
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"max_turns",
"stuck_timeout_seconds"
],
"type": "object"
},
"ExecutionRealism": {
"enum": [
"synthetic",
"realistic_simulator",
"frozen_real_artifact"
],
"type": "string"
},
"FailureDomain": {
"enum": [
"subject",
"resource",
"e2e_infrastructure"
],
"type": "string"
},
"FailurePhase": {
"enum": [
"setup",
"execute",
"collect",
"evaluate",
"cleanup"
],
"type": "string"
},
"FailureRecord": {
"properties": {
"code": {
"default": "",
"type": "string"
},
"contamination": {
"allOf": [
{
"$ref": "#/definitions/ContaminationScope"
}
],
"default": "none"
},
"domain": {
"$ref": "#/definitions/FailureDomain"
},
"message": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/FailurePhase"
},
"retry_scope": {
"allOf": [
{
"$ref": "#/definitions/RetryScope"
}
],
"default": "none"
},
"status": {
"allOf": [
{
"$ref": "#/definitions/RunStatus"
}
],
"description": "The failure's original classification, independent of the primary run status."
}
},
"required": [
"domain",
"message",
"phase",
"status"
],
"type": "object"
},
"FunctionContractEvidence": {
"properties": {
"function_id": {
"type": "string"
},
"request_schema": true,
"response_schema": true,
"sha256": {
"type": "string"
}
},
"required": [
"function_id",
"request_schema",
"response_schema",
"sha256"
],
"type": "object"
},
"HumanHorizon": {
"properties": {
"basis": {
"allOf": [
{
"$ref": "#/definitions/HumanHorizonBasis"
}
],
"default": "unknown"
},
"max_minutes": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"min_minutes": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"HumanHorizonBasis": {
"enum": [
"unknown",
"author_estimate",
"measured"
],
"type": "string"
},
"InvariantSpec": {
"properties": {
"description": {
"type": "string"
},
"id": {
"type": "string"
}
},
"required": [
"description",
"id"
],
"type": "object"
},
"ModelArtifact": {
"properties": {
"agent": {
"anyOf": [
{
"$ref": "#/definitions/AgentProfileArtifact"
},
{
"type": "null"
}
]
},
"context_window": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"model": {
"type": "string"
},
"provider": {
"type": "string"
},
"supports_tools": {
"type": [
"boolean",
"null"
]
},
"supports_vision": {
"type": [
"boolean",
"null"
]
}
},
"required": [
"context_window",
"max_output_tokens",
"model",
"provider"
],
"type": "object"
},
"ObjectiveOutcome": {
"enum": [
"passed",
"failed",
"inconclusive"
],
"type": "string"
},
"ObservationCorrelation": {
"additionalProperties": false,
"properties": {
"deployment_id": {
"type": "string"
},
"operation_id": {
"type": "string"
},
"system": {
"type": "string"
}
},
"required": [
"deployment_id",
"operation_id",
"system"
],
"type": "object"
},
"ObservationDataAvailability": {
"enum": [
"complete",
"partial",
"unavailable"
],
"type": "string"
},
"ObservationDecision": {
"enum": [
"observe_only"
],
"type": "string"
},
"ObservationEnvironment": {
"enum": [
"demonstration"
],
"type": "string"
},
"ObservationEvidence": {
"additionalProperties": false,
"properties": {
"artifacts": {
"items": {
"$ref": "#/definitions/ArtifactReference"
},
"type": "array"
},
"manifest_sha256": {
"type": [
"string",
"null"
]
},
"results_sha256": {
"type": [
"string",
"null"
]
}
},
"type": "object"
},
"ObservationExecutionIdentity": {
"additionalProperties": false,
"properties": {
"attempt": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"completed_at": {
"type": "string"
},
"id": {
"type": "string"
},
"request_sha256": {
"type": "string"
},
"run_contract_sha256": {
"type": "string"
},
"started_at": {
"type": "string"
}
},
"required": [
"attempt",
"completed_at",
"id",
"request_sha256",
"run_contract_sha256",
"started_at"
],
"type": "object"
},
"ObservationIdentity": {
"additionalProperties": false,
"properties": {
"execution": {
"$ref": "#/definitions/ObservationExecutionIdentity"
},
"plan": {
"$ref": "#/definitions/ObservationPlanIdentity"
},
"runner": {
"$ref": "#/definitions/RunnerIdentity"
},
"target": {
"$ref": "#/definitions/ObservationTargetIdentity"
}
},
"required": [
"execution",
"plan",
"runner",
"target"
],
"type": "object"
},
"ObservationMetric": {
"additionalProperties": false,
"description": "A metric is explicit about its unit and availability so a recorded zero cannot be mistaken for a missing measurement. The legacy `metrics` object remains alongside this additive projection for current consumers.",
"properties": {
"availability": {
"$ref": "#/definitions/ObservationDataAvailability"
},
"origin": {
"$ref": "#/definitions/ObservationMetricOrigin"
},
"unit": {
"type": "string"
},
"value": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"required": [
"availability",
"origin",
"unit"
],
"type": "object"
},
"ObservationMetricDerivation": {
"additionalProperties": false,
"properties": {
"formula": {
"type": "string"
},
"formula_version": {
"type": "string"
},
"metric": {
"type": "string"
},
"origin": {
"$ref": "#/definitions/ObservationMetricOrigin"
}
},
"required": [
"formula",
"formula_version",
"metric",
"origin"
],
"type": "object"
},
"ObservationMetricOrigin": {
"enum": [
"observed",
"derived_from_observed"
],
"type": "string"
},
"ObservationMode": {
"additionalProperties": false,
"properties": {
"decision": {
"$ref": "#/definitions/ObservationDecision"
},
"environment": {
"$ref": "#/definitions/ObservationEnvironment"
}
},
"required": [
"decision",
"environment"
],
"type": "object"
},
"ObservationObjective": {
"enum": [
"passed",
"hard_gate_failed",
"technical_failed",
"infrastructure_failed",
"cancelled",
"unsupported_plan"
],
"type": "string"
},
"ObservationOutcome": {
"additionalProperties": false,
"properties": {
"control_phase": {
"type": "string"
},
"data_availability": {
"$ref": "#/definitions/ObservationDataAvailability"
},
"error": {
"type": "string"
},
"objective": {
"$ref": "#/definitions/ObservationObjective"
},
"passed": {
"type": [
"boolean",
"null"
]
}
},
"required": [
"control_phase",
"data_availability",
"objective"
],
"type": "object"
},
"ObservationPlanIdentity": {
"additionalProperties": false,
"properties": {
"catalog_sha256": {
"type": "string"
},
"id": {
"type": "string"
},
"revision": {
"type": "string"
},
"sha256": {
"type": "string"
}
},
"required": [
"catalog_sha256",
"id",
"revision",
"sha256"
],
"type": "object"
},
"ObservationProvenance": {
"additionalProperties": false,
"properties": {
"subject_model": {
"type": "string"
},
"subject_provider": {
"type": "string"
},
"system_under_test": {
"anyOf": [
{
"$ref": "#/definitions/SystemUnderTestIdentity"
},
{
"type": "null"
}
]
}
},
"required": [
"subject_model",
"subject_provider"
],
"type": "object"
},
"ObservationRunContract": {
"additionalProperties": false,
"properties": {
"attempt": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"correlation": {
"$ref": "#/definitions/ObservationCorrelation"
},
"mode": {
"$ref": "#/definitions/ObservationMode"
},
"plan": {
"$ref": "#/definitions/ObservationPlanIdentity"
},
"runner": {
"$ref": "#/definitions/RunnerIdentity"
},
"selected_cases": {
"items": {
"$ref": "#/definitions/ObservationSelectedCase"
},
"type": "array"
},
"target": {
"$ref": "#/definitions/ObservationTargetIdentity"
}
},
"required": [
"attempt",
"correlation",
"mode",
"plan",
"runner",
"selected_cases",
"target"
],
"type": "object"
},
"ObservationSample": {
"additionalProperties": false,
"properties": {
"attempt_id": {
"type": "string"
},
"behavior_sha256": {
"type": [
"string",
"null"
]
},
"case_id": {
"type": "string"
},
"data_availability": {
"$ref": "#/definitions/ObservationDataAvailability"
},
"derivations": {
"items": {
"$ref": "#/definitions/ObservationMetricDerivation"
},
"type": "array"
},
"metric_values": {
"additionalProperties": {
"$ref": "#/definitions/ObservationMetric"
},
"default": {},
"type": "object"
},
"metrics": {
"anyOf": [
{
"$ref": "#/definitions/EfficiencyReport"
},
{
"type": "null"
}
]
},
"origin": {
"$ref": "#/definitions/ObservationMetricOrigin"
},
"run_id": {
"type": "string"
},
"scenario_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"status": {
"$ref": "#/definitions/RunStatus"
}
},
"required": [
"attempt_id",
"case_id",
"data_availability",
"origin",
"run_id",
"scenario_id",
"seed",
"status"
],
"type": "object"
},
"ObservationSelectedCase": {
"additionalProperties": false,
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"inputs_sha256": {
"type": "string"
},
"scenario_id": {
"$ref": "#/definitions/ScenarioId"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"contract_sha256",
"inputs_sha256",
"scenario_id",
"seed"
],
"type": "object"
},
"ObservationTargetIdentity": {
"additionalProperties": false,
"properties": {
"application": {
"type": "string"
},
"stack": {
"$ref": "#/definitions/StackIdentity"
},
"version": {
"type": "string"
}
},
"required": [
"application",
"stack",
"version"
],
"type": "object"
},
"ObservedComplexityReport": {
"properties": {
"ambiguity_level": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"artifact_count": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"coordination_edges": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"dependency_depth": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"external_systems": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"parallel_branches": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"planning_depth": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"state_transitions": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"validation_loops": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wake_cycles": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"ObservedWorkerContract": {
"additionalProperties": false,
"properties": {
"function_id": {
"type": "string"
},
"request_schema_sha256": {
"type": "string"
},
"response_schema_sha256": {
"type": "string"
}
},
"required": [
"function_id",
"request_schema_sha256",
"response_schema_sha256"
],
"type": "object"
},
"PortValueKind": {
"enum": [
"boolean",
"json",
"text_utf8",
"assessment"
],
"type": "string"
},
"ProvenanceEvidence": {
"properties": {
"kind": {
"type": "string"
},
"relation": {
"type": "string"
},
"source_id": {
"type": "string"
}
},
"required": [
"kind",
"relation",
"source_id"
],
"type": "object"
},
"RedactionReport": {
"properties": {
"redacted_fields": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"redacted_values": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"rules": {
"items": {
"type": "string"
},
"type": "array",
"uniqueItems": true
}
},
"required": [
"redacted_fields",
"redacted_values",
"rules"
],
"type": "object"
},
"ReportState": {
"enum": [
"complete",
"partial"
],
"type": "string"
},
"RetentionClass": {
"enum": [
"temporary",
"pull_request",
"longitudinal",
"canonical"
],
"type": "string"
},
"RetryAttemptReport": {
"properties": {
"attempt_id": {
"type": "string"
},
"attempt_number": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"completion": {
"$ref": "#/definitions/CompletionState"
},
"cost": {
"$ref": "#/definitions/CostReport"
},
"deliverables": {
"items": {
"$ref": "#/definitions/DeliverableReport"
},
"type": "array"
},
"dimensions": {
"items": {
"$ref": "#/definitions/DimensionReport"
},
"type": "array"
},
"efficiency": {
"anyOf": [
{
"$ref": "#/definitions/EfficiencyReport"
},
{
"type": "null"
}
]
},
"evaluators": {
"$ref": "#/definitions/EvaluatorStates"
},
"evidence": {
"items": {
"$ref": "#/definitions/ArtifactReference"
},
"type": "array"
},
"failures": {
"items": {
"$ref": "#/definitions/FailureRecord"
},
"type": "array"
},
"metrics": {
"anyOf": [
{
"$ref": "#/definitions/SessionMetricsPayload"
},
{
"type": "null"
}
]
},
"run_id": {
"type": "string"
},
"scenario_flow": {
"anyOf": [
{
"$ref": "#/definitions/ScenarioFlowEvidence"
},
{
"type": "null"
}
]
},
"score": {
"description": "The points the attempt's evaluated criteria awarded, as on the run.",
"format": "uint8",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"semantic_tests": {
"items": {
"$ref": "#/definitions/WorkflowStepReport"
},
"type": "array"
},
"session_id": {
"type": "string"
},
"status": {
"$ref": "#/definitions/RunStatus"
},
"technical": {
"$ref": "#/definitions/TechnicalState"
},
"transcript": true,
"wall_time_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"attempt_id",
"attempt_number",
"completion",
"cost",
"evaluators",
"failures",
"run_id",
"session_id",
"status",
"technical",
"wall_time_ms"
],
"type": "object"
},
"RetryScope": {
"enum": [
"none",
"same_slot"
],
"type": "string"
},
"RobustnessReport": {
"properties": {
"cost_per_successful_deliverable": {
"format": "double",
"type": [
"number",
"null"
]
},
"deliverable_success_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"eligible": {
"type": "boolean"
},
"flaky_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"median_wall_time_ms": {
"format": "double",
"type": [
"number",
"null"
]
},
"minimum_sample_size": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"p95_wall_time_ms": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"sample_size": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"structural_integrity_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"tail_minimum_sample_size": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"technical_failure_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"unavailable": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"wall_time_variance": {
"format": "double",
"type": [
"number",
"null"
]
}
},
"required": [
"eligible",
"minimum_sample_size",
"sample_size",
"tail_minimum_sample_size"
],
"type": "object"
},
"RunAssessmentContract": {
"properties": {
"assessments": {
"items": {
"$ref": "#/definitions/AssessmentResult"
},
"type": "array"
},
"assets": {
"items": {
"$ref": "#/definitions/AssetValidationResult"
},
"type": "array"
},
"attempt_id": {
"type": "string"
},
"run_id": {
"type": "string"
},
"system_status": {
"$ref": "#/definitions/SystemStatus"
}
},
"required": [
"attempt_id",
"run_id",
"system_status"
],
"type": "object"
},
"RunStatus": {
"enum": [
"passed",
"hard_gate_failed",
"subject_error",
"resource_limit",
"infrastructure_error"
],
"type": "string"
},
"RunnerIdentity": {
"additionalProperties": false,
"properties": {
"name": {
"type": "string"
},
"revision": {
"type": "string"
},
"version": {
"type": "string"
}
},
"required": [
"name",
"revision",
"version"
],
"type": "object"
},
"ScenarioAggregate": {
"properties": {
"completed_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"completion_evidence_coverage": {
"format": "double",
"type": [
"number",
"null"
]
},
"completion_rate": {
"format": "double",
"type": [
"number",
"null"
]
},
"cost": {
"$ref": "#/definitions/CostReport"
},
"deferred_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"execution_reliability": {
"format": "double",
"type": [
"number",
"null"
]
},
"failed_attempt_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"mean_score": {
"description": "Mean score of the technically valid runs that have one; null when none.",
"format": "double",
"type": [
"number",
"null"
]
},
"observed_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"pass_rate": {
"format": "double",
"type": "number"
},
"passed_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"planned_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"required_passes": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"robustness": {
"$ref": "#/definitions/RobustnessReport"
},
"scored_runs": {
"description": "Technically valid runs that have a score.",
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"task_incomplete_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"technical_failures": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"technical_invalid_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"technical_valid_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"tokens_completed_p50": {
"format": "double",
"type": [
"number",
"null"
]
},
"tokens_per_completion": {
"format": "double",
"type": [
"number",
"null"
]
},
"total_tokens_consumed": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"undetermined_runs": {
"format": "uint32",
"minimum": 0,
"type": "integer"
}
},
"required": [
"completed_runs",
"cost",
"deferred_runs",
"observed_runs",
"pass_rate",
"passed_runs",
"planned_runs",
"required_passes",
"robustness",
"scored_runs",
"task_incomplete_runs",
"technical_failures",
"technical_invalid_runs",
"technical_valid_runs",
"undetermined_runs"
],
"type": "object"
},
"ScenarioCase": {
"properties": {
"behavior_sha256": {
"description": "Digest of the scenario definition this case was materialized from: prompt, execution policy, criteria, and the case-independent contract. Two cases with the same digest were evaluated by the same definition. Sealed by `ScenarioId::materialize`; empty only before sealing.",
"type": "string"
},
"case_id": {
"type": "string"
},
"characterization": {
"allOf": [
{
"$ref": "#/definitions/ScenarioCharacterization"
}
],
"default": {
"human_horizon": {
"basis": "unknown"
},
"realism": {
"execution": "synthetic",
"shadow": "none"
}
}
},
"deliverable_contract": {
"$ref": "#/definitions/DeliverableContract"
},
"inputs": true,
"inputs_sha256": {
"type": "string"
},
"required_capabilities": {
"items": {
"$ref": "#/definitions/Capability"
},
"type": "array"
},
"scenario_id": {
"type": "string"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"deliverable_contract",
"inputs",
"inputs_sha256",
"required_capabilities",
"scenario_id",
"seed"
],
"type": "object"
},
"ScenarioCharacterization": {
"properties": {
"human_horizon": {
"allOf": [
{
"$ref": "#/definitions/HumanHorizon"
}
],
"default": {
"basis": "unknown"
}
},
"realism": {
"allOf": [
{
"$ref": "#/definitions/ScenarioRealism"
}
],
"default": {
"execution": "synthetic",
"shadow": "none"
}
}
},
"type": "object"
},
"ScenarioFlowEvidence": {
"additionalProperties": false,
"properties": {
"checkpoint": {
"$ref": "#/definitions/ArtifactReference"
},
"cleanup": {
"$ref": "#/definitions/WorkflowCleanupReport"
},
"definition_sha256": {
"type": "string"
},
"snapshot": true
},
"required": [
"checkpoint",
"cleanup",
"definition_sha256",
"snapshot"
],
"type": "object"
},
"ScenarioId": {
"enum": [
"registry_planning",
"registry_implementation",
"registry_environment",
"registry_verification",
"kanban_c1_foundation",
"kanban_c2_persistence",
"kanban_c3_board",
"kanban_c4_ticket_flow",
"kanban_c5_edit_move",
"kanban_c6_discussion",
"kanban_c7_live",
"linkly_tutorial",
"context_pressure",
"minimal_path",
"persistent_state",
"insert_record",
"sequential_pipeline",
"database_migration_recovery",
"shell_coder_sandbox",
"research_pipeline",
"fanout_ladder",
"security_review",
"incident_response",
"todo_worker_simple",
"todo_worker_planned",
"engineering_ticket",
"engineering_ticket_git_handoff",
"engineering_endurance_ladder",
"git_regression_forensics",
"mechanical_reaction",
"timer_wake",
"receiving_operation",
"validation_loop",
"subagent_validation",
"subagent_validation_failure",
"validation_self_repair",
"validation_scope_enforcement",
"validation_chain",
"secret_hygiene",
"prompt_injection_resilience",
"moving_target",
"poison_message",
"cleanup_under_failure",
"depth_ladder",
"quorum_fan_in",
"contention_ledger",
"wake_chain_soak",
"chess_engine_build",
"chess_play_ladder",
"trend_blog",
"trending_topics_build",
"typescript_chat_service",
"tool_contract_recovery",
"policy_bound_action",
"cross_app_transaction",
"performance_regression",
"browser_cross_site",
"release_train_recovery",
"cross_repo_contract_migration",
"swe_config_isolation",
"swe_cache_invalidation",
"swe_batch_replay",
"swe_replay_recovery",
"swe_contract_migration",
"swe_tenant_isolation",
"swe_replay_performance",
"swe_release_handoff",
"swe_service_journey"
],
"type": "string"
},
"ScenarioMeasurement": {
"additionalProperties": false,
"properties": {
"id": {
"type": "string"
},
"origin": {
"$ref": "#/definitions/ObservationMetricOrigin"
},
"unit": {
"type": "string"
},
"value": {
"format": "double",
"type": "number"
}
},
"required": [
"id",
"origin",
"unit",
"value"
],
"type": "object"
},
"ScenarioRealism": {
"properties": {
"execution": {
"allOf": [
{
"$ref": "#/definitions/ExecutionRealism"
}
],
"default": "synthetic"
},
"shadow": {
"allOf": [
{
"$ref": "#/definitions/ShadowMode"
}
],
"default": "none"
}
},
"type": "object"
},
"SessionMetricsPayload": {
"properties": {
"by_session": {
"items": {
"$ref": "#/definitions/SessionUsage"
},
"type": "array"
},
"complete": {
"type": "boolean"
},
"root_session_id": {
"type": "string"
},
"totals": {
"$ref": "#/definitions/SessionUsageTotals"
},
"traces": {
"default": null
}
},
"required": [
"by_session",
"complete",
"root_session_id",
"totals"
],
"type": "object"
},
"SessionUsage": {
"properties": {
"cache_read_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"context": {
"default": null
},
"cost_usd": {
"default": null,
"format": "double",
"type": [
"number",
"null"
]
},
"depth": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"function_call_errors": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"function_calls": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"input_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"parent_session_id": {
"default": null,
"type": [
"string",
"null"
]
},
"reasoning_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"session_id": {
"type": "string"
},
"transient_resumes": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"turns": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"validation_retries": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wake_resumes": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"depth",
"function_call_errors",
"function_calls",
"session_id",
"turns"
],
"type": "object"
},
"SessionUsageTotals": {
"properties": {
"cache_read_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cache_write_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"cost_usd": {
"default": null,
"format": "double",
"type": [
"number",
"null"
]
},
"function_call_errors": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"function_calls": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"input_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"output_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"reasoning_tokens": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"sessions": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"transient_resumes": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"turns": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"validation_retries": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"wake_resumes": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"required": [
"function_call_errors",
"function_calls",
"sessions",
"turns"
],
"type": "object"
},
"ShadowMode": {
"enum": [
"none",
"read_only"
],
"type": "string"
},
"StackIdentity": {
"oneOf": [
{
"properties": {
"mode": {
"enum": [
"source"
],
"type": "string"
},
"workers_repository": {
"type": "string"
},
"workers_revision": {
"type": "string"
}
},
"required": [
"mode",
"workers_repository",
"workers_revision"
],
"type": "object"
},
{
"properties": {
"mode": {
"enum": [
"registry"
],
"type": "string"
},
"stack_lock_digest": {
"type": "string"
},
"stack_versions": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"required": [
"mode",
"stack_lock_digest",
"stack_versions"
],
"type": "object"
}
]
},
"StorageObjectReference": {
"properties": {
"bucket": {
"type": "string"
},
"key": {
"type": "string"
},
"media_type": {
"type": "string"
},
"sha256": {
"type": "string"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"uri": {
"type": "string"
}
},
"required": [
"bucket",
"key",
"media_type",
"sha256",
"size_bytes",
"uri"
],
"type": "object"
},
"SystemStatus": {
"enum": [
"unavailable",
"passed",
"hard_gate_failed",
"subject_error",
"resource_limit",
"infrastructure_error"
],
"type": "string"
},
"SystemUnderTestIdentity": {
"properties": {
"contract_hashes": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"e2e_repository": {
"type": "string"
},
"e2e_revision": {
"type": "string"
},
"engine_revision": {
"type": [
"string",
"null"
]
},
"engine_version": {
"type": "string"
},
"harness_version": {
"type": "string"
},
"stack": {
"$ref": "#/definitions/StackIdentity"
}
},
"required": [
"contract_hashes",
"e2e_repository",
"e2e_revision",
"engine_version",
"harness_version",
"stack"
],
"type": "object"
},
"TechnicalState": {
"enum": [
"valid",
"technical_invalid"
],
"type": "string"
},
"TypedPortValue": {
"additionalProperties": false,
"properties": {
"kind": {
"$ref": "#/definitions/PortValueKind"
},
"value": true
},
"required": [
"kind",
"value"
],
"type": "object"
},
"WorkflowAssetReport": {
"additionalProperties": false,
"properties": {
"artifact": {
"$ref": "#/definitions/ArtifactReference"
},
"content_sha256": {
"type": "string"
},
"id": {
"type": "string"
},
"kind": {
"type": "string"
},
"media_type": {
"type": "string"
},
"namespaced_id": {
"type": "string"
},
"preview": true,
"preview_truncated": {
"type": "boolean"
},
"size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"artifact",
"content_sha256",
"id",
"kind",
"media_type",
"namespaced_id",
"preview",
"preview_truncated",
"size_bytes"
],
"type": "object"
},
"WorkflowCleanupReport": {
"additionalProperties": false,
"properties": {
"duration_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"failure": {
"type": [
"string",
"null"
]
},
"status": {
"$ref": "#/definitions/WorkflowCleanupStatus"
}
},
"required": [
"duration_ms",
"status"
],
"type": "object"
},
"WorkflowCleanupStatus": {
"enum": [
"succeeded",
"failed"
],
"type": "string"
},
"WorkflowEvaluationOutcome": {
"enum": [
"passed",
"failed",
"advisory",
"not_evaluated"
],
"type": "string"
},
"WorkflowEvaluationResult": {
"additionalProperties": false,
"properties": {
"evidence_ids": {
"default": [],
"items": {
"type": "string"
},
"type": "array"
},
"id": {
"type": "string"
},
"outcome": {
"$ref": "#/definitions/WorkflowEvaluationOutcome"
},
"score": {
"format": "double",
"type": [
"number",
"null"
]
},
"summary": {
"type": "string"
}
},
"required": [
"id",
"outcome",
"summary"
],
"type": "object"
},
"WorkflowFailurePhase": {
"enum": [
"preflight",
"execute",
"capture",
"persist",
"evaluate",
"cleanup",
"cancel"
],
"type": "string"
},
"WorkflowGateResult": {
"additionalProperties": false,
"properties": {
"evidence_ids": {
"default": [],
"items": {
"type": "string"
},
"type": "array"
},
"id": {
"type": "string"
},
"passed": {
"type": "boolean"
},
"reason": {
"type": "string"
}
},
"required": [
"id",
"passed",
"reason"
],
"type": "object"
},
"WorkflowStepFailure": {
"additionalProperties": false,
"properties": {
"message": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/WorkflowFailurePhase"
},
"technical": {
"type": "boolean"
}
},
"required": [
"message",
"phase",
"technical"
],
"type": "object"
},
"WorkflowStepReport": {
"additionalProperties": false,
"properties": {
"activation": {
"$ref": "#/definitions/ActivationPolicy"
},
"assets": {
"items": {
"$ref": "#/definitions/WorkflowAssetReport"
},
"type": "array"
},
"completed_at": {
"type": [
"string",
"null"
]
},
"cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"dependencies": {
"items": {
"type": "string"
},
"type": "array"
},
"dependency_policy": {
"$ref": "#/definitions/DependencyPolicy"
},
"duration_ms": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"evaluations": {
"items": {
"$ref": "#/definitions/WorkflowEvaluationResult"
},
"type": "array"
},
"failures": {
"items": {
"$ref": "#/definitions/WorkflowStepFailure"
},
"type": "array"
},
"hard_gates": {
"items": {
"$ref": "#/definitions/WorkflowGateResult"
},
"type": "array"
},
"harness_session_id": {
"type": [
"string",
"null"
]
},
"metrics": true,
"node_id": {
"type": "string"
},
"outputs": {
"additionalProperties": {
"$ref": "#/definitions/TypedPortValue"
},
"type": "object"
},
"redaction": {
"allOf": [
{
"$ref": "#/definitions/RedactionReport"
}
],
"default": {
"redacted_fields": 0,
"redacted_values": 0,
"rules": []
}
},
"required": {
"type": "boolean"
},
"skip_reason": {
"type": [
"string",
"null"
]
},
"started_at": {
"type": [
"string",
"null"
]
},
"status": {
"$ref": "#/definitions/WorkflowStepStatus"
},
"step_type": {
"type": "string"
},
"transcript": true
},
"required": [
"activation",
"dependencies",
"dependency_policy",
"duration_ms",
"node_id",
"required",
"status",
"step_type"
],
"type": "object"
},
"WorkflowStepStatus": {
"enum": [
"pending",
"running",
"succeeded",
"hard_gate_failed",
"failed",
"skipped",
"cancelled"
],
"type": "string"
}
},
"properties": {
"archive": {
"anyOf": [
{
"$ref": "#/definitions/DurableArchiveReference"
},
{
"type": "null"
}
]
},
"execution_id": {
"type": "string"
},
"manifest": {
"anyOf": [
{
"$ref": "#/definitions/E2eManifest"
},
{
"type": "null"
}
]
},
"observation": {
"anyOf": [
{
"$ref": "#/definitions/E2eObservationEnvelope"
},
{
"type": "null"
}
]
},
"observation_artifact": {
"anyOf": [
{
"$ref": "#/definitions/ArtifactReference"
},
{
"type": "null"
}
]
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
},
"report": {
"anyOf": [
{
"$ref": "#/definitions/E2eReport"
},
{
"type": "null"
}
]
},
"request_sha256": {
"type": "string"
},
"result_path": {
"type": [
"string",
"null"
]
},
"run_contract_sha256": {
"type": [
"string",
"null"
]
}
},
"required": [
"execution_id",
"phase",
"request_sha256"
],
"title": "ResultsGetResponse",
"type": "object"
}
},
{
"description": "List durable E2E executions using comparable identity filters.",
"metadata": {
"contract": {
"capabilities": [
"results-list"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::results-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ScenarioId": {
"enum": [
"registry_planning",
"registry_implementation",
"registry_environment",
"registry_verification",
"kanban_c1_foundation",
"kanban_c2_persistence",
"kanban_c3_board",
"kanban_c4_ticket_flow",
"kanban_c5_edit_move",
"kanban_c6_discussion",
"kanban_c7_live",
"linkly_tutorial",
"context_pressure",
"minimal_path",
"persistent_state",
"insert_record",
"sequential_pipeline",
"database_migration_recovery",
"shell_coder_sandbox",
"research_pipeline",
"fanout_ladder",
"security_review",
"incident_response",
"todo_worker_simple",
"todo_worker_planned",
"engineering_ticket",
"engineering_ticket_git_handoff",
"engineering_endurance_ladder",
"git_regression_forensics",
"mechanical_reaction",
"timer_wake",
"receiving_operation",
"validation_loop",
"subagent_validation",
"subagent_validation_failure",
"validation_self_repair",
"validation_scope_enforcement",
"validation_chain",
"secret_hygiene",
"prompt_injection_resilience",
"moving_target",
"poison_message",
"cleanup_under_failure",
"depth_ladder",
"quorum_fan_in",
"contention_ledger",
"wake_chain_soak",
"chess_engine_build",
"chess_play_ladder",
"trend_blog",
"trending_topics_build",
"typescript_chat_service",
"tool_contract_recovery",
"policy_bound_action",
"cross_app_transaction",
"performance_regression",
"browser_cross_site",
"release_train_recovery",
"cross_repo_contract_migration",
"swe_config_isolation",
"swe_cache_invalidation",
"swe_batch_replay",
"swe_replay_recovery",
"swe_contract_migration",
"swe_tenant_isolation",
"swe_replay_performance",
"swe_release_handoff",
"swe_service_journey"
],
"type": "string"
}
},
"properties": {
"lane": {
"default": null,
"type": [
"string",
"null"
]
},
"limit": {
"default": 50,
"format": "uint16",
"minimum": 0,
"type": "integer"
},
"scenario_id": {
"anyOf": [
{
"$ref": "#/definitions/ScenarioId"
},
{
"type": "null"
}
],
"default": null
}
},
"title": "ResultsListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ActiveAttempt": {
"properties": {
"attempt_id": {
"type": "string"
},
"resume_state_path": {
"type": [
"string",
"null"
]
},
"resume_state_sha256": {
"type": [
"string",
"null"
]
},
"run_id": {
"default": "",
"type": "string"
},
"scenario_id": {
"$ref": "#/definitions/ScenarioId"
},
"session_id": {
"type": "string"
}
},
"required": [
"attempt_id",
"scenario_id",
"session_id"
],
"type": "object"
},
"ExecutionPhase": {
"enum": [
"requested",
"admitted",
"preflighting",
"materializing",
"setting_up",
"executing",
"collecting",
"evaluating",
"persisting",
"cleaning_up",
"finalizing",
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
},
"JournalProgress": {
"properties": {
"active_attempt_id": {
"type": [
"string",
"null"
]
},
"attempts_finished": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"attempts_started": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"committed_events": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"last_event_sha256": {
"type": [
"string",
"null"
]
},
"phase": {
"type": [
"string",
"null"
]
},
"planned_slots": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"runs_committed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"slots_deferred": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"subject_observations_committed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"terminal": {
"type": "boolean"
},
"terminal_reason": {
"type": [
"string",
"null"
]
},
"terminal_state": {
"anyOf": [
{
"$ref": "#/definitions/JournalTerminalState"
},
{
"type": "null"
}
]
}
},
"required": [
"attempts_finished",
"attempts_started",
"committed_events",
"planned_slots",
"runs_committed",
"slots_deferred",
"subject_observations_committed",
"terminal"
],
"type": "object"
},
"JournalTerminalState": {
"enum": [
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
},
"PhaseTransition": {
"properties": {
"at": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
},
"reason": {
"type": "string"
}
},
"required": [
"at",
"phase",
"reason"
],
"type": "object"
},
"ScenarioId": {
"enum": [
"registry_planning",
"registry_implementation",
"registry_environment",
"registry_verification",
"kanban_c1_foundation",
"kanban_c2_persistence",
"kanban_c3_board",
"kanban_c4_ticket_flow",
"kanban_c5_edit_move",
"kanban_c6_discussion",
"kanban_c7_live",
"linkly_tutorial",
"context_pressure",
"minimal_path",
"persistent_state",
"insert_record",
"sequential_pipeline",
"database_migration_recovery",
"shell_coder_sandbox",
"research_pipeline",
"fanout_ladder",
"security_review",
"incident_response",
"todo_worker_simple",
"todo_worker_planned",
"engineering_ticket",
"engineering_ticket_git_handoff",
"engineering_endurance_ladder",
"git_regression_forensics",
"mechanical_reaction",
"timer_wake",
"receiving_operation",
"validation_loop",
"subagent_validation",
"subagent_validation_failure",
"validation_self_repair",
"validation_scope_enforcement",
"validation_chain",
"secret_hygiene",
"prompt_injection_resilience",
"moving_target",
"poison_message",
"cleanup_under_failure",
"depth_ladder",
"quorum_fan_in",
"contention_ledger",
"wake_chain_soak",
"chess_engine_build",
"chess_play_ladder",
"trend_blog",
"trending_topics_build",
"typescript_chat_service",
"tool_contract_recovery",
"policy_bound_action",
"cross_app_transaction",
"performance_regression",
"browser_cross_site",
"release_train_recovery",
"cross_repo_contract_migration",
"swe_config_isolation",
"swe_cache_invalidation",
"swe_batch_replay",
"swe_replay_recovery",
"swe_contract_migration",
"swe_tenant_isolation",
"swe_replay_performance",
"swe_release_handoff",
"swe_service_journey"
],
"type": "string"
},
"StatusResponse": {
"properties": {
"active_attempt": {
"anyOf": [
{
"$ref": "#/definitions/ActiveAttempt"
},
{
"type": "null"
}
]
},
"cancel_requested": {
"type": "boolean"
},
"error": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"journal_progress": {
"$ref": "#/definitions/JournalProgress"
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
},
"terminal": {
"type": "boolean"
},
"transitions": {
"items": {
"$ref": "#/definitions/PhaseTransition"
},
"type": "array"
},
"updated_at": {
"type": "string"
}
},
"required": [
"cancel_requested",
"execution_id",
"journal_progress",
"phase",
"terminal",
"transitions",
"updated_at"
],
"type": "object"
}
},
"properties": {
"executions": {
"items": {
"$ref": "#/definitions/StatusResponse"
},
"type": "array"
}
},
"required": [
"executions"
],
"title": "ResultsListResponse",
"type": "object"
}
},
{
"description": "Delete expired E2E objects and tombstone their history records.",
"metadata": {
"contract": {
"capabilities": [
"retention-sweep"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::retention-sweep",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"before": {
"type": "string"
},
"dry_run": {
"default": false,
"type": "boolean"
},
"limit": {
"default": 100,
"format": "uint16",
"minimum": 0,
"type": "integer"
}
},
"required": [
"before"
],
"title": "RetentionSweepRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"deleted_archives": {
"items": {
"type": "string"
},
"type": "array"
},
"dry_run": {
"type": "boolean"
},
"inspected": {
"format": "uint32",
"minimum": 0,
"type": "integer"
}
},
"required": [
"deleted_archives",
"dry_run",
"inspected"
],
"title": "RetentionSweepResponse",
"type": "object"
}
},
{
"description": "Admit an asynchronous, idempotent E2E execution.",
"metadata": {
"contract": {
"capabilities": [
"run"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::run",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"definitions": {
"ObservationCorrelation": {
"additionalProperties": false,
"properties": {
"deployment_id": {
"type": "string"
},
"operation_id": {
"type": "string"
},
"system": {
"type": "string"
}
},
"required": [
"deployment_id",
"operation_id",
"system"
],
"type": "object"
},
"ObservationDecision": {
"enum": [
"observe_only"
],
"type": "string"
},
"ObservationEnvironment": {
"enum": [
"demonstration"
],
"type": "string"
},
"ObservationMode": {
"additionalProperties": false,
"properties": {
"decision": {
"$ref": "#/definitions/ObservationDecision"
},
"environment": {
"$ref": "#/definitions/ObservationEnvironment"
}
},
"required": [
"decision",
"environment"
],
"type": "object"
},
"ObservationPlanIdentity": {
"additionalProperties": false,
"properties": {
"catalog_sha256": {
"type": "string"
},
"id": {
"type": "string"
},
"revision": {
"type": "string"
},
"sha256": {
"type": "string"
}
},
"required": [
"catalog_sha256",
"id",
"revision",
"sha256"
],
"type": "object"
},
"ObservationRunContract": {
"additionalProperties": false,
"properties": {
"attempt": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"correlation": {
"$ref": "#/definitions/ObservationCorrelation"
},
"mode": {
"$ref": "#/definitions/ObservationMode"
},
"plan": {
"$ref": "#/definitions/ObservationPlanIdentity"
},
"runner": {
"$ref": "#/definitions/RunnerIdentity"
},
"selected_cases": {
"items": {
"$ref": "#/definitions/ObservationSelectedCase"
},
"type": "array"
},
"target": {
"$ref": "#/definitions/ObservationTargetIdentity"
}
},
"required": [
"attempt",
"correlation",
"mode",
"plan",
"runner",
"selected_cases",
"target"
],
"type": "object"
},
"ObservationSelectedCase": {
"additionalProperties": false,
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"contract_sha256": {
"type": "string"
},
"inputs_sha256": {
"type": "string"
},
"scenario_id": {
"$ref": "#/definitions/ScenarioId"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"contract_sha256",
"inputs_sha256",
"scenario_id",
"seed"
],
"type": "object"
},
"ObservationTargetIdentity": {
"additionalProperties": false,
"properties": {
"application": {
"type": "string"
},
"stack": {
"$ref": "#/definitions/StackIdentity"
},
"version": {
"type": "string"
}
},
"required": [
"application",
"stack",
"version"
],
"type": "object"
},
"RunnerIdentity": {
"additionalProperties": false,
"properties": {
"name": {
"type": "string"
},
"revision": {
"type": "string"
},
"version": {
"type": "string"
}
},
"required": [
"name",
"revision",
"version"
],
"type": "object"
},
"ScenarioId": {
"enum": [
"registry_planning",
"registry_implementation",
"registry_environment",
"registry_verification",
"kanban_c1_foundation",
"kanban_c2_persistence",
"kanban_c3_board",
"kanban_c4_ticket_flow",
"kanban_c5_edit_move",
"kanban_c6_discussion",
"kanban_c7_live",
"linkly_tutorial",
"context_pressure",
"minimal_path",
"persistent_state",
"insert_record",
"sequential_pipeline",
"database_migration_recovery",
"shell_coder_sandbox",
"research_pipeline",
"fanout_ladder",
"security_review",
"incident_response",
"todo_worker_simple",
"todo_worker_planned",
"engineering_ticket",
"engineering_ticket_git_handoff",
"engineering_endurance_ladder",
"git_regression_forensics",
"mechanical_reaction",
"timer_wake",
"receiving_operation",
"validation_loop",
"subagent_validation",
"subagent_validation_failure",
"validation_self_repair",
"validation_scope_enforcement",
"validation_chain",
"secret_hygiene",
"prompt_injection_resilience",
"moving_target",
"poison_message",
"cleanup_under_failure",
"depth_ladder",
"quorum_fan_in",
"contention_ledger",
"wake_chain_soak",
"chess_engine_build",
"chess_play_ladder",
"trend_blog",
"trending_topics_build",
"typescript_chat_service",
"tool_contract_recovery",
"policy_bound_action",
"cross_app_transaction",
"performance_regression",
"browser_cross_site",
"release_train_recovery",
"cross_repo_contract_migration",
"swe_config_isolation",
"swe_cache_invalidation",
"swe_batch_replay",
"swe_replay_recovery",
"swe_contract_migration",
"swe_tenant_isolation",
"swe_replay_performance",
"swe_release_handoff",
"swe_service_journey"
],
"type": "string"
},
"StackIdentity": {
"oneOf": [
{
"properties": {
"mode": {
"enum": [
"source"
],
"type": "string"
},
"workers_repository": {
"type": "string"
},
"workers_revision": {
"type": "string"
}
},
"required": [
"mode",
"workers_repository",
"workers_revision"
],
"type": "object"
},
{
"properties": {
"mode": {
"enum": [
"registry"
],
"type": "string"
},
"stack_lock_digest": {
"type": "string"
},
"stack_versions": {
"additionalProperties": {
"type": "string"
},
"type": "object"
}
},
"required": [
"mode",
"stack_lock_digest",
"stack_versions"
],
"type": "object"
}
]
}
},
"properties": {
"agent": {
"type": [
"string",
"null"
]
},
"idempotency_key": {
"type": "string"
},
"label": {
"default": "",
"type": "string"
},
"lane": {
"default": "local",
"type": "string"
},
"model": {
"type": "string"
},
"progress_interval_seconds": {
"default": 15,
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"provider": {
"type": "string"
},
"rotating_seeds": {
"default": [],
"items": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"type": "array"
},
"run_contract": {
"anyOf": [
{
"$ref": "#/definitions/ObservationRunContract"
},
{
"type": "null"
}
]
},
"runs": {
"default": 1,
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"scenarios": {
"default": [],
"items": {
"$ref": "#/definitions/ScenarioId"
},
"type": "array"
},
"seed": {
"default": null,
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"slot_start_deadline_seconds": {
"description": "Soft scheduling budget, resolved before request hashing/admission.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"technical_retries": {
"default": 1,
"format": "uint8",
"minimum": 0,
"type": "integer"
}
},
"required": [
"idempotency_key",
"model",
"provider"
],
"title": "RunRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ExecutionPhase": {
"enum": [
"requested",
"admitted",
"preflighting",
"materializing",
"setting_up",
"executing",
"collecting",
"evaluating",
"persisting",
"cleaning_up",
"finalizing",
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
}
},
"properties": {
"duplicate": {
"type": "boolean"
},
"execution_id": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
},
"request_sha256": {
"type": "string"
},
"run_contract_sha256": {
"type": [
"string",
"null"
]
}
},
"required": [
"duplicate",
"execution_id",
"phase",
"request_sha256"
],
"title": "RunAccepted",
"type": "object"
}
},
{
"description": "List every built-in E2E test with its immutable contract, capabilities, complexity, and materialized case.",
"metadata": {
"agent": {
"audience": "harness",
"category": "e2e_local_test_authoring",
"operation": "list_tests",
"starts_execution": false,
"writes_local_state": false
},
"contract": {
"capabilities": [
"scenarios-list"
],
"name": "e2e-control-plane"
},
"internal": false
},
"name": "e2e::scenarios-list",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"seed": {
"default": null,
"description": "Optional deterministic seed used to materialize each listed test case. Omit it to use each scenario's canonical seed.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"title": "ScenariosListRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ArtifactExpectation": {
"properties": {
"id": {
"type": "string"
},
"kind": {
"type": "string"
},
"max_size_bytes": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"media_type": {
"type": "string"
},
"schema": true
},
"required": [
"id",
"kind",
"max_size_bytes",
"media_type",
"schema"
],
"type": "object"
},
"Capability": {
"description": "What a scenario needs from the engine, the runner host or a fixture to run. The closed list keeps the catalog honest: a new need is a new variant, and every id is spelled once.",
"enum": [
"browser::interactive",
"cross_repo_contract_simulator::v1",
"curl",
"docker",
"e2e::adaptive-flow-v1",
"e2e::control-plane-v1",
"e2e::filesystem",
"e2e::git",
"e2e::run-scoped-fixtures",
"e2e::shell",
"e2e::subagents",
"e2e::workflow-resume-v1",
"fixture::multi-origin-http",
"git",
"git::deterministic-fixture-v1",
"git::offline-bundle",
"github::security-read",
"github::trusted-handoff",
"harness::independent_session",
"harness::post-turn-validation",
"harness::scripted-dialogue-v1",
"iii::coder",
"iii::compose",
"iii::database",
"iii::functions",
"iii::registry",
"iii::shell",
"iii::state",
"iii::triggers",
"iii::workers",
"incident_fixture::v1",
"node",
"playwright",
"python3",
"release_shadow::read-only-v1",
"release_train_simulator::v1",
"security_scan::on-demand",
"security_scan::v1",
"swe::isolated-python-workspace"
],
"type": "string"
},
"DeliverableContract": {
"properties": {
"artifacts": {
"items": {
"$ref": "#/definitions/ArtifactExpectation"
},
"type": "array"
},
"capture_before_cleanup": {
"type": "boolean"
},
"invariants": {
"items": {
"$ref": "#/definitions/InvariantSpec"
},
"type": "array"
},
"provenance_required": {
"type": "boolean"
}
},
"required": [
"artifacts",
"capture_before_cleanup",
"invariants",
"provenance_required"
],
"type": "object"
},
"ExecutionPolicy": {
"properties": {
"max_output_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_total_tokens": {
"description": "Shared Harness token budget. `None` leaves the Harness budget unbounded for this scenario.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"max_turns": {
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"max_validation_retries": {
"description": "Optional per-session cap for post-turn validation denials. `None` preserves the Harness-configured default.",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"stuck_timeout_seconds": {
"description": "Stop only after this many seconds without observable useful progress. Large scenarios have no fixed wall-clock deadline.",
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"max_turns",
"stuck_timeout_seconds"
],
"type": "object"
},
"ExecutionRealism": {
"enum": [
"synthetic",
"realistic_simulator",
"frozen_real_artifact"
],
"type": "string"
},
"HumanHorizon": {
"properties": {
"basis": {
"allOf": [
{
"$ref": "#/definitions/HumanHorizonBasis"
}
],
"default": "unknown"
},
"max_minutes": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"min_minutes": {
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
}
},
"type": "object"
},
"HumanHorizonBasis": {
"enum": [
"unknown",
"author_estimate",
"measured"
],
"type": "string"
},
"InvariantSpec": {
"properties": {
"description": {
"type": "string"
},
"id": {
"type": "string"
}
},
"required": [
"description",
"id"
],
"type": "object"
},
"RunnerIdentity": {
"additionalProperties": false,
"properties": {
"name": {
"type": "string"
},
"revision": {
"type": "string"
},
"version": {
"type": "string"
}
},
"required": [
"name",
"revision",
"version"
],
"type": "object"
},
"ScenarioCharacterization": {
"properties": {
"human_horizon": {
"allOf": [
{
"$ref": "#/definitions/HumanHorizon"
}
],
"default": {
"basis": "unknown"
}
},
"realism": {
"allOf": [
{
"$ref": "#/definitions/ScenarioRealism"
}
],
"default": {
"execution": "synthetic",
"shadow": "none"
}
}
},
"type": "object"
},
"ScenarioDescriptor": {
"properties": {
"behavior_sha256": {
"type": "string"
},
"case_id": {
"type": "string"
},
"characterization": {
"$ref": "#/definitions/ScenarioCharacterization"
},
"contract_sha256": {
"type": "string"
},
"deliverable_contract": {
"$ref": "#/definitions/DeliverableContract"
},
"inputs_sha256": {
"type": "string"
},
"required_capabilities": {
"items": {
"$ref": "#/definitions/Capability"
},
"type": "array"
},
"resource_envelope": {
"$ref": "#/definitions/ScenarioResourceEnvelope"
},
"scenario_id": {
"$ref": "#/definitions/ScenarioId"
},
"seed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"behavior_sha256",
"case_id",
"characterization",
"contract_sha256",
"deliverable_contract",
"inputs_sha256",
"required_capabilities",
"resource_envelope",
"scenario_id",
"seed"
],
"type": "object"
},
"ScenarioId": {
"enum": [
"registry_planning",
"registry_implementation",
"registry_environment",
"registry_verification",
"kanban_c1_foundation",
"kanban_c2_persistence",
"kanban_c3_board",
"kanban_c4_ticket_flow",
"kanban_c5_edit_move",
"kanban_c6_discussion",
"kanban_c7_live",
"linkly_tutorial",
"context_pressure",
"minimal_path",
"persistent_state",
"insert_record",
"sequential_pipeline",
"database_migration_recovery",
"shell_coder_sandbox",
"research_pipeline",
"fanout_ladder",
"security_review",
"incident_response",
"todo_worker_simple",
"todo_worker_planned",
"engineering_ticket",
"engineering_ticket_git_handoff",
"engineering_endurance_ladder",
"git_regression_forensics",
"mechanical_reaction",
"timer_wake",
"receiving_operation",
"validation_loop",
"subagent_validation",
"subagent_validation_failure",
"validation_self_repair",
"validation_scope_enforcement",
"validation_chain",
"secret_hygiene",
"prompt_injection_resilience",
"moving_target",
"poison_message",
"cleanup_under_failure",
"depth_ladder",
"quorum_fan_in",
"contention_ledger",
"wake_chain_soak",
"chess_engine_build",
"chess_play_ladder",
"trend_blog",
"trending_topics_build",
"typescript_chat_service",
"tool_contract_recovery",
"policy_bound_action",
"cross_app_transaction",
"performance_regression",
"browser_cross_site",
"release_train_recovery",
"cross_repo_contract_migration",
"swe_config_isolation",
"swe_cache_invalidation",
"swe_batch_replay",
"swe_replay_recovery",
"swe_contract_migration",
"swe_tenant_isolation",
"swe_replay_performance",
"swe_release_handoff",
"swe_service_journey"
],
"type": "string"
},
"ScenarioRealism": {
"properties": {
"execution": {
"allOf": [
{
"$ref": "#/definitions/ExecutionRealism"
}
],
"default": "synthetic"
},
"shadow": {
"allOf": [
{
"$ref": "#/definitions/ShadowMode"
}
],
"default": "none"
}
},
"type": "object"
},
"ScenarioResourceEnvelope": {
"properties": {
"execution": {
"$ref": "#/definitions/ExecutionPolicy"
},
"workflow": {
"anyOf": [
{
"$ref": "#/definitions/WorkflowResourceEnvelope"
},
{
"type": "null"
}
]
}
},
"required": [
"execution"
],
"type": "object"
},
"ShadowMode": {
"enum": [
"none",
"read_only"
],
"type": "string"
},
"WorkflowResourceEnvelope": {
"properties": {
"max_cost_usd": {
"format": "double",
"type": [
"number",
"null"
]
},
"max_nodes": {
"format": "uint16",
"minimum": 0,
"type": "integer"
},
"max_parallel": {
"format": "uint16",
"minimum": 0,
"type": "integer"
},
"max_total_tokens": {
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"step_timeout_seconds": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"technical_retries": {
"format": "uint8",
"minimum": 0,
"type": "integer"
},
"workflow_timeout_seconds": {
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"max_nodes",
"max_parallel",
"step_timeout_seconds",
"technical_retries",
"workflow_timeout_seconds"
],
"type": "object"
}
},
"properties": {
"catalog_sha256": {
"type": "string"
},
"runner": {
"$ref": "#/definitions/RunnerIdentity"
},
"scenarios": {
"items": {
"$ref": "#/definitions/ScenarioDescriptor"
},
"type": "array"
},
"schema": {
"type": "string"
}
},
"required": [
"catalog_sha256",
"runner",
"scenarios",
"schema"
],
"title": "ScenariosListResponse",
"type": "object"
}
},
{
"description": "Read the durable phase and progress of an E2E execution.",
"metadata": {
"contract": {
"capabilities": [
"status"
],
"name": "e2e-control-plane"
},
"internal": true
},
"name": "e2e::status",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"execution_id": {
"type": "string"
}
},
"required": [
"execution_id"
],
"title": "ExecutionRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ActiveAttempt": {
"properties": {
"attempt_id": {
"type": "string"
},
"resume_state_path": {
"type": [
"string",
"null"
]
},
"resume_state_sha256": {
"type": [
"string",
"null"
]
},
"run_id": {
"default": "",
"type": "string"
},
"scenario_id": {
"$ref": "#/definitions/ScenarioId"
},
"session_id": {
"type": "string"
}
},
"required": [
"attempt_id",
"scenario_id",
"session_id"
],
"type": "object"
},
"ExecutionPhase": {
"enum": [
"requested",
"admitted",
"preflighting",
"materializing",
"setting_up",
"executing",
"collecting",
"evaluating",
"persisting",
"cleaning_up",
"finalizing",
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
},
"JournalProgress": {
"properties": {
"active_attempt_id": {
"type": [
"string",
"null"
]
},
"attempts_finished": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"attempts_started": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"committed_events": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"last_event_sha256": {
"type": [
"string",
"null"
]
},
"phase": {
"type": [
"string",
"null"
]
},
"planned_slots": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"runs_committed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"slots_deferred": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"subject_observations_committed": {
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"terminal": {
"type": "boolean"
},
"terminal_reason": {
"type": [
"string",
"null"
]
},
"terminal_state": {
"anyOf": [
{
"$ref": "#/definitions/JournalTerminalState"
},
{
"type": "null"
}
]
}
},
"required": [
"attempts_finished",
"attempts_started",
"committed_events",
"planned_slots",
"runs_committed",
"slots_deferred",
"subject_observations_committed",
"terminal"
],
"type": "object"
},
"JournalTerminalState": {
"enum": [
"completed",
"failed",
"cancelled",
"needs_reconciliation",
"unsupported"
],
"type": "string"
},
"PhaseTransition": {
"properties": {
"at": {
"type": "string"
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
},
"reason": {
"type": "string"
}
},
"required": [
"at",
"phase",
"reason"
],
"type": "object"
},
"ScenarioId": {
"enum": [
"registry_planning",
"registry_implementation",
"registry_environment",
"registry_verification",
"kanban_c1_foundation",
"kanban_c2_persistence",
"kanban_c3_board",
"kanban_c4_ticket_flow",
"kanban_c5_edit_move",
"kanban_c6_discussion",
"kanban_c7_live",
"linkly_tutorial",
"context_pressure",
"minimal_path",
"persistent_state",
"insert_record",
"sequential_pipeline",
"database_migration_recovery",
"shell_coder_sandbox",
"research_pipeline",
"fanout_ladder",
"security_review",
"incident_response",
"todo_worker_simple",
"todo_worker_planned",
"engineering_ticket",
"engineering_ticket_git_handoff",
"engineering_endurance_ladder",
"git_regression_forensics",
"mechanical_reaction",
"timer_wake",
"receiving_operation",
"validation_loop",
"subagent_validation",
"subagent_validation_failure",
"validation_self_repair",
"validation_scope_enforcement",
"validation_chain",
"secret_hygiene",
"prompt_injection_resilience",
"moving_target",
"poison_message",
"cleanup_under_failure",
"depth_ladder",
"quorum_fan_in",
"contention_ledger",
"wake_chain_soak",
"chess_engine_build",
"chess_play_ladder",
"trend_blog",
"trending_topics_build",
"typescript_chat_service",
"tool_contract_recovery",
"policy_bound_action",
"cross_app_transaction",
"performance_regression",
"browser_cross_site",
"release_train_recovery",
"cross_repo_contract_migration",
"swe_config_isolation",
"swe_cache_invalidation",
"swe_batch_replay",
"swe_replay_recovery",
"swe_contract_migration",
"swe_tenant_isolation",
"swe_replay_performance",
"swe_release_handoff",
"swe_service_journey"
],
"type": "string"
}
},
"properties": {
"active_attempt": {
"anyOf": [
{
"$ref": "#/definitions/ActiveAttempt"
},
{
"type": "null"
}
]
},
"cancel_requested": {
"type": "boolean"
},
"error": {
"type": "string"
},
"execution_id": {
"type": "string"
},
"journal_progress": {
"$ref": "#/definitions/JournalProgress"
},
"phase": {
"$ref": "#/definitions/ExecutionPhase"
},
"terminal": {
"type": "boolean"
},
"transitions": {
"items": {
"$ref": "#/definitions/PhaseTransition"
},
"type": "array"
},
"updated_at": {
"type": "string"
}
},
"required": [
"cancel_requested",
"execution_id",
"journal_progress",
"phase",
"terminal",
"transitions",
"updated_at"
],
"title": "StatusResponse",
"type": "object"
}
},
{
"description": "Serve the Harness E2E injectable Console assets.",
"metadata": {
"internal": true
},
"name": "e2e::ui-content",
"request_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"path": {
"type": "string"
}
},
"required": [
"path"
],
"title": "UiContentRequest",
"type": "object"
},
"response_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"content": {
"type": "string"
},
"content_type": {
"type": "string"
}
},
"required": [
"content",
"content_type"
],
"title": "UiContentResponse",
"type": "object"
}
}
],
"triggers": [
{
"description": "A local E2E execution started, produced progress, or reached a terminal state.",
"invocation_schema": {
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": false,
"title": "EventBindingConfig",
"type": "object"
},
"metadata": {},
"name": "e2e::dashboard::changed",
"return_schema": {}
}
]
}