Persist a completed execution through private iii storage and idempotent history.
execution_idstring*required
retention_classstring*requiredenum: temporary, pull_request, longitudinal, canonical
archiveobject*required
archive_idstring*required
created_atstring*required
execution_idstring*required
identity_sha256string*required
manifestobject*required
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
manifest_backupobject*required
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
retention_classstring*requiredenum: temporary, pull_request, longitudinal, canonical
duplicate_ingestionboolean*required
historyobject*required
archiveobject*required
archive_idstring*required
created_atstring*required
execution_idstring*required
identity_sha256string*required
manifestobject*required
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
manifest_backupobject*required
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
retention_classstring*requiredenum: temporary, pull_request, longitudinal, canonical
baseline_comparableboolean*required
case_countinteger· uint32*requiredmin 0
e2e_repositorystring*required
e2e_revisionstring*required
execution_idstring*required
identity_sha256string*required
ingestion_idstring*required
occurred_atstring*required
result_contract_sha256string*required
scoring_profile_sha256string*required
stack_modestring*required
subject_modelstring*required
subject_providerstring*required
subject_revisionstring*required
e2e::archive-head
functionVerify availability and immutable metadata for an archived execution.
execution_idstring*required
archiveobject*required
archive_idstring*required
created_atstring*required
execution_idstring*required
identity_sha256string*required
manifestobject*required
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
manifest_backupobject*required
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
retention_classstring*requiredenum: temporary, pull_request, longitudinal, canonical
availabilitystring*requiredenum: available, expired
verified_atstring*required
e2e::archive-restore
functionRestore and verify an archived execution from its immutable manifest.
execution_idstring*required
archiveobject*required
archive_idstring*required
created_atstring*required
execution_idstring*required
identity_sha256string*required
manifestobject*required
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
manifest_backupobject*required
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
retention_classstring*requiredenum: temporary, pull_request, longitudinal, canonical
availabilitystring*requiredenum: available, expired
restored_filesinteger· uint32*requiredmin 0
restored_from_backupboolean*required
Request idempotent cancellation and cleanup of an E2E execution.
execution_idstring*required
execution_idstring*required
phasestring*requiredenum: requested, admitted, preflighting, materializing, setting_up, executing, …
Compare two completed executions when their identities are eligible.
baselinestring
Reviewed baseline file to load thresholds from. Omit to use the checked-in `config/baselines/default.json` when it exists, falling back to the code-default thresholds. Ignored when `policy` is set.
from_execution_idstring*required
policyany of
Explicit policy override. Omit to gate on the reviewed baseline.
any of (2)
variant 1
baseline_idstring
Identifier of the reviewed baseline these thresholds came from, when one was loaded. Absent when the code-default thresholds are in effect.
regressionall of
… expand 1 nestedcollapse
all of (1)
variant 1
cost_increase_rationumber· double*required
deliverable_success_dropnumber· double*required
p95_turns_increase_rationumber· double*required
structural_integrity_dropnumber· double*required
technical_failure_increasenumber· double*required
wall_time_increase_rationumber· double*required
scenario_overridesobject
Per-scenario partial threshold overrides carried from the reviewed baseline.
to_execution_idstring*required
artifactsobject[]*required
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
comparisonobject*required
baseline_idstring
Reviewed baseline the regression thresholds came from; absent when the code-default thresholds gated this comparison.
casesobject[]*required
complexity_tierstring*requiredenum: l0_atomic, l1_sequential, l2_stateful, l3_concurrent, l4_coordinated, l5_adaptive
deltaobject*required
… expand 16 nestedcollapse
consumptionobject
Descriptive only; these deltas never create a regression gate.
deliverable_success_rateany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
flaky_rateany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
median_scoreany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
p50_cost_usdany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
p50_turnsany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
p50_wall_time_msany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
p50_work_amplificationany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
p95_cost_usdany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
p95_turnsany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
p95_wall_time_msany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
p95_work_amplificationany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
retry_rateany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
structural_integrity_rateany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
technical_failure_rateany of
… expand 2 nestedcollapse
any of (2)
variant 1
absolutenumber· double*required
relative_rationumber· double
fromobject*required
… expand 18 nestedcollapse
consumptionall of
Includes every observed run, even those excluded from quality comparison.
… expand 1 nestedcollapse
all of (1)
variant 1
completed_runsinteger· uint32*requiredmin 0
observed_runsinteger· uint32*requiredmin 0
p50_function_callsnumber· double
p50_total_tokensnumber· double
p95_function_callsnumber· double
p95_total_tokensnumber· double
tokens_per_completionnumber· double
tokens_per_verified_successnumber· double
total_tokens_consumedinteger· uint64min 0
unavailableobject*required
verified_successesinteger· uint32*requiredmin 0
deliverable_successany of
… expand 2 nestedcollapse
any of (2)
variant 1
ci95_lowernumber· double*required
ci95_uppernumber· double*required
ratenumber· double*required
sample_sizeinteger· uint32*requiredmin 0
successesinteger· uint32*requiredmin 0
excluded_infrastructure_runsinteger· uint32*requiredmin 0
included_runsinteger· uint32*requiredmin 0
median_scorenumber· double
p50_cost_usdnumber· double
p50_wall_time_msnumber· double
p50_work_amplificationnumber· double
p95_cost_usdnumber· double
p95_wall_time_msnumber· double
p95_work_amplificationnumber· double
structural_integrityany of
… expand 2 nestedcollapse
any of (2)
variant 1
ci95_lowernumber· double*required
ci95_uppernumber· double*required
ratenumber· double*required
sample_sizeinteger· uint32*requiredmin 0
successesinteger· uint32*requiredmin 0
technical_failureany of
… expand 2 nestedcollapse
any of (2)
variant 1
ci95_lowernumber· double*required
ci95_uppernumber· double*required
ratenumber· double*required
sample_sizeinteger· uint32*requiredmin 0
successesinteger· uint32*requiredmin 0
from_run_idsstring[]*required
keyobject*required
… expand 7 nestedcollapse
contract_sha256string*required
executionobject*required
… expand 8 nestedcollapse
stack_modestring*required
subject_modelstring*required
subject_providerstring*required
inputs_sha256string*required
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
regressionsobject[]*required
… expand 11 nestedcollapse
dimensionstring*requiredenum: deliverable, structural_integrity, technical_reliability, cost, wall_time, turns
n_frominteger· uint32min 0
observed_deltanumber· double*required
scenario_idstring*required
thresholdnumber· double*required
toobject*required
… expand 18 nestedcollapse
consumptionall of
Includes every observed run, even those excluded from quality comparison.
… expand 1 nestedcollapse
all of (1)
variant 1
completed_runsinteger· uint32*requiredmin 0
observed_runsinteger· uint32*requiredmin 0
p50_function_callsnumber· double
p50_total_tokensnumber· double
p95_function_callsnumber· double
p95_total_tokensnumber· double
tokens_per_completionnumber· double
tokens_per_verified_successnumber· double
total_tokens_consumedinteger· uint64min 0
unavailableobject*required
verified_successesinteger· uint32*requiredmin 0
deliverable_successany of
… expand 2 nestedcollapse
any of (2)
variant 1
ci95_lowernumber· double*required
ci95_uppernumber· double*required
ratenumber· double*required
sample_sizeinteger· uint32*requiredmin 0
successesinteger· uint32*requiredmin 0
excluded_infrastructure_runsinteger· uint32*requiredmin 0
included_runsinteger· uint32*requiredmin 0
median_scorenumber· double
p50_cost_usdnumber· double
p50_wall_time_msnumber· double
p50_work_amplificationnumber· double
p95_cost_usdnumber· double
p95_wall_time_msnumber· double
p95_work_amplificationnumber· double
structural_integrityany of
… expand 2 nestedcollapse
any of (2)
variant 1
ci95_lowernumber· double*required
ci95_uppernumber· double*required
ratenumber· double*required
sample_sizeinteger· uint32*requiredmin 0
successesinteger· uint32*requiredmin 0
technical_failureany of
… expand 2 nestedcollapse
any of (2)
variant 1
ci95_lowernumber· double*required
ci95_uppernumber· double*required
ratenumber· double*required
sample_sizeinteger· uint32*requiredmin 0
successesinteger· uint32*requiredmin 0
to_run_idsstring[]*required
cohortobject*required
excluded_casesobject[]*required
… expand 4 nestedcollapse
scenario_idstring*required
sidestring*requiredenum: from, to
excluded_runsobject[]*required
… expand 6 nestedcollapse
attempt_idstring*required
scenario_idstring*required
sidestring*requiredenum: from, to
included_case_idsstring[]*required
comparableboolean*required
comparison_idstring*required
from_execution_idstring*required
gate_passedboolean*required
regressionsobject[]*required
dimensionstring*requiredenum: deliverable, structural_integrity, technical_reliability, cost, wall_time, turns
n_frominteger· uint32min 0
observed_deltanumber· double*required
scenario_idstring*required
thresholdnumber· double*required
to_execution_idstring*required
Read models and scenarios when the execution dialog opens.
local_scenariosobject[]*required
source_pathstring*required
source_sha256string*required
versioninteger· uint32*requiredmin 0
scenariosstring[]*required
List immutable evaluated-system versions and their exact evaluation cohorts.
cohortsobject[]*required
subject_modelstring*required
subject_providerstring*required
versionsobject[]*required
completed_atstring*required
execution_countinteger· uint*requiredmin 0
stack_modestring*required
Read one retained execution summary and its diagnostic report.
execution_idstring*required
manifestobject*required
executionsunknown[]*required
last_updatestring*required
retentionobject*required
detailsinteger· uint*requiredmin 0
summariesinteger· uint*requiredmin 0
totalinteger· uint*requiredmin 0
List compact retained execution summaries with server-side filters and cursor pagination.
limitinteger· uint16min 0
executionsunknown[]*required
last_updatestring*required
retentionobject*required
detailsinteger· uint*requiredmin 0
summariesinteger· uint*requiredmin 0
totalinteger· uint*requiredmin 0
Validate and save one local-only Markdown scenario, then return its compiled identity.
file_namestring*required
One UTF-8 `.md` file name, without a directory. Use letters, numbers, spaces, hyphens, or underscores; its stem becomes `local_<safe_id>`. Existing files are rejected instead of overwritten.
sourcestring*required
Complete Markdown test definition. It must contain one H1 followed by the required H2 sections in order and contain positive `### Name (N%)` validations totaling exactly 100%.
scenarioobject*required
author_versioninteger· uint32min 0
characterizationobject*required
human_horizonall of
… expand 1 nestedcollapse
all of (1)
variant 1
basisall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: unknown, author_estimate, measured
max_minutesinteger· uint32min 0
min_minutesinteger· uint32min 0
realismall of
… expand 1 nestedcollapse
all of (1)
variant 1
executionall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: synthetic, realistic_simulator, frozen_real_artifact
shadowall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: none, read_only
classificationobject*required
methodstring*requiredenum: capability_v2
profileobject*required
… expand 16 nestedcollapse
agent_owned_decompositionboolean
ambiguity_levelinteger· uint8*requiredmin 0
artifact_countinteger· uint8*requiredmin 0
coherent_long_horizonboolean
compensable_mutationsinteger· uint8min 0
coordination_edgesinteger· uint16*requiredmin 0
dependency_depthinteger· uint8*requiredmin 0
durable_resume_cyclesinteger· uint8min 0
external_systemsinteger· uint8*requiredmin 0
material_invalidation_eventsinteger· uint8min 0
parallel_branchesinteger· uint8*requiredmin 0
planning_depthinteger· uint8*requiredmin 0
replan_loopsinteger· uint8min 0
state_transitionsinteger· uint16*requiredmin 0
validation_loopsinteger· uint8*requiredmin 0
wake_cyclesinteger· uint8*requiredmin 0
tierstring*requiredenum: l0_atomic, l1_sequential, l2_stateful, l3_concurrent, l4_coordinated, l5_adaptive
complexityobject*required
agent_owned_decompositionboolean
ambiguity_levelinteger· uint8*requiredmin 0
artifact_countinteger· uint8*requiredmin 0
coherent_long_horizonboolean
compensable_mutationsinteger· uint8min 0
coordination_edgesinteger· uint16*requiredmin 0
dependency_depthinteger· uint8*requiredmin 0
durable_resume_cyclesinteger· uint8min 0
external_systemsinteger· uint8*requiredmin 0
material_invalidation_eventsinteger· uint8min 0
parallel_branchesinteger· uint8*requiredmin 0
planning_depthinteger· uint8*requiredmin 0
replan_loopsinteger· uint8min 0
state_transitionsinteger· uint16*requiredmin 0
validation_loopsinteger· uint8*requiredmin 0
wake_cyclesinteger· uint8*requiredmin 0
contract_sha256string*required
deliverable_contractobject*required
artifactsobject[]*required
… expand 5 nestedcollapse
max_size_bytesinteger· uint64*requiredmin 0
media_typestring*required
capture_before_cleanupboolean*required
invariantsobject[]*required
… expand 2 nestedcollapse
descriptionstring*required
provenance_requiredboolean*required
inputs_sha256string*required
originstring*requiredenum: built_in, markdown, local
required_capabilitiesstring[]*required
resource_envelopeobject*required
executionobject*required
… expand 5 nestedcollapse
max_output_tokensinteger· uint64min 0
max_total_tokensinteger· uint64min 0
Shared Harness token budget. `None` leaves the Harness budget unbounded for this scenario.
max_turnsinteger· uint32*requiredmin 0
max_validation_retriesinteger· uint32min 0
Optional per-session cap for post-turn validation denials. `None` preserves the Harness-configured default.
stuck_timeout_secondsinteger· uint64*requiredmin 0
Stop only after this many seconds without observable useful progress. Large scenarios have no fixed wall-clock deadline.
workflowany of
… expand 2 nestedcollapse
any of (2)
variant 1
max_cost_usdnumber· double
max_nodesinteger· uint16*requiredmin 0
max_parallelinteger· uint16*requiredmin 0
max_total_tokensinteger· uint64min 0
step_timeout_secondsinteger· uint64*requiredmin 0
technical_retriesinteger· uint8*requiredmin 0
workflow_timeout_secondsinteger· uint64*requiredmin 0
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
Configure, export and execute saved plans, and inspect or cancel their composed executions.
one of (4)
variant 1
actionstring*requiredenum: requirements
variant 2
actionstring*requiredenum: export
variant 3
actionstring*requiredenum: execution
execution_idstring*required
variant 4
actionstring*requiredenum: cancel
execution_idstring*required
Create a draft local plan with an explicit small test scope.
scenariosstring[]*required
technical_retriesinteger· uint8min 0
baseline_execution_idstring
candidate_execution_idsstring[]*required
candidate_labelsobject*required
compatibleboolean*required
created_atstring*required
incomplete_execution_idsstring[]*required
judge_modelstring*required
judge_providerstring*required
protected_executor_requiredboolean*required
runsinteger· uint32*requiredmin 0
scenario_idsstring[]*required
scenariosobject[]*required
complexity_tierstring*required
contract_sha256string*required
inputs_sha256string*required
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
schema_versioninteger· uint32*requiredmin 0
scope_hashstring*required
statestring*requiredenum: draft, baseline_running, baseline_ready, candidate_running, comparison_ready
technical_retriesinteger· uint8*requiredmin 0
updated_atstring*required
e2e::dashboard::plan-get
functionRead one local plan.
baseline_execution_idstring
candidate_execution_idsstring[]*required
candidate_labelsobject*required
compatibleboolean*required
created_atstring*required
incomplete_execution_idsstring[]*required
judge_modelstring*required
judge_providerstring*required
protected_executor_requiredboolean*required
runsinteger· uint32*requiredmin 0
scenario_idsstring[]*required
scenariosobject[]*required
complexity_tierstring*required
contract_sha256string*required
inputs_sha256string*required
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
schema_versioninteger· uint32*requiredmin 0
scope_hashstring*required
statestring*requiredenum: draft, baseline_running, baseline_ready, candidate_running, comparison_ready
technical_retriesinteger· uint8*requiredmin 0
updated_atstring*required
Start a baseline or candidate from a locked local plan.
idempotency_keystring*required
rolestring*requiredenum: baseline, candidate
baseline_execution_idstring
candidate_execution_idsstring[]*required
candidate_labelsobject*required
compatibleboolean*required
created_atstring*required
incomplete_execution_idsstring[]*required
judge_modelstring*required
judge_providerstring*required
protected_executor_requiredboolean*required
runsinteger· uint32*requiredmin 0
scenario_idsstring[]*required
scenariosobject[]*required
complexity_tierstring*required
contract_sha256string*required
inputs_sha256string*required
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
schema_versioninteger· uint32*requiredmin 0
scope_hashstring*required
statestring*requiredenum: draft, baseline_running, baseline_ready, candidate_running, comparison_ready
technical_retriesinteger· uint8*requiredmin 0
updated_atstring*required
Update an unlocked local plan or rename retained candidates.
technical_retriesinteger· uint8min 0
baseline_execution_idstring
candidate_execution_idsstring[]*required
candidate_labelsobject*required
compatibleboolean*required
created_atstring*required
incomplete_execution_idsstring[]*required
judge_modelstring*required
judge_providerstring*required
protected_executor_requiredboolean*required
runsinteger· uint32*requiredmin 0
scenario_idsstring[]*required
scenariosobject[]*required
complexity_tierstring*required
contract_sha256string*required
inputs_sha256string*required
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
schema_versioninteger· uint32*requiredmin 0
scope_hashstring*required
statestring*requiredenum: draft, baseline_running, baseline_ready, candidate_running, comparison_ready
technical_retriesinteger· uint8*requiredmin 0
updated_atstring*required
List local plans and their baseline/candidate lifecycle.
master_planunknown*required
plansobject[]*required
baseline_execution_idstring
candidate_execution_idsstring[]*required
candidate_labelsobject*required
compatibleboolean*required
created_atstring*required
incomplete_execution_idsstring[]*required
judge_modelstring*required
judge_providerstring*required
protected_executor_requiredboolean*required
runsinteger· uint32*requiredmin 0
scenario_idsstring[]*required
scenariosobject[]*required
complexity_tierstring*required
contract_sha256string*required
inputs_sha256string*required
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
schema_versioninteger· uint32*requiredmin 0
scope_hashstring*required
statestring*requiredenum: draft, baseline_running, baseline_ready, candidate_running, comparison_ready
technical_retriesinteger· uint8*requiredmin 0
updated_atstring*required
Cancel the active local E2E execution.
defaultsobject*required
judge_modelstring*required
judge_providerstring*required
runsinteger· uint32*requiredmin 0
technical_retriesinteger· uint8*requiredmin 0
jobany of
any of (2)
variant 1
completed_atstring*required
log_frominteger· uint64*requiredmin 0
log_offsetinteger· uint64*requiredmin 0
log_truncatedboolean*required
requestobject*required
… expand 10 nestedcollapse
runsinteger· uint32*requiredmin 0
scenariosstring[]*required
technical_retriesinteger· uint8*requiredmin 0
started_atstring*required
statusstring*requiredenum: running, cancelling, cancelled, completed, failed
Start one local E2E execution.
runsinteger· uint32*requiredmin 0
scenariosstring[]*required
technical_retriesinteger· uint8*requiredmin 0
defaultsobject*required
judge_modelstring*required
judge_providerstring*required
runsinteger· uint32*requiredmin 0
technical_retriesinteger· uint8*requiredmin 0
jobany of
any of (2)
variant 1
completed_atstring*required
log_frominteger· uint64*requiredmin 0
log_offsetinteger· uint64*requiredmin 0
log_truncatedboolean*required
requestobject*required
… expand 10 nestedcollapse
runsinteger· uint32*requiredmin 0
scenariosstring[]*required
technical_retriesinteger· uint8*requiredmin 0
started_atstring*required
statusstring*requiredenum: running, cancelling, cancelled, completed, failed
Read local execution state and only the unread log suffix.
afterinteger· uint64min 0
defaultsobject*required
judge_modelstring*required
judge_providerstring*required
runsinteger· uint32*requiredmin 0
technical_retriesinteger· uint8*requiredmin 0
jobany of
any of (2)
variant 1
completed_atstring*required
log_frominteger· uint64*requiredmin 0
log_offsetinteger· uint64*requiredmin 0
log_truncatedboolean*required
requestobject*required
… expand 10 nestedcollapse
runsinteger· uint32*requiredmin 0
scenariosstring[]*required
technical_retriesinteger· uint8*requiredmin 0
started_atstring*required
statusstring*requiredenum: running, cancelling, cancelled, completed, failed
Read local metric history for one test version, with provider-grouped execution and judge models, without comparison actions.
limitinteger· uint16min 0
test_versioninteger· uint32min 0
available_versionsobject[]*required
execution_countinteger· uint*requiredmin 0
run_countinteger· uint*requiredmin 0
versioninteger· uint32*requiredmin 0
current_versioninteger· uint32min 0
The contract's current version, which may have no executions yet.
judge_modelsobject[]*required
observationsobject[]*required
analyzer_profile_sha256string*required
assessment_profile_sha256string*required
assessment_summaryobject*required
ai_availabilityobject*required
… expand 6 nestedcollapse
availableinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_requestedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
ai_verdictsobject*required
… expand 4 nestedcollapse
failinteger· uint*requiredmin 0
inconclusiveinteger· uint*requiredmin 0
passinteger· uint*requiredmin 0
pass_with_concernsinteger· uint*requiredmin 0
assessment_countinteger· uint*requiredmin 0
assessment_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_countinteger· uint*requiredmin 0
asset_qualitative_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_validation_outcomesobject*required
… expand 10 nestedcollapse
invalidinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_producedinteger· uint*requiredmin 0
oversizedinteger· uint*requiredmin 0
removed_during_cleanupinteger· uint*requiredmin 0
unexpectedinteger· uint*requiredmin 0
unreadableinteger· uint*requiredmin 0
unsafe_pathinteger· uint*requiredmin 0
validinteger· uint*requiredmin 0
effective_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
evidence_reference_countinteger· uint*requiredmin 0
median_confidencenumber· double
median_quality_scorenumber· double
run_countinteger· uint*requiredmin 0
system_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
completed_atstring*required
contract_sha256string*required
evaluated_version_idstring
execution_idstring*required
median_cost_usdnumber· double
median_duration_secondsnumber· double
median_function_call_errorsnumber· double
median_function_callsnumber· double
median_scorenumber· double
median_tokensnumber· double
median_turnsnumber· double
run_countinteger· uint*requiredmin 0
scenario_versioninteger· uint32*requiredmin 0
scored_runsinteger· uint*requiredmin 0
stack_modestring*required
subject_modelstring*required
subject_providerstring*required
system_labelstring*required
seriesobject[]*required
analyzer_profile_sha256string*required
assessment_profile_sha256string*required
contract_sha256string*required
execution_countinteger· uint*requiredmin 0
median_cost_usdnumber· double
median_duration_secondsnumber· double
median_function_call_errorsnumber· double
median_function_callsnumber· double
median_scorenumber· double
median_tokensnumber· double
median_turnsnumber· double
run_countinteger· uint*requiredmin 0
scenario_versioninteger· uint32*requiredmin 0
stack_modestring*required
subject_modelstring*required
subject_providerstring*required
system_labelstring*required
subject_modelsobject[]*required
subjectsstring[]*required
test_versioninteger· uint32*requiredmin 0
The version whose executions are shown (the latest with evidence by default).
totalinteger· uint*requiredmin 0
Read one test version across two immutable evaluated-system versions.
from_version_idstring*required
test_versioninteger· uint32*requiredmin 0
to_version_idstring*required
compatibilitystring*required
compatibility_reasonsstring[]*required
deltaobject*required
duration_secondsnumber· double
fromany of
any of (2)
variant 1
assessment_summaryobject*required
… expand 13 nestedcollapse
ai_availabilityobject*required
… expand 6 nestedcollapse
availableinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_requestedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
ai_verdictsobject*required
… expand 4 nestedcollapse
failinteger· uint*requiredmin 0
inconclusiveinteger· uint*requiredmin 0
passinteger· uint*requiredmin 0
pass_with_concernsinteger· uint*requiredmin 0
assessment_countinteger· uint*requiredmin 0
assessment_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_countinteger· uint*requiredmin 0
asset_qualitative_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_validation_outcomesobject*required
… expand 10 nestedcollapse
invalidinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_producedinteger· uint*requiredmin 0
oversizedinteger· uint*requiredmin 0
removed_during_cleanupinteger· uint*requiredmin 0
unexpectedinteger· uint*requiredmin 0
unreadableinteger· uint*requiredmin 0
unsafe_pathinteger· uint*requiredmin 0
validinteger· uint*requiredmin 0
effective_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
evidence_reference_countinteger· uint*requiredmin 0
median_confidencenumber· double
median_quality_scorenumber· double
run_countinteger· uint*requiredmin 0
system_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
case_countinteger· uint*requiredmin 0
evaluated_version_idstring*required
execution_countinteger· uint*requiredmin 0
median_cost_usdnumber· double
median_duration_secondsnumber· double
median_scorenumber· double
median_tokensnumber· double
outcomesobject*required
… expand 4 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infra_failedinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
technical_failedinteger· uint*requiredmin 0
samplesobject*required
… expand 5 nestedcollapse
cost_usdinteger· uint*requiredmin 0
duration_secondsinteger· uint*requiredmin 0
scoreinteger· uint*requiredmin 0
tokensinteger· uint*requiredmin 0
turnsinteger· uint*requiredmin 0
scored_runsinteger· uint*requiredmin 0
total_runsinteger· uint*requiredmin 0
from_observationsobject[]*required
analyzer_profile_sha256string*required
assessment_profile_sha256string*required
assessment_summaryobject*required
ai_availabilityobject*required
… expand 6 nestedcollapse
availableinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_requestedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
ai_verdictsobject*required
… expand 4 nestedcollapse
failinteger· uint*requiredmin 0
inconclusiveinteger· uint*requiredmin 0
passinteger· uint*requiredmin 0
pass_with_concernsinteger· uint*requiredmin 0
assessment_countinteger· uint*requiredmin 0
assessment_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_countinteger· uint*requiredmin 0
asset_qualitative_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_validation_outcomesobject*required
… expand 10 nestedcollapse
invalidinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_producedinteger· uint*requiredmin 0
oversizedinteger· uint*requiredmin 0
removed_during_cleanupinteger· uint*requiredmin 0
unexpectedinteger· uint*requiredmin 0
unreadableinteger· uint*requiredmin 0
unsafe_pathinteger· uint*requiredmin 0
validinteger· uint*requiredmin 0
effective_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
evidence_reference_countinteger· uint*requiredmin 0
median_confidencenumber· double
median_quality_scorenumber· double
run_countinteger· uint*requiredmin 0
system_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
completed_atstring*required
contract_sha256string*required
evaluated_version_idstring
execution_idstring*required
median_cost_usdnumber· double
median_duration_secondsnumber· double
median_function_call_errorsnumber· double
median_function_callsnumber· double
median_scorenumber· double
median_tokensnumber· double
median_turnsnumber· double
run_countinteger· uint*requiredmin 0
scenario_versioninteger· uint32*requiredmin 0
scored_runsinteger· uint*requiredmin 0
stack_modestring*required
subject_modelstring*required
subject_providerstring*required
system_labelstring*required
test_versioninteger· uint32*requiredmin 0
toany of
any of (2)
variant 1
assessment_summaryobject*required
… expand 13 nestedcollapse
ai_availabilityobject*required
… expand 6 nestedcollapse
availableinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_requestedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
ai_verdictsobject*required
… expand 4 nestedcollapse
failinteger· uint*requiredmin 0
inconclusiveinteger· uint*requiredmin 0
passinteger· uint*requiredmin 0
pass_with_concernsinteger· uint*requiredmin 0
assessment_countinteger· uint*requiredmin 0
assessment_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_countinteger· uint*requiredmin 0
asset_qualitative_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_validation_outcomesobject*required
… expand 10 nestedcollapse
invalidinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_producedinteger· uint*requiredmin 0
oversizedinteger· uint*requiredmin 0
removed_during_cleanupinteger· uint*requiredmin 0
unexpectedinteger· uint*requiredmin 0
unreadableinteger· uint*requiredmin 0
unsafe_pathinteger· uint*requiredmin 0
validinteger· uint*requiredmin 0
effective_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
evidence_reference_countinteger· uint*requiredmin 0
median_confidencenumber· double
median_quality_scorenumber· double
run_countinteger· uint*requiredmin 0
system_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
case_countinteger· uint*requiredmin 0
evaluated_version_idstring*required
execution_countinteger· uint*requiredmin 0
median_cost_usdnumber· double
median_duration_secondsnumber· double
median_scorenumber· double
median_tokensnumber· double
outcomesobject*required
… expand 4 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infra_failedinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
technical_failedinteger· uint*requiredmin 0
samplesobject*required
… expand 5 nestedcollapse
cost_usdinteger· uint*requiredmin 0
duration_secondsinteger· uint*requiredmin 0
scoreinteger· uint*requiredmin 0
tokensinteger· uint*requiredmin 0
turnsinteger· uint*requiredmin 0
scored_runsinteger· uint*requiredmin 0
total_runsinteger· uint*requiredmin 0
to_observationsobject[]*required
analyzer_profile_sha256string*required
assessment_profile_sha256string*required
assessment_summaryobject*required
ai_availabilityobject*required
… expand 6 nestedcollapse
availableinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_requestedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
ai_verdictsobject*required
… expand 4 nestedcollapse
failinteger· uint*requiredmin 0
inconclusiveinteger· uint*requiredmin 0
passinteger· uint*requiredmin 0
pass_with_concernsinteger· uint*requiredmin 0
assessment_countinteger· uint*requiredmin 0
assessment_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_countinteger· uint*requiredmin 0
asset_qualitative_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_validation_outcomesobject*required
… expand 10 nestedcollapse
invalidinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_producedinteger· uint*requiredmin 0
oversizedinteger· uint*requiredmin 0
removed_during_cleanupinteger· uint*requiredmin 0
unexpectedinteger· uint*requiredmin 0
unreadableinteger· uint*requiredmin 0
unsafe_pathinteger· uint*requiredmin 0
validinteger· uint*requiredmin 0
effective_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
evidence_reference_countinteger· uint*requiredmin 0
median_confidencenumber· double
median_quality_scorenumber· double
run_countinteger· uint*requiredmin 0
system_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
completed_atstring*required
contract_sha256string*required
evaluated_version_idstring
execution_idstring*required
median_cost_usdnumber· double
median_duration_secondsnumber· double
median_function_call_errorsnumber· double
median_function_callsnumber· double
median_scorenumber· double
median_tokensnumber· double
median_turnsnumber· double
run_countinteger· uint*requiredmin 0
scenario_versioninteger· uint32*requiredmin 0
scored_runsinteger· uint*requiredmin 0
stack_modestring*required
subject_modelstring*required
subject_providerstring*required
system_labelstring*required
List the versioned test catalog with compact A/B results and stable pagination.
limitinteger· uint16min 0
rowsobject[]*required
available_versionsobject[]*required
execution_countinteger· uint*requiredmin 0
run_countinteger· uint*requiredmin 0
versioninteger· uint32*requiredmin 0
calibrationany of
any of (2)
variant 1
compatible_sample_countinteger· uint*requiredmin 0
characterizationany of
any of (2)
variant 1
human_horizonall of
… expand 1 nestedcollapse
all of (1)
variant 1
basisall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: unknown, author_estimate, measured
max_minutesinteger· uint32min 0
min_minutesinteger· uint32min 0
realismall of
… expand 1 nestedcollapse
all of (1)
variant 1
executionall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: synthetic, realistic_simulator, frozen_real_artifact
shadowall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: none, read_only
complexityany of
any of (2)
variant 1
methodstring*requiredenum: capability_v2
profileobject*required
… expand 16 nestedcollapse
agent_owned_decompositionboolean
ambiguity_levelinteger· uint8*requiredmin 0
artifact_countinteger· uint8*requiredmin 0
coherent_long_horizonboolean
compensable_mutationsinteger· uint8min 0
coordination_edgesinteger· uint16*requiredmin 0
dependency_depthinteger· uint8*requiredmin 0
durable_resume_cyclesinteger· uint8min 0
external_systemsinteger· uint8*requiredmin 0
material_invalidation_eventsinteger· uint8min 0
parallel_branchesinteger· uint8*requiredmin 0
planning_depthinteger· uint8*requiredmin 0
replan_loopsinteger· uint8min 0
state_transitionsinteger· uint16*requiredmin 0
validation_loopsinteger· uint8*requiredmin 0
wake_cyclesinteger· uint8*requiredmin 0
tierstring*requiredenum: l0_atomic, l1_sequential, l2_stateful, l3_concurrent, l4_coordinated, l5_adaptive
current_versioninteger· uint32min 0
resultany of
any of (2)
variant 1
compatibilitystring*required
compatibility_reasonsstring[]*required
deltaobject*required
… expand 4 nestedcollapse
duration_secondsnumber· double
fromany of
… expand 2 nestedcollapse
any of (2)
variant 1
assessment_summaryobject*required
… expand 13 nestedcollapse
ai_availabilityobject*required
… expand 6 nestedcollapse
availableinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_requestedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
ai_verdictsobject*required
… expand 4 nestedcollapse
failinteger· uint*requiredmin 0
inconclusiveinteger· uint*requiredmin 0
passinteger· uint*requiredmin 0
pass_with_concernsinteger· uint*requiredmin 0
assessment_countinteger· uint*requiredmin 0
assessment_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_countinteger· uint*requiredmin 0
asset_qualitative_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_validation_outcomesobject*required
… expand 10 nestedcollapse
invalidinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_producedinteger· uint*requiredmin 0
oversizedinteger· uint*requiredmin 0
removed_during_cleanupinteger· uint*requiredmin 0
unexpectedinteger· uint*requiredmin 0
unreadableinteger· uint*requiredmin 0
unsafe_pathinteger· uint*requiredmin 0
validinteger· uint*requiredmin 0
effective_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
evidence_reference_countinteger· uint*requiredmin 0
median_confidencenumber· double
median_quality_scorenumber· double
run_countinteger· uint*requiredmin 0
system_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
case_countinteger· uint*requiredmin 0
evaluated_version_idstring*required
execution_countinteger· uint*requiredmin 0
median_cost_usdnumber· double
median_duration_secondsnumber· double
median_scorenumber· double
median_tokensnumber· double
outcomesobject*required
… expand 4 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infra_failedinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
technical_failedinteger· uint*requiredmin 0
samplesobject*required
… expand 5 nestedcollapse
cost_usdinteger· uint*requiredmin 0
duration_secondsinteger· uint*requiredmin 0
scoreinteger· uint*requiredmin 0
tokensinteger· uint*requiredmin 0
turnsinteger· uint*requiredmin 0
scored_runsinteger· uint*requiredmin 0
total_runsinteger· uint*requiredmin 0
from_observationsobject[]*required
… expand 32 nestedcollapse
analyzer_profile_sha256string*required
assessment_profile_sha256string*required
assessment_summaryobject*required
… expand 13 nestedcollapse
ai_availabilityobject*required
… expand 6 nestedcollapse
availableinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_requestedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
ai_verdictsobject*required
… expand 4 nestedcollapse
failinteger· uint*requiredmin 0
inconclusiveinteger· uint*requiredmin 0
passinteger· uint*requiredmin 0
pass_with_concernsinteger· uint*requiredmin 0
assessment_countinteger· uint*requiredmin 0
assessment_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_countinteger· uint*requiredmin 0
asset_qualitative_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_validation_outcomesobject*required
… expand 10 nestedcollapse
invalidinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_producedinteger· uint*requiredmin 0
oversizedinteger· uint*requiredmin 0
removed_during_cleanupinteger· uint*requiredmin 0
unexpectedinteger· uint*requiredmin 0
unreadableinteger· uint*requiredmin 0
unsafe_pathinteger· uint*requiredmin 0
validinteger· uint*requiredmin 0
effective_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
evidence_reference_countinteger· uint*requiredmin 0
median_confidencenumber· double
median_quality_scorenumber· double
run_countinteger· uint*requiredmin 0
system_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
completed_atstring*required
contract_sha256string*required
evaluated_version_idstring
execution_idstring*required
median_cost_usdnumber· double
median_duration_secondsnumber· double
median_function_call_errorsnumber· double
median_function_callsnumber· double
median_scorenumber· double
median_tokensnumber· double
median_turnsnumber· double
run_countinteger· uint*requiredmin 0
scenario_versioninteger· uint32*requiredmin 0
scored_runsinteger· uint*requiredmin 0
stack_modestring*required
subject_modelstring*required
subject_providerstring*required
system_labelstring*required
test_versioninteger· uint32*requiredmin 0
toany of
… expand 2 nestedcollapse
any of (2)
variant 1
assessment_summaryobject*required
… expand 13 nestedcollapse
ai_availabilityobject*required
… expand 6 nestedcollapse
availableinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_requestedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
ai_verdictsobject*required
… expand 4 nestedcollapse
failinteger· uint*requiredmin 0
inconclusiveinteger· uint*requiredmin 0
passinteger· uint*requiredmin 0
pass_with_concernsinteger· uint*requiredmin 0
assessment_countinteger· uint*requiredmin 0
assessment_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_countinteger· uint*requiredmin 0
asset_qualitative_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_validation_outcomesobject*required
… expand 10 nestedcollapse
invalidinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_producedinteger· uint*requiredmin 0
oversizedinteger· uint*requiredmin 0
removed_during_cleanupinteger· uint*requiredmin 0
unexpectedinteger· uint*requiredmin 0
unreadableinteger· uint*requiredmin 0
unsafe_pathinteger· uint*requiredmin 0
validinteger· uint*requiredmin 0
effective_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
evidence_reference_countinteger· uint*requiredmin 0
median_confidencenumber· double
median_quality_scorenumber· double
run_countinteger· uint*requiredmin 0
system_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
case_countinteger· uint*requiredmin 0
evaluated_version_idstring*required
execution_countinteger· uint*requiredmin 0
median_cost_usdnumber· double
median_duration_secondsnumber· double
median_scorenumber· double
median_tokensnumber· double
outcomesobject*required
… expand 4 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infra_failedinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
technical_failedinteger· uint*requiredmin 0
samplesobject*required
… expand 5 nestedcollapse
cost_usdinteger· uint*requiredmin 0
duration_secondsinteger· uint*requiredmin 0
scoreinteger· uint*requiredmin 0
tokensinteger· uint*requiredmin 0
turnsinteger· uint*requiredmin 0
scored_runsinteger· uint*requiredmin 0
total_runsinteger· uint*requiredmin 0
to_observationsobject[]*required
… expand 32 nestedcollapse
analyzer_profile_sha256string*required
assessment_profile_sha256string*required
assessment_summaryobject*required
… expand 13 nestedcollapse
ai_availabilityobject*required
… expand 6 nestedcollapse
availableinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_requestedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
ai_verdictsobject*required
… expand 4 nestedcollapse
failinteger· uint*requiredmin 0
inconclusiveinteger· uint*requiredmin 0
passinteger· uint*requiredmin 0
pass_with_concernsinteger· uint*requiredmin 0
assessment_countinteger· uint*requiredmin 0
assessment_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_countinteger· uint*requiredmin 0
asset_qualitative_outcomesobject*required
… expand 6 nestedcollapse
errorinteger· uint*requiredmin 0
failedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
partialinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
asset_validation_outcomesobject*required
… expand 10 nestedcollapse
invalidinteger· uint*requiredmin 0
malformedinteger· uint*requiredmin 0
not_evaluatedinteger· uint*requiredmin 0
not_producedinteger· uint*requiredmin 0
oversizedinteger· uint*requiredmin 0
removed_during_cleanupinteger· uint*requiredmin 0
unexpectedinteger· uint*requiredmin 0
unreadableinteger· uint*requiredmin 0
unsafe_pathinteger· uint*requiredmin 0
validinteger· uint*requiredmin 0
effective_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
evidence_reference_countinteger· uint*requiredmin 0
median_confidencenumber· double
median_quality_scorenumber· double
run_countinteger· uint*requiredmin 0
system_statusesobject*required
… expand 8 nestedcollapse
hard_gate_failedinteger· uint*requiredmin 0
infrastructure_errorinteger· uint*requiredmin 0
judge_errorinteger· uint*requiredmin 0
passedinteger· uint*requiredmin 0
passed_with_concernsinteger· uint*requiredmin 0
resource_limitinteger· uint*requiredmin 0
subject_errorinteger· uint*requiredmin 0
unavailableinteger· uint*requiredmin 0
completed_atstring*required
contract_sha256string*required
evaluated_version_idstring
execution_idstring*required
median_cost_usdnumber· double
median_duration_secondsnumber· double
median_function_call_errorsnumber· double
median_function_callsnumber· double
median_scorenumber· double
median_tokensnumber· double
median_turnsnumber· double
run_countinteger· uint*requiredmin 0
scenario_versioninteger· uint32*requiredmin 0
scored_runsinteger· uint*requiredmin 0
stack_modestring*required
subject_modelstring*required
subject_providerstring*required
system_labelstring*required
selected_versioninteger· uint32min 0
specany of
any of (2)
variant 1
criteriaobject[]*required
… expand 7 nestedcollapse
descriptionstring*required
dimensionstring*requiredenum: deliverable, structural_integrity, efficiency, robustness, e2e_infrastructure
kindstring*requiredenum: required_check, signal, asset_validation, asset_quality
policystring*requiredenum: hard_gate, advisory
sourcestring*requiredenum: deterministic, judge, asset_analyzer
weightinteger· uint8*requiredmin 0
denied_functionsstring[]*required
executionobject*required
… expand 5 nestedcollapse
max_output_tokensinteger· uint64min 0
max_total_tokensinteger· uint64min 0
Shared Harness token budget. `None` leaves the Harness budget unbounded for this scenario.
max_turnsinteger· uint32*requiredmin 0
max_validation_retriesinteger· uint32min 0
Optional per-session cap for post-turn validation denials. `None` preserves the Harness-configured default.
stuck_timeout_secondsinteger· uint64*requiredmin 0
Stop only after this many seconds without observable useful progress. Large scenarios have no fixed wall-clock deadline.
promptstring*required
The prompt as materialized for a representative run: run-scoped paths resolve against a catalog namespace, never a real execution.
summarystring
Editorial description; absent until the scenario defines a `SUMMARY`.
totalinteger· uint*requiredmin 0
e2e::fault-evaluate
functionClassify a protected supervisor's fault journal against canonical execution evidence.
execution_idstring*required
journalobject*required
actionsobject[]*required
kindstring*requiredenum: delay_call, fail_first_attempt, malformed_result, throttle_burst, duplicate_event, child_timeout, …
statusstring*requiredenum: recovered, unrecovered, not_triggered, injector_failed
cancellation_observedboolean*required
cleanup_completedboolean*required
completed_atstring*required
execution_idstring*required
infrastructure_failuresstring[]
observed_work_amplificationnumber· double
profile_sha256string*required
started_atstring*required
planobject*required
actionsobject[]*required
duration_msinteger· uint64min 0
kindstring*requiredenum: delay_call, fail_first_attempt, malformed_result, throttle_burst, duplicate_event, child_timeout, …
sequenceinteger· uint32*requiredmin 0
trigger_attemptinteger· uint32min 0
expected_outcomeone of*required
one of (2)
variant 1
valuestringenum: recovered, cancelled
variant 2
valuestringenum: degraded
profile_idstring*required
profile_sha256string*required
seedinteger· uint64*requiredmin 0
profileobject*required
cancellationany of
any of (2)
variant 1
after_actionstring*required
grace_period_msinteger· uint64min 0
child_timeout_msinteger· uint64min 0
delayed_callsobject[]
delay_msinteger· uint64*requiredmin 0
occurrencesinteger· uint8min 0
expected_outcomeall of
all of (1)
variant 1
one of (2)
variant 1
valuestringenum: recovered, cancelled
variant 2
valuestringenum: degraded
fail_first_attemptstring[]
malformed_resultsstring[]
The supervisor corrupts the result payload of the first call to each named target: the subject receives undecodable, contract-breaking bytes exactly once per target.
max_work_amplificationnumber· double*required
out_of_order_resultsboolean
profile_idstring*required
seedinteger· uint64*requiredmin 0
throttle_burstany of
any of (2)
variant 1
responsesinteger· uint8*requiredmin 0
spacing_msinteger· uint64*requiredmin 0
transient_disconnectany of
any of (2)
variant 1
after_actionstring*required
duration_msinteger· uint64*requiredmin 0
cancellation_observedboolean*required
classificationstring*requiredenum: correct_recovery, excessive_recovery, incorrect_result, structural_failure, infrastructure_failure
cleanup_completedboolean*required
deliverable_correctboolean
execution_idstring*required
max_work_amplificationnumber· double*required
observed_work_amplificationnumber· double
planned_actionsinteger· uint32*requiredmin 0
profile_idstring*required
profile_sha256string*required
recovered_actionsinteger· uint32*requiredmin 0
structural_integrityboolean
Materialize a deterministic, versioned fault plan for a protected supervisor.
profileobject*required
cancellationany of
any of (2)
variant 1
after_actionstring*required
grace_period_msinteger· uint64min 0
child_timeout_msinteger· uint64min 0
delayed_callsobject[]
delay_msinteger· uint64*requiredmin 0
occurrencesinteger· uint8min 0
expected_outcomeall of
all of (1)
variant 1
one of (2)
variant 1
valuestringenum: recovered, cancelled
variant 2
valuestringenum: degraded
fail_first_attemptstring[]
malformed_resultsstring[]
The supervisor corrupts the result payload of the first call to each named target: the subject receives undecodable, contract-breaking bytes exactly once per target.
max_work_amplificationnumber· double*required
out_of_order_resultsboolean
profile_idstring*required
seedinteger· uint64*requiredmin 0
throttle_burstany of
any of (2)
variant 1
responsesinteger· uint8*requiredmin 0
spacing_msinteger· uint64*requiredmin 0
transient_disconnectany of
any of (2)
variant 1
after_actionstring*required
duration_msinteger· uint64*requiredmin 0
actionsobject[]*required
duration_msinteger· uint64min 0
kindstring*requiredenum: delay_call, fail_first_attempt, malformed_result, throttle_burst, duplicate_event, child_timeout, …
sequenceinteger· uint32*requiredmin 0
trigger_attemptinteger· uint32min 0
expected_outcomeone of*required
one of (2)
variant 1
valuestringenum: recovered, cancelled
variant 2
valuestringenum: degraded
profile_idstring*required
profile_sha256string*required
seedinteger· uint64*requiredmin 0
e2e::history-list
functionList hash-validated E2E history records from the iii database worker.
limitinteger· uint16min 0
recordsobject[]*required
archiveobject*required
archive_idstring*required
created_atstring*required
execution_idstring*required
identity_sha256string*required
manifestobject*required
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
manifest_backupobject*required
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
retention_classstring*requiredenum: temporary, pull_request, longitudinal, canonical
baseline_comparableboolean*required
case_countinteger· uint32*requiredmin 0
e2e_repositorystring*required
e2e_revisionstring*required
execution_idstring*required
identity_sha256string*required
ingestion_idstring*required
occurred_atstring*required
result_contract_sha256string*required
scoring_profile_sha256string*required
stack_modestring*required
subject_modelstring*required
subject_providerstring*required
subject_revisionstring*required
Get the immutable result and manifest of an E2E execution.
execution_idstring*required
archiveany of
any of (2)
variant 1
archive_idstring*required
created_atstring*required
execution_idstring*required
identity_sha256string*required
manifestobject*required
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
manifest_backupobject*required
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
retention_classstring*requiredenum: temporary, pull_request, longitudinal, canonical
execution_idstring*required
manifestany of
any of (2)
variant 1
control_planeobject*required
… expand 1 nestedcollapse
functionsobject[]*required
… expand 4 nestedcollapse
function_idstring*required
request_schemaunknown*required
response_schemaunknown*required
executionobject*required
… expand 4 nestedcollapse
completed_atstring*required
execution_idstring*required
started_atstring*required
judgeany of
… expand 2 nestedcollapse
any of (2)
variant 1
context_windowinteger· uint64*requiredmin 0
max_output_tokensinteger· uint64*requiredmin 0
observation_contractany of
… expand 2 nestedcollapse
any of (2)
variant 1
attemptinteger· uint32*requiredmin 0
correlationobject*required
… expand 3 nestedcollapse
deployment_idstring*required
operation_idstring*required
modeobject*required
… expand 2 nestedcollapse
decisionstring*requiredenum: observe_only
environmentstring*requiredenum: demonstration
planobject*required
… expand 4 nestedcollapse
catalog_sha256string*required
runnerobject*required
… expand 3 nestedcollapse
selected_casesobject[]*required
… expand 6 nestedcollapse
contract_sha256string*required
inputs_sha256string*required
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
targetobject*required
… expand 3 nestedcollapse
applicationstring*required
stackone of*required
… expand 2 nestedcollapse
one of (2)
variant 1
modestring*requiredenum: source
workers_repositorystring*required
workers_revisionstring*required
variant 2
modestring*requiredenum: registry
stack_lock_digeststring*required
stack_versionsobject*required
subjectobject*required
… expand 6 nestedcollapse
context_windowinteger· uint64*requiredmin 0
max_output_tokensinteger· uint64*requiredmin 0
system_under_testobject*required
… expand 7 nestedcollapse
contract_hashesobject*required
e2e_repositorystring*required
e2e_revisionstring*required
engine_versionstring*required
harness_versionstring*required
stackone of*required
… expand 2 nestedcollapse
one of (2)
variant 1
modestring*requiredenum: source
workers_repositorystring*required
workers_revisionstring*required
variant 2
modestring*requiredenum: registry
stack_lock_digeststring*required
stack_versionsobject*required
worker_contractsobject[]
… expand 3 nestedcollapse
function_idstring*required
request_schema_sha256string*required
response_schema_sha256string*required
observationany of
any of (2)
variant 1
evidenceobject*required
… expand 3 nestedcollapse
artifactsobject[]
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
identityobject*required
… expand 4 nestedcollapse
executionobject*required
… expand 6 nestedcollapse
attemptinteger· uint32*requiredmin 0
completed_atstring*required
request_sha256string*required
run_contract_sha256string*required
started_atstring*required
planobject*required
… expand 4 nestedcollapse
catalog_sha256string*required
runnerobject*required
… expand 3 nestedcollapse
targetobject*required
… expand 3 nestedcollapse
applicationstring*required
stackone of*required
… expand 2 nestedcollapse
one of (2)
variant 1
modestring*requiredenum: source
workers_repositorystring*required
workers_revisionstring*required
variant 2
modestring*requiredenum: registry
stack_lock_digeststring*required
stack_versionsobject*required
modeobject*required
… expand 2 nestedcollapse
decisionstring*requiredenum: observe_only
environmentstring*requiredenum: demonstration
outcomeobject*required
… expand 5 nestedcollapse
control_phasestring*required
data_availabilitystring*requiredenum: complete, partial, unavailable
objectivestring*requiredenum: passed, quality_advisory, hard_gate_failed, technical_failed, infrastructure_failed, cancelled, …
provenanceobject*required
… expand 3 nestedcollapse
subject_modelstring*required
subject_providerstring*required
system_under_testany of
… expand 2 nestedcollapse
any of (2)
variant 1
contract_hashesobject*required
e2e_repositorystring*required
e2e_revisionstring*required
engine_versionstring*required
harness_versionstring*required
stackone of*required
… expand 2 nestedcollapse
one of (2)
variant 1
modestring*requiredenum: source
workers_repositorystring*required
workers_revisionstring*required
variant 2
modestring*requiredenum: registry
stack_lock_digeststring*required
stack_versionsobject*required
samplesobject[]*required
… expand 12 nestedcollapse
attempt_idstring*required
data_availabilitystring*requiredenum: complete, partial, unavailable
derivationsobject[]
… expand 4 nestedcollapse
formula_versionstring*required
originstring*requiredenum: observed, derived_from_observed
metricsany of
… expand 2 nestedcollapse
any of (2)
variant 1
child_sessionsinteger· uint64min 0
child_turnsinteger· uint64min 0
critical_path_msinteger· uint64min 0
effective_fan_outinteger· uint64min 0
function_call_errorsinteger· uint64min 0
function_callsinteger· uint64min 0
input_tokensinteger· uint64min 0
minimum_expected_workinteger· uint64*requiredmin 0
observed_complexityobject*required
… expand 11 nestedcollapse
ambiguity_levelinteger· uint64min 0
artifact_countinteger· uint64min 0
coordination_edgesinteger· uint64min 0
dependency_depthinteger· uint64min 0
external_systemsinteger· uint64min 0
parallel_branchesinteger· uint64min 0
planning_depthinteger· uint64min 0
state_transitionsinteger· uint64min 0
validation_loopsinteger· uint64min 0
wake_cyclesinteger· uint64min 0
observed_workinteger· uint64min 0
output_tokensinteger· uint64min 0
root_turnsinteger· uint64min 0
technical_attemptsinteger· uint32*requiredmin 0
total_tokensinteger· uint64min 0
transient_resumesinteger· uint64min 0
validation_retriesinteger· uint64min 0
wake_resumesinteger· uint64min 0
wall_time_msinteger· uint64*requiredmin 0
work_amplificationnumber· double
originstring*requiredenum: observed, derived_from_observed
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
statusstring*requiredenum: passed, hard_gate_failed, subject_error, judge_error, resource_limit, infrastructure_error
observation_artifactany of
any of (2)
variant 1
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
phasestring*requiredenum: requested, admitted, preflighting, materializing, setting_up, executing, …
reportany of
any of (2)
variant 1
assessment_contractobject*required
… expand 1 nestedcollapse
runsobject[]*required
… expand 7 nestedcollapse
ai_final_assessmentobject*required
… expand 5 nestedcollapse
analyzerany of
… expand 2 nestedcollapse
any of (2)
variant 1
input_sha256string*required
analyzer_usageany of
… expand 2 nestedcollapse
any of (2)
variant 1
input_tokensinteger· uint64min 0
latency_msinteger· uint64min 0
output_tokensinteger· uint64min 0
availabilitystring*requiredenum: not_requested, not_evaluated, available, unavailable, malformed, failed
resultany of
… expand 2 nestedcollapse
any of (2)
variant 1
concernsstring[]*required
confidencenumber· double*required
diagnosisstring
What actually occurred in this execution — the decisive outcome and its proximate cause. Defaulted so reports persisted before this field existed still load; fresh judge responses are required to fill it.
evidenceobject[]*required
… expand 3 nestedcollapse
artifact_idstring*required
artifact_sha256string*required
limitationsstring[]*required
quality_scoreinteger· uint8*requiredmin 0
recommendationstring*required
strengthsstring[]*required
verdictstring*requiredenum: pass, pass_with_concerns, fail, inconclusive
assessmentsobject[]
… expand 13 nestedcollapse
analyzerany of
… expand 2 nestedcollapse
any of (2)
variant 1
input_sha256string*required
analyzer_usageany of
… expand 2 nestedcollapse
any of (2)
variant 1
input_tokensinteger· uint64min 0
latency_msinteger· uint64min 0
output_tokensinteger· uint64min 0
criterion_idstring*required
dimensionstring*requiredenum: deliverable, structural_integrity, efficiency, robustness, e2e_infrastructure
evidenceobject[]
… expand 3 nestedcollapse
artifact_idstring*required
artifact_sha256string*required
kindstring*requiredenum: required_check, signal, asset_validation, asset_quality
outcomestring*requiredenum: passed, failed, partial, not_evaluated, unavailable, error
policystring*requiredenum: hard_gate, advisory
scoreany of
… expand 2 nestedcollapse
any of (2)
variant 1
awardedinteger· uint8*requiredmin 0
possibleinteger· uint8*requiredmin 0
sourcestring*requiredenum: deterministic, judge, asset_analyzer
targetobject*required
… expand 2 nestedcollapse
kindstring*requiredenum: criterion, asset
assetsobject[]
… expand 2 nestedcollapse
qualitative_assessmentobject*required
… expand 13 nestedcollapse
analyzerany of
… expand 2 nestedcollapse
any of (2)
variant 1
input_sha256string*required
analyzer_usageany of
… expand 2 nestedcollapse
any of (2)
variant 1
input_tokensinteger· uint64min 0
latency_msinteger· uint64min 0
output_tokensinteger· uint64min 0
criterion_idstring*required
dimensionstring*requiredenum: deliverable, structural_integrity, efficiency, robustness, e2e_infrastructure
evidenceobject[]
… expand 3 nestedcollapse
artifact_idstring*required
artifact_sha256string*required
kindstring*requiredenum: required_check, signal, asset_validation, asset_quality
outcomestring*requiredenum: passed, failed, partial, not_evaluated, unavailable, error
policystring*requiredenum: hard_gate, advisory
scoreany of
… expand 2 nestedcollapse
any of (2)
variant 1
awardedinteger· uint8*requiredmin 0
possibleinteger· uint8*requiredmin 0
sourcestring*requiredenum: deterministic, judge, asset_analyzer
targetobject*required
… expand 2 nestedcollapse
kindstring*requiredenum: criterion, asset
validationobject*required
… expand 4 nestedcollapse
evidenceobject[]
… expand 3 nestedcollapse
artifact_idstring*required
artifact_sha256string*required
outcomestring*requiredenum: valid, invalid, malformed, oversized, not_produced, unreadable, …
attempt_idstring*required
effective_statusstring*requiredenum: unavailable, passed, passed_with_concerns, hard_gate_failed, subject_error, judge_error, …
system_statusstring*requiredenum: unavailable, passed, hard_gate_failed, subject_error, judge_error, resource_limit, …
executionobject*required
… expand 4 nestedcollapse
completed_atstring*required
execution_idstring*required
started_atstring*required
judgeany of
… expand 2 nestedcollapse
any of (2)
variant 1
context_windowinteger· uint64*requiredmin 0
max_output_tokensinteger· uint64*requiredmin 0
manifestany of
… expand 2 nestedcollapse
any of (2)
variant 1
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
objective_outcomestring*requiredenum: passed, failed, inconclusive
observation_contractany of
… expand 2 nestedcollapse
any of (2)
variant 1
attemptinteger· uint32*requiredmin 0
correlationobject*required
… expand 3 nestedcollapse
deployment_idstring*required
operation_idstring*required
modeobject*required
… expand 2 nestedcollapse
decisionstring*requiredenum: observe_only
environmentstring*requiredenum: demonstration
planobject*required
… expand 4 nestedcollapse
catalog_sha256string*required
runnerobject*required
… expand 3 nestedcollapse
selected_casesobject[]*required
… expand 6 nestedcollapse
contract_sha256string*required
inputs_sha256string*required
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
targetobject*required
… expand 3 nestedcollapse
applicationstring*required
stackone of*required
… expand 2 nestedcollapse
one of (2)
variant 1
modestring*requiredenum: source
workers_repositorystring*required
workers_revisionstring*required
variant 2
modestring*requiredenum: registry
stack_lock_digeststring*required
stack_versionsobject*required
persistence_errorsstring[]
redactionall of
… expand 1 nestedcollapse
all of (1)
variant 1
redacted_fieldsinteger· uint32*requiredmin 0
redacted_valuesinteger· uint32*requiredmin 0
report_statestring*requiredenum: complete, partial
result_contract_sha256string*required
scenariosobject[]*required
… expand 9 nestedcollapse
aggregateobject*required
… expand 32 nestedcollapse
completed_runsinteger· uint32*requiredmin 0
completion_evidence_coveragenumber· double
completion_ratenumber· double
costobject*required
… expand 3 nestedcollapse
subject_usdnumber· double
deferred_runsinteger· uint32*requiredmin 0
execution_reliabilitynumber· double
failed_attempt_tokensinteger· uint64min 0
hard_gate_failuresinteger· uint32*requiredmin 0
judge_tokens_consumedinteger· uint64min 0
median_scorenumber· double
Compatibility alias for `objective_median_score`.
objective_median_scorenumber· double
objective_score_coveragenumber· double
objective_scored_runsinteger· uint32*requiredmin 0
observed_runsinteger· uint32*requiredmin 0
pass_ratenumber· double*required
passed_runsinteger· uint32*requiredmin 0
planned_runsinteger· uint32*requiredmin 0
quality_coveragenumber· double
quality_score_completednumber· double
quality_scored_completed_runsinteger· uint32*requiredmin 0
required_passesinteger· uint32*requiredmin 0
robustnessobject*required
… expand 13 nestedcollapse
cost_per_successful_deliverablenumber· double
deliverable_success_ratenumber· double
median_wall_time_msnumber· double
minimum_sample_sizeinteger· uint32*requiredmin 0
p95_wall_time_msinteger· uint64min 0
sample_sizeinteger· uint32*requiredmin 0
structural_integrity_ratenumber· double
tail_minimum_sample_sizeinteger· uint32*requiredmin 0
technical_failure_ratenumber· double
wall_time_variancenumber· double
runsinteger· uint32*requiredmin 0
Compatibility alias for `observed_runs`.
scored_runsinteger· uint32*requiredmin 0
Compatibility alias for `objective_scored_runs`.
task_incomplete_runsinteger· uint32*requiredmin 0
technical_failuresinteger· uint32*requiredmin 0
technical_invalid_runsinteger· uint32*requiredmin 0
technical_valid_runsinteger· uint32*requiredmin 0
tokens_completed_p50number· double
tokens_per_completionnumber· double
total_tokens_consumedinteger· uint64min 0
undetermined_runsinteger· uint32*requiredmin 0
caseany of
… expand 2 nestedcollapse
any of (2)
variant 1
characterizationall of
… expand 1 nestedcollapse
all of (1)
variant 1
human_horizonall of
… expand 1 nestedcollapse
all of (1)
variant 1
basisall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: unknown, author_estimate, measured
max_minutesinteger· uint32min 0
min_minutesinteger· uint32min 0
realismall of
… expand 1 nestedcollapse
all of (1)
variant 1
executionall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: synthetic, realistic_simulator, frozen_real_artifact
shadowall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: none, read_only
complexityobject*required
… expand 3 nestedcollapse
methodstring*requiredenum: capability_v2
profileobject*required
… expand 16 nestedcollapse
agent_owned_decompositionboolean
ambiguity_levelinteger· uint8*requiredmin 0
artifact_countinteger· uint8*requiredmin 0
coherent_long_horizonboolean
compensable_mutationsinteger· uint8min 0
coordination_edgesinteger· uint16*requiredmin 0
dependency_depthinteger· uint8*requiredmin 0
durable_resume_cyclesinteger· uint8min 0
external_systemsinteger· uint8*requiredmin 0
material_invalidation_eventsinteger· uint8min 0
parallel_branchesinteger· uint8*requiredmin 0
planning_depthinteger· uint8*requiredmin 0
replan_loopsinteger· uint8min 0
state_transitionsinteger· uint16*requiredmin 0
validation_loopsinteger· uint8*requiredmin 0
wake_cyclesinteger· uint8*requiredmin 0
tierstring*requiredenum: l0_atomic, l1_sequential, l2_stateful, l3_concurrent, l4_coordinated, l5_adaptive
deliverable_contractobject*required
… expand 4 nestedcollapse
artifactsobject[]*required
… expand 5 nestedcollapse
max_size_bytesinteger· uint64*requiredmin 0
media_typestring*required
capture_before_cleanupboolean*required
invariantsobject[]*required
… expand 2 nestedcollapse
descriptionstring*required
provenance_requiredboolean*required
inputs_sha256string*required
required_capabilitiesstring[]*required
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
workobject*required
… expand 1 nestedcollapse
minimum_expected_workinteger· uint64*requiredmin 0
execution_policyobject*required
… expand 5 nestedcollapse
max_output_tokensinteger· uint64min 0
max_total_tokensinteger· uint64min 0
Shared Harness token budget. `None` leaves the Harness budget unbounded for this scenario.
max_turnsinteger· uint32*requiredmin 0
max_validation_retriesinteger· uint32min 0
Optional per-session cap for post-turn validation denials. `None` preserves the Harness-configured default.
stuck_timeout_secondsinteger· uint64*requiredmin 0
Stop only after this many seconds without observable useful progress. Large scenarios have no fixed wall-clock deadline.
runsobject[]*required
… expand 34 nestedcollapse
attempt_idstring*required
attempt_numberinteger· uint32*requiredmin 0
auditany of
Advisory behavioral audit over the captured transcript and metrics. Never contributes to score, status, gates, or longitudinal inputs.
… expand 2 nestedcollapse
any of (2)
variant 1
analyzerany of
… expand 2 nestedcollapse
any of (2)
variant 1
input_sha256string*required
analyzer_statusone of*required
… expand 3 nestedcollapse
one of (3)
variant 1
valuestringenum: completed, failed
variant 2
valuestringenum: not_configured
variant 3
valuestringenum: no_transcript
analyzer_usageany of
… expand 2 nestedcollapse
any of (2)
variant 1
input_tokensinteger· uint64min 0
latency_msinteger· uint64min 0
output_tokensinteger· uint64min 0
flagsobject[]*required
… expand 6 nestedcollapse
evidenceobject[]
… expand 3 nestedcollapse
detailstring*required
Sanitized, bounded description. Never contains secret material.
message_indexinteger· uintmin 0
Index of the transcript message the observation anchors to.
kindone of*required
… expand 5 nestedcollapse
one of (5)
variant 1
valuestringenum: verifier_tampering
variant 2
valuestringenum: out_of_scope_session_access
variant 3
valuestringenum: secret_exposure
variant 4
valuestringenum: destructive_action
variant 5
valuestringenum: anomalous_work
severitystring*requiredenum: info, warning, critical
sourcestring*requiredenum: deterministic, analyzer
completionstring*requiredenum: completed, task_incomplete, undetermined
costobject*required
… expand 3 nestedcollapse
subject_usdnumber· double
criteriaobject[]*required
… expand 4 nestedcollapse
awardedinteger· uint8min 0
possibleinteger· uint8*requiredmin 0
deliverablesobject[]
… expand 12 nestedcollapse
artifactany of
… expand 2 nestedcollapse
any of (2)
variant 1
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
content_formatstring*requiredenum: json, text_utf8
content_sha256string*required
content_size_bytesinteger· uint64*requiredmin 0
invariantsobject[]*required
… expand 3 nestedcollapse
media_typestring*required
provenanceobject[]*required
… expand 3 nestedcollapse
provenance_validboolean*required
schema_validboolean*required
dimensionsobject[]
… expand 3 nestedcollapse
dimensionstring*requiredenum: deliverable, structural_integrity, efficiency, robustness, e2e_infrastructure
efficiencyany of
… expand 2 nestedcollapse
any of (2)
variant 1
child_sessionsinteger· uint64min 0
child_turnsinteger· uint64min 0
critical_path_msinteger· uint64min 0
effective_fan_outinteger· uint64min 0
function_call_errorsinteger· uint64min 0
function_callsinteger· uint64min 0
input_tokensinteger· uint64min 0
minimum_expected_workinteger· uint64*requiredmin 0
observed_complexityobject*required
… expand 11 nestedcollapse
ambiguity_levelinteger· uint64min 0
artifact_countinteger· uint64min 0
coordination_edgesinteger· uint64min 0
dependency_depthinteger· uint64min 0
external_systemsinteger· uint64min 0
parallel_branchesinteger· uint64min 0
planning_depthinteger· uint64min 0
state_transitionsinteger· uint64min 0
validation_loopsinteger· uint64min 0
wake_cyclesinteger· uint64min 0
observed_workinteger· uint64min 0
output_tokensinteger· uint64min 0
root_turnsinteger· uint64min 0
technical_attemptsinteger· uint32*requiredmin 0
total_tokensinteger· uint64min 0
transient_resumesinteger· uint64min 0
validation_retriesinteger· uint64min 0
wake_resumesinteger· uint64min 0
wall_time_msinteger· uint64*requiredmin 0
work_amplificationnumber· double
evaluatorsobject*required
… expand 3 nestedcollapse
completionstring*requiredenum: not_required, pending, available, unavailable
final_advisorystring*requiredenum: not_required, pending, available, unavailable
qualitystring*requiredenum: not_required, pending, available, unavailable
evidenceobject[]
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
failuresobject[]
… expand 7 nestedcollapse
contaminationall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: none, case, execution
domainstring*requiredenum: subject, judge, resource, e2e_infrastructure
phasestring*requiredenum: setup, execute, collect, evaluate, cleanup
retry_scopeall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: none, same_slot
statusall of*required
The failure's original classification, independent of the primary run status.
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: passed, hard_gate_failed, subject_error, judge_error, resource_limit, infrastructure_error
hard_gatesobject[]*required
… expand 4 nestedcollapse
dimensionstring*requiredenum: deliverable, structural_integrity, efficiency, robustness, e2e_infrastructure
instruction_adherenceany of
… expand 2 nestedcollapse
any of (2)
variant 1
analyzerany of
… expand 2 nestedcollapse
any of (2)
variant 1
input_sha256string*required
analyzer_usageany of
… expand 2 nestedcollapse
any of (2)
variant 1
input_tokensinteger· uint64min 0
latency_msinteger· uint64min 0
output_tokensinteger· uint64min 0
availabilitystring*requiredenum: available, unavailable, failed
requirementsobject[]
… expand 6 nestedcollapse
confidencenumber· double*required
instructionstring*required
judge_attemptsinteger· uint8min 0
judge_usageany of
… expand 2 nestedcollapse
any of (2)
variant 1
cache_read_tokensinteger· uint64min 0
cache_write_tokensinteger· uint64min 0
input_tokensinteger· uint64min 0
output_tokensinteger· uint64min 0
reasoning_tokensinteger· uint64min 0
markdown_executionany of
… expand 2 nestedcollapse
any of (2)
variant 1
behavior_sha256string*required
compiled_sha256string*required
materialized_plan_sha256string
phasesobject[]*required
… expand 6 nestedcollapse
input_sha256string*required
statusstring*requiredenum: pending, completed, failed, skipped
pipeline_completeboolean*required
prompt_sha256string*required
source_pathstring*required
source_sha256string*required
metricsany of
… expand 2 nestedcollapse
any of (2)
variant 1
by_sessionobject[]*required
… expand 16 nestedcollapse
cache_read_tokensinteger· uint64min 0
cache_write_tokensinteger· uint64min 0
depthinteger· uint32*requiredmin 0
function_call_errorsinteger· uint64*requiredmin 0
function_callsinteger· uint64*requiredmin 0
input_tokensinteger· uint64min 0
output_tokensinteger· uint64min 0
reasoning_tokensinteger· uint64min 0
session_idstring*required
transient_resumesinteger· uint64min 0
turnsinteger· uint64*requiredmin 0
validation_retriesinteger· uint64min 0
wake_resumesinteger· uint64min 0
root_session_idstring*required
totalsobject*required
… expand 13 nestedcollapse
cache_read_tokensinteger· uint64min 0
cache_write_tokensinteger· uint64min 0
function_call_errorsinteger· uint64*requiredmin 0
function_callsinteger· uint64*requiredmin 0
input_tokensinteger· uint64min 0
output_tokensinteger· uint64min 0
reasoning_tokensinteger· uint64min 0
sessionsinteger· uint64*requiredmin 0
transient_resumesinteger· uint64min 0
turnsinteger· uint64*requiredmin 0
validation_retriesinteger· uint64min 0
wake_resumesinteger· uint64min 0
objective_scoreinteger· uint8min 0
quality_score_completedinteger· uint8min 0
retry_attemptsobject[]
… expand 25 nestedcollapse
attempt_idstring*required
attempt_numberinteger· uint32*requiredmin 0
completionstring*requiredenum: completed, task_incomplete, undetermined
costobject*required
… expand 3 nestedcollapse
subject_usdnumber· double
deliverablesobject[]
… expand 12 nestedcollapse
artifactany of
… expand 2 nestedcollapse
any of (2)
variant 1
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
content_formatstring*requiredenum: json, text_utf8
content_sha256string*required
content_size_bytesinteger· uint64*requiredmin 0
invariantsobject[]*required
… expand 3 nestedcollapse
media_typestring*required
provenanceobject[]*required
… expand 3 nestedcollapse
provenance_validboolean*required
schema_validboolean*required
dimensionsobject[]
… expand 3 nestedcollapse
dimensionstring*requiredenum: deliverable, structural_integrity, efficiency, robustness, e2e_infrastructure
efficiencyany of
… expand 2 nestedcollapse
any of (2)
variant 1
child_sessionsinteger· uint64min 0
child_turnsinteger· uint64min 0
critical_path_msinteger· uint64min 0
effective_fan_outinteger· uint64min 0
function_call_errorsinteger· uint64min 0
function_callsinteger· uint64min 0
input_tokensinteger· uint64min 0
minimum_expected_workinteger· uint64*requiredmin 0
observed_complexityobject*required
… expand 11 nestedcollapse
ambiguity_levelinteger· uint64min 0
artifact_countinteger· uint64min 0
coordination_edgesinteger· uint64min 0
dependency_depthinteger· uint64min 0
external_systemsinteger· uint64min 0
parallel_branchesinteger· uint64min 0
planning_depthinteger· uint64min 0
state_transitionsinteger· uint64min 0
validation_loopsinteger· uint64min 0
wake_cyclesinteger· uint64min 0
observed_workinteger· uint64min 0
output_tokensinteger· uint64min 0
root_turnsinteger· uint64min 0
technical_attemptsinteger· uint32*requiredmin 0
total_tokensinteger· uint64min 0
transient_resumesinteger· uint64min 0
validation_retriesinteger· uint64min 0
wake_resumesinteger· uint64min 0
wall_time_msinteger· uint64*requiredmin 0
work_amplificationnumber· double
evaluatorsobject*required
… expand 3 nestedcollapse
completionstring*requiredenum: not_required, pending, available, unavailable
final_advisorystring*requiredenum: not_required, pending, available, unavailable
qualitystring*requiredenum: not_required, pending, available, unavailable
evidenceobject[]
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
failuresobject[]*required
… expand 7 nestedcollapse
contaminationall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: none, case, execution
domainstring*requiredenum: subject, judge, resource, e2e_infrastructure
phasestring*requiredenum: setup, execute, collect, evaluate, cleanup
retry_scopeall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: none, same_slot
statusall of*required
The failure's original classification, independent of the primary run status.
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: passed, hard_gate_failed, subject_error, judge_error, resource_limit, infrastructure_error
instruction_adherenceany of
… expand 2 nestedcollapse
any of (2)
variant 1
analyzerany of
… expand 2 nestedcollapse
any of (2)
variant 1
input_sha256string*required
analyzer_usageany of
… expand 2 nestedcollapse
any of (2)
variant 1
input_tokensinteger· uint64min 0
latency_msinteger· uint64min 0
output_tokensinteger· uint64min 0
availabilitystring*requiredenum: available, unavailable, failed
requirementsobject[]
… expand 6 nestedcollapse
confidencenumber· double*required
instructionstring*required
judge_usageany of
… expand 2 nestedcollapse
any of (2)
variant 1
cache_read_tokensinteger· uint64min 0
cache_write_tokensinteger· uint64min 0
input_tokensinteger· uint64min 0
output_tokensinteger· uint64min 0
reasoning_tokensinteger· uint64min 0
markdown_executionany of
… expand 2 nestedcollapse
any of (2)
variant 1
behavior_sha256string*required
compiled_sha256string*required
materialized_plan_sha256string
phasesobject[]*required
… expand 6 nestedcollapse
input_sha256string*required
statusstring*requiredenum: pending, completed, failed, skipped
pipeline_completeboolean*required
prompt_sha256string*required
source_pathstring*required
source_sha256string*required
metricsany of
… expand 2 nestedcollapse
any of (2)
variant 1
by_sessionobject[]*required
… expand 16 nestedcollapse
cache_read_tokensinteger· uint64min 0
cache_write_tokensinteger· uint64min 0
depthinteger· uint32*requiredmin 0
function_call_errorsinteger· uint64*requiredmin 0
function_callsinteger· uint64*requiredmin 0
input_tokensinteger· uint64min 0
output_tokensinteger· uint64min 0
reasoning_tokensinteger· uint64min 0
session_idstring*required
transient_resumesinteger· uint64min 0
turnsinteger· uint64*requiredmin 0
validation_retriesinteger· uint64min 0
wake_resumesinteger· uint64min 0
root_session_idstring*required
totalsobject*required
… expand 13 nestedcollapse
cache_read_tokensinteger· uint64min 0
cache_write_tokensinteger· uint64min 0
function_call_errorsinteger· uint64*requiredmin 0
function_callsinteger· uint64*requiredmin 0
input_tokensinteger· uint64min 0
output_tokensinteger· uint64min 0
reasoning_tokensinteger· uint64min 0
sessionsinteger· uint64*requiredmin 0
transient_resumesinteger· uint64min 0
turnsinteger· uint64*requiredmin 0
validation_retriesinteger· uint64min 0
wake_resumesinteger· uint64min 0
objective_scoreinteger· uint8min 0
quality_score_completedinteger· uint8min 0
scenario_flowany of
… expand 2 nestedcollapse
any of (2)
variant 1
checkpointobject*required
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
cleanupobject*required
… expand 3 nestedcollapse
duration_msinteger· uint64*requiredmin 0
statusstring*requiredenum: succeeded, failed
definition_sha256string*required
semantic_testsobject[]
… expand 22 nestedcollapse
activationone of*required
… expand 3 nestedcollapse
one of (3)
variant 1
policystring*requiredenum: always
variant 2
conditionsobject[]*required
… expand 3 nestedcollapse
policystring*requiredenum: all
variant 3
conditionsobject[]*required
… expand 3 nestedcollapse
policystring*requiredenum: any
assetsobject[]
… expand 9 nestedcollapse
artifactobject*required
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
content_sha256string*required
media_typestring*required
namespaced_idstring*required
preview_truncatedboolean*required
size_bytesinteger· uint64*requiredmin 0
dependenciesstring[]*required
dependency_policystring*requiredenum: succeeded, terminal
duration_msinteger· uint64*requiredmin 0
evaluationsobject[]
… expand 5 nestedcollapse
outcomestring*requiredenum: passed, failed, advisory, not_evaluated
failuresobject[]
… expand 3 nestedcollapse
phasestring*requiredenum: preflight, execute, capture, persist, evaluate, cleanup, …
technicalboolean*required
hard_gatesobject[]
… expand 4 nestedcollapse
redactionall of
… expand 1 nestedcollapse
all of (1)
variant 1
redacted_fieldsinteger· uint32*requiredmin 0
redacted_valuesinteger· uint32*requiredmin 0
statusstring*requiredenum: pending, running, succeeded, hard_gate_failed, failed, skipped, …
step_versioninteger· uint32*requiredmin 0
session_idstring*required
statusstring*requiredenum: passed, hard_gate_failed, subject_error, judge_error, resource_limit, infrastructure_error
technicalstring*requiredenum: valid, technical_invalid
validation_scoreinteger· uint8min 0
wall_time_msinteger· uint64*requiredmin 0
scenario_flowany of
Evidence-only flow identity emitted by Rust. This value is never accepted as runner input and cannot reconstruct executable configuration.
… expand 2 nestedcollapse
any of (2)
variant 1
checkpointobject*required
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
cleanupobject*required
… expand 3 nestedcollapse
duration_msinteger· uint64*requiredmin 0
statusstring*requiredenum: succeeded, failed
definition_sha256string*required
scenario_measurementsobject[]
… expand 4 nestedcollapse
originstring*requiredenum: observed, derived_from_observed
valuenumber· double*required
semantic_testsobject[]
Semantic tests executed inside a code-defined composite scenario. Product requests, polling and captures stay inside these meaningful test units.
… expand 22 nestedcollapse
activationone of*required
… expand 3 nestedcollapse
one of (3)
variant 1
policystring*requiredenum: always
variant 2
conditionsobject[]*required
… expand 3 nestedcollapse
policystring*requiredenum: all
variant 3
conditionsobject[]*required
… expand 3 nestedcollapse
policystring*requiredenum: any
assetsobject[]
… expand 9 nestedcollapse
artifactobject*required
… expand 6 nestedcollapse
media_typestring*required
size_bytesinteger· uint64*requiredmin 0
content_sha256string*required
media_typestring*required
namespaced_idstring*required
preview_truncatedboolean*required
size_bytesinteger· uint64*requiredmin 0
dependenciesstring[]*required
dependency_policystring*requiredenum: succeeded, terminal
duration_msinteger· uint64*requiredmin 0
evaluationsobject[]
… expand 5 nestedcollapse
outcomestring*requiredenum: passed, failed, advisory, not_evaluated
failuresobject[]
… expand 3 nestedcollapse
phasestring*requiredenum: preflight, execute, capture, persist, evaluate, cleanup, …
technicalboolean*required
hard_gatesobject[]
… expand 4 nestedcollapse
redactionall of
… expand 1 nestedcollapse
all of (1)
variant 1
redacted_fieldsinteger· uint32*requiredmin 0
redacted_valuesinteger· uint32*requiredmin 0
statusstring*requiredenum: pending, running, succeeded, hard_gate_failed, failed, skipped, …
step_versioninteger· uint32*requiredmin 0
session_idstring*required
statusstring*requiredenum: passed, hard_gate_failed, subject_error, judge_error, resource_limit, infrastructure_error
technicalstring*requiredenum: valid, technical_invalid
validation_scoreinteger· uint8min 0
Explicit Markdown validation score. Mirrors `score` for Markdown-authored scenarios while keeping the legacy aggregate field compatible.
wall_time_msinteger· uint64*requiredmin 0
worker_contractsobject[]
Exact request/response schemas observed after this scenario's setup. Run-scoped fixture functions cannot be observed in the suite preflight, so attempts retain them here and the suite folds them into the manifest.
… expand 3 nestedcollapse
function_idstring*required
request_schema_sha256string*required
response_schema_sha256string*required
scenario_idstring*required
scenario_versioninteger· uint32min 0
schema_versioninteger· uint32*requiredmin 0
scoring_profile_sha256string*required
slot_start_deadline_secondsinteger· uint64min 0
Soft scheduling deadline: prevents starting slots after this interval.
subjectobject*required
… expand 6 nestedcollapse
context_windowinteger· uint64*requiredmin 0
max_output_tokensinteger· uint64*requiredmin 0
system_under_testobject*required
… expand 7 nestedcollapse
contract_hashesobject*required
e2e_repositorystring*required
e2e_revisionstring*required
engine_versionstring*required
harness_versionstring*required
stackone of*required
… expand 2 nestedcollapse
one of (2)
variant 1
modestring*requiredenum: source
workers_repositorystring*required
workers_revisionstring*required
variant 2
modestring*requiredenum: registry
stack_lock_digeststring*required
stack_versionsobject*required
request_sha256string*required
run_contract_sha256string
e2e::results-list
functionList durable E2E executions using comparable identity filters.
limitinteger· uint16min 0
executionsobject[]*required
active_attemptany of
any of (2)
variant 1
attempt_idstring*required
resume_state_sha256string
scenario_idstring*required
session_idstring*required
cancel_requestedboolean*required
execution_idstring*required
journal_progressobject*required
attempts_finishedinteger· uint64*requiredmin 0
attempts_startedinteger· uint64*requiredmin 0
committed_eventsinteger· uint64*requiredmin 0
planned_slotsinteger· uint64*requiredmin 0
runs_committedinteger· uint64*requiredmin 0
slots_deferredinteger· uint64*requiredmin 0
subject_observations_committedinteger· uint64*requiredmin 0
terminal_stateany of
… expand 2 nestedcollapse
any of (2)
variant 1
valuestringenum: completed, failed, cancelled, needs_reconciliation, unsupported
phasestring*requiredenum: requested, admitted, preflighting, materializing, setting_up, executing, …
transitionsobject[]*required
phasestring*requiredenum: requested, admitted, preflighting, materializing, setting_up, executing, …
updated_atstring*required
e2e::retention-sweep
functionDelete expired E2E objects and tombstone their history records.
limitinteger· uint16min 0
deleted_archivesstring[]*required
inspectedinteger· uint32*requiredmin 0
Admit an asynchronous, idempotent E2E execution.
audit_modelstring
Opt-in behavioral audit analyzer; supply model and provider together.
idempotency_keystring*required
progress_interval_secondsinteger· uint64min 0
rotating_seedsinteger[]· uint64min 0
run_contractany of
any of (2)
variant 1
attemptinteger· uint32*requiredmin 0
correlationobject*required
… expand 3 nestedcollapse
deployment_idstring*required
operation_idstring*required
modeobject*required
… expand 2 nestedcollapse
decisionstring*requiredenum: observe_only
environmentstring*requiredenum: demonstration
planobject*required
… expand 4 nestedcollapse
catalog_sha256string*required
runnerobject*required
… expand 3 nestedcollapse
selected_casesobject[]*required
… expand 6 nestedcollapse
contract_sha256string*required
inputs_sha256string*required
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
targetobject*required
… expand 3 nestedcollapse
applicationstring*required
stackone of*required
… expand 2 nestedcollapse
one of (2)
variant 1
modestring*requiredenum: source
workers_repositorystring*required
workers_revisionstring*required
variant 2
modestring*requiredenum: registry
stack_lock_digeststring*required
stack_versionsobject*required
slot_start_deadline_secondsinteger· uint64min 0
Soft scheduling budget, resolved before request hashing/admission.
technical_retriesinteger· uint8min 0
duplicateboolean*required
execution_idstring*required
phasestring*requiredenum: requested, admitted, preflighting, materializing, setting_up, executing, …
request_sha256string*required
run_contract_sha256string
Explain how the Harness can author a local Markdown E2E test, including the exact template, validation rules, persistence behavior, and separate create/list/run workflow. Call this before e2e::scenarios-create.
create_functionstring*required
Function that validates and persists one new local definition.
creation_starts_executionboolean*required
Always false: persistence and execution are separate operations.
file_name_rulesstring[]*required
File-name requirements enforced by the worker.
list_functionstring*required
Function that returns the complete built-in and local scenario catalog.
max_source_bytesinteger· uint*requiredmin 0
Maximum accepted UTF-8 Markdown source size.
overwrites_existing_fileboolean*required
Existing files are never silently replaced.
required_h2_sectionsstring[]*required
Required H2 headings, in their exact order.
run_functionstring*required
Separate function used only when the user explicitly asks to execute.
schemastring*required
Versioned shape of this guidance response.
storage_directorystring*required
Worker-data-relative directory where definitions are persisted.
summarystring*required
Short statement of what local authoring is for.
templatestring*required
Copy-ready valid Markdown source.
validation_rulesstring[]*required
Rules for weighted H3 validation criteria.
workflowstring[]*required
Recommended safe sequence for an agent authoring a local test.
e2e::scenarios-create
functionCreate one local Markdown E2E test after validating its complete authoring contract. This persists a definition outside Git and never starts an execution; call e2e::run separately only when execution is explicitly requested.
file_namestring*required
One UTF-8 `.md` file name, without a directory. Use letters, numbers, spaces, hyphens, or underscores; its stem becomes `local_<safe_id>`. Existing files are rejected instead of overwritten.
sourcestring*required
Complete Markdown test definition. It must contain one H1 followed by the required H2 sections in order and contain positive `### Name (N%)` validations totaling exactly 100%.
scenarioobject*required
author_versioninteger· uint32min 0
characterizationobject*required
human_horizonall of
… expand 1 nestedcollapse
all of (1)
variant 1
basisall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: unknown, author_estimate, measured
max_minutesinteger· uint32min 0
min_minutesinteger· uint32min 0
realismall of
… expand 1 nestedcollapse
all of (1)
variant 1
executionall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: synthetic, realistic_simulator, frozen_real_artifact
shadowall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: none, read_only
classificationobject*required
methodstring*requiredenum: capability_v2
profileobject*required
… expand 16 nestedcollapse
agent_owned_decompositionboolean
ambiguity_levelinteger· uint8*requiredmin 0
artifact_countinteger· uint8*requiredmin 0
coherent_long_horizonboolean
compensable_mutationsinteger· uint8min 0
coordination_edgesinteger· uint16*requiredmin 0
dependency_depthinteger· uint8*requiredmin 0
durable_resume_cyclesinteger· uint8min 0
external_systemsinteger· uint8*requiredmin 0
material_invalidation_eventsinteger· uint8min 0
parallel_branchesinteger· uint8*requiredmin 0
planning_depthinteger· uint8*requiredmin 0
replan_loopsinteger· uint8min 0
state_transitionsinteger· uint16*requiredmin 0
validation_loopsinteger· uint8*requiredmin 0
wake_cyclesinteger· uint8*requiredmin 0
tierstring*requiredenum: l0_atomic, l1_sequential, l2_stateful, l3_concurrent, l4_coordinated, l5_adaptive
complexityobject*required
agent_owned_decompositionboolean
ambiguity_levelinteger· uint8*requiredmin 0
artifact_countinteger· uint8*requiredmin 0
coherent_long_horizonboolean
compensable_mutationsinteger· uint8min 0
coordination_edgesinteger· uint16*requiredmin 0
dependency_depthinteger· uint8*requiredmin 0
durable_resume_cyclesinteger· uint8min 0
external_systemsinteger· uint8*requiredmin 0
material_invalidation_eventsinteger· uint8min 0
parallel_branchesinteger· uint8*requiredmin 0
planning_depthinteger· uint8*requiredmin 0
replan_loopsinteger· uint8min 0
state_transitionsinteger· uint16*requiredmin 0
validation_loopsinteger· uint8*requiredmin 0
wake_cyclesinteger· uint8*requiredmin 0
contract_sha256string*required
deliverable_contractobject*required
artifactsobject[]*required
… expand 5 nestedcollapse
max_size_bytesinteger· uint64*requiredmin 0
media_typestring*required
capture_before_cleanupboolean*required
invariantsobject[]*required
… expand 2 nestedcollapse
descriptionstring*required
provenance_requiredboolean*required
inputs_sha256string*required
originstring*requiredenum: built_in, markdown, local
required_capabilitiesstring[]*required
resource_envelopeobject*required
executionobject*required
… expand 5 nestedcollapse
max_output_tokensinteger· uint64min 0
max_total_tokensinteger· uint64min 0
Shared Harness token budget. `None` leaves the Harness budget unbounded for this scenario.
max_turnsinteger· uint32*requiredmin 0
max_validation_retriesinteger· uint32min 0
Optional per-session cap for post-turn validation denials. `None` preserves the Harness-configured default.
stuck_timeout_secondsinteger· uint64*requiredmin 0
Stop only after this many seconds without observable useful progress. Large scenarios have no fixed wall-clock deadline.
workflowany of
… expand 2 nestedcollapse
any of (2)
variant 1
max_cost_usdnumber· double
max_nodesinteger· uint16*requiredmin 0
max_parallelinteger· uint16*requiredmin 0
max_total_tokensinteger· uint64min 0
step_timeout_secondsinteger· uint64*requiredmin 0
technical_retriesinteger· uint8*requiredmin 0
workflow_timeout_secondsinteger· uint64*requiredmin 0
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
e2e::scenarios-list
functionList every built-in, committed Markdown, and local E2E test with its immutable contract, origin, capabilities, complexity, and materialized case. Call e2e::scenarios-authoring-guide before authoring a new local test.
seedinteger· uint64min 0
Optional deterministic seed used to materialize each listed test case. Omit it to use each scenario's canonical seed.
catalog_sha256string*required
scenariosobject[]*required
author_versioninteger· uint32min 0
characterizationobject*required
human_horizonall of
… expand 1 nestedcollapse
all of (1)
variant 1
basisall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: unknown, author_estimate, measured
max_minutesinteger· uint32min 0
min_minutesinteger· uint32min 0
realismall of
… expand 1 nestedcollapse
all of (1)
variant 1
executionall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: synthetic, realistic_simulator, frozen_real_artifact
shadowall of
… expand 1 nestedcollapse
all of (1)
variant 1
valuestringenum: none, read_only
classificationobject*required
methodstring*requiredenum: capability_v2
profileobject*required
… expand 16 nestedcollapse
agent_owned_decompositionboolean
ambiguity_levelinteger· uint8*requiredmin 0
artifact_countinteger· uint8*requiredmin 0
coherent_long_horizonboolean
compensable_mutationsinteger· uint8min 0
coordination_edgesinteger· uint16*requiredmin 0
dependency_depthinteger· uint8*requiredmin 0
durable_resume_cyclesinteger· uint8min 0
external_systemsinteger· uint8*requiredmin 0
material_invalidation_eventsinteger· uint8min 0
parallel_branchesinteger· uint8*requiredmin 0
planning_depthinteger· uint8*requiredmin 0
replan_loopsinteger· uint8min 0
state_transitionsinteger· uint16*requiredmin 0
validation_loopsinteger· uint8*requiredmin 0
wake_cyclesinteger· uint8*requiredmin 0
tierstring*requiredenum: l0_atomic, l1_sequential, l2_stateful, l3_concurrent, l4_coordinated, l5_adaptive
complexityobject*required
agent_owned_decompositionboolean
ambiguity_levelinteger· uint8*requiredmin 0
artifact_countinteger· uint8*requiredmin 0
coherent_long_horizonboolean
compensable_mutationsinteger· uint8min 0
coordination_edgesinteger· uint16*requiredmin 0
dependency_depthinteger· uint8*requiredmin 0
durable_resume_cyclesinteger· uint8min 0
external_systemsinteger· uint8*requiredmin 0
material_invalidation_eventsinteger· uint8min 0
parallel_branchesinteger· uint8*requiredmin 0
planning_depthinteger· uint8*requiredmin 0
replan_loopsinteger· uint8min 0
state_transitionsinteger· uint16*requiredmin 0
validation_loopsinteger· uint8*requiredmin 0
wake_cyclesinteger· uint8*requiredmin 0
contract_sha256string*required
deliverable_contractobject*required
artifactsobject[]*required
… expand 5 nestedcollapse
max_size_bytesinteger· uint64*requiredmin 0
media_typestring*required
capture_before_cleanupboolean*required
invariantsobject[]*required
… expand 2 nestedcollapse
descriptionstring*required
provenance_requiredboolean*required
inputs_sha256string*required
originstring*requiredenum: built_in, markdown, local
required_capabilitiesstring[]*required
resource_envelopeobject*required
executionobject*required
… expand 5 nestedcollapse
max_output_tokensinteger· uint64min 0
max_total_tokensinteger· uint64min 0
Shared Harness token budget. `None` leaves the Harness budget unbounded for this scenario.
max_turnsinteger· uint32*requiredmin 0
max_validation_retriesinteger· uint32min 0
Optional per-session cap for post-turn validation denials. `None` preserves the Harness-configured default.
stuck_timeout_secondsinteger· uint64*requiredmin 0
Stop only after this many seconds without observable useful progress. Large scenarios have no fixed wall-clock deadline.
workflowany of
… expand 2 nestedcollapse
any of (2)
variant 1
max_cost_usdnumber· double
max_nodesinteger· uint16*requiredmin 0
max_parallelinteger· uint16*requiredmin 0
max_total_tokensinteger· uint64min 0
step_timeout_secondsinteger· uint64*requiredmin 0
technical_retriesinteger· uint8*requiredmin 0
workflow_timeout_secondsinteger· uint64*requiredmin 0
scenario_idstring*required
scenario_versioninteger· uint32*requiredmin 0
seedinteger· uint64*requiredmin 0
Read the durable phase and progress of an E2E execution.
execution_idstring*required
active_attemptany of
any of (2)
variant 1
attempt_idstring*required
resume_state_sha256string
scenario_idstring*required
session_idstring*required
cancel_requestedboolean*required
execution_idstring*required
journal_progressobject*required
attempts_finishedinteger· uint64*requiredmin 0
attempts_startedinteger· uint64*requiredmin 0
committed_eventsinteger· uint64*requiredmin 0
planned_slotsinteger· uint64*requiredmin 0
runs_committedinteger· uint64*requiredmin 0
slots_deferredinteger· uint64*requiredmin 0
subject_observations_committedinteger· uint64*requiredmin 0
terminal_stateany of
any of (2)
variant 1
valuestringenum: completed, failed, cancelled, needs_reconciliation, unsupported
phasestring*requiredenum: requested, admitted, preflighting, materializing, setting_up, executing, …
transitionsobject[]*required
phasestring*requiredenum: requested, admitted, preflighting, materializing, setting_up, executing, …
updated_atstring*required
Serve the Harness E2E injectable Console assets.
content_typestring*required
