skip to content
$worker

iii-observability

v0.19.7-next.1

OpenTelemetry-based traces, metrics, logs, alerts, and sampling.

iiiverified
8,665 installs64 in 7d0 today
install
$iii worker add iii-observability@0.19.7-next.1

exact versions are immutable; binary and bundle artifacts are digest-pinned.

engine module
baked into the iii engine; no separate install required.

functions

21

engine::alerts::evaluate

function

Force an immediate alert-rule evaluation against current metrics and return any triggered_alerts, bypassing the periodic tick. Returns evaluated:false when the alert manager is not initialized; produces nothing without configured rules.

request
empty object
response
  • evaluatedbooleanrequired
  • messagestring
  • timestampinteger· int64required
  • triggered_alertsunknown[]

    Alerts triggered by this evaluation pass (present when the manager is initialized).

engine::alerts::list

function

List configured alert rules with their current evaluated states and a firing_count. Returns empty when no alert rules are configured or the alert manager is not initialized.

request
empty object
response
  • alertsunknown[]required

    Current evaluated alert states (serialized).

  • firing_countinteger· uintrequiredmin 0
  • messagestring
  • rulesobject[]

    Configured alert rules (present when the alert manager is initialized).

    • actionall of

      Action to take when alert triggers (Log, Webhook { url }, Function { path })

      all of (1)
      variant 1
      one of (3)
      variant 1
      • typestringrequiredenum: log
      variant 2
      • typestringrequiredenum: webhook
      • urlstringrequired

        The webhook URL to send the alert to

      variant 3
      • pathstringrequired

        The function path to invoke

      • typestringrequiredenum: function
    • cooldown_secondsinteger· uint64min 1

      Minimum interval between alert triggers in seconds (default: 60)

    • enabledboolean

      Whether the alert is enabled (default: true)

    • metricstringrequired

      Metric name to monitor (e.g., "iii.invocations.error")

    • namestringrequired

      Name of the alert (for identification)

    • operatorall of

      Comparison operator (>, >=, <, <=, ==, !=)

      all of (1)
      variant 1
      valuestringenum: greaterthan, greaterthanorequal, lessthan, lessthanorequal, equal, notequal
    • thresholdnumber· doublerequired

      Threshold value for the alert

    • window_secondsinteger· uint64min 1

      Time window in seconds to evaluate the metric (default: 60)

  • timestampinteger· int64required

engine::baggage::get

function

Read one baggage entry by key from the current OTEL context, returning { value } (null if unset). Diagnostic only: reads ambient process context, not per-invocation baggage.

request
  • keystringrequired

    The baggage key to retrieve

response
  • valuestring

engine::baggage::get_all

function

Read all baggage entries from the current OTEL context as a { baggage } map. Diagnostic only: reflects ambient process context, not per-invocation baggage.

request
empty object
response
  • baggageobjectrequired

engine::baggage::set

function

Set a baggage key/value on a fresh OTEL context for verification only; the new context is not propagated to the caller or global state. Use SDK-level headers for real propagation.

request
  • keystringrequired

    The baggage key to set

  • valuestringrequired

    The baggage value to set

response
  • notestringrequired
  • successbooleanrequired

engine::health::check

function

Report observability subsystem health: per-component status (otel, metrics, logs, spans) marked healthy with counts or disabled, plus engine version. Always succeeds regardless of configuration.

request
empty object
response
  • componentsobjectrequired
    • logsunknownrequired
    • metricsunknownrequired
    • otelunknownrequired

      Each component is `{ status: "healthy"|"disabled", details: <object|null> }`.

    • spansunknownrequired
  • statusstringrequired
  • timestampinteger· int64required
  • versionstringrequired

engine::log::debug

function

Record a DEBUG-level OTEL log (severity 5) with optional trace/span correlation and structured data. No-op when logs_enabled is false or dropped by logs_sampling_ratio.

request
  • dataunknown

    Additional structured data/attributes

  • messagestringrequired

    The log message

  • service_namestring

    Service name (defaults to function name if not provided)

  • span_idstring

    Optional span ID for correlation

  • trace_idstring

    Optional trace ID for correlation

response
valueunknown

engine::log::error

function

Record an ERROR-level OTEL log (severity 17) with optional trace/span correlation and structured data. No-op when logs_enabled is false or dropped by logs_sampling_ratio.

request
  • dataunknown

    Additional structured data/attributes

  • messagestringrequired

    The log message

  • service_namestring

    Service name (defaults to function name if not provided)

  • span_idstring

    Optional span ID for correlation

  • trace_idstring

    Optional trace ID for correlation

response
valueunknown

engine::log::info

function

Record an INFO-level OTEL log (severity 9) with optional trace/span correlation and structured data. No-op when logs_enabled is false or dropped by logs_sampling_ratio.

request
  • dataunknown

    Additional structured data/attributes

  • messagestringrequired

    The log message

  • service_namestring

    Service name (defaults to function name if not provided)

  • span_idstring

    Optional span ID for correlation

  • trace_idstring

    Optional trace ID for correlation

response
valueunknown

engine::log::trace

function

Record a TRACE-level OTEL log (severity 1) with optional trace/span correlation and structured data. No-op when logs_enabled is false or dropped by logs_sampling_ratio.

request
  • dataunknown

    Additional structured data/attributes

  • messagestringrequired

    The log message

  • service_namestring

    Service name (defaults to function name if not provided)

  • span_idstring

    Optional span ID for correlation

  • trace_idstring

    Optional trace ID for correlation

response
valueunknown

engine::log::warn

function

Record a WARN-level OTEL log (severity 13) with optional trace/span correlation and structured data. No-op when logs_enabled is false or dropped by logs_sampling_ratio.

request
  • dataunknown

    Additional structured data/attributes

  • messagestringrequired

    The log message

  • service_namestring

    Service name (defaults to function name if not provided)

  • span_idstring

    Optional span ID for correlation

  • trace_idstring

    Optional trace ID for correlation

response
valueunknown

engine::logs::clear

function

Drop every stored OTEL log, returning { success: true }. Succeeds as a no-op when log storage was never initialized (logs_enabled false).

request
empty object
response
  • messagestring
  • successbooleanrequired

engine::logs::list

function

List stored OTEL logs filtered by trace/span id, severity, and time range, with pagination and a total count. Returns an empty result when logs_enabled is false (no log storage).

request
  • end_timeinteger· uint64min 0

    End time in Unix timestamp milliseconds

  • limitinteger· uintmin 0

    Maximum number of logs to return

  • offsetinteger· uintmin 0

    Pagination offset (default: 0)

  • severity_mininteger· int32

    Minimum severity number (1-24, higher = more severe)

  • severity_textstring

    Filter by severity text (e.g., "ERROR", "WARN", "INFO")

  • span_idstring

    Filter by span ID

  • start_timeinteger· uint64min 0

    Start time in Unix timestamp milliseconds

  • trace_idstring

    Filter by trace ID

response
  • logsunknown[]required

    Stored OTEL log records (serialized).

  • queryany of

    Echo of the applied filters (present only when storage exists).

    any of (2)
    variant 1
    • end_timeinteger· uint64min 0
    • limitinteger· uintmin 0
    • offsetinteger· uintmin 0
    • severity_mininteger· int32
    • severity_textstring
    • span_idstring
    • start_timeinteger· uint64min 0
    • trace_idstring
    variant 2
    valuenull
  • timestampinteger· int64required
  • totalinteger· uintrequiredmin 0

    Total matching logs before pagination.

engine::metrics::list

function

Return engine invocation/worker counters and span-derived latency percentiles, plus stored SDK metrics filtered by name/time and optionally aggregated by interval. engine_metrics is always present; sdk_metrics is empty when no metric storage exists.

request
  • aggregate_intervalinteger· uint64min 0

    Aggregate interval in seconds

  • end_timeinteger· uint64min 0

    End time in Unix timestamp milliseconds

  • metric_namestring

    Filter by metric name

  • start_timeinteger· uint64min 0

    Start time in Unix timestamp milliseconds

response
  • aggregated_metricsunknown[]required

    Time-bucketed aggregates (present only when an aggregate_interval was requested and produced data).

  • engine_metricsobjectrequired
    • invocationsobjectrequired
      • by_functionobjectrequired

        Per-function invocation counts.

      • deferredinteger· uint64requiredmin 0
      • errorinteger· uint64requiredmin 0
      • successinteger· uint64requiredmin 0
      • totalinteger· uint64requiredmin 0
    • performanceobjectrequired
      • avg_duration_msnumber· doublerequired
      • max_duration_msnumber· doublerequired
      • min_duration_msnumber· doublerequired
      • p50_duration_msnumber· doublerequired
      • p95_duration_msnumber· doublerequired
      • p99_duration_msnumber· doublerequired
    • workersobjectrequired
      • activeinteger· uint64requiredmin 0
      • deathsinteger· uint64requiredmin 0
      • spawnsinteger· uint64requiredmin 0
  • queryany of

    Echo of the applied query filters (present only when a filter was supplied).

    any of (2)
    variant 1
    • aggregate_intervalinteger· uint64min 0
    • end_timeinteger· uint64min 0
    • metric_namestring
    • start_timeinteger· uint64min 0
    variant 2
    valuenull
  • sdk_metricsunknown[]required

    Stored SDK metric points (serialized).

  • timestampinteger· int64required

engine::rollups::list

function

Return pre-aggregated metric rollups and histograms for a level (0=1m, 1=5m, 2=1h) over a time range (default last hour), optionally by metric name. Falls back to on-the-fly aggregation over metric storage when no rollup storage exists.

request
  • end_timeinteger· uint64min 0

    End time in Unix timestamp milliseconds

  • levelinteger· uintmin 0

    Rollup level index (0 = 1 min, 1 = 5 min, 2 = 1 hour)

  • metric_namestring

    Filter by metric name

  • start_timeinteger· uint64min 0

    Start time in Unix timestamp milliseconds

response
  • histogram_rollupsunknown[]required

    Pre-aggregated histogram rollups (serialized).

  • levelinteger· uintmin 0
  • messagestring
  • queryany of
    any of (2)
    variant 1
    • end_timeinteger· uint64min 0
    • metric_namestring
    • start_timeinteger· uint64min 0
    variant 2
    valuenull
  • rollupsunknown[]required

    Pre-aggregated metric rollups (serialized).

  • sourcestring

    "on_the_fly" when computed live because no rollup storage exists.

  • timestampinteger· int64required

engine::sampling::rules

function

Report the active trace sampling config (default ratio, per-operation/service rules, parent_based) and the logs sampling_ratio, read from live config. Defaults to ratio 1.0 with no rules when sampling is unconfigured.

request
empty object
response
  • logsobjectrequired
    • sampling_rationumber· doublerequired
  • timestampinteger· int64required
  • tracesobjectrequired
    • default_rationumber· doublerequired
    • parent_basedbooleanrequired
    • rulesobject[]required
      • operationstring
      • ratenumber· doublerequired
      • servicestring

engine::traces::clear

function

Drop every span from the in-memory trace store, returning { success: true }. Requires exporter memory or both, else fails memory_exporter_not_enabled.

request
empty object
response
  • successbooleanrequired

engine::traces::group_by

function

Aggregate stored spans by one attribute into groups (trace_ids, span_count, duration, error_count), newest-first, capped at limit (default 100); skips spans lacking the attribute and engine-internal spans unless include_internal. Requires exporter memory or both.

request
  • attributestringrequired

    Span attribute key to group by. Spans without this attribute are skipped.

  • include_internalboolean

    Include engine-internal spans. Defaults to false, matching `traces::list`.

  • limitinteger· uint32min 0

    Max groups returned after sorting by `first_seen_ms` descending. Default 100.

  • since_msinteger· uint64min 0

    Earliest end_time (ms since epoch) to include.

response
  • groupsobject[]required
    • duration_msinteger· uint32requiredmin 0
    • error_countinteger· uint32requiredmin 0
    • first_seen_msinteger· uint64requiredmin 0
    • last_seen_msinteger· uint64requiredmin 0
    • span_countinteger· uint32requiredmin 0
    • trace_idsstring[]required
    • valuestringrequired

engine::traces::list

function

List root spans of stored traces with filtering (service, name, status, duration, time, attributes), pagination, and sort; hides engine-internal spans unless include_internal. Requires exporter memory or both, else fails memory_exporter_not_enabled.

request
  • attributesstring[][]

    Filter by span attributes (array of [key, value] pairs, AND logic, exact match)

  • end_timeinteger· uint64min 0

    End time in unix timestamp milliseconds (include spans overlapping before this)

  • include_internalboolean

    Include internal engine traces (engine.* functions). Defaults to false.

  • limitinteger· uintmin 0

    Pagination limit (default: 100)

  • max_duration_msnumber· double

    Maximum span duration in milliseconds (sub-ms precision)

  • min_duration_msnumber· double

    Minimum span duration in milliseconds (sub-ms precision)

  • namestring

    Filter by span name (case-insensitive substring match)

  • offsetinteger· uintmin 0

    Pagination offset (default: 0)

  • search_all_spansboolean

    Search across all spans in each trace, not just root spans. When true and a `name` filter is set, traces are matched if ANY span in the trace matches the name filter. Defaults to false.

  • service_namestring

    Filter by service name (case-insensitive substring match)

  • sort_bystring

    Sort field: "start_time" | "duration" (alias "duration_ms") | "service_name" | "name" (default: "start_time"). Unknown values fall back to "start_time".

  • sort_orderstring

    Sort order: "asc" | "desc" (default: "asc")

  • start_timeinteger· uint64min 0

    Start time in unix timestamp milliseconds (include spans overlapping after this)

  • statusstring

    Filter by status (case-insensitive substring match)

  • trace_idstring

    Filter by specific trace ID

response
  • limitinteger· uintrequiredmin 0
  • offsetinteger· uintrequiredmin 0
  • spansunknown[]required

    Stored spans (each a serialized span record).

  • totalinteger· uintrequiredmin 0

    Total matching spans before pagination.

engine::traces::tree

function

Build the nested span tree for one trace_id as { roots }, pruning no-op trigger wrappers and collapsing configured pass-through spans. Requires exporter memory or both, else fails memory_exporter_not_enabled.

request
  • trace_idstringrequired

    Trace ID to build the tree for

response
  • rootsunknown[]required

    Root span-tree nodes (each a serialized, flattened span with nested children).

iii-observability::on-config-change

function

Internal: re-apply the iii-observability configuration when the authoritative configuration entry changes.

request
valueunknown
response
valueunknown

triggers

0
no triggers registered