# computer

> Drive a full desktop on the iii bus. Start a session on the local machine, a sandboxed desktop, or a remote one, screenshot it, click and type by coordinate, read app accessibility trees and act on elements by reference (native sessions), and stream the live screen.

| field | value |
|-------|-------|
| version | 0.1.20 |
| type | binary |
| license | Apache-2.0 |
| repo | https://github.com/iii-hq/workers |
| supported_targets | x86_64-apple-darwin, aarch64-apple-darwin, x86_64-pc-windows-msvc, aarch64-pc-windows-msvc, x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-unknown-linux-musl, armv7-unknown-linux-gnueabihf |
| author | iii |

## installation

```sh
iii trigger compose::add worker=computer@0.1.20
```

## readme

# computer

Drive a full desktop on the iii bus. Where the
[browser](https://github.com/iii-hq/workers/tree/main/browser) worker gives an
agent a Chromium tab, `computer` gives it a whole screen: it hands the model a
screenshot, clicks and scrolls at the coordinates it reads off that image, and
types wherever the desktop's focus is. The harness discovers
`computer::*` as functions automatically, so a model sees the screen and acts on
it with no glue, and the console gets a live viewport of whatever the desktop is
doing.

The worker does the one thing no other worker can: capture the screen and move
the cursor. Three drivers sit behind one surface, picked per session:

- **native** (no `endpoint`, no `image`): drive the local machine this worker
  runs on, capture and input injection built in.
- **sandbox** (an `image`): boot a fresh desktop inside an
  [iii-sandbox](https://github.com/iii-hq/workers/tree/main/iii-sandbox) microVM
  and drive it through iii primitives alone (`sandbox::exec` / `sandbox::fs`),
  with no socket into the guest. A fixed virtual display means 1:1 coordinates,
  no HiDPI or multi-monitor ambiguity, and nothing to grant on the host.
- **remote** (an `endpoint`): drive an already-running desktop through the
  executor inside it, over a WebSocket. `computer` connects; that desktop boots
  out of band.

Everything else composes with workers that already exist: run commands and touch
files with the [shell](https://github.com/iii-hq/workers/tree/main/ide)
worker, persist with `state`, schedule with `cron`.

## Install

```bash
iii trigger compose::add worker=computer
```

To drive a throwaway desktop instead of your own machine, add the sandbox worker
too — it boots the microVM the `image` driver runs in:

```bash
iii trigger compose::add worker=iii-sandbox
```

**macOS, native driver only.** Capture and input are permission-gated and macOS
degrades both silently, so the worker checks and fails loud. Grant the app that
runs the worker **Screen Recording** (without it every screenshot is the
wallpaper with all windows stripped) and **Accessibility** (without it clicks and
typing are dropped) in System Settings > Privacy & Security, then restart it. The
sandbox driver needs neither.

## Quickstart

Start a session, look at the screen, click what you see:

```bash
# drive this machine: a bare start picks the native driver when neither
# sandbox_image nor default_endpoint is configured
iii trigger computer::sessions::start --json '{}'
# -> { "session_id": "c1", "endpoint": "native", "os": "macos", "screen": { "width": 1512, "height": 945 } }

# see the screen: an image block the vision model renders inline
iii trigger computer::screenshot --json '{"session_id":"c1"}'

# click a pixel read off that screenshot (top-left origin)
iii trigger computer::act --json '{"session_id":"c1","action":"click","x":640,"y":360}'

iii trigger computer::sessions::stop --json '{"session_id":"c1"}'
```

Swap the first call for `{"image":"desktop"}` to boot a sandboxed Linux desktop
instead (see [Sandbox desktop image](#sandbox-desktop-image)), or
`{"endpoint":"ws://host:8000"}` to drive a remote one. Everything after that call
is identical: the session id is the only handle.

The same from a sibling worker:

```rust
use iii_sdk::protocol::TriggerRequest;
use iii_sdk::{register_worker, InitOptions};
use serde_json::json;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let iii = register_worker("ws://localhost:49134", InitOptions::default());

    let started = iii
        .trigger(TriggerRequest {
            function_id: "computer::sessions::start".into(),
            payload: json!({}),
            action: None,
            timeout_ms: Some(30_000),
        })
        .await?;
    let session_id = started["session_id"].as_str().unwrap();

    iii.trigger(TriggerRequest {
        function_id: "computer::act".into(),
        payload: json!({ "session_id": session_id, "action": "click", "x": 640, "y": 360 }),
        action: None,
        timeout_ms: Some(10_000),
    })
    .await?;

    Ok(())
}
```

The rest of the surface: `computer::act` — pointer actions addressed by
coordinate (click, right_click, double_click, move, drag, scroll) and keyboard
actions that land wherever the desktop's focus is (`type` with `text`,
`press` / `hotkey` with `keys`, e.g. `["cmd","c"]`) —
`computer::observe` (screenshot plus the accessibility tree on macOS guests),
`computer::displays`, `computer::sessions::list` and `computer::sessions::stop`.
For commands and files, mind which machine you mean: on a native session the
desktop IS this host, so the
[shell](https://github.com/iii-hq/workers/tree/main/ide) worker is the right
tool. A sandboxed desktop is reached with `sandbox::exec` / `sandbox::fs`, and
a remote one through its own guest executor — `shell` would run on the host
instead, which is a different machine entirely.

## Console

The worker ships its own console page (page `computer`): a session rail, a live
viewport fed by the live screencast, and click / type / scroll forwarding
straight into the desktop. It is injected into any running console at
registration time — nothing to install, nothing to rebuild. Every `computer::*`
call in chat and traces renders through the same asset.

## Configuration

Stored in the `configuration` worker under the `computer` key; every field is
editable live from the console. The screencast rate applies to running
sessions; everything a driver is built with — endpoint, OS label, timeouts,
capture limits, and the sandbox display and network settings — is read at
`sessions::start`, so a change reaches sessions started after it.

```yaml
computer:
  default_endpoint: ''      # guest-executor endpoint for a remote desktop; empty = drive the local machine
  os: linux                 # guest OS label recorded on sessions
  max_sessions: 2           # concurrent desktop connections
  idle_stop_ms: 300000      # stop sessions idle this long; 0 disables
  screencast_fps: 15        # live-view frame rate cap
  max_screenshot_dimension: 1280 # downscale screenshots/frames to this longest edge (native)
  screenshot_quality: 70    # JPEG quality 1-100 (native)
  command_timeout_ms: 120000 # timeout for each driver action (fixed at connect)
  connect_timeout_ms: 15000  # driver connect timeout at session start
  sandbox_image: ''          # iii-sandbox image for a sandbox session; empty = name it per call
  sandbox_width: 1280        # sandbox virtual display width
  sandbox_height: 800        # sandbox virtual display height
  sandbox_network: true      # give the sandboxed desktop network access; false keeps it offline
  sandbox_idle_timeout_secs: 86400 # idle_timeout_secs for sandbox::create; kept high, the worker owns teardown
  screen_capture_preflight: true # macOS: ask for Screen Recording at native start, fail loud if denied
```

## Sandbox desktop image

The sandbox driver needs a desktop inside the guest (Xvfb, xdotool, imagemagick,
openbox). A prebaked image lives in [`images/desktop`](images/desktop): build it,
push it, register it as a `custom_images` entry on the iii-sandbox worker, then
start a session with `{ "image": "desktop" }`. iii-sandbox boots on macOS Apple
Silicon (libkrun) and Linux (`/dev/kvm`).

## Custom trigger types

Sibling workers (and the console page) subscribe to session activity instead of
polling. Both bindings accept an optional `{ "session_id": "..." }` filter.

| Trigger type | Fires when | Payload to subscribers |
|---|---|---|
| `computer::session-started` | A session connected and is ready | `{ session_id, endpoint, os, screen, timestamp }` |
| `computer::session-stopped` | A session ended | `{ session_id, reason: "stopped" \| "idle", timestamp }` |

## Live screen

`computer::screencast::start` / `stop`, `computer::frame` and the
`computer::frame-changed` trigger type drive the live viewport. The worker
captures at `screencast_fps` and keeps only the newest frame per session in
memory (no history). After each frame it fires `computer::frame-changed` with a
small, image-free notification; the viewer reads the frame with
`computer::frame` (notify, then fetch), so the console and any number of
watchers follow the desktop without polling, and a slow watcher skips frames
instead of queueing them.

| Trigger type | Fires when | Payload to subscribers |
|---|---|---|
| `computer::frame-changed` | A new frame was stored (`change: "updated"`), or the frame was dropped because the screencast or session stopped (`change: "cleared"`) | `{ session_id, epoch, frame_seq, change, width, height, mime?, bytes?, timestamp }` |

The binding config `{ "session_id": "..." }` is **required** (a viewer follows
one desktop). Order frames by `(epoch, frame_seq)`; `epoch` changes when the
worker restarts or restores the session. Bind first, then read
`computer::frame` once for the current frame; read it again after each
notification (pass `since_frame` to skip an unchanged image) and after a
reconnect. These are internal console plumbing rather than agent surface, and
stay out of agent function lists.

## skills

This version ships 1 skill file for agents, also browsable on the worker page:

- [`SKILL.md`](https://workers.iii.dev/workers/computer?tab=skills&skill=SKILL.md&version=0.1.20)

### SKILL.md

# computer

The computer worker turns a live desktop into iii functions. Start a session
with neither `image` nor `endpoint` to drive the local machine this worker runs
on (unless the operator configured a default image or endpoint, which is
resolved first), pass an `image` to boot a sandboxed desktop, or an `endpoint`
to drive a remote one. Take a
screenshot to see the screen, then act on it by pixel coordinate: the screenshot
is the source of truth for where things are, and `computer::act` clicks and
types at those coordinates. The session stays alive, so you can act, screenshot
the result, and act again.

Sessions usually survive a worker restart (it reconnects them best-effort, so
an id can still go away — start a new session if one does) and cost one driver
connection each; the configured session cap is small. Stop sessions when a task
is done. Screenshots are large; take one when you need to see the screen, not
after every action.

## When to Use

- Operating a desktop GUI application that is not a web page (an installer, a
  native app, a settings panel).
- End-to-end tasks that span the whole screen: move a window, drag between
  apps, use a system dialog.
- Watching the effect of a change on screen: run the command with the `shell`
  worker, then `computer::screenshot` to see the result.

## Boundaries

- Web-only tasks belong to the
  [browser](https://github.com/iii-hq/workers/tree/main/browser) worker (a real
  Chromium tab with an accessibility outline and page console) or `web::fetch`
  for a one-shot page. Do not start a desktop session just to open a URL.
- Shell and files are not this worker's job; `computer` only sees the screen
  and drives the cursor. Reach for `shell::exec` / `shell::fs::*` when the
  session is native (the desktop is that host), `sandbox::exec` /
  `sandbox::fs` for a sandboxed desktop, and the guest's own executor for a
  remote one. `shell` always runs on the worker's host, not inside a guest.
- With neither argument it drives whatever the configuration defaults to, and
  the local machine when nothing is configured; an `image` boots a sandboxed
  desktop through the iii-sandbox worker; an `endpoint` connects to a desktop
  somebody else booted.
- `computer::screencast::*` and `computer::frame` are console-UI plumbing, not
  agent surface.
- Coordinates are integer pixels, top-left origin, in the space of the most
  recent screenshot. Re-screenshot after the screen changes before acting.

## Functions

- `computer::sessions::start` — connect a desktop session; returns the
  session_id every session-scoped function needs (all but
  `computer::sessions::list` and `computer::displays`), plus the screen size.
- `computer::sessions::list` — live sessions with endpoint, OS, and screen.
- `computer::sessions::stop` — stop a session; idempotent.
- `computer::screenshot` — the desktop as a viewable image; how you see the
  screen before acting.
- `computer::observe` — screenshot plus, on macOS guests, the accessibility
  tree (`include_a11y: true`).
- `computer::act` — click, right_click, double_click, move, drag and scroll are
  addressed by pixel coordinates; type, press and hotkey carry their own `text`
  or `keys` and land wherever the desktop's focus is.

## Keeping context small

Desktop screenshots are large and land in the transcript, so a few careless
captures fill the context window. Screenshot when you need to see the screen,
not reflexively after each action; a `computer::act` returns a short confirming
`detail` on its own. Reuse one session across steps and stop it when done.

## Reactive triggers

Bind a `computer::*` trigger when another function should react to session
activity instead of polling. The types: `computer::session-started` (payload
carries `endpoint`, `os`, `screen`) and `computer::session-stopped` (payload
carries `reason`). Both accept an optional `session_id` equality filter.

### How to bind

1. Register a handler: `registerFunction('mywatcher::on-desktop', handler)`.
2. Register the trigger:

```typescript
iii.registerTrigger({
  type: 'computer::session-stopped',
  function_id: 'mywatcher::on-desktop',
  config: { session_id: 'c1' },
})
```

Omit `session_id` to receive events for all sessions. For event payload shapes,
call `get function info` on the trigger type.

## api reference

```json
{
  "functions": [
    {
      "description": "Drive the desktop: click, right_click, double_click, move, drag, scroll, type, press, or hotkey. Address by pixel coordinates read off the screenshot (top-left origin).",
      "metadata": {},
      "name": "computer::act",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "action": {
            "description": "What to do: `click`, `right_click`, `double_click`, `move`, `drag`, `scroll`, `type`, `press`, or `hotkey`.",
            "type": "string"
          },
          "button": {
            "default": null,
            "description": "Mouse button for `drag` (`left`, `right`, `middle`). Default `left`.",
            "type": [
              "string",
              "null"
            ]
          },
          "keys": {
            "default": null,
            "description": "Keys for `press`/`hotkey`: a single key (`[\"enter\"]`) or a chord (`[\"ctrl\", \"c\"]`).",
            "items": {
              "type": "string"
            },
            "type": [
              "array",
              "null"
            ]
          },
          "scroll_x": {
            "default": null,
            "description": "Horizontal scroll amount for `action=scroll`. Default 0.",
            "format": "int64",
            "type": [
              "integer",
              "null"
            ]
          },
          "scroll_y": {
            "default": null,
            "description": "Vertical scroll amount for `action=scroll`; positive scrolls down. Default 3.",
            "format": "int64",
            "type": [
              "integer",
              "null"
            ]
          },
          "session_id": {
            "type": "string"
          },
          "text": {
            "default": null,
            "description": "Text to type, for `action=type`.",
            "type": [
              "string",
              "null"
            ]
          },
          "to_x": {
            "default": null,
            "description": "Drag end x (with `to_y`), for `action=drag`.",
            "format": "int64",
            "type": [
              "integer",
              "null"
            ]
          },
          "to_y": {
            "default": null,
            "description": "Drag end y.",
            "format": "int64",
            "type": [
              "integer",
              "null"
            ]
          },
          "x": {
            "default": null,
            "description": "Target x (pixels). Required for click/right_click/double_click/move and the drag start; the scroll anchor defaults to screen center.",
            "format": "int64",
            "type": [
              "integer",
              "null"
            ]
          },
          "y": {
            "default": null,
            "description": "Target y (pixels).",
            "format": "int64",
            "type": [
              "integer",
              "null"
            ]
          }
        },
        "required": [
          "action",
          "session_id"
        ],
        "title": "ActInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "detail": {
            "description": "What was done, for the transcript.",
            "type": "string"
          },
          "ok": {
            "type": "boolean"
          }
        },
        "required": [
          "detail",
          "ok"
        ],
        "title": "ActOutput",
        "type": "object"
      }
    },
    {
      "description": "List the local displays (index, name, primary, size). Pass the chosen index as `monitor` to computer::sessions::start to drive that display; omit it to use the display under the cursor. Native host only.",
      "metadata": {},
      "name": "computer::displays",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "title": "DisplaysInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "DisplayInfo": {
            "description": "One display available to the native host (from `computer::displays`).",
            "properties": {
              "builtin": {
                "type": "boolean"
              },
              "height": {
                "description": "Logical height in points.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "index": {
                "description": "Index to pass as `monitor` to `computer::sessions::start`.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "name": {
                "type": "string"
              },
              "primary": {
                "type": "boolean"
              },
              "width": {
                "description": "Logical width in points.",
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "builtin",
              "height",
              "index",
              "name",
              "primary",
              "width"
            ],
            "type": "object"
          }
        },
        "properties": {
          "displays": {
            "items": {
              "$ref": "#/definitions/DisplayInfo"
            },
            "type": "array"
          }
        },
        "required": [
          "displays"
        ],
        "title": "DisplaysOutput",
        "type": "object"
      }
    },
    {
      "description": "Internal: newest screencast frame, or nothing when since_frame is still current. No capture round-trip; read it on each computer::frame-changed notification. Not an agent function.",
      "metadata": {
        "internal": true
      },
      "name": "computer::frame",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "session_id": {
            "type": "string"
          },
          "since_frame": {
            "default": null,
            "description": "Frame cursor from the previous read; when the newest frame still has this seq the response omits `frame` (nothing changed).",
            "format": "uint64",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          }
        },
        "required": [
          "session_id"
        ],
        "title": "FrameInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "active": {
            "description": "False when no screencast is running (call screencast::start first).",
            "type": "boolean"
          },
          "epoch": {
            "description": "Session incarnation in this worker process; `frame_seq` restarts when it changes. Order frames by `(epoch, frame_seq)`.",
            "format": "int64",
            "type": "integer"
          },
          "frame": {
            "description": "Base64 image of the newest frame; absent when `since_frame` is still current or no frame has arrived yet.",
            "type": [
              "string",
              "null"
            ]
          },
          "frame_seq": {
            "format": "uint64",
            "minimum": 0,
            "type": "integer"
          },
          "height": {
            "format": "uint32",
            "minimum": 0,
            "type": "integer"
          },
          "mime": {
            "description": "Image mime of the frame bytes.",
            "type": "string"
          },
          "timestamp": {
            "format": "int64",
            "type": "integer"
          },
          "width": {
            "format": "uint32",
            "minimum": 0,
            "type": "integer"
          }
        },
        "required": [
          "active",
          "epoch",
          "frame_seq",
          "height",
          "mime",
          "timestamp",
          "width"
        ],
        "title": "FrameOutput",
        "type": "object"
      }
    },
    {
      "description": "Capture the desktop plus, optionally, the accessibility tree. Use include_a11y on macOS guests for a machine-readable element tree; elsewhere prefer computer::screenshot.",
      "metadata": {},
      "name": "computer::observe",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "include_a11y": {
            "default": null,
            "description": "Also fetch the accessibility tree. macOS returns a real tree; other guests may return a stub or nothing (the field is then omitted).",
            "type": [
              "boolean",
              "null"
            ]
          },
          "session_id": {
            "type": "string"
          }
        },
        "required": [
          "session_id"
        ],
        "title": "ObserveInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "ContentBlock": {
            "description": "One block of a viewable response: an image block or a text line.",
            "properties": {
              "data": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "mime": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "text": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "type": {
                "type": "string"
              }
            },
            "required": [
              "type"
            ],
            "type": "object"
          },
          "ObserveDetails": {
            "properties": {
              "mime": {
                "type": "string"
              },
              "screen": {
                "$ref": "#/definitions/Screen"
              },
              "session_id": {
                "type": "string"
              }
            },
            "required": [
              "mime",
              "screen",
              "session_id"
            ],
            "type": "object"
          },
          "Screen": {
            "description": "Desktop pixel dimensions. Coordinates handed to pointer actions are in this space: integer pixels, top-left origin, 1:1 with the screenshot.",
            "properties": {
              "height": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "width": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "height",
              "width"
            ],
            "type": "object"
          }
        },
        "properties": {
          "accessibility": {
            "description": "Accessibility tree, present only when requested and the guest exposes one."
          },
          "content": {
            "items": {
              "$ref": "#/definitions/ContentBlock"
            },
            "type": "array"
          },
          "details": {
            "$ref": "#/definitions/ObserveDetails"
          }
        },
        "required": [
          "content",
          "details"
        ],
        "title": "ObserveOutput",
        "type": "object"
      }
    },
    {
      "description": "Internal: reload computer settings from the authoritative configuration on change.",
      "metadata": {
        "internal": true
      },
      "name": "computer::on-config-change",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "title": "OnConfigChangeRequest",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "ok": {
            "type": "boolean"
          }
        },
        "required": [
          "ok"
        ],
        "title": "OnConfigChangeResponse",
        "type": "object"
      }
    },
    {
      "description": "Internal: start capturing live desktop frames for the console viewport; each new frame fires computer::frame-changed and is read with computer::frame. Console-UI plumbing; agents use computer::screenshot. Not an agent function.",
      "metadata": {
        "internal": true
      },
      "name": "computer::screencast::start",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "session_id": {
            "type": "string"
          }
        },
        "required": [
          "session_id"
        ],
        "title": "ScreencastStartInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "ok": {
            "type": "boolean"
          }
        },
        "required": [
          "ok"
        ],
        "title": "AckOutput",
        "type": "object"
      }
    },
    {
      "description": "Internal: stop the live frame capture and clear the stored frame. Idempotent. Not an agent function.",
      "metadata": {
        "internal": true
      },
      "name": "computer::screencast::stop",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Stopping the screencast on an unknown session succeeds.",
        "properties": {
          "session_id": {
            "type": "string"
          }
        },
        "required": [
          "session_id"
        ],
        "title": "ScreencastStopInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "ok": {
            "type": "boolean"
          }
        },
        "required": [
          "ok"
        ],
        "title": "AckOutput",
        "type": "object"
      }
    },
    {
      "description": "Take a screenshot of the desktop screen. This is how you see the screen before acting; the coordinate space of computer::act is this image's pixels (top-left origin).",
      "metadata": {},
      "name": "computer::screenshot",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "session_id": {
            "type": "string"
          }
        },
        "required": [
          "session_id"
        ],
        "title": "ScreenshotInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "ContentBlock": {
            "description": "One block of a viewable response: an image block or a text line.",
            "properties": {
              "data": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "mime": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "text": {
                "type": [
                  "string",
                  "null"
                ]
              },
              "type": {
                "type": "string"
              }
            },
            "required": [
              "type"
            ],
            "type": "object"
          },
          "ScreenshotDetails": {
            "properties": {
              "height": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "mime": {
                "description": "Detected image mime (`image/png` or `image/jpeg`).",
                "type": "string"
              },
              "session_id": {
                "type": "string"
              },
              "width": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "height",
              "mime",
              "session_id",
              "width"
            ],
            "type": "object"
          }
        },
        "properties": {
          "content": {
            "items": {
              "$ref": "#/definitions/ContentBlock"
            },
            "type": "array"
          },
          "details": {
            "$ref": "#/definitions/ScreenshotDetails"
          }
        },
        "required": [
          "content",
          "details"
        ],
        "title": "ScreenshotOutput",
        "type": "object"
      }
    },
    {
      "description": "List live computer sessions with their endpoint, guest OS, and screen size.",
      "metadata": {},
      "name": "computer::sessions::list",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "title": "ListInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "Screen": {
            "description": "Desktop pixel dimensions. Coordinates handed to pointer actions are in this space: integer pixels, top-left origin, 1:1 with the screenshot.",
            "properties": {
              "height": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "width": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "height",
              "width"
            ],
            "type": "object"
          },
          "SessionInfo": {
            "properties": {
              "created_ms": {
                "format": "int64",
                "type": "integer"
              },
              "endpoint": {
                "type": "string"
              },
              "last_used_ms": {
                "format": "int64",
                "type": "integer"
              },
              "os": {
                "type": "string"
              },
              "screen": {
                "$ref": "#/definitions/Screen"
              },
              "screencast_active": {
                "description": "True while a live screen stream is running for this session.",
                "type": "boolean"
              },
              "session_id": {
                "type": "string"
              }
            },
            "required": [
              "created_ms",
              "endpoint",
              "last_used_ms",
              "os",
              "screen",
              "screencast_active",
              "session_id"
            ],
            "type": "object"
          }
        },
        "properties": {
          "sessions": {
            "items": {
              "$ref": "#/definitions/SessionInfo"
            },
            "type": "array"
          }
        },
        "required": [
          "sessions"
        ],
        "title": "ListOutput",
        "type": "object"
      }
    },
    {
      "description": "Start a computer-use session and return its session_id. Pass `image` to boot a fresh desktop in an iii-sandbox microVM, `endpoint` to drive a remote desktop's guest executor, or neither to drive the local machine. Stop it with computer::sessions::stop.",
      "metadata": {},
      "name": "computer::sessions::start",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "endpoint": {
            "default": null,
            "description": "Remote desktop guest executor (`ws`/`wss`/`http`/`https` url or `host:port`). Omit this and `image` to drive the local machine.",
            "type": [
              "string",
              "null"
            ]
          },
          "image": {
            "default": null,
            "description": "OCI image (sandbox preset name or `custom_images` key) for a fresh iii-sandbox microVM desktop; overrides `endpoint`.",
            "type": [
              "string",
              "null"
            ]
          },
          "monitor": {
            "default": null,
            "description": "Display index from `computer::displays` for a native session; omit for the display under the cursor. Ignored with `endpoint` or `image`.",
            "format": "uint32",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          },
          "os": {
            "default": null,
            "description": "Guest OS label recorded on the session: `linux`, `macos`, `windows` or `android`. Omit to let the driver label itself.",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "StartInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "Screen": {
            "description": "Desktop pixel dimensions. Coordinates handed to pointer actions are in this space: integer pixels, top-left origin, 1:1 with the screenshot.",
            "properties": {
              "height": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              },
              "width": {
                "format": "uint32",
                "minimum": 0,
                "type": "integer"
              }
            },
            "required": [
              "height",
              "width"
            ],
            "type": "object"
          }
        },
        "properties": {
          "endpoint": {
            "description": "What the session drives: `native` for the local machine, or the normalized remote endpoint.",
            "type": "string"
          },
          "os": {
            "type": "string"
          },
          "screen": {
            "allOf": [
              {
                "$ref": "#/definitions/Screen"
              }
            ],
            "description": "Desktop pixel dimensions; the coordinate space for `computer::act`."
          },
          "session_id": {
            "description": "Pass this to every other computer function.",
            "type": "string"
          }
        },
        "required": [
          "endpoint",
          "os",
          "screen",
          "session_id"
        ],
        "title": "StartOutput",
        "type": "object"
      }
    },
    {
      "description": "Stop a computer session and close its driver connection. Idempotent: stopping an unknown or already-stopped session succeeds with was_running=false.",
      "metadata": {},
      "name": "computer::sessions::stop",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "session_id": {
            "description": "Session to stop. Stopping an unknown or already-stopped id succeeds.",
            "type": "string"
          }
        },
        "required": [
          "session_id"
        ],
        "title": "StopInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "properties": {
          "ok": {
            "type": "boolean"
          },
          "was_running": {
            "description": "False when the session was already gone.",
            "type": "boolean"
          }
        },
        "required": [
          "ok",
          "was_running"
        ],
        "title": "StopOutput",
        "type": "object"
      }
    },
    {
      "description": "Serve the computer worker's injected console UI assets (content function for its console:script / console:style / console:module triggers).",
      "metadata": {
        "internal": true
      },
      "name": "computer::ui-content",
      "request_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Input of the content function: the console asks for one asset by path.",
        "properties": {
          "path": {
            "description": "The asset path from the trigger config (e.g. `state/page.js`).",
            "type": "string"
          }
        },
        "required": [
          "path"
        ],
        "title": "UiContentInput",
        "type": "object"
      },
      "response_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "description": "Output of the content function.",
        "properties": {
          "content": {
            "description": "The asset source, verbatim.",
            "type": "string"
          },
          "content_type": {
            "description": "MIME type the console should serve the asset with.",
            "type": "string"
          }
        },
        "required": [
          "content",
          "content_type"
        ],
        "title": "UiContentResult",
        "type": "object"
      }
    }
  ],
  "triggers": [
    {
      "description": "The newest screencast frame of one session changed (a new frame was stored, or the frame was cleared because the screencast or session stopped). Config: { session_id } (required). Payload: { session_id, epoch, frame_seq, change, width, height, mime?, bytes?, timestamp }; no image: read it with computer::frame. Latest state only: intermediate notifications are coalesced for slow consumers, nothing is stored or replayed.",
      "invocation_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "description": "Binding filter for `computer::frame-changed`. `session_id` is required: a viewer follows one desktop; unknown keys fail the registration.",
        "properties": {
          "session_id": {
            "description": "The computer session whose frames to follow.",
            "type": "string"
          }
        },
        "required": [
          "session_id"
        ],
        "title": "FrameBindingConfig",
        "type": "object"
      },
      "metadata": {},
      "name": "computer::frame-changed",
      "return_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "definitions": {
          "FrameChangeKind": {
            "oneOf": [
              {
                "description": "A new frame was stored; read it with `computer::frame`.",
                "enum": [
                  "updated"
                ],
                "type": "string"
              },
              {
                "description": "The stored frame was dropped (screencast stopped, capture failed, or the session ended). Nothing to read until the next `updated`.",
                "enum": [
                  "cleared"
                ],
                "type": "string"
              }
            ]
          }
        },
        "description": "The notification. Small and image-free.",
        "properties": {
          "bytes": {
            "description": "Encoded image size in bytes (`updated` only).",
            "format": "uint64",
            "minimum": 0,
            "type": [
              "integer",
              "null"
            ]
          },
          "change": {
            "$ref": "#/definitions/FrameChangeKind"
          },
          "epoch": {
            "description": "Identifies this incarnation of the session in this worker process (ms when it was created or restored). `frame_seq` restarts at 1 when `epoch` changes; compare `(epoch, frame_seq)` to order frames.",
            "format": "int64",
            "type": "integer"
          },
          "frame_seq": {
            "description": "Sequence of the newest stored frame (monotonic within an epoch). For `cleared`, the sequence of the last frame that was stored.",
            "format": "uint64",
            "minimum": 0,
            "type": "integer"
          },
          "height": {
            "format": "uint32",
            "minimum": 0,
            "type": "integer"
          },
          "mime": {
            "description": "Image mime of the stored frame (`updated` only).",
            "type": [
              "string",
              "null"
            ]
          },
          "session_id": {
            "type": "string"
          },
          "timestamp": {
            "format": "int64",
            "type": "integer"
          },
          "width": {
            "format": "uint32",
            "minimum": 0,
            "type": "integer"
          }
        },
        "required": [
          "change",
          "epoch",
          "frame_seq",
          "height",
          "session_id",
          "timestamp",
          "width"
        ],
        "title": "FrameChange",
        "type": "object"
      }
    },
    {
      "description": "A desktop session connected and is ready to drive.",
      "invocation_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "description": "Config accepted by every `computer::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
        "properties": {
          "session_id": {
            "description": "Only deliver events for this computer session.",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "BindingConfig",
        "type": "object"
      },
      "metadata": {},
      "name": "computer::session-started",
      "return_schema": {}
    },
    {
      "description": "A desktop session ended (stopped or idle).",
      "invocation_schema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "additionalProperties": false,
        "description": "Config accepted by every `computer::*` trigger binding. The only filter is an optional session-id equality match; unknown fields fail at registration so a misspelled filter key fails loudly instead of silently receiving nothing.",
        "properties": {
          "session_id": {
            "description": "Only deliver events for this computer session.",
            "type": [
              "string",
              "null"
            ]
          }
        },
        "title": "BindingConfig",
        "type": "object"
      },
      "metadata": {},
      "name": "computer::session-stopped",
      "return_schema": {}
    }
  ]
}
```