Browser Studio

Choose the browser surface

Know which Chrome you are controlling.

Planner generation, Planner testing, Studio authoring, and production execution use separate Chrome processes. Code moves between them; page state is never transferred implicitly. A workflow may deliberately restore its own configured browser profile, but never the Studio process or tab.

Surface	Use it when	Lifetime	Durable result
Planner generation browser	Observe Copilot inspecting the portal while it generates or repairs browser code.	Only while generation is running.	Generated draft code.
Planner step-test browser	Observe the real `run_browser_workflow` activity executing the partial workflow test.	Only while that test is running.	The test result, followed by the explicit Planner step decision.
Browser Studio	Create a separate authoring Chrome to inspect, execute, and repair one existing `run_browser_workflow` step.	Retained until idle expiry, explicit close, successful credential-session cleanup, or service restart.	Only after `/apply` returns code to Planner TEST.
Workflow run browser	Execute the accepted browser activity in a promoted workflow.	One browser-step execution. A configured browser profile may persist, but the Studio process does not.	The activity output and workflow state.

noVNC

Request a link, then view the run in your browser.

noVNC is a browser-based screen and input bridge to an already-running Chrome. Asking for a link mints an expiring capability for that display; it does not launch, clone, or retain Chrome.

Mint or refresh noVNC links

# Planner: request fresh links for the browser that is running now.
# This returns 404 when no browser generation/test is active for the step.
curl --fail-with-body \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  "${XY_API_BASE}/planner/builds/${BUILD_ID}/steps/1/browser-session"

# Browser Studio: refresh links for the retained authoring session.
curl --fail-with-body \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  "${XY_API_BASE}/browser/sessions/${SESSION_ID}"

Planner

The active build stream can include vnc_url. REST callers can request a fresh one with the browser-session route; MCP callers useplanner_get_browser_session. The route finds only the browser currently registered for that build and step—generation when needed, then testing—and returns 404 when neither is active. Each browser operation has a fresh Chrome and capability. Wait for viewer_ready: true before opening a link minted while the browser is still starting.

With browser:read, session creation returnscapabilities.viewer.url. Read the session—or callbrowser_get_session through MCP—to refresh it while retained Chrome still exists.

In an MCP client, “show me the browser for this build and step” means: call the session lookup tool, return its URL, and let the operator open that URL. The MCP response does not contain a video stream or create browser state; the noVNC page displays the live pixels after it connects to the already-running browser session.

Create a session

Discover the tools, then create authoring Chrome for the step.

Start with a Planner build paused on a browser step. When existing_code is omitted, Studio loads the current step code. A supplied credential must match the reference already bound to that step.

Tool discovery and retained session

# Discover the exact commands deployed with the browser runtime.
curl --fail-with-body \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  "${XY_API_BASE}/browser/tools"

# Create retained authoring Chrome for one existing Planner browser step.
curl --fail-with-body --request POST \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  --header "Content-Type: application/json" \
  "${XY_API_BASE}/browser/sessions" \
  --data "{
    "build_id": "${BUILD_ID}",
    "step_number": 1,
    "url": "https://portal.example.com/open-records",
    "credential_id": "cred_REPLACE"
  }"

Direct or agentic

Use both authoring modes on the same Chrome.

Direct commands and Browser Copilot share one serialized session and one page state. Choose direct commands when you know the exact operation; choose Copilot when inspection, code generation, execution, and repair need a conversational loop.

Direct SDK commands

curl --fail-with-body --request POST \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  --header "Content-Type: application/json" \
  "${XY_API_BASE}/browser/sessions/${SESSION_ID}/commands" \
  --data '{"tool":"scan_page","input":{}}'

curl --fail-with-body --request POST \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  --header "Content-Type: application/json" \
  "${XY_API_BASE}/browser/sessions/${SESSION_ID}/commands" \
  --data '{
    "tool": "update_workflow_code",
    "input": {
      "code": "navigate(url="https://portal.example.com/open-records")\nscan_page()"
    }
  }'

Stateful Browser Copilot

curl --fail-with-body --no-buffer --request POST \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  --header "Content-Type: application/json" \
  "${XY_API_BASE}/browser/sessions/${SESSION_ID}/prompts" \
  --data '{
    "prompt": "Inspect the open-record flow. Write deterministic browser workflow code, execute it, repair failures, and save the passing draft with update_workflow_code."
  }'

Inspect

Scan the page, elements, tabs, screenshots, and current workflow code.

Interact

Navigate, click, type, upload/download, and call deployed SDK tools.

Author

Update, validate, execute, and inspect the draft and execution logs.

Extract

Read structured page/table data and persist safe workflow memory.

Portal endpoints

Use the page like a Network panel.

A portal's own authenticated HTTP calls can be more stable than repeated DOM scraping. The endpoint observer records the live tab's requests while you exercise a flow, then an API map keeps inferred paths and shapes—not captured values.

Capture, inspect, and map

# Arm capture before performing the interesting action.
curl --fail-with-body --request POST \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  --header "Content-Type: application/json" \
  "${XY_API_BASE}/browser/sessions/${SESSION_ID}/commands" \
  --data '{"tool":"set_endpoint_observer","input":{"enabled":true}}'

# After navigating/clicking, list a compact index.
curl --fail-with-body --request POST \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  --header "Content-Type: application/json" \
  "${XY_API_BASE}/browser/sessions/${SESSION_ID}/commands" \
  --data '{
    "tool": "get_observed_endpoints",
    "input": {"urlContains":"/api/","limit":25,"includePayloads":false}
  }'

# Build a reusable shape map after exercising the flow.
curl --fail-with-body --request POST \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  --header "Content-Type: application/json" \
  "${XY_API_BASE}/browser/sessions/${SESSION_ID}/commands" \
  --data '{"tool":"build_api_map","input":{"host":"portal.example.com"}}'

Recommended sequence

Enable observation before the interesting page action.
Exercise every relevant list, detail, pagination, and write flow.
List compact observations and inspect selected details by storage path.
Build the host map and use it to author a deterministic browser activity.
Run the activity against a fresh state and verify its actual business output.

Apply

A passing Studio draft is not an accepted workflow step.

The retained session is a development environment. For an existing step definition, applying a draft changes its browser code and invokes the normal Planner TEST action in a fresh Planner test browser. If generation failed before producing a definition, apply creates the minimal native browser step needed for that test.

Return the draft to Planner

# Read exactly what will be sent back to Planner.
curl --fail-with-body \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  "${XY_API_BASE}/browser/sessions/${SESSION_ID}/draft"

# Run it through Planner TEST in a fresh test browser.
curl --fail-with-body --no-buffer --request POST \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  --header "Content-Type: application/json" \
  "${XY_API_BASE}/browser/sessions/${SESSION_ID}/apply" \
  --data '{}'

# Close retained Chrome when authoring is done.
curl --fail-with-body --request DELETE \
  --header "Authorization: Bearer ${XY_API_KEY}" \
  "${XY_API_BASE}/browser/sessions/${SESSION_ID}"

Preserved

When step YAML already exists: URL, queue, credential reference, timeout, result mapping, and its other settings.

Changed

The existing step's activity code; or, when no YAML exists after failed generation, a minimal native browser step containing that code.

Still required

Inspect the fresh test result, explicitly ACCEPT the Planner step, finish the build, and promote it.

Return to the Planner lifecycle Understand browser credentials Use Studio through MCP