EmbeddedBrowser

Selectors are ranked by how likely they are to survive the next refactor. QA test hooks (data-testid, data-qa, data-cy, data-test, data-ui) come first: they exist precisely to be stable identifiers. Other data-* attributes come next: usually author-controlled and intentional. HTML ids follow, stable when hand-written, but framework-generated ids churn between renders. A positional CSS path with :nth-of-type() is the last resort. The inspector shows all four as clickable chips; the user picks which one the AI should target, and their typed instruction is preserved across chip swaps.

• data-testid / data-qa / data-cy / data-test / data-ui: test-framework conventions first
• Other data-* attributes: author-controlled, usually semantic
• #id: exact when hand-written, noisy when framework-generated
• Positional CSS path with :nth-of-type() collision handling: always valid, rarely stable
• Active selector chip is highlighted; clicking a different chip never wipes the instruction in progress

The prompt Plexon sends carries more than just the selector. The viewport dimensions go with every message so the AI can reason about responsive breakpoints. A computed-styles digest covers the properties that actually matter for a typical edit (display, position, color, background-color, font-size, font-weight, padding, margin, width, height) filtered out of the thousands that getComputedStyle returns. The outer HTML of the element is truncated to 4 KB and included in a collapsible <details> block so it is visible when the model needs it but does not dominate the context window. And a cropped PNG of the element itself rides along as an image attachment for vision-capable models.

• Selector + inner text (truncated to 200 characters) + full page URL
• Viewport width × height: essential for responsive-design instructions
• Computed styles digest: 10 high-signal properties, not the 300+ getComputedStyle returns
• Outer HTML capped at 4 KB with ellipsis marker, wrapped in <details> so it does not crowd the prompt
• Cropped element screenshot attached as image_url: same channel as pasted images, so any vision model sees it

Click the crosshair icon in the URL bar to enter inspect mode. A blue highlight follows the mouse across the page with an int32-max z-index so it outranks any host page stacking context, including modal dialogs and fullscreen surfaces. Click an element; inspect mode exits automatically and a docked popover appears under the viewport with the captured metadata. Type your instruction (Enter sends, Shift+Enter inserts a newline, Escape inside the webview exits inspect mode without selecting) and Plexon pushes the message into your current chat session. The AI uses its normal read/edit/grep tools to locate the source component and apply the change; your dev server hot-reloads; you click reload in the browser pane, re-inspect, and go again.

• Crosshair icon toggles inspect mode; the button flips primary-coloured while active
• Hover highlight uses a lightweight DOM overlay, not a heavyweight debugging protocol: no network cost, no frame drops
• Plain click = single element edit and exit inspect mode; Ctrl+Click = add to selection and keep picking
• The inspector docks at the bottom of the browser pane: always in the same place, never overlapping the element you clicked
• Esc, the × button, or toggling the crosshair off all exit inspect mode cleanly
• Back / forward / reload, smart URL normalisation (localhost → http://, example.com → https://, anything else → DuckDuckGo search)

Hold Ctrl while clicking to stack multiple selections. Each one renders as its own row in the inspector with its screenshot, selector chips, and a × to drop just that entry. One instruction field applies to every selected element ("make all of these cards the same height and pin the CTA to the bottom") and the prompt switches to a numbered list with one cropped screenshot per element attached. The plus button in the inspector header re-enters inspect mode for more selections without a trip to the URL bar.

• Ctrl+Click appends; plain click replaces: familiar conventions from file managers and editors
• Each selection is an independent row with its own selector chips and remove button
• Sent prompt numbers every entry and attaches one image_url block per captured screenshot
• One instruction → many elements. Same-height card rows, matching button styles, consistent padding, all with a single send

Some sites publish actions for an assistant to call: filter this catalog, price this plan, start this support request. A page registers them with document.modelContext.registerTool, and Plexon reads that catalog over the debugging protocol without injecting a script into the page. A "Page tools" button on the browser toolbar counts what the current page offers and opens the list under the URL bar. When the AI wants to use a site, and again before every individual call, Plexon shows you the request with Allow and Deny, naming the tool, the origin and the arguments. Answer it, or do nothing: after three minutes (your setting) the request is allowed and the work goes on, so a task you left running does not stall at the first page tool. What comes back is marked untrusted and stays an ordinary tool result in the transcript, and the page catalog never joins the model’s own tool list.

• Four actions on the browser tool: status, list_tools, call_tool, cancel_call
• Site access is per origin and lasts for that chat session; Revoke access ends it on the spot, and a denied site stays denied until you allow it yourself
• Every site request and every call shows a toast and a row in Page tools with Allow and Deny. Unanswered requests are allowed after a wait you set (default 3 minutes, 0 to always wait for you)
• Tool references expire when the page commits a navigation, so a stale reference fails instead of hitting a different page
• Switching chat session or revoking the site drops the binding and any calls waiting on it
• Page results are labelled untrusted; the catalog never becomes part of the model’s global tools

Ask for something on a site you have open and the AI does it in this pane, in the session you are signed in to, where you can see every step. It reads the page as a nested outline from the accessibility tree, each element with its state and every link with its target, or as plain text when the job is to collect things (the posts on a profile, the rows of a table, the results of a search come back in one read instead of one click each, and a feed that loads as you scroll is read to its end in one call). Then it clicks, types, presses keys and key combinations, picks options out of dropdowns, scrolls, hovers, drags one element onto another, and goes back and forward through the page’s own history. Every one of those comes back with the page it left behind, what changed or that nothing did, so it acts and reads the answer in one step instead of looking again after each one. Boxes embedded from another site (a card field, a captcha, a sign-in) are read and filled like the rest of the page, and a link that wants a new window opens in the pane while the AI is driving, one Back from where it was. Nothing is asked first: the moment it acts, a strip appears over the page with Take the wheel. While you hold the wheel, every action the AI tries is refused, not queued. When the page asks for a password, a captcha, a code or a card number, the AI hands you the page itself, with the field already focused, and says what it needs; you type, and Give control back sends the chat on by itself. A newer opt-in setting takes this further: give the page one whole goal, plus any text it needs typed, and Plexon works through it on its own with a fast decision service. It stays off until you turn it on in Settings, Advanced. It does not start from nothing on every site, either: Plexon ships driving notes for some popular ones, the address that lands on the job and which control does it, and hands them to the AI with the page. They are hints it checks against what is on screen, never a recorded script, because sites change. When a run works out a site with no notes, it offers what it did as a draft you approve or reject under Skills, and nothing typed into a page is ever in one.

• Nineteen driving actions on the one browser tool: snapshot, read, collect, find, click, type, key, select, scroll, hover, drag, back, forward, wait, console, screenshot, dialog, handoff, task
• Plexon carries driving notes for some popular sites, so the AI knows which control does a common job before it starts clicking, and offers to save what it learns on a new one
• The handoff goes one way only: the AI can give you the page, never take it back; that stays your button
• Every acting step answers with the page: what changed, or that nothing did, or the whole outline after a navigation
• What the AI has already seen stays usable: a scroll or a closed panel does not make it re-read the page, and a link it has seen is opened by address
• A page’s own question box can be answered by you or by the AI, whichever gets there first, and the reply says which and what the answer was
• Every action goes through the debugging protocol; nothing is injected into the page and no script is ever evaluated in it
• Typing never submits and never echoes the text back; sending a form is a separate Enter the AI has to choose
• Console capture starts only when the AI asks for it, so a page merely previewed is not being listened to
• A screenshot reaches the model as an image; clicks are still addressed by outline reference, never by coordinates
• Take the wheel refuses the AI’s actions outright instead of saving them up for when you hand control back
• When its turn ends the strip comes down, and a pane the AI opened for its own job closes again; the page stays a click away

A web page is not a trusted source of schemas, so its catalog is treated as input. Every tool schema is validated with Ajv inside a worker with its own time budget, and both the arguments going out and the reply coming back are size-capped before they reach the chat. A call that passes its deadline is not thrown away: cancel_call hands back a pending receipt, so status and a later cancel still act on the same call instead of on nothing. A document that is not http or https, a local file for instance, is reported as an unsupported scheme rather than failing quietly.

• Ajv validation in a bounded worker: a hostile schema costs a rejection, not the app
• 64 KiB of input, 16 KiB per tool schema, 256 KiB of output, 128 KiB of catalog
• 128 entries per page catalog, 8 calls in flight, the 20 most recent results kept
• cancel_call returns a pending receipt at the deadline, so status and cancel keep working on it
• file:// and other non-http(s) documents report an unsupported scheme, with no guessing
• Reconnect re-binds the panel after a guest reload, with no restart of the app

The embedded page runs fully isolated. The guest page has no access to Plexon’s internals and no way to reach the backend. A minimal inspection layer installs the hover / click / Esc listeners and posts element metadata back to the host over a narrow, one-way channel. The main window runs sandboxed, exposing none of its privileged APIs to guest content. The pane also has its own storage, so its cookies and logins are separate from Plexon’s own and from the browser you use for everything else, and clearing it clears the pane and nothing else. A page that tries to open a new window never gets one: it is loaded in the pane while the AI is driving, handed to your normal browser while you are, and refused otherwise. For a dev tool that loads user-chosen pages on the user’s own machine, the threat model is intentionally minimal, and explicitly documented.

• Full isolation on every embedded page: guest content cannot read Plexon’s internals
• Its own cookie jar: sign in to a site in the pane and it stays signed in there, and nowhere else in Plexon
• Embedded views are opt-in for the main window only; a guest page never spawns a second browser window
• Inspection uses a capture-phase listener and an isolated overlay; it never mutates the host page’s DOM subtree
• Screenshot capture runs at CSS-pixel resolution, clamped to the viewport, with no debugging-protocol exposure
• The inspection layer ships with the app: no setup, and no extension to install

The send path is the same pipeline typed messages use. The inspector’s "Send to AI" button builds a ContentBlock[] with a text block (selector + viewport + outer HTML + styles) and an image_url block (the cropped screenshot as a base64 data URL), calls chatStore.addMessage and DaemonService.sendMessage, and lets the daemon’s normal streaming response come back to the regular chat view. No new provider code, no new message shape, no new storage. The browser is a richer input, not a separate conversation. When the daemon streams a file edit tool call, you see it in the chat while your browser pane shows the live page. Hit reload after the hot-reload kicks in; inspect the next element; send the next instruction.

• ContentBlock[] with text + image_url: identical to the shape pasted images use, so every provider already handles it
• Session auto-creation follows the same fallback chain as typed messages (client-side UUID if createSession RPC transiently fails)
• No changes to the daemon, system prompts, or provider adapters: pure client-side feature
• User messages show up in the chat transcript with the embedded screenshot thumbnail, so the history stays readable
• Works in every chat mode (Code, Plan, Ask, Auto, Spec) without per-mode branching