Pins and comments

Concepts

Element pin

A pin that comes from a live output element, by a click, or from code with add, addAll or setElement. Its kind is element.

Region pin

A pin that comes from a dragged rectangle, or from code with addRect or setRect. Its kind is rect and its label is Rect.

Linked and detached

An element pin is linked by default (linked: true): it finds its live element again and follows it, and its geometry is always tracked. A detached pin (linked: false) keeps the element's metadata but behaves like a region pin. Only element pins have linked.

Auto-track

autoTrack, true by default, controls geometry tracking for region and detached pins. A tracked pin stays attached to the same spot of the scrolled output document: it follows the output window's own scroll position, not scrolling inside nested scroll containers. An untracked pin stays fixed in the viewport. Linked element pins ignore it. The icons are 🛰️ for tracking and 🪨 for fixed.

Auto-capture

autoCapture, false by default, means refresh this pin's PNG automatically: 📸, Automatic image refresh on. false (🖼️, Preserve existing image) keeps the current image. A forced capture, from capture() or a double-click, works either way.

Comment

Plain text. In code, values are converted with String(), and null and undefined become an empty string.

Id

pin_<time in base36>_<6 base36 characters>, for example pin_mutrqtc0_a1b2c3. It stays the same through reordering, renumbering, moves and retargeting.

All pin geometry is in output CSS pixels. A pin's rect is relative to the output viewport, and elements in nested same-origin frames are mapped into the output's coordinates. The viewport that a pin stores is a historical record: overlays and crops use the output frame's live layout size, so marks and images stay correct at any output zoom.

Pins do not add elements or attributes to the output document. The panel, the curtain, the marks and the hover boxes live in the playground document, and images with a scale or a background use temporary stylesheets that are removed afterwards. The output does receive helper scripts that add listeners and window globals. Pins are not the same as the cursor pins of Locate, which map editor cursors to output elements; they share only the word.

Requirements

• Agent Folder Share. Pins are stored only through an active share, and the work folder is their storage. Without a share, every mutating o.pins call, and pause() and resume(), rejects with Pin API requires an active Agent Folder Share.
• Chrome or Edge on a desktop. Pins, o.pins and the Agent commands exist only where the browser has FileSystemObserver and the playground is not itself framed.
• An Output capture, for images. A pin image needs an active Output capture. This applies to 📍.<N>.png, the panel thumbnails, the ⚙️ options and the 📸, 🧬 and row 📸 buttons. Without an Output capture, pins still work, and their JSON files are still written. A person has to start the capture, because getDisplayMedia() needs a user gesture: an agent cannot start it from io.eval.js.
• The inline output. Pins target only the output frame inside the playground. Without it, starting Pins fails with Pin requires the inline output iframe.
• Playground only. Nothing about pins is deployed with a route. o.pins exists only in the editor window, which is where io.eval.js runs, and not in output code (o.eval.js). The screenshot that RTEdge.net keeps for each deploy is a separate feature; see Social previews.

Starting and stopping Pins

• The Pins and Pins Stop labels end with an invisible filler character, so type the visible part.
• Pins + 🔴 Output… depends on the state. Without a share, it shares with an Output capture, with no chooser and no picture-in-picture, and starts picking. With a share, it makes sure an Output capture runs, and starts picking. If an Output capture already runs, the command is disabled: use Ctrl+P / ⌘+P. If a different kind of capture is active, a toast says Capture is already active. Stop it before switching to Output capture. and picking still starts.
• The Code menu has Agent ▸ Pins, with a pin-map icon and the Ctrl+P hint. It appears only while a folder is shared, and clicking it toggles Pins. The tooltip reads Mark Output elements or regions, add comments, track geometry, and capture images for agent context.
• Outside the editor, Ctrl+P and Ctrl+Alt+P work too, and the browser's own Ctrl+P (Print) is suppressed. When Pins starts, a hotkey script goes into the output, so Ctrl+P and Ctrl+Alt+Shift+P also work while the output has focus. That script stays in the output document after Pins Stop. Unshare removes it, and so does an output reload.

Stopping

Ctrl+P while Pins is open, the ✕ button, or Shift+click on the state button closes the panel. It turns picking off, removes the overlays, destroys the output bridge and cancels image work in progress. Pins stay in memory, in o.pins and on disk.

Unshare

Closes Pins, removes o.pins from the output frame element and clears the pins from memory. The 📍.<N>.json files stay in the work folder, unless you used Unshare with Shift (Empty) or Ctrl (Delete). Pin images, 📍.<N>.png, are deleted when Unshare stops a capture that the share started.

Restoring at Share

With Share ← Folder on, pins on disk win. If the folder has none, the pins in the page's memory are used. With Share ← Folder off, the pins in the page's memory are written over the folder. If there are none, which is so after a reload or an Unshare, the folder's 📍.<N>.json files are loaded.

Tab icon

The third character of the tab icon shows Pins: _ closed, P paused, or the pin count from 0 to 9 while picking.

Picking

While Pins is picking, a transparent curtain with a crosshair cursor covers exactly the visible output. It forwards pointer and wheel events to the bridge in the output, which hit-tests there.

Click

Pins the hovered element, the deepest one that is hit-tested through shadow roots and frames. A coral hover box, labelled with the element's selector, label or tag, follows the pointer. Only the primary button picks.

Drag

Creates a region pin when both the width and the height of the dragged box exceed 5 px. A dashed cyan box shows the region during the drag. A release with a side of 5 px or less pins the hovered element instead. Regions are clamped to the output viewport.

Shift+wheel

Walks the hover target through the element's ancestry: wheel up moves toward the ancestors, wheel down moves back toward the original element. The ancestry crosses shadow hosts and frame elements.

Wheel

A plain wheel still scrolls the output.

Esc

With focus in the output, pauses picking. The Pins panel itself does not intercept Esc.

Picker hit-testing enters open shadow roots and same-origin iframes, up to the 🩻 Picker pierce depth: -1, the default, is unlimited; 0 stays in the output document; n from 1 up crosses at most n boundaries. Closed shadow roots and cross-origin frames cannot be entered, but the iframe element itself can be pinned. The depth affects human picking only, never the selectors of agents, and it works without a capture.

A newly picked pin takes autoTrack from the header's 🛰️/🪨 default, autoCapture from the header's 📸/🖼️ default, and linked: true. The list scrolls to the new row and focuses its comment. With an Output capture active, an initial PNG is captured for the new pin and for any other pin that still has no image.

Marks over the output

Existing pins are drawn as marks over the output: coral for element pins, cyan for region pins. Tracked pins get a solid border. Fixed pins get a dashed border with an outline. A label at the top right holds the state buttons and the pin's index. Marks are clipped to the visible output, and they take pointer events only while you pick or while Shift or Alt is held. Otherwise clicks pass through to the output. Shift, Alt and Ctrl pressed while the output has focus are relayed to the playground, so marks react to them too.

The Pins panel

The panel is a non-modal dialog without a focus trap, and it follows light and dark mode. It sits at the bottom right by default, with 20 px insets, and is 340 px wide. It does not steal focus when it opens. Drag its header, outside the buttons and inputs, to move it. Eight resize handles (n, e, s, w, nw, ne, se, sw) resize it unless it is minimized. It stays 4 px inside the window, and the browser remembers its geometry. Minimized, it is 96 px wide and shows only the LED, the pin count and the restore button. The hint reads Click output elements or drag a rectangle., and an empty list reads No pins yet.

Image capture options

⚙️ opens the popover ⚙️ Image capture, anchored to the button. It opens only while an Output capture is active, and the button is hidden otherwise. Esc closes it. The options set how pin images are made when a person picks, double-clicks or uses the panel's buttons. Agents can pass the same options per call, and can inherit them; see Enforce as agent pin defaults.

The footnote of the popover reads Scale and backgrounds apply to linked elements. Rectangle and detached pins keep their original crop. The settings, the header defaults and the 🩻 depth belong to the page, and survive Pins Stop and Unshare. They reset when the page reloads, and changing a setting does not recapture existing images.

Scale

Temporarily applies transform: translate(…) scale(…) to the element, composed with any transform it has. Transitions are switched off, and inline elements become inline-block. The applied scale is the smallest of the requested scale, (visible output width − 16 px) / element width and (visible output height − 16 px) / element height, and enclosing iframes limit it as well. With origin: 'auto' the element is scaled about its centre and shifted to fit inside the visible output with 8 px margins. An explicit origin is used as the CSS transform-origin.

Lift

Temporarily sets position: relative, only if the element is static, and z-index: 2147483647. Like the transition and inline-block changes, it applies only to scaled captures. A background-only capture at scale 1 is not lifted. It is also skipped for Backdrop captures of inner SVG graphics.

Unclip

Applies only to scaled captures. Before scaling, ancestors with idle auto or overlay overflow get overflow: visible. After scaling, ancestors that would clip the enlarged box (overflow: hidden or clip, or contain: paint, strict or content) are unclipped. Real scrollers are left alone and recorded in clippedBy. If unclipping moves anything, the capture fails with Unclipping an idle overflow container changed the pin layout or scroll position.

Settle

An optional delay, in milliseconds, that starts after two animation frames. It applies only to the enhanced path: scaled or background captures. settle: 0 removes only the delay. The freshness and restoration checks still run.

Background

transparent, or an absolute sRGB color: #rgb, #rgba, #rrggbb, #rrggbbaa, rgb(), rgba() or color(srgb …). The value is trimmed and lowercased, and 3- or 4-digit hex is expanded, so '#fff' becomes '#ffffff'. Other forms are re-serialized by the browser, for example 'rgb(255, 255, 255)'. Named colors, hsl() and oklch() are rejected.

Custom background

It is checked automatically when the scale is not 1, and unchecked at 1, until you toggle it. Your choice then sticks regardless of the scale, until you reset it. Its color is white in a light output and black in a dark output, following the output's resolved color-scheme, including ocs, until you pick a color. A color you picked is remembered, even while the checkbox is unchecked, until you reset it.

Background method

Backdrop, the default, places the color behind the target. It hides the surrounding paint: sibling branches get opacity 0, the paint and pseudo-elements of ancestors are hidden, iframe chrome becomes transparent, unrelated open popovers and modals are hidden, and the top document root is painted with the color. This works for masked icons and inner SVG shapes. Element sets the target's own background-color and keeps its background images.

Wheel over a field or its label

Scale changes by ±1 above 1 (wheel up at 1 goes to 2), and by ±0.1 below 1 (wheel down at 1 goes to 0.9). Settle changes by ±10 ms, and 🩻 depth by ±1, with a minimum of −1. Origin cycles the presets; a custom value stays unchanged unless Alt is held, and Alt+wheel enters the presets at auto. Selects cycle, checkboxes toggle and the color stays. Ctrl/⌘+wheel is ignored, because the browser uses it to zoom.

Reset

Every field has a reset button ↺, which is disabled while the field is at its default. A right-click on a field's row resets that field, and a right-click on any ↺ resets every pin setting. Resetting Enforce turns it on again, resetting 🩻 restores −1, resetting Custom background restores the automatic behaviour that follows the scale, and resetting the color forgets the chosen color.

Enforce as agent pin defaults

It starts checked. When it is checked, agent calls inherit any capture option that they omit from the ⚙️ settings, including the automatic Custom background that follows the call's effective scale. When it is unchecked, omitted options use the built-in defaults, so the background stays transparent even for scaled captures. Options that a call passes explicitly always win and never change the settings.

How pin images are made

Two paths

The direct crop path handles region pins, detached pins, and linked element pins at scale 1 with a transparent background. It crops one fresh frame of the Output capture stream, with no temporary styles and no settle delay. The enhanced path handles linked element pins with a scale other than 1 or a non-transparent background. It temporarily restyles the element, waits for a frame that was captured after that paint, crops it, restores the element, and waits again until the restored paint has reached the stream. A batch with both kinds uses both paths.

The crop

Crops use the output frame's live viewport, so they match at any output zoom, and they cover every bitmap pixel that the pin touches, snapped to whole pixels. When part of the pin lies outside the visible output, the crop is first filled with a hatched placeholder, with the text outside viewport in the middle when the crop is larger than 96 × 36 px, and the visible part is drawn over it. Crops of more than 2,000,000 pixels are scaled down proportionally. Otherwise the PNG's pixels are the capture stream's pixels, whose density relative to CSS pixels depends on the browser and the device pixel ratio.

Freshness

A direct-path crop waits for a newly delivered video frame, and fails after 2 seconds with Timed out waiting for a fresh capture frame. An enhanced capture waits, also up to 2 seconds, for a frame that is newer than the temporary paint, and fails with Capture frame timestamps are unavailable. when the browser gives no timestamps.

One lane

Image work runs one request at a time in a single lane per page. Full-output screenshots, 📸.png and Image…, queue in the same lane, so they never show a temporarily enlarged element. They refuse to run while a restoration is unconfirmed (Pin image restoration has not been confirmed.).

Failures

A failed capture keeps the pin's previous PNG. If the pin has no PNG yet, the error text is stored and written as captureError in its JSON file, and a later success clears it. The panel's thumbnail shows the error. Explicit calls from code reject with the first error, after they have processed every requested pin.

Cancelling

A capture in progress is cancelled by source edits and by changes of the output viewport or zoom; by clearing or closing Pins, output reloads and target changes; by your input in the output (pointer, click, key, wheel, a real scroll, resize, page hide and unload) and by picking input; and by pointer, click or wheel input for Locate. A presentation that never settles is cancelled by a watchdog after the longer of 10 seconds and the settle delay plus 10 seconds.

Automatic refresh

The images of autoCapture pins refresh after the 📸.png refresh that follows a source edit (linked element pins only when mutation observation is on); after DOM mutations; after changes of the output's zoom, viewport or color scheme (ocs), after 80 ms and two frames; after region edits, for the edited pin only; when an Output capture is retargeted after a layout swap or an orientation change; and after a save of the pins that is not about specific pins, which can re-crop all autoCapture pins. Such saves can follow tracked geometry, scroll, layout or zoom changes, comment edits, button toggles, a right-click that applies a default to all pins, and deletions. Starting an Output capture refreshes every pin's image.

Mutation observation

It needs all of these: an Output capture, 🧬 on (the default), and linked element pins with autoCapture: true. The output then observes the document, and the shadow roots on the path of each eligible element, for child, attribute and text changes. Any mutation in those roots queues a refresh for all eligible pins, not only the changed one.

Capture start and stop

When an Output capture starts, all pin images are captured again. When it ends while a folder is shared, all 📍.<N>.png files are deleted and the JSON files are rewritten without image references.

Pin files

Each pin is written as 📍.<N>.json, and as 📍.<N>.png when it has an image. The numbers start at 0 and follow the live pin order. Every save rewrites all the files, so deleting pin 0 renumbers the rest. The writes are quiet, with no toasts, and the playground ignores edits to the files while it runs: it reads them only when a share starts. To change pins, use o.pins. Agent files describes the envelope, every field of the pin, and the metadata of images.

Pin files copy what the output shows into the work folder: the element's text and its full source, its computed styles, the CSS rules that match it, and the output's URL and title. Agents that read the files can read all of it.

Pins from code

An agent drives pins through o.pins from io.eval.js. It is installed on the output frame element in the editor window, at startup and again on every share, and Unshare removes it. Write one request, wait for and read io.eval.json, and only then write the next; top-level await and an implicit final-expression return work as described in File evals. The methods and options are documented in PinList. The shared skill, .agents/skills/pin-use/SKILL.md and .claude/skills/pin-use/SKILL.md, teaches the same to the agent.

• o.pins is a live Array of pin handles: length, indexing, map and iteration work, and it updates in place. JSON.stringify(o.pins) gives o.pins.toJSON().
• Every method except at, toJSON, paused and curtain returns a promise. It resolves to the updated aggregate payload, or rejects with an Error.
• Every mutating call, even a call that changes nothing, such as setOptions(), opens the Pins panel if it is closed and turns picking on. That resumes a paused session and puts the curtain over the output. Call pause() afterwards to hand the output back to the person.
• Pass CSS selectors as targets. From io.eval.js there is no usable Element: the output is a separate document, and DOM nodes cannot cross to it. add pins the first match of each selector, addAll pins all matches.
• Use pierceDepth to look into open shadow roots and same-origin frames: 0 searches the output document only, -1 is unlimited, and n crosses at most n boundaries. Without it, a plain document.querySelector runs in the output document. It applies to add, addAll and setElement.
• Targets of the list methods are indexes, which can be negative, or pin ids. With no target, they act on all pins. Pass several targets as separate arguments: o.pins.delete(0, 2).
• When an Output capture is active, add, addAll and addRect wait for the new pins' PNGs. If the capture fails, the call can reject although the pins were added, and their JSON files then carry captureError. Do not retry such a call blindly.
• The result of a call includes every pin's outerHTML, innerText and cssRules. Return a projection, such as pins.map(({ id, label, comment }) => …), and not the whole payload.
• A pin handle, o.pins.at(i), has read-only getters for the fields of the pin, and settable comment, autoTrack and autoCapture. Those setters are not awaitable, and a failure is only logged to the console. The methods up() and down() move a linked element pin along its ancestors, setElement() points a pin at a new element, and setRect() turns an element pin into a region pin.
• o.pins.default holds the session defaults. Do not pass it, or spread it, into setOptions: it also sends comment: '' and linked: true, so the call can clear comments and relink detached pins.

<style>
  :root    { color-scheme: light dark; }
  main     { display: grid; gap: 1rem; padding: 1rem; }
  #badge   { justify-self: start; padding: .25rem .6rem; border-radius: 1rem; background: oklch(.75 .15 150); }
  x-card   { display: block; }
</style>
<main>
  <h1>Pins demo</h1>
  <span id=badge>new</span>
  <ul id=list><li>One<li>Two<li>Three</ul>
  <x-card></x-card>
</main>
<script>
  customElements.define('x-card', class extends HTMLElement {
    connectedCallback() { this.shadowRoot ?? this.attachShadow({ mode: 'open' }).append(Object.assign(document.createElement('button'), { id: 'inner', textContent: 'Buy' })) }
  })
</script>

const { pins } = await o.pins.add('#badge', { comment: 'Make this pill smaller' })
pins.map(({ id, label, rect, comment }) => ({ id, label, rect, comment }))

await o.pins.addAll('#list li', { comment: 'row' })
await o.pins.addRect({ x: 0, y: 0, width: 320, height: 80, comment: 'header area', autoTrack: false })
o.pins.map(p => `${p.index} ${p.kind} ${p.label}: ${p.comment}`)

await o.pins.add('#inner', { pierceDepth: -1, comment: 'inside x-card' })

await o.pins.at(-1).up()      // button#inner → x-card
await o.pins.at(-1).down(-1)  // back to the original button

await o.pins.at(0).capture({ scale: 8, settle: 0 })

o.pins.default.scale = 3                    // UI setting; omitted agent options inherit it while Enforce is on
o.pins.default.enforceAgentDefaults = false // agents fall back to built-ins (scale 1, transparent)

o.pins.toJSON().pins.map(({ id, kind, selector, label, comment, autoCapture }) => ({ id, kind, selector, label, comment, autoCapture }))

Messages

RTCode.ai pins Coming soon

RTCode.ai is announced to ship its own Pins plugin, with the same capture options. It is not available yet. See Coming soon.

Availability

See also

• Agent 📂 Share™
• Agent files: pin files
• File evals
• Screen capture
• PinList
• o
• Pairing with an AI agent
• Keyboard shortcuts