File evals (io.eval.js and o.eval.js)

Two channels

There are two request and result pairs. Both need an active share, and the request must be a top-level file in the work folder.

Both channels are independent of each other. See Agent files for where the four files sit in the folder.

The request and result protocol

printf '%s' 'Code.D' > io.eval.tmp && mv io.eval.tmp io.eval.js
while [ -e io.eval.js ]; do sleep 0.05; done
until [ -e io.eval.json ] && [ ! -e io.eval.json.crswap ]; do sleep 0.05; done
cat io.eval.json

What a request may contain

RTCode compiles the request, without running it, to pick a form:

• If (async()=>(<source>\n))() compiles, the whole request is one expression and its value is returned. A bare { answer: 42 } returns an object.
• Otherwise the request runs as statements in the body of an async function. A newline is appended when it is missing, so a trailing // comment is safe.

For statements, RTCode inserts return before the last non-empty top-level statement when that statement is an expression. The parser that does this is loaded when the share starts. A declaration, a block, an if, a loop or any other statement in last place returns nothing. An explicit top-level return works anywhere, and automatic semicolon insertion applies: return\n42 returns undefined. If the parser cannot load, the console warns agent eval final expression unavailable once. Single expressions still return, but statement requests return only through an explicit return.

• Top-level await and dynamic import(url) work in both channels. Static import and export declarations are syntax errors, because the request runs inside a function: SyntaxError: Cannot use import statement outside a module.
• Invalid syntax never executes. The request is still consumed, and the result is the text of the SyntaxError, for example SyntaxError: Unexpected token '='.
• Runtime errors execute exactly once, including a SyntaxError thrown at runtime, and rejected promises or final awaits.
• A multi-property object literal followed by ;, such as { a: 1, b: 2 };, is parsed as a block and fails with SyntaxError: Unexpected token ':'. Wrap object results in parentheses: ({ a: 1, b: 2 }).

const before = sel;
sel = [[1, 1, 1, 1, 0]];
await new Promise(r => setTimeout(r, 50));
({ before, after: sel })

io.eval.js: the playground

io.eval.js is evaluated as the body of a function in strict mode, in the playground's global scope. The globals and accessors of the Playground API are reachable: sel, Code, codeTitle, edge, o, s, i, ovp, oz and so on.

• Assigning an undeclared name throws a ReferenceError. Write window.name = … instead.
• this is undefined.
• const, let and var stay local to one request.
• document is the playground page, not the output.
• There is no timeout.
• Console calls made here never reach 📜.json.

Reading the playground's state is a single expression. The routes, the title and the selections:

({ routes: Code.D, title: codeTitle, selections: sel })

io.eval.json then holds two-space JSON without a final newline, for example:

{
  "routes": [
    "demo.rt.ht"
  ],
  "title": "Demo",
  "selections": [
    [
      3,
      1,
      3,
      1,
      0
    ]
  ]
}

Because each request runs in its own scope and in strict mode, keep state across requests on window. A bare count2 = 1 throws ReferenceError: count2 is not defined, while this form returns 1, then 2, and so on:

window.count = (window.count ?? 0) + 1

o.eval.js: the output

Before an o.eval.js runs, RTCode waits for the editor's latest input-to-output sync, and for the output to initialize and become ready. If an earlier initialization had failed, it initializes the output again, once per request. If the output is not ready in time, the result is an error text, for example a stack that starts with Error: Output document readiness timed out. The request has been consumed by then and does not run.

<button id=x>Save</button>

window.x.textContent

[...document.querySelectorAll('h1, h2')].map(h => ({ tag: h.localName, text: h.textContent.trim(), box: h.getBoundingClientRect().toJSON() }))

window.job ??= { done: false, started: Date.now() };
setTimeout(() => job.done = true, 30_000);
'started'

Failures, retries and overlapping evals

• Read failures. If reading the request fails while the file still exists (it is locked, or in the middle of a replace, or NotReadableError), the read error is written to the result file and the run is retried after 1, 2, 4, 8 and 16 seconds. That makes at most five re-runs, each with the file system backoff of about 2.15 seconds. A re-run is skipped while another run of the same request name is active. Success overwrites the error. A request that disappears stops the retries quietly. After five failures the request stays until its next save.
• Runtime errors are never retried. The request is already consumed and has run once.
• A request that is already gone when its read fails, for example because a duplicate notification consumed it, returns quietly without an error file, after up to about 2.15 seconds of read retries.
• Evals overlap. Only consumption is serialized. A new request runs while an earlier one is still awaiting. Results are written in completion order, so whichever eval finishes last owns the result file. The io and o channels are independent. Writing identical content twice runs it twice. Duplicate notifications for one write run it only once.
• Unshare and a new share cancel pending retries. Runs still in flight from the old share write no results, and waiting claims are dropped.

Diagnostics: Test Playground and Output File Evals

When evals do not seem to run, use Diagnostics: Test Playground and Output File Evals in the command palette. It is available while a folder is shared, and has no default shortcut. It runs a playground round trip (io.eval.js to io.eval.json) and then an output round trip (o.eval.js to o.eval.json) through the real files, each with a timeout of 1 second.

Each round trip works like this:

1. It deletes the existing result file, so your last result is lost.
2. It writes the request return { token: "<uuid>", context: window === top ? 'playground' : 'output', value: 6 * 7 };.
3. It polls for the result every 100 ms.
4. It requires JSON with the same token, the expected context and the value 42.

The probe's files are quiet: no transfer toasts and no screenshot refresh. It removes its own request if it is still there. Calling the command again while it runs joins the running check. Each context shows one toast: Playground eval: or Output eval: followed by Success, Failure or Busy, then <input> → <output>, then the message and the time in milliseconds.

Cautions

Deploying from an eval, with await edge.save.limit({ message }), is part of what the agent guide teaches agents. See Agent files.

Availability

See also

• Agent 📂 Share™
• Agent files
• Playground API
• o
• PinList
• edge.save.limit()
• Console and network
• Pairing with an AI agent