fix: support preview annotations in same-origin iframes

[StiensWout]

Summary

• Keep the preview annotation picker running only in the top document while adding a transparent capture surface so iframe areas still deliver picker pointer input.
• Add same-origin/srcdoc iframe-aware hit testing and viewport-rect translation for hover outlines, selected outlines, marquee selection, erasing, screenshot crop rects, and capture payloads.
• Treat inaccessible cross-origin iframes conservatively as the iframe element/region surface.
• Replaces [codex] fix: Support iframe preview annotations #3160 with a fresh branch based on current main.

Root cause

The picker listened for pointer events and used document.elementsFromPoint only in the top document. Pointer events over iframe content were delivered to the frame instead, so the top picker never saw the interaction and could not show the selection editor for elements inside same-origin/srcdoc iframes.

Impact

Preview annotations can now select elements inside accessible same-origin/srcdoc iframes while preserving a single top-frame toolbar/editor and the existing preview webview security preferences. Drawing and region workflows still operate from the top overlay, including over iframe areas.

Validation

• PATH="$HOME/.vite-plus/bin:$PATH" vp run --filter @t3tools/desktop typecheck
• PATH="$HOME/.vite-plus/bin:$PATH" vp run --filter @t3tools/desktop test
• PATH="$HOME/.vite-plus/bin:$PATH" vp check
• PATH="$HOME/.vite-plus/bin:$PATH" vp run typecheck

vp check reports the repository's existing unrelated warnings and no errors.

Closes #3141

---

Note

Medium Risk
Large change to preview preload pointer/wheel/scroll behavior and coordinate math; regressions could affect top-frame-only flows, cross-origin iframe edge cases, or teardown of frame scroll listeners.

Overview
Preview element picking now works for same-origin / srcdoc iframes while the annotation UI stays in the top document. Hit testing recurses into accessible frame documents (pickFromDocument), skips opaque cross-origin frames (treats the iframe as the target), and maps element geometry into top-level viewport coordinates for hover/selection outlines, marquee select, erase, screenshot bounds, and capture rects.

Input handling moves from capture-phase window listeners to a full-screen transparent captureSurface with pointer capture, so clicks over iframe areas still reach the picker. Wheel events are forwarded through that surface (including nested frames) so the page can scroll during annotation; nested frame scroll listeners trigger repaints so outlines stay aligned.

main.ts only switches node:os to a namespace import for homedir().

^{Reviewed by Cursor Bugbot for commit 5c01d2e. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Support preview annotations in same-origin iframes

• Adds recursive frame traversal to element picking (pickFromDocument) and marquee selection (collectElementsInRect), enabling annotation interactions inside same-origin iframes.
• Introduces viewport offset calculation (getDocumentViewportOffset) that climbs the frame hierarchy to produce accurate overlay positions for elements in nested frames.
• Replaces window-level pointer/wheel listeners with a dedicated captureSurface element that captures pointer events without blocking hover visuals; wheel events are forwarded to underlying scrollable elements across frames.
• Attaches scroll listeners to nested frame documents (refreshFrameScrollListeners) so selected outlines and visuals stay in sync when inner frames scroll.
• Cross-origin iframes are safely skipped via try/catch guards in getAccessibleFrameDocument.
• Behavioral Change: annotation payloads now include viewport-relative rects instead of local getBoundingClientRect values; disconnected or out-of-viewport elements are skipped during submission.

^{Macroscope summarized 5c01d2e.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 2bd7cd68-dc31-45b0-887e-8025d38046f6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR introduces significant new capability (iframe annotation support) with ~480 lines of new logic including recursive document traversal, pointer capture, and wheel event proxying. Additionally, there are 4 unresolved review comments identifying potential bugs in the wheel handling and URL attribution for iframe elements.

^{You can customize Macroscope's approvability policy. Learn more.}

[cursor]

Top-frame wheel delta scaling

Medium Severity

The normalizeWheelDelta function incorrectly scales DOM_DELTA_LINE and DOM_DELTA_PAGE wheel events using the top-level document's metrics. This leads to nested same-origin iframes scrolling by the parent's calculated pixel delta instead of their own viewport-relative values.

 

^{Reviewed by Cursor Bugbot for commit 2bc5b04. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

There are 4 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 5c01d2e. Configure here.}

[cursor]

Cross-origin iframe wheel blocked

Medium Severity

The new full-screen captureSurface handles all wheel input, and scrolling depends on scrollDocumentAtPoint, which only enters frames via getAccessibleFrameDocument. Cross-origin iframe documents stay inaccessible, so wheel gestures over embedded cross-origin content no longer scroll that content (and may scroll the top page instead), unlike before when the overlay did not intercept wheel events.

Additional Locations (1)

• apps/desktop/src/preview/PickPreload.ts#L1233-L1259

 

^{Reviewed by Cursor Bugbot for commit 5c01d2e. Configure here.}

[cursor]

Iframe picks use top page URL

Low Severity

Elements selected inside a same-origin iframe still get pageUrl and pageTitle from the top-frame location and document.title in captureElement and the annotation payload. Downstream consumers can associate iframe-local selectors and stack frames with the wrong page after this iframe picking support.

Additional Locations (1)

• apps/desktop/src/preview/PickPreload.ts#L1586-L1590

 

^{Reviewed by Cursor Bugbot for commit 5c01d2e. Configure here.}

[cursor]

Wheel scrolls parent through iframe

Medium Severity

In scrollDocumentAtPoint, when the pointer is over an accessible iframe’s content and nested scrolling does not consume the wheel delta, the loop keeps walking elementsFromPoint hits below the iframe and may scroll parent scrollables or the parent defaultView. That can move the top document while the cursor is over iframe content, which conflicts with treating iframe regions as the interaction surface.

 

^{Reviewed by Cursor Bugbot for commit 5c01d2e. Configure here.}