Skip to content

feat: add TV remote command#1147

Open
thymikee wants to merge 4 commits into
mainfrom
agent/tv-remote-support
Open

feat: add TV remote command#1147
thymikee wants to merge 4 commits into
mainfrom
agent/tv-remote-support

Conversation

@thymikee

@thymikee thymikee commented Jul 7, 2026

Copy link
Copy Markdown
Member

Summary

Add first-class tv-remote support with a d-pad CLI alias across CLI, runtime, client, MCP, command descriptors, and platform capability routing.

Wire Android TV to D-pad keyevents and tvOS to the Apple remote press path, then improve the operator/API ergonomics around that surface: data-driven TV button semantics with ok/center/enter aliases, explicit Android longpress vs tvOS hold-duration docs, a dedicated help tv topic, website/client examples, reusable focus capture policy, and a narrow system-command Node client projection from command facets.

Validation

  • pnpm format
  • pnpm build
  • pnpm check:quick
  • pnpm exec vitest run src/commands/system/index.test.ts src/core/__tests__/dispatch-tv-remote.test.ts src/commands/system/runtime/system.test.ts src/mcp/__tests__/command-tools.test.ts src/cli/parser/__tests__/cli-help-topics.test.ts src/commands/interaction/runtime/selector-read.test.ts src/commands/interaction/runtime/selector-capture-policy.test.ts src/__tests__/cli-client-commands.test.ts src/core/command-descriptor
  • pnpm test:skillgym:case android-tv-focus-uses-tv-remote
  • pnpm exec vitest run src/platforms/android/__tests__/input-actions.test.ts
  • pnpm exec vitest run src/daemon/__tests__/runtime-hints.test.ts src/platforms/__tests__/install-source.test.ts src/platforms/apple/core/__tests__/runner-client.test.ts src/platforms/apple/core/__tests__/runner-session.test.ts
  • Manual Android TV emulator: tv-remote, d-pad, select, and screenshot --overlay-refs
  • Manual tvOS simulator: tv-remote, focused predicate, and screenshot --overlay-refs

Full pnpm check:unit was rerun and hit unrelated host-load timing/slow-test failures; every failed file passed when rerun in isolation.

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.6 MB 1.6 MB +8.2 kB
JS gzip 502.9 kB 505.2 kB +2.2 kB
npm tarball 601.5 kB 603.2 kB +1.7 kB
npm unpacked 2.1 MB 2.1 MB +3.1 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 17.1 ms 16.7 ms -0.3 ms
CLI --help 32.4 ms 32.3 ms -0.1 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/tv-remote.js +73.6 kB +23.7 kB
dist/src/cli-help.js +3.0 kB +831 B
dist/src/registry.js +1.4 kB +463 B
dist/src/internal/daemon.js +1.0 kB +225 B
dist/src/runtime.js +644 B +147 B

@thymikee

thymikee commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

CI has two actionable blockers:

  • Fallow: src/core/tv-remote.ts:27 isTvRemoteButton is an unused export. Remove it or wire it to a real consumer.
  • Integration progress: pnpm test:integration:progress:check reports missing provider-backed integration command coverage for public command tv-remote (51/52 covered).

Please add the provider-backed integration coverage/classification for tv-remote, rerun pnpm check:fallow --base origin/main and pnpm test:integration:progress:check, and reply with what changed plus any remaining blocker.

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://callstack.github.io/agent-device/pr-preview/pr-1147/

Built to branch gh-pages at 2026-07-07 16:38 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@thymikee

thymikee commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

Review: feat: add TV remote command (#1147)

Adds a first-class tv-remote command (with d-pad CLI alias) that routes Android TV to D-pad keyevents and tvOS to the Apple remote press path, plus a focused selector/predicate, a help tv topic, and a data-driven button model. The command wiring is thorough and consistently propagated across CLI → daemon → client → MCP → capability routing, and it's well tested. I verified end-to-end that durationMs survives the daemon writer (rides metadataFlags, not positionals), handleTvRemoteCommand always receives a single normalized positional (the CLI reader/daemon writer strip press/aliases before dispatch), the back → menu Apple mapping matches the runner remoteButton contract, the projected system client preserves all 8 methods, and focused was added consistently across the predicate, selector parser, matcher, and CLI validator. The includeRects refactor is behavior-preserving (all backends compare === true).

Two things worth a look, one behavioral and one cleanup:

1. focused reads request an interactive-only snapshot, which can drop the focused node on Android TV (false negative)

src/commands/interaction/runtime/selector-capture-policy.ts:19deriveSelectorCapturePolicy sets interactiveOnly: true for predicate: 'focused' and for any focused=true selector, and isCommand/waitForSelector/resolveSelectorNode now honor it.

On Android, a snapshot node's hittable is derived as attrs.clickable ?? attrs.focusable (src/platforms/android/ui-hierarchy.ts:495). Because uiautomator dumps always emit a clickable attribute, the ?? never falls through — hittable collapses to the clickable value. A node that is focusable="true" focused="true" clickable="false" (a focusable row/container in a leanback list, a focused text label, etc.) is therefore hittable:false. shouldIncludeInteractiveAndroidNode (ui-hierarchy.ts:730) keeps such a node only if it has meaningful text/id and a hittable ancestor/descendant or a collection ancestor — otherwise it's filtered out of the interactive tree.

Failure scenario: move focus onto a focusable-but-not-clickable Android TV container, then run agent-device is focused '<selector>' (or wait focused=true, or target focused=true). The focused node is absent from the interactiveOnly snapshot, so is reports pass:false, wait times out, and a focused=true target fails to resolve — even though the element is genuinely focused. This is exactly the verification workflow the PR documents in help tv ("If a fresh snapshot exposes a focused node, verify it with is focused"). The full (non-interactive) tree that focus reads would use without this policy contains the node. Worth confirming the design intent — reading focus off the full tree (or including focused nodes in the interactive filter) would avoid the narrowing.

2. durationMode is uniform dead generality

src/core/tv-remote.ts — all 8 entries in TV_REMOTE_BUTTON_DEFINITIONS carry an identical durationMode: { android: 'longpress', apple: 'exact' }, and the sole reader tvRemoteDurationMode hardcodes TV_REMOTE_BUTTON_DEFINITIONS.select.durationMode[platform] — it never varies by button. The actual Android longpress decision lives in pressAndroidTvRemote (durationMs && durationMs > 0), so this field is purely help-text data dressed up as per-button config. A single module-level constant would say the same thing with 7 fewer copies to keep in sync.

(Minor, related: the hint string 'tv-remote is supported only on Android TV targets.' is duplicated verbatim in register-builtins.ts and apple/plugin.ts, and '...only on tvOS devices.' in apple/plugin.ts — a TV_REMOTE_*_HINT constant in core/tv-remote.ts would collapse the wording-drift risk, though the parity-test copies are somewhat deliberate as independent oracles.)


🤖 Generated with Claude Code · /review

@thymikee

thymikee commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

Re-review after 2bab15a: no actionable blockers found. The focused-selector issue is addressed by keeping focused reads on full snapshots and preserving focused Android nodes in the interactive filter; the durationMode cleanup is now a module-level constant; provider-backed tv-remote coverage is present; and CI is green across 22 checks. The PR also includes manual Android TV emulator and tvOS simulator evidence. Labeling ready-for-human for maintainer judgment.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant