Skip to content

fix(telemetry): keep telemetry inert during shell completion (BE-4274)#586

Open
mattmillerai wants to merge 2 commits into
mainfrom
matt/be-4274-posthog-completion
Open

fix(telemetry): keep telemetry inert during shell completion (BE-4274)#586
mattmillerai wants to merge 2 commits into
mainfrom
matt/be-4274-posthog-completion

Conversation

@mattmillerai

Copy link
Copy Markdown
Collaborator

ELI-5

When you press Tab to autocomplete a comfy ... command, the CLI runs a tiny hidden invocation just to list the possible completions. It should do nothing else — but a fish user reported (GitHub #506) that this hidden run was still spinning up the PostHog analytics client (a background thread + network setup). This PR makes sure telemetry stays completely asleep during tab-completion, and adds a test so it never wakes up there again.

What changed

  • comfy_cli/tracking.py: added _in_shell_completion() (detects the _<PROG>_COMPLETE env var Click/Typer set during completion, across comfy / comfy-cli / comfycli and any alias, for fish/bash/zsh) and an early-return guard in _get_providers() so no telemetry provider (PostHog or Mixpanel) is constructed on the completion path. The guard is uncached, so a real invocation is never affected.
  • tests/comfy_cli/test_tracking_completion.py: new regression tests — detection matrix across shells/entrypoints, a gateway test asserting _get_providers() builds nothing under completion, and an end-to-end test that drives the real Typer completion resolution for fish/bash/zsh and asserts the PostHog client is never constructed.

Honest note on scope (please read)

While triaging I found the behavior was already fixed incidentally by #535 (BE-3289, "build telemetry providers lazily"), which landed on main earlier the same day — after #506 was filed (2026-07-11). With lazy construction, providers are built only inside _dispatch(), which never runs during completion. I verified this empirically: driving the actual Typer fish completion on clean origin/main resolves the full command list and constructs zero telemetry providers (no posthog import, no PostHogProvider.__init__, no _get_providers call).

So this PR is deliberately not a behavior fix on top of a broken main. Its value is:

  1. The regression test the ticket explicitly asks for (acceptance criterion 3) — which did not exist. This is the real, lasting deliverable: it locks PostHog client is initialized unconditionally during fish shell completion, adding ~10 seconds to every completion #506's fix so a future change that eagerly builds providers can't silently re-break completion.
  2. An explicit guard that encodes completion-inertness intentionally at the construction gateway, rather than leaving it an incidental side effect of fix: build telemetry providers lazily so comfy install can upgrade pydantic_core (BE-3289) #535's lazy-init (whose motivation was a Windows pydantic_core DLL lock, unrelated to completion). The two concerns are only coupled by accident today.

Acceptance criteria

Testing

  • pytest tests/comfy_cli/test_tracking_completion.py tests/comfy_cli/test_tracking_providers.py tests/comfy_cli/test_tracking.py → 142 passed.
  • ruff check + ruff format --diff (CI-pinned 0.15.15), repo-wide → clean.

Add an explicit shell-completion guard to the telemetry provider gateway
so no PostHog/Mixpanel client is stood up while fish/bash/zsh resolve tab
completion (GitHub #506). The lazy provider construction from #535 already
made completion inert as a side effect (providers build only on dispatch,
which never runs during completion); this makes the guarantee explicit and
independent of when providers get built, and adds the regression test the
ticket asks for.
@mattmillerai mattmillerai added agent-coded PR authored by the agent-work loop cursor-review Request Cursor bot review labels Jul 24, 2026
@mattmillerai
mattmillerai marked this pull request as ready for review July 24, 2026 05:20
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 22 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 15d742ab-bb59-48d2-a555-fee5d700270e

📥 Commits

Reviewing files that changed from the base of the PR and between 85b62da and cb46e4b.

📒 Files selected for processing (2)
  • comfy_cli/tracking.py
  • tests/comfy_cli/test_tracking_completion.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch matt/be-4274-posthog-completion
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch matt/be-4274-posthog-completion

Comment @coderabbitai help to get the list of available commands.

@dosubot dosubot Bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Jul 24, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 2 finding(s).

Severity Count
🟢 Low 2

Panel: 6/8 reviewers contributed findings.

Reviewers that did not contribute: kimi-k2.5:adversarial (empty), kimi-k2.5:edge-case (empty)

Comment thread comfy_cli/tracking.py Outdated
Comment thread comfy_cli/tracking.py Outdated
…E-4274)

- Set COMP_WORDS/COMP_CWORD in the end-to-end completion test so Typer's
  bash arg-resolution path (which reads them directly, unlike zsh/fish's
  _TYPER_COMPLETE_ARGS) no longer raises KeyError — fixes the red `build` job.
- _in_shell_completion(): require a recognized completion instruction token
  in the _*_COMPLETE value so a stray/empty user-exported _FOO_COMPLETE can't
  silently suppress telemetry on a real run, and snapshot os.environ keys with
  list() to avoid RuntimeError under concurrent env mutation (cursor-review).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-coded PR authored by the agent-work loop cursor-review Request Cursor bot review size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant