Align CLI help/grouping and flag semantics for command consistency

[Copilot]

CLI consistency review found drift between gh aw --help, command flag behavior, and setup docs (group placement, shorthand collisions, and inconsistent wording/default semantics). This PR tightens command grouping and normalizes high-impact flag/help inconsistencies to match established CLI patterns.

• Command grouping + root help alignment

  • Assigns outcomes to the analysis group so it no longer appears under “Additional Commands”.
  • Moves status/list analysis assignments into the analysis section in main.go (code organization consistency).
  • Updates root “Common Tasks” to include status and improves audit task wording to reflect broader audit/diff use.

• Flag semantics and shorthand normalization

  • Adds missing shorthand parity:
    • compile --force → -f
    • compile --logical-repo → -l
    • env get/update --repo → -r
  • Removes the trial --repo alias (host-repo alias with conflicting meaning) to eliminate cross-command --repo semantic ambiguity.
  • Removes -e shorthand from logs --engine (filter-only flag) to avoid collision with override-style --engine/-e used by mutating commands.

• Help text consistency cleanup

  • Normalizes selected descriptions for consistency:
    • forecast short description format/terminology ([EXPERIMENTAL], AIC wording)
    • logs short/long description alignment
    • compile --allow-action-refs wording alignment with validation language
    • upgrade --pre-releases hyphenation
    • fix --disable-codemod wording alignment with upgrade flow
    • trial --dry-run/--timeout phrasing clarity.

• Docs + focused test updates

  • Updates docs/src/content/docs/setup/cli.md to match actual CLI output/options (including shorthand changes and corrected add --dir example).
  • Extends command/flag tests for:
    • grouped outcomes (and hidden-but-grouped analysis command pattern),
    • logs --engine no-shorthand behavior,
    • env --repo/-r,
    • compile --force/-f and --logical-repo/-l.

// analysis grouping now includes outcomes
outcomesCmd := cli.NewOutcomesCommand()
outcomesCmd.GroupID = "analysis"

// compile shorthand parity
compileCmd.Flags().BoolP("force", "f", false, "...")
compileCmd.Flags().StringP("logical-repo", "l", "", "...")

// logs engine is filter-only: no shorthand
func addEngineFilterFlag(cmd *cobra.Command) {
    cmd.Flags().String("engine", "", engineFlagUsage("Filter logs by AI engine"))
}

[github-actions]

✅ Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #41727 does not have the 'implementation' label and has ≤100 new lines of code in business logic directories (49 additions detected, threshold is 100).

[github-actions]

✅ PR Code Quality Reviewer completed the code quality review.

[github-actions]

✅ Test Quality Sentinel completed test quality analysis.

[github-actions]

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

[Copilot]

Pull request overview

This PR tightens gh aw CLI consistency by aligning command grouping in root help, normalizing high-impact flag/shorthand semantics, and updating docs/tests to match the intended UX patterns across commands.

Changes:

• Re-group analysis commands (including outcomes) and refresh root “Common Tasks” wording for status/audit.
• Normalize/clarify flag semantics and help text (notably: add compile -f/-l, add env --repo/-r, remove shorthand for logs --engine, remove trial --repo alias).
• Update docs and extend tests to lock in grouping + shorthand behavior.

Show a summary per file

File	Description
pkg/cli/upgrade_command.go	Minor help-text hyphenation consistency for --pre-releases.
pkg/cli/trial_command.go	Removes --repo alias for --host-repo and clarifies --dry-run/--timeout descriptions.
pkg/cli/logs_filtering_test.go	Updates expectations to ensure logs --engine has no shorthand.
pkg/cli/logs_command.go	Aligns logs short description with artifacts-focused behavior.
pkg/cli/logs_command_test.go	Updates logs help assertions; adds explicit shorthand check for --engine.
pkg/cli/forecast_command.go	Normalizes [EXPERIMENTAL] short description wording/format.
pkg/cli/flags.go	Removes shorthand from engine filter flag helper (now --engine only).
pkg/cli/flags_test.go	Adds/updates shorthand consistency tests (including compile -f/-l and logs engine no-shorthand).
pkg/cli/fix_command.go	Clarifies --disable-codemod help text to match intended flow.
pkg/cli/env_command.go	Adds -r shorthand for env get/update --repo and clarifies defaulting behavior.
pkg/cli/env_command_test.go	Verifies env get/update --repo shorthand is -r.
docs/src/content/docs/setup/cli.md	Updates docs to reflect new semantics/shorthands and clearer examples.
cmd/gh-aw/main.go	Moves/normalizes analysis group assignments and adds status to “Common Tasks”; adds compile -f/-l.
cmd/gh-aw/compile_flags_test.go	Adds a targeted test ensuring compile --force/-f and --logical-repo/-l exist.
cmd/gh-aw/command_groups_test.go	Extends grouping tests to ensure outcomes is in the analysis group.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 15/15 changed files
• Comments generated: 2
• Review effort level: Low

[github-actions]

🧪 Test Quality Sentinel Report

✅ Test Quality Score: 85/100 — Excellent

Analyzed 6 test(s): 6 design, 0 implementation, 1 guideline warning (missing assertion messages — does not trigger failure).

📊 Metrics & Test Classification (6 tests analyzed)

Metric	Value
New/modified tests analyzed	6
✅ Design tests (behavioral contracts)	6 (100%)
⚠️ Implementation tests (low value)	0 (0%)
Tests with error/edge cases	5 (83%)
Duplicate test clusters	0
Test inflation detected	Yes (env_command_test.go 4:1, flags_test.go 26:1)
🚨 Coding-guideline violations	1 (missing assertion messages in env_command_test.go — warning only)

Test	File	Classification	Issues Detected
TestCommandGroupAssignments (+2 rows)	cmd/gh-aw/command_groups_test.go:46	✅ Design	Happy-path rows only (no error row in additions)
TestCompileCommandShortFlags (new)	cmd/gh-aw/compile_flags_test.go:7	✅ Design	—
TestNewEnvCommand (additions)	pkg/cli/env_command_test.go:48	✅ Design	⚠️ assert.Equal missing descriptive message
TestShortFlagConsistency (+3 rows)	pkg/cli/flags_test.go:62	✅ Design	—
TestNewLogsCommand (additions)	pkg/cli/logs_command_test.go:18	✅ Design	—
TestLogsCommandFlags (modification)	pkg/cli/logs_filtering_test.go:74	✅ Design	—

Go: 6 (*_test.go); JavaScript: 0. No other languages detected.

⚠️ Flagged Tests — Requires Review (1 issue)

TestNewEnvCommand (pkg/cli/env_command_test.go:51–54) — ⚠️ Missing assertion messages: assert.Equal(t, "r", getRepoFlag.Shorthand) and assert.Equal(t, "r", updateRepoFlag.Shorthand) have no descriptive context string. When these fail, the output will just say Not equal: expected "r" with no hint about which command or flag. Suggested fix: add a message, e.g. assert.Equal(t, "r", getRepoFlag.Shorthand, "env get --repo should have -r shorthand").

Verdict

✅ Check passed. 0% implementation tests (threshold: 30%). All 6 tests verify observable CLI contracts (flag shorthands, command group assignments, command descriptions). No mock-library violations and no missing build tags detected. The new compile_flags_test.go correctly carries //go:build !integration.

Note on inflation: flags_test.go shows a 26:1 ratio vs flags.go, but this is a cross-cutting consistency test that validates flag behavior across several production files — the ratio is expected and not a quality concern.

🧪 Test quality analysis by Test Quality Sentinel · 55.9 AIC · ⌖ 13.5 AIC · ⊞ 8.4K · ◷

[github-actions]

✅ Test Quality Sentinel: 85/100. Test quality is acceptable — 0% of new tests are implementation tests (threshold: 30%).

[github-actions]

REQUEST_CHANGES — one blocking issue, one non-blocking concern

Blocking: trial --repo was a hidden but working alias for --host-repo. This PR removes it outright, causing unknown flag: --repo for any script or pipeline that relied on it. The cobra MarkDeprecated path (keep-flag + print warning) is a one-line fix and gives users a migration window.

Non-blocking observations

• Dropped --timeout 0 concern: The description change ('set to 0 to disable timeout') is actually correct — PollWithSignalHandling explicitly checks options.Timeout > 0 before applying the deadline, so passing 0 polls indefinitely. No issue there.
• compile_flags_test.go design: Tests the package-global compileCmd directly, while flags_test.go already covers the same flags via createCompileCommandStub(). Both files need updating on every shorthand change. Consider a factory function to enable isolated testing, but this is not blocking.
• The shorthand additions (-f, -l on compile; -r on env get/update), grouping changes, and help text cleanups are well-structured and the test coverage is appropriate.

🔎 Code quality review by PR Code Quality Reviewer · 102.6 AIC · ⌖ 7.04 AIC · ⊞ 5.2K

[github-actions]

Silent removal of hidden --repo alias breaks scripts without a deprecation warning.

The --repo flag was a working alias for --host-repo — hidden, but still functional. Removing it outright causes gh aw trial ... --repo myorg/myrepo to fail with unknown flag: --repo rather than guiding users to the correct flag.

💡 Suggested fix

Use MarkDeprecated to preserve compatibility while discouraging use — cobra prints a warning on stderr but the flag still works:

cmd.Flags().String("repo", "", "Alias for --host-repo")
_ = cmd.Flags().MarkHidden("repo")
_ = cmd.Flags().MarkDeprecated("repo", "use --host-repo instead")

This gives users a migration window without a silent break in automation scripts or CI pipelines that relied on the hidden alias.

[github-actions]

Testing the real compileCmd global duplicates assertions already in flags_test.go and couples tests to shared init state.

The new entries in flags_test.go already verify --force/-f and --logical-repo/-l via createCompileCommandStub(). This file then re-tests the same properties against the package-global compileCmd — two different mechanisms asserting the same invariant means two files that must be updated together on every shorthand change.

💡 Details

The two-layer approach (stub for pattern consistency + real command for registration correctness) is reasonable in principle, but the implementation has a subtle risk: compileCmd is shared mutable global state initialized by init(). Any test in package main that registers or removes a flag on compileCmd (even in a t.Cleanup) would silently corrupt this test's expectations.

A safer approach is to extract command construction into a factory function:

// in main.go
func newCompileCommand() *cobra.Command { ... }

That lets both this test and the stub test use an isolated instance with no shared-state risk. Non-blocking, but worth addressing before the pattern spreads to other command tests.

[github-actions]

Skills-Based Review 🧠

Applied /improve-codebase-architecture, /tdd, /zoom-out, and /grill-with-docs — observations only, no blocking issues.

📋 Key Themes & Highlights

Key Themes

• Stale comment (flags.go): addEngineFilterFlag doc-comment still says --engine/-e after -e was removed — a 1-line fix.
• Missing regression test: The trial --repo alias removal has no corresponding "flag is absent" test; easy to add and documents the API contract.
• Stub drift risk: createCompileCommandStub in flags_test.go now needs to be manually kept in sync with the real compileCmd in two places. A clarifying comment would help.
• Description precision: logs Short drops "aggregated metrics" — worth confirming whether that capability should remain discoverable.

Positive Highlights

• ✅ Clean split of addEngineFlag (override, keeps -e) vs addEngineFilterFlag (filter-only, no shorthand) — the semantic distinction is now enforced at the type level, not just by convention.
• ✅ trial --repo removal is surgical: the hidden alias, its conditional logic, and the mutual-exclusion constraint are all cleaned up together.
• ✅ New shorthand parity (-f, -l, -r) is well-covered by both stub-based consistency tests and a direct real-command test in compile_flags_test.go.
• ✅ outcomes grouping fix (analysis instead of Additional Commands) is a small change with good UX impact.
• ✅ PR description is thorough — clearly explains every change category and why.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 103.7 AIC · ⌖ 7.24 AIC · ⊞ 6.5K

[github-actions]

[/improve-codebase-architecture] Stale comment: addEngineFilterFlag now registers --engine without a shorthand, but the doc-comment on line 17 still says --engine/-e. This directly contradicts the implementation after the PR removes -e.

💡 Suggested fix

Change line 17 from:

// addEngineFilterFlag adds the --engine/-e flag to a command for filtering.

to:

// addEngineFilterFlag adds the --engine flag (no shorthand) to a command for filtering.

The intent to distinguish filter-only from override-style flags is clear in the implementation — the comment just needs to match.

[github-actions]

[/tdd] Missing regression test for --repo flag removal. The --repo alias was deliberately removed to eliminate cross-command semantic ambiguity — there's no test asserting the flag is absent, so it could be silently re-introduced.

💡 Suggested test addition

Add to pkg/cli/trial_command_test.go (or a new test file):

func TestTrialCommandRepoFlagRemoved(t *testing.T) {
    cmd := NewTrialCommand()
    if cmd.Flags().Lookup("repo") != nil {
        t.Error("trial command should not have --repo flag (semantic conflict with other commands)")
    }
}

This is a deliberate API contract: trial uses --host-repo, not --repo. A regression test documents that intent and prevents accidental re-introduction.

[github-actions]

[/zoom-out] createCompileCommandStub manually mirrors the real compileCmd flag set. With -f and -l added here AND tested directly against the real compileCmd in cmd/gh-aw/compile_flags_test.go, there are now two places to update when compile flags change. If the stub drifts (e.g. a new shorthand is added to the real command but not the stub), the consistency tests in this file can report a false pass.

💡 Consideration

The stub exists because pkg/cli tests can't import cmd/gh-aw. That's a valid constraint. But it's worth adding a comment near the stub noting it must be kept in sync with cmd/gh-aw/main.go, and that compile_flags_test.go is the authoritative test for the real command:

// createCompileCommandStub mirrors the shorthand flags defined on compileCmd
// in cmd/gh-aw/main.go. Keep this in sync when adding new shorthands.
// The authoritative test for the real command is cmd/gh-aw/compile_flags_test.go.
func createCompileCommandStub() *cobra.Command {

[github-actions]

[/grill-with-docs] The new short description "Download and analyze agentic workflow logs and artifacts" drops "with aggregated metrics". If the command still surfaces aggregated metrics (token counts, cost summaries, latency), users searching for that capability won't find it via --help or tab-completion. Consider keeping the analytics signal:

"Download and analyze agentic workflow logs, artifacts, and metrics"

If artifacts is the key new capability being highlighted, leading with it rather than appending and artifacts might make it clearer why the description changed.

[pelikhan]

@copilot run pr-finisher

[Copilot]

@copilot run pr-finisher

Done. Addressed the two reviewer findings in commit fix: update addEngineFilterFlag comment and align logs docs description:

• pkg/cli/flags.go: Updated addEngineFilterFlag doc comment to remove the stale /-e shorthand reference and clarify why no shorthand is registered.
• docs/src/content/docs/setup/cli.md: Aligned the gh aw logs table description with the command's updated short description ("Download and analyze agentic workflow logs and artifacts").

make fmt, make lint, and make test-unit all pass locally.