Skip to content

aitools: bug bash round 2 (uninstall confirm, version scope, official marketplace)#5746

Merged
simonfaltum merged 6 commits into
mainfrom
simonfaltum/aitools-bugbash2
Jun 29, 2026
Merged

aitools: bug bash round 2 (uninstall confirm, version scope, official marketplace)#5746
simonfaltum merged 6 commits into
mainfrom
simonfaltum/aitools-bugbash2

Conversation

@simonfaltum

@simonfaltum simonfaltum commented Jun 26, 2026

Copy link
Copy Markdown
Member

Stack

Part of the aitools plugin-first redesign. Open PRs, merge bottom to top:

Merged: #5734, #5736, #5737, #5738, #5739.

Why

A second round of testing the plugin-first databricks aitools CLI. Findings are
tracked in documents/proposals/aitools-bugbash-findings.md.

Changes

  • uninstall confirms in interactive mode. It removed skills and de-registered
    plugins immediately; now (on a TTY) it summarizes what will be removed and asks
    "Proceed?". Non-interactive runs are unaffected.
  • version labels the scope. Skills and plugins are now always labeled
    global/project (e.g. Plugin (Claude Code, global): latest).
  • Claude installs from the official marketplace. The databricks plugin is in
    Claude's built-in claude-plugins-official marketplace, so Claude installs
    databricks@claude-plugins-official. A built-in marketplace (empty
    PluginSpec.Source) is never added or de-registered. Codex/Copilot keep our own
    marketplace.
  • Picker only offers actionable agents. Every detected agent still shows in the
    detection list with its state/reason, but only agents that will do something
    (plugin or skills) are selectable; agents skipped in the scope are no longer
    checkboxes.
  • Cursor is a plain skills agent. Cursor was a confusing "manual, run
    /add-plugin" option that did nothing. Since the Cursor plugin can't be installed
    or interacted with headlessly, all references to it are removed: Cursor now
    installs raw skills like OpenCode/Antigravity, labeled simply "skills". This
    removed the whole manual-only concept (PluginSpec.ManualOnly/ManualInstructions,
    StateManualOnly, ReasonManualOnly, the manual_add_plugin list status, and the
    deliveryManualCursor path).
  • Help links the source repo. databricks aitools help points at
    https://github.com/databricks/databricks-agent-skills so users know where skills
    and plugins come from.

Test plan

  • Unit: uninstall confirm; version scope labels; built-in marketplace install
    (no marketplace add) and uninstall (no de-register); picker offers only
    actionable agents; Cursor plan = skills; no-plugin install guard.
  • go test ./libs/aitools/... ./cmd/aitools/... and acceptance experimental/aitools pass.
  • ./task fmt-q, ./task lint-q (0 issues), ./task checks (no dead code) clean.

This pull request and its description were written by Isaac.

@simonfaltum simonfaltum changed the title aitools: bug bash round 2 (uninstall confirm, ...) aitools: bug bash round 2 (uninstall confirm, version scope, official marketplace) Jun 26, 2026
github-merge-queue Bot pushed a commit that referenced this pull request Jun 29, 2026
<!-- aitools-stack -->
### Stack

Part of the `aitools` plugin-first redesign. Open PRs, merge bottom to
top:

- #5740 Uninstall teardown + legacy skills cleanup
- **#5741 list + version plugin state  ← this PR**
- #5745 Bug bash (wording, latest-by-default skills, project-scope plan,
version reporting)
- #5746 Bug bash round 2 (uninstall confirm, picker, Cursor, official
marketplace, version scope)

Merged: #5734, #5736, #5737, #5738, #5739.
<!-- /aitools-stack -->

## Why

`list` and `version` only knew about raw skills. Now that the CLI
installs plugins, they should report the real per-agent plugin state so
users (and the plugin's SessionStart nudge) can see what's installed and
whether it's current.

Stacked on #5740 (uninstall).

## Changes

- **`list --output json`** gains an additive `agents[]` array: each
plugin agent with a recorded install (its `version` and an `up_to_date`
/ `update_available` status, `managed: true`), plus Cursor as
`manual_add_plugin` (`managed: false`) when it's present. The existing
`release` / `skills` / `summary` keys and shapes are byte-identical
(additive `omitempty` field). The text view adds an `AGENT / STATUS`
block only when there are agent entries, leaving the skills table and
summary line unchanged.
- **`version`** prints a `Plugin (<Agent>): vX` line per recorded
plugin, and only prints the skills line when skills are actually
installed, so a plugin-only install no longer reports "0 skills".
Existing skills output is unchanged.

## Test plan

- [x] Unit tests: `list` JSON includes `agents[]` with the right
statuses/managed flags while keeping `release`/`skills`/`summary`
intact; `buildAgentEntries` maps recorded plugins to
up_to_date/update_available and surfaces Cursor as manual; `version`
prints the plugin line for a plugin-only install.
- [x] Existing `list`/`version` tests pass unchanged (JSON round-trip,
both-scopes skills output).
- [x] `go test ./libs/aitools/... ./cmd/aitools/...` passes.
- [x] `./task fmt-q`, `./task lint-q` (0 issues), `./task checks` (no
dead code) clean.

This pull request and its description were written by Isaac.
@simonfaltum simonfaltum force-pushed the simonfaltum/aitools-bugbash2 branch from b67c4b4 to 6564b5a Compare June 29, 2026 08:55
@simonfaltum simonfaltum force-pushed the simonfaltum/aitools-bugbash branch 2 times, most recently from f7b4b43 to 77612a9 Compare June 29, 2026 09:02
@simonfaltum simonfaltum force-pushed the simonfaltum/aitools-bugbash2 branch from 6564b5a to ab3a37b Compare June 29, 2026 09:02
@simonfaltum simonfaltum force-pushed the simonfaltum/aitools-bugbash branch from 77612a9 to be9dc54 Compare June 29, 2026 09:21
@simonfaltum simonfaltum force-pushed the simonfaltum/aitools-bugbash2 branch from ab3a37b to 5e2d6bb Compare June 29, 2026 09:22

@mihaimitrea-db mihaimitrea-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Base automatically changed from simonfaltum/aitools-bugbash to main June 29, 2026 11:22
uninstall removed skills and de-registered plugins immediately, with no
confirmation, which is easy to trigger by accident. When running interactively
(a TTY), it now prints what will be removed (counts of skills and the plugin
agents, or the named --skills, plus the scope) and asks "Proceed?" first.
Declining cancels with no changes. Non-interactive runs are unaffected so
automation does not break, matching install. When nothing is recorded in the
scope, the prompt is skipped so the installer surfaces its own guidance.

Co-authored-by: Isaac
version showed "Skills (global)" / "Skills (project)" only when both scopes were
present, and plugin lines carried no scope at all, so you could not always tell
where something was installed. Now the scope is always labeled for both skills
and plugins (e.g. "Plugin (Claude Code, global): latest"), making version an
unambiguous view of what is installed where.

Co-authored-by: Isaac
…marketplace

The databricks plugin is published in Claude's built-in claude-plugins-official
marketplace, so Claude should install from there instead of the CLI registering
its own databricks-agent-skills marketplace.

Claude's PluginSpec now targets claude-plugins-official with an empty Source,
which marks a built-in marketplace: the install never runs `marketplace add` for
it, and uninstall never de-registers it (even if a stale record claims we did),
so we can't disturb shared marketplace infra. Codex, Copilot, and Cursor keep our
own databricks-agent-skills marketplace.

Co-authored-by: Isaac
Two picker fixes:

- The interactive picker now only offers agents that will do something (plugin or
  skills delivery). Every detected agent still appears in the detection list with
  its state/reason, but agents that would be skipped in the chosen scope (e.g.
  files-only or user-only agents under project scope) are no longer selectable
  checkboxes. If nothing is actionable, the picker isn't shown.
- Cursor is no longer a confusing "manual, run /add-plugin" checkbox that does
  nothing. It installs raw skills like a files-only agent (labeled "skills (plugin
  recommended)") and, after installing, prints a recommendation to add the plugin
  with /add-plugin. The deliveryManualCursor path is removed.

Co-authored-by: Isaac
…concept

We can't install or interact with the Cursor plugin headlessly, so referencing it
anywhere was just confusing (a "manual, run /add-plugin" picker option that did
nothing, a "(plugin recommended)" label, a manual_add_plugin list status).

Cursor is now a plain skills-only agent (Plugin nil), exactly like OpenCode and
Antigravity: it installs raw skills, shows as "skills", and carries no plugin
references. Since Cursor was the only manual-only agent, this removes the whole
concept: PluginSpec.ManualOnly/ManualInstructions, StateManualOnly,
ReasonManualOnly, statusManualAddPlugin, and the deliveryManualCursor path. The
nil-plugin guards now return ReasonNoPlugin.

Co-authored-by: Isaac
Add the source repo to `databricks aitools` help so users know where the skills
and plugins come from: https://github.com/databricks/databricks-agent-skills

Co-authored-by: Isaac
@simonfaltum simonfaltum force-pushed the simonfaltum/aitools-bugbash2 branch from 5e2d6bb to 0380123 Compare June 29, 2026 11:29
@simonfaltum simonfaltum added this pull request to the merge queue Jun 29, 2026
Merged via the queue into main with commit c845286 Jun 29, 2026
19 checks passed
@simonfaltum simonfaltum deleted the simonfaltum/aitools-bugbash2 branch June 29, 2026 12:51
bradleyjamrozik-origindigital pushed a commit to bradleyjamrozik-origindigital/databricks-cli that referenced this pull request Jun 29, 2026
…all (databricks#5740)

<!-- aitools-stack -->
### Stack

Part of the `aitools` plugin-first redesign. Open PRs, merge bottom to
top:

- **databricks#5740 Uninstall teardown + legacy skills cleanup  ← this PR**
- databricks#5741 list + version plugin state
- databricks#5745 Bug bash (wording, latest-by-default skills, project-scope plan,
version reporting)
- databricks#5746 Bug bash round 2 (uninstall confirm, picker, Cursor, official
marketplace, version scope)

Merged: databricks#5734, databricks#5736, databricks#5737, databricks#5738, databricks#5739.
<!-- /aitools-stack -->

## Why

To finish the lifecycle: `uninstall` has to remove the plugin we
installed (not just skills), and `install` has to clean up any raw
skills a user had from the old skills-everywhere model, so a plugin
agent doesn't end up with the plugin *and* leftover loose skill files
showing the same skills twice.

Stacked on databricks#5739 (update).

## Changes

- **Uninstall tears down plugins.** On a full uninstall (no `--skills`
filter), each agent recorded in `state.Plugins` has its plugin removed
through the agent's own CLI, and the marketplace is de-registered, but
only when this CLI registered it and `--keep-marketplace` wasn't passed,
so we never remove a marketplace another plugin may share. Skills
teardown is unchanged for file agents; the state file is removed only
when no skills and no plugins remain in the scope.
- **Install cleans up legacy raw skills.** After installing the plugin
for an agent, `RemoveLegacyRawSkills` removes skill dirs the CLI
previously dropped there: a symlink pointing into our canonical dir, or
a copy whose files all match the recorded checksums. User-modified dirs,
third-party dirs, and copies with no recorded provenance are left
untouched.
- New `--keep-marketplace` flag on `uninstall`.

## Test plan

- [x] Unit tests: uninstall tears down the plugin and de-registers the
marketplace, removing the state file when nothing remains;
`--keep-marketplace` skips the de-register; `RemoveLegacyRawSkills`
removes our symlink and a checksum-matched copy while keeping a
user-modified copy and a third-party dir.
- [x] Existing skills-uninstall tests still pass byte-for-byte
("Uninstalled N skills.", selective removal, orphan cleanup, "no skills
installed").
- [x] `go test ./libs/aitools/... ./cmd/aitools/...` passes.
- [x] `./task fmt-q`, `./task lint-q` (0 issues), `./task checks` (no
dead code) clean.

This pull request and its description were written by Isaac.
bradleyjamrozik-origindigital pushed a commit to bradleyjamrozik-origindigital/databricks-cli that referenced this pull request Jun 29, 2026
…lan, version reporting) (databricks#5745)

<!-- aitools-stack -->
### Stack

Part of the `aitools` plugin-first redesign. Open PRs, merge bottom to
top:

- databricks#5740 Uninstall teardown + legacy skills cleanup
- databricks#5741 list + version plugin state
- **databricks#5745 Bug bash (wording, latest-by-default skills, project-scope
plan, version reporting) ← this PR**
- databricks#5746 Bug bash round 2 (uninstall confirm, picker, Cursor, official
marketplace, version scope)

Merged: databricks#5734, databricks#5736, databricks#5737, databricks#5738, databricks#5739.
<!-- /aitools-stack -->

## Why

Testing the built CLI surfaced several issues in `databricks aitools`:

1. Inconsistent, plugin-blind copy ("AI Tools" / "AI skills") even
though most agents now get a plugin.
2. Skills pinned to a cli-compat version (e.g. `0.2.5`) while plugin
agents installed latest (e.g. `0.2.7`). Two different versions for the
same content, and list/version misreported the plugin version.
3. Project-scope install offered files-only agents (OpenCode,
Antigravity) that can't do project scope, then failed after the fact
without installing anything.

## Changes

**Wording.** Everything under `databricks aitools` now reads "skills and
plugins" (help, version output, install log); empty-state errors say "no
skills or plugins installed". The legacy skills alias and `experimental
aitools` are untouched.

**Skills track latest by default.** Plugin agents' CLIs have no version
flag, so they only ever install the marketplace's latest. Skills now
match that:

- `GetSkillsRef` precedence: `DATABRICKS_SKILLS_REF` (exact ref, for
evals), then cli-compat.json, then default latest (`main`).
- cli-compat.json is the remote safety valve: its `skills` field is now
`"latest"` (track latest), but it can be edited to a concrete version to
pin older CLIs after a breaking change, with no CLI release needed.
AppKit version resolution is unchanged (cli-compat still pins it via its
own field).
- Plugins (and unpinned skills) report `latest` instead of a stale
concrete version, so list/version are honest.
- `update` reconciles against latest when unpinned instead of falsely
reporting "already up to date".

**Project-scope plan is honest.** `buildPlan` skips files-only agents
that don't support project scope (the way plugin agents already were),
and the picker is scope-aware, so incompatible agents are labeled and
not pre-checked. No more "offer then fail with nothing installed".

## Test plan

- [x] Unit: `GetSkillsRef` (default latest / env pin / cli-compat pin),
`DisplaySkillsVersion`, `buildPlan` project-scope skip for files-only
agents; clicompat allows the `latest` sentinel.
- [x] Acceptance `experimental/aitools` unchanged (tests pin the ref via
env).
- [x] `go test ./libs/aitools/... ./cmd/aitools/...
./libs/clicompat/...` passes.
- [x] `./task fmt-q`, `./task lint-q` (0 issues), `./task checks` (no
dead code) clean.

This pull request and its description were written by Isaac.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants