Vv/trace mode parity

.gitignore

@@ -39,8 +39,22 @@ packages/nightwatch-devtools/nightwatch-video-*.webm
 trace-*.zip
 examples/**/trace-*/
 
+# test results
+examples/**/test-results*/
+
 # vitest --coverage output
 coverage/
 
 # pnpm state, cache, logs, and debug files
 /packages/**/*.mjs
+
+.claude
+
+# Local Allure exploration / trace-parity comparison (throwaway — not committed)
+examples/**/allure-results*/
+examples/**/allure-report*/
+packages/**/allure-results*/
+packages/**/allure-report*/
+examples/playwright-allure/
+examples/wdio/cucumber/wdio.allure.conf.ts
+

ARCHITECTURE.md

@@ -144,7 +144,7 @@ The execution environment is the browser, not Node, so this package cannot impor
 
 Per-framework demo projects used for manual verification.
 
-- `examples/wdio/` — WebdriverIO with Mocha (default). Run via `pnpm demo:wdio`.
+- `examples/wdio/` — WebdriverIO, split into `cucumber/` and `mocha/` (shared page objects in `pageobjects/`). Run via `pnpm demo:wdio` (Cucumber) or `pnpm demo:wdio:mocha`.
 - `examples/nightwatch/` — Nightwatch (both vanilla and Cucumber). Run via `pnpm demo:nightwatch`.
 - `examples/selenium/` — Selenium with subdirs for `mocha-test/`, `jest-test/`, `cucumber-test/`, `jasmine-test/`, `vitest-test/`. `pnpm demo:selenium` runs mocha; `pnpm --filter @wdio/selenium-devtools example:<runner>` runs the others.
 

README.md

@@ -70,7 +70,7 @@ When BiDi is active in Selenium or Nightwatch, the per-command Chrome performanc
 
 ### 📦 Trace mode (trace.zip)
 
-Headless capture path — no DevTools UI window opens. At session end the adapter writes a trace artifact next to the user's output directory, suitable for offline replay, AI-agent diffing, or any consumer that prefers a portable artifact over a live UI.
+Headless capture path — no DevTools UI window opens. At session end the adapter writes trace artifacts into a `test-results/` folder (created next to the resolved spec/config directory), suitable for offline replay, AI-agent diffing, or any consumer that prefers a portable artifact over a live UI.
 
 | Adapter | How to enable |
 |---|---|
@@ -79,7 +79,7 @@ Headless capture path — no DevTools UI window opens. At session end the adapte
 | **Nightwatch** | `globals: nightwatchDevtools({ mode: 'trace' })` |
 
 The trace artifact contains:
-- `trace.trace` — NDJSON `context-options` + `before`/`after` action events. When test hooks are available (Mocha's `it()` / Cucumber's `Scenario()`), each test becomes a [`Tracing.tracingGroup`](https://github.com/VibiumDev/vibium/blob/main/docs/explanation/recording-format.md#action-groups-user-defined) span — an open/close `before`/`after` pair with `method: "tracingGroup"` and `params.name` set to the test title. Child actions inside the group carry `parentId` pointing back to the group's `callId`, so timeline viewers render tests as labelled spans wrapping their commands.
+- `trace.trace` — NDJSON `context-options` + `before`/`after` action events. When test hooks are available (Mocha's `it()` / Cucumber's `Scenario()`), each test becomes a `Tracing.tracingGroup` span — an open/close `before`/`after` pair with `method: "tracingGroup"` and `params.name` set to the test title. Child actions inside the group carry `parentId` pointing back to the group's `callId`, so timeline viewers render tests as labelled spans wrapping their commands.
 - `trace.network` — HAR-style network entries derived from the existing capture
 - `resources/page@<id>-<ts>.jpeg` — screenshot per user-facing action
 - `resources/elements-page@<id>-<ts>.json` — flat interactable element list extracted by the page-injected scripts in `@wdio/devtools-core/element-scripts`
@@ -103,15 +103,17 @@ npx show-trace path/to/trace.zip      # in a project that installs an adapter
 
 The `show-trace` bin is exposed by each adapter (`@wdio/devtools-service`, `@wdio/nightwatch-devtools`, `@wdio/selenium-devtools`), so `pnpm show-trace <zip>` / `npx show-trace <zip>` work in any project that installs one — no extra dependency.
 
-**Other viewers.** Because the format is unchanged, the same `.zip` also drops into [player.vibium.dev](https://player.vibium.dev) or `npx playwright show-trace <path>`. The format follows the [Vibium recording format](https://github.com/VibiumDev/vibium/blob/main/docs/explanation/recording-format.md) spec — a Playwright-compatible NDJSON schema that the ecosystem already renders. This is the same format [`@wdio/mcp`](https://webdriver.io/docs/mcp) uses for AI-driven session recording.
+**Other viewers.** The trace uses a portable NDJSON schema, so the same `.zip` also opens in other compatible trace viewers that read the format. This is the same format [`@wdio/mcp`](https://webdriver.io/docs/mcp) uses for AI-driven session recording.
 
 #### Options
 
 | Option | Values | Default | Description |
 |--------|--------|---------|-------------|
 | `mode` | `'live'` \| `'trace'` | `'live'` | `'live'` launches the DevTools UI; `'trace'` writes an offline artifact. |
 | `traceFormat` | `'zip'` \| `'ndjson-directory'` | `'zip'` | Output layout. `'zip'` writes a single archive; `'ndjson-directory'` unpacks into `trace-<id>/`. |
-| `traceGranularity` | `'session'` \| `'spec'` | `'session'` | `'session'` writes one trace per worker; `'spec'` writes one trace per spec file — smaller artifacts, easier to navigate. |
+| `traceGranularity` | `'session'` \| `'spec'` \| `'test'` | `'session'` | `'session'` writes one trace per worker; `'spec'` one per spec file; `'test'` one per test into its own `<spec>-<title>-<browser>[-retryN]/trace.zip` folder — the smallest, most navigable artifacts, and the best pairing for a retention policy. |
+| `tracePolicy` | `'on'` \| `'retain-on-failure'` \| `'retain-on-first-failure'` \| `'on-first-retry'` \| `'on-all-retries'` \| `'retain-on-failure-and-retries'` | `'on'` | Which traces to keep. `'on'` keeps every trace; the rest keep only failing/retried tests — pairs well with `traceGranularity: 'test'`. |
+| `captureAssertions` | `boolean` | `true` | Capture assertions as action rows: `node:assert` (all adapters), WebdriverIO `expect(...)` matchers, and Nightwatch `browser.assert`/`verify`. Set `false` to opt out. |
 
 WDIO config example:
 
@@ -129,7 +131,7 @@ services: [[DevToolsHookService, {
 
 Adapters detect mobile sessions via `platformName: 'android' | 'ios'` (case-insensitive) and adjust the per-action snapshot to extract elements from the mobile XML tree instead of the DOM. The trace's `context-options` records `title: 'android' — <deviceName>` / `'ios' — <deviceName>` so the viewer labels frames correctly.
 
-A reference WDIO config is at [examples/wdio/wdio.mobile.conf.ts](examples/wdio/wdio.mobile.conf.ts). Prereqs to run it end-to-end with a local emulator:
+A reference WDIO config is at [examples/wdio/cucumber/wdio.mobile.conf.ts](examples/wdio/cucumber/wdio.mobile.conf.ts). Prereqs to run it end-to-end with a local emulator:
 
 1. **Java JDK** — `brew install --cask temurin`
 2. **Android SDK** — `brew install --cask android-commandlinetools` then `yes | sdkmanager --licenses && sdkmanager "platform-tools" "emulator" "system-images;android-34;google_apis_playstore;arm64-v8a"`. The brew cask installs sdkmanager under `/opt/homebrew/share/android-commandlinetools/`, and sdkmanager downloads other SDK pieces alongside it — set `ANDROID_HOME` to that path (not `~/Library/Android/sdk/`).

eslint.config.cjs

@@ -9,7 +9,13 @@ const security = require('eslint-plugin-security')
 
 module.exports = [
   {
-    ignores: ['node_modules/**', '**/dist/**', '**/.tsup/**']
+    ignores: [
+      'node_modules/**',
+      '**/dist/**',
+      '**/.tsup/**',
+      '**/allure-report/**',
+      '**/allure-results/**'
+    ]
   },
   // Base JS config
   {

examples/nightwatch/nightwatch.conf.cjs

@@ -37,16 +37,24 @@ module.exports = {
         },
         'goog:loggingPrefs': { performance: 'ALL' }
       },
-      // Simple configuration - just call the function to get globals.
-      // - screencast: polling-mode .webm written to cwd as
-      //   nightwatch-video-<sessionId>.webm.
-      // - bidi: opt-in WebDriver BiDi capture for console + network. When
-      //   attached, the per-command Chrome perf-log network path is gated
-      //   off to avoid duplicate entries.
+      // bidi: opt-in WebDriver BiDi capture for console + network. When
+      // attached, the per-command Chrome perf-log network path is gated off to
+      // avoid duplicate entries.
       globals: nightwatchDevtools({
         port: 3000,
-        mode: 'live',
-        screencast: { enabled: true, pollIntervalMs: 200 },
+        // ── Config ladder — change ONLY this block per rung ───────────────
+        // 1 live:     mode: 'live'
+        // 2 trace:    mode: 'trace'
+        // 3 per-test: mode: 'trace', traceGranularity: 'test'
+        // 4 fail:     mode: 'trace', traceGranularity: 'test', tracePolicy: 'retain-on-failure'
+        // 5 retry:    mode: 'trace', traceGranularity: 'test', tracePolicy: 'on-first-retry'
+        //             (rung 5 needs retries → run `pnpm demo:nightwatch:retry`)
+        // NOTE: the BDD describe/it interface fires the plugin's beforeEach once
+        // per module (no per-`it` hook), so traceGranularity:'test' collapses to
+        // a single session-scoped slice here. See CLAUDE.md § Known debt.
+        mode: 'trace',
+        traceGranularity: 'session',
+        tracePolicy: 'retain-on-first-failure',
         bidi: true
       })
     }

examples/nightwatch/tests/smoke-test.js

@@ -0,0 +1,48 @@
+/**
+ * Example + config-sweep harness for @wdio/nightwatch-devtools.
+ *
+ * Walk the live/trace ladder by editing ONLY the mode/traceGranularity/
+ * tracePolicy block in ../nightwatch.conf.cjs. The suite carries a passing
+ * pair, an always-failing test (retain-on-failure target), and a flaky
+ * fail-then-pass test (on-first-retry / attempt-capture target).
+ *
+ * Native asserts (browser.assert.*) double as the assertion-capture check:
+ * the passing ones must render green ✓, the failing one red ✗.
+ *
+ * Run from repo root:
+ *   pnpm demo:nightwatch          (rungs 1-4)
+ *   pnpm demo:nightwatch:retry    (rung 5 — adds --retries 1)
+ */
+
+// Survives Nightwatch's testcase retry so the flaky test fails once, then passes.
+let flakyAttempts = 0
+
+describe('nightwatch-devtools smoke test', function () {
+  it('loads example.com and reads the heading', async function (browser) {
+    await browser.url('https://example.com')
+    await browser.waitForElementVisible('body', 5000)
+    browser.assert.titleContains('Example')
+  })
+
+  it('navigates and reads the page title', async function (browser) {
+    await browser.url('https://example.org')
+    await browser.waitForElementVisible('body', 5000)
+    browser.assert.titleContains('Example')
+  })
+
+  it('fails on a wrong title (retain-on-failure target)', async function (browser) {
+    await browser.url('https://example.com')
+    await browser.waitForElementVisible('body', 5000)
+    browser.assert.titleContains('This Is Not The Title')
+  })
+
+  it('flaky: fails the first attempt, then passes (retry target)', async function (browser) {
+    await browser.url('https://example.com')
+    await browser.waitForElementVisible('body', 5000)
+    flakyAttempts += 1
+    if (flakyAttempts === 1) {
+      throw new Error('intentional first-attempt failure — should retry')
+    }
+    browser.assert.titleContains('Example')
+  })
+})

examples/selenium/mocha-test/test/example.js

@@ -1,5 +1,10 @@
 /**
- * Smoke test for @wdio/selenium-devtools.
+ * Example + config-sweep harness for @wdio/selenium-devtools (Mocha runner).
+ *
+ * Walk the live/trace ladder by editing ONLY the DevTools.configure({...})
+ * block below. The suite carries a passing pair, an always-failing test (for
+ * retain-on-failure), and a flaky fail-then-pass test with a per-test retry
+ * (for on-first-retry / attempt capture).
  *
  * Run from the package root:  pnpm example:mocha
  */
@@ -8,11 +13,22 @@ import { strict as assert } from 'node:assert'
 import { Builder, By, until } from 'selenium-webdriver'
 import { DevTools } from '@wdio/selenium-devtools'
 
+// ── Config ladder — change this block per rung ──────────────────────────────
+// 1 live:    { mode: 'live' }
+// 2 trace:   { mode: 'trace' }
+// 3 per-test:{ mode: 'trace', traceGranularity: 'test' }
+// 4 fail:    { mode: 'trace', traceGranularity: 'test', tracePolicy: 'retain-on-failure' }
+// 5 retry:   { mode: 'trace', traceGranularity: 'test', tracePolicy: 'on-first-retry' }
 DevTools.configure({
-  screencast: { enabled: true, quality: 70, maxWidth: 1280, maxHeight: 720 },
+  mode: 'trace',
+  traceGranularity: 'session',
+  tracePolicy: 'retain-on-first-failure',
   headless: true
 })
 
+// Survives Mocha's in-process retry so the flaky test fails once, then passes.
+let flakyAttempts = 0
+
 describe('selenium-devtools smoke test', function () {
   let driver
 
@@ -40,4 +56,24 @@ describe('selenium-devtools smoke test', function () {
     const title = await driver.getTitle()
     assert.match(title, /Example/i)
   })
+
+  it('fails on a wrong heading (retain-on-failure target)', async function () {
+    await driver.get('https://example.com')
+    await driver.sleep(1000)
+    const heading = await driver.wait(until.elementLocated(By.css('h1')), 10000)
+    const text = await heading.getText()
+    assert.equal(text, 'This Is Not The Heading')
+  })
+
+  it('flaky: fails the first attempt, then passes (retry target)', async function () {
+    this.retries(1)
+    await driver.get('https://example.com')
+    await driver.sleep(1000)
+    flakyAttempts += 1
+    if (flakyAttempts === 1) {
+      throw new Error('intentional first-attempt failure — should retry')
+    }
+    const title = await driver.getTitle()
+    assert.match(title, /Example/i)
+  })
 })

examples/wdio/cucumber/features/login-fail.feature

@@ -0,0 +1,9 @@
+Feature: Retention check — deliberately failing login
+
+  # Logs in with bad credentials but asserts the SUCCESS message, so the Then
+  # step fails. Used to verify retain-on-failure keeps this spec's trace while
+  # dropping the passing login.feature. Reuses the existing step definitions.
+  Scenario: Failing assertion exercises retain-on-failure
+    Given I am on the login page
+    When I login with foobar and barfoo
+    Then I should see a flash message saying You logged into a secure area!

examples/wdio/features/step-definitions/steps.ts → examples/wdio/cucumber/features/step-definitions/steps.ts

@@ -1,8 +1,8 @@
 import { Given, When, Then, After } from '@wdio/cucumber-framework'
 import { browser, expect } from '@wdio/globals'
 
-import LoginPage from '../pageobjects/login.page.js'
-import SecurePage from '../pageobjects/secure.page.js'
+import LoginPage from '../../../pageobjects/login.page.js'
+import SecurePage from '../../../pageobjects/secure.page.js'
 
 const pages = {
   login: LoginPage
@@ -26,9 +26,9 @@ When(/^I login with (\w+) and (.+)$/, async (username, password) => {
 
 Then(/^I should see a flash message saying (.*)$/, async (message) => {
   console.log(`[TEST] Verifying flash message: ${message}`)
-  const el = await SecurePage.flashAlert
-  await expect(el).toBeExisting()
-  await expect(el).toHaveText(expect.stringContaining(message))
+  await expect(SecurePage.flashAlert).toBeExisting()
+  await expect(SecurePage.flashAlert).toHaveText(
+    expect.stringContaining(message)
+  )
   console.log('[TEST] Flash message verified successfully')
-  // await browser.pause(15000)
 })