Skip to content

feat(eval): add environment recipe schema loader (av-noh3.2.2)#1670

Merged
christso merged 2 commits into
mainfrom
feat/environment-schema-loader
Jul 5, 2026
Merged

feat(eval): add environment recipe schema loader (av-noh3.2.2)#1670
christso merged 2 commits into
mainfrom
feat/environment-schema-loader

Conversation

@christso

@christso christso commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

Beads remain the source of truth for this work: av-noh3.2.2.

Summary

  • Add AgentV environment recipe loading for authored eval YAML at suite and case scope.
  • Support inline and file:// host/docker recipes; referenced recipe-relative paths are normalized relative to the recipe file directory.
  • Carry normalized recipes on EvalTest.environment for future host/Docker runtime workers.
  • Reject target-level environment/testbed setup fields and hard-deprecate public workspace.repos, workspace.scope, workspace.template, and workspace.docker authoring.
  • Preserve top-level env and Promptfoo-compatible extensions behavior.
  • Regenerate the AI-facing eval JSON schema.

Validation

  • bun install
  • bun test packages/core/test/evaluation/loaders/environment-recipe.test.ts packages/core/test/evaluation/loaders/config-loader.test.ts
  • bun --filter @agentv/core typecheck
  • bun test packages/core/test/evaluation/validation/eval-file-schema.test.ts
  • bun --filter @agentv/core lint
  • bun --filter @agentv/core build
  • git diff --check

No live dogfood was run because this bead is schema/loader/validation only; runtime execution is deferred to av-noh3.2.3 and av-noh3.2.4.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 5, 2026

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 88a798b
Status: ✅  Deploy successful!
Preview URL: https://addd9704.agentv.pages.dev
Branch Preview URL: https://feat-environment-schema-load.agentv.pages.dev

View logs

@christso christso force-pushed the feat/environment-schema-loader branch from 0aa8c5e to 187ba6b Compare July 5, 2026 08:59
@christso christso force-pushed the feat/environment-schema-loader branch from 187ba6b to 88a798b Compare July 5, 2026 09:05
@christso christso merged commit a84b978 into main Jul 5, 2026
8 checks passed
@christso christso deleted the feat/environment-schema-loader branch July 5, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant