Skip to content

feat(runtime): execute host environment recipes#1671

Merged
christso merged 4 commits into
mainfrom
environment-host-runtime
Jul 5, 2026
Merged

feat(runtime): execute host environment recipes#1671
christso merged 4 commits into
mainfrom
environment-host-runtime

Conversation

@christso

@christso christso commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

Host environment recipes are now executable at eval runtime. A type: host environment creates its resolved workdir, runs optional setup from the recipe source directory with typed JSON args on stdin, and makes that workdir the cwd for target providers and supported script graders.

This keeps testbed setup on the eval environment instead of the selected target. Generic CLI targets and the Pi CLI coding-agent path now receive the prepared host workdir through the existing provider cwd contract, while Docker recipes remain schema-only with a clear unsupported-runtime error.

Related: av-noh3.2.3

Validation

  • bun test packages/core/test/evaluation/environment-host-runtime.test.ts packages/core/test/evaluation/loaders/environment-recipe.test.ts packages/core/test/evaluation/providers/cli.test.ts packages/core/test/evaluation/providers/pi-runtime.test.ts
  • bun test ./packages/core/test/evaluation/graders.test.ts --grep "passes workspace_path"
  • bun test ./apps/cli/test/commands/eval/bundle.test.ts --grep "bundles inherited"
  • bun --filter @agentv/core test
  • bun --filter agentv test -- --grep "agentv eval bundle"
  • bunx biome check apps/cli/src/commands/eval/task-bundle.ts apps/cli/test/commands/eval/bundle.test.ts packages/core/src/evaluation/environment/host.ts packages/core/src/evaluation/graders/script-grader.ts packages/core/src/evaluation/loaders/environment-recipe.ts packages/core/src/evaluation/workspace/setup.ts packages/core/test/evaluation/environment-host-runtime.test.ts packages/core/test/evaluation/graders.test.ts packages/core/test/evaluation/loaders/environment-recipe.test.ts packages/core/test/evaluation/providers/pi-runtime.test.ts
  • bun --filter @agentv/core typecheck
  • bun --filter @agentv/core build
  • Live dogfood via local OpenAI-compatible proxy + pi-cli: run 2026-07-05T09-42-10-563Z passed 1/1 with score 100%, target cwd set to the host environment workdir, and live grader accepted HOST_ENV_READY.
  • GitHub CI run 28736988713: Build, Check Links, Cloudflare Pages, Lint, Test, Typecheck, Validate Evals, and Validate Marketplace all passed.

Evidence

Private evidence branch: EntityProcess/agentv-private@997584c9066fd1dabaf69f9160c3931c78f01584 on evidence/av-noh3-2-3-host-runtime.

Review Notes

  • Pre-PR review: manual fallback from ce-code-review because generic subagent spawning is gated to explicit delegation requests in this harness. The scan found and fixed shared environment/workspace owner-key interaction before this PR was opened.
  • Known residuals: none.

Post-Deploy Monitoring & Validation

Watch early CI and any downstream dogfood runs that use environment.type: host with setup commands. Healthy signals are successful setup execution, provider target_execution.command.cwd matching the resolved environment.workdir, bundled eval YAML omitting internal source_dir metadata, and script grader raw requests showing the same cwd when no explicit grader cwd is set. Failure signals are unsupported_environment for host recipes, environment_setup_error on valid setup scripts, target cwd falling back to the eval repo root, or bundles containing absolute source paths. Rollback is reverting this PR; owner is the AgentV runtime worker for the first 24 hours after merge.


Compound Engineering
GPT-5

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 5, 2026

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: a83057b
Status: ✅  Deploy successful!
Preview URL: https://e6532dbd.agentv.pages.dev
Branch Preview URL: https://environment-host-runtime.agentv.pages.dev

View logs

@christso christso merged commit 4897bfa into main Jul 5, 2026
8 checks passed
@christso christso deleted the environment-host-runtime branch July 5, 2026 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant