Skip to content

feat(zk-host): enable all configured proving backends#3920

Open
mw2000 wants to merge 13 commits into
mw2000/zk-host-routingfrom
mw2000/zk-host-multi-backend
Open

feat(zk-host): enable all configured proving backends#3920
mw2000 wants to merge 13 commits into
mw2000/zk-host-routingfrom
mw2000/zk-host-multi-backend

Conversation

@mw2000

@mw2000 mw2000 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Enables one ZK host to initialize every configured proving backend and route each request to its selected backend. Backend availability is inferred from configuration presence, with shared witness-provider setup centralized in base-proof-zk-backend and the legacy ZK_BACKEND selector removed.

Discover backend availability from its existing settings so one host can serve mock, dry-run, cluster, and network requests without a selector environment variable.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread bin/prover/zk-host/src/cli.rs Outdated
Comment thread bin/prover/zk-host/src/cli.rs Outdated
Comment thread bin/prover/zk-host/src/cli.rs Outdated
Comment thread bin/prover/zk-host/README.md Outdated
Comment thread bin/prover/zk-host/src/cli.rs Outdated
Preserve mock-only compatibility for stale partial RPC settings, reuse parsed RPC configuration, and clarify presence-based backend enablement.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread bin/prover/zk-host/src/cli.rs Outdated
Comment thread bin/prover/zk-host/src/cli.rs Outdated
Exercise mock, dry-run, cluster, and network enablement together while making optional string parsing accept borrowed string options directly.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread bin/prover/zk-host/src/cli.rs Outdated
Comment thread bin/prover/zk-host/src/cli.rs Outdated
Keep binary wiring minimal by moving backend validation and shared prover setup into the backend crate, while removing stale selector configuration.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread crates/proof/zk/backend/src/succinct/builder.rs Outdated
Treat either S3 setting as cluster intent while keeping the default dry-run deployment free of implicit cluster configuration.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread crates/proof/zk/backend/src/succinct/builder.rs
Comment thread crates/proof/zk/backend/src/succinct/builder.rs
mw2000 and others added 2 commits July 10, 2026 14:11
Keep backend-specific invariants in focused resolvers and derive shared witness RPC settings from the resolved backend configs to prevent drift.

Co-authored-by: Cursor <cursoragent@cursor.com>
Removed outdated information about RPC configuration and backend settings.
Comment thread crates/proof/zk/backend/src/succinct/builder.rs
Comment thread crates/proof/zk/backend/src/succinct/builder.rs Outdated
Comment thread crates/proof/zk/backend/src/succinct/builder.rs Outdated
mw2000 and others added 2 commits July 10, 2026 14:17
Report backend-specific validation failures before tolerated mock-only RPC errors, and warn when incomplete RPC settings are intentionally ignored.

Co-authored-by: Cursor <cursoragent@cursor.com>
Build independent configured backends in parallel so expensive key setup does not serialize multi-backend startup.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread crates/proof/zk/backend/src/succinct/builder.rs Outdated
Comment thread crates/proof/zk/backend/src/succinct/builder.rs
Comment thread crates/proof/zk/backend/src/succinct/builder.rs Outdated
Drain parallel initialization tasks on failure or cancellation and reject mismatched RPC settings before sharing a witness provider.

Co-authored-by: Cursor <cursoragent@cursor.com>
Keep canonical RPC resolution alongside backend configs and consolidate parallel task error handling without changing startup behavior.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

✅ base-std fork tests: all 616 passed

base/base is fully in sync with the base-std spec.

Dependency Ref Commit
base-std main 4658f1b7
base-anvil 0092692587d8d064dd2c6923ce26a682c58f3694 00926925

@github-actions

Copy link
Copy Markdown
Contributor

Review Summary

The PR cleanly replaces the single-backend ZK_BACKEND enum selector with a presence-based multi-backend model. The new SuccinctZkProversConfig centralizes configuration validation and the shared witness provider is correctly built once and shared across backends via JoinSet parallelism. The CLI simplification is a clear improvement — the binary now acts as a thin adapter to the library config struct, which is the right layering.

Key findings (from inline comments)

Correctness / Operator UX (most impactful):

  • Dry-run is implicitly enabled whenever RPC URLs are present. An operator configuring only the cluster or network backend will also get a dry-run prover consuming CPU/memory on every job. This should either be gated behind an explicit opt-in flag or clearly documented.
  • cluster_requested() triggers on stale S3 env varsCLI_S3_BUCKET alone (without SP1_CLUSTER_API_ENDPOINT) returns true, which can cause confusing RPC validation errors for mock-only operators with leftover env vars. The primary trigger should be cluster_rpc only.
  • Suppressed RPC errors are silent — when enable_mock=true with a partial RPC config and no cluster/network triggers, the RPC error is logged at warn level (good), but operators may still not notice. The warning is adequate but worth verifying it surfaces clearly in production logging configs.

Design / Maintainability:

  • Dual ownership of SuccinctRpcConfig — the RPC config is both returned in the BackendConfigs tuple and cloned into each non-Mock SuccinctZkBackendConfig variant. This creates an implicit invariant that both copies must match. Consider returning rpc only in the tuple and extracting it from there in build_until_cancelled, rather than embedding it in each config.
  • BackendConfigs type alias could be a named struct for clarity, given it's the primary return type of the configuration pipeline.

Testing:

  • The test at line 520 exercises single-backend scenarios and the all-backends-at-once case, but doesn't verify the multi-backend interaction that is the core purpose of this PR (e.g., that RPC + cluster produces exactly dry-run + cluster without mock, or that RPC + network + mock produces 3 backends).

No new issues found beyond existing inline comments

The existing inline review comments comprehensively cover the significant findings. The overall architecture is sound and the change meaningfully simplifies the operator deployment model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant