perf(config): memoize successful OCI reads to halve warm startup latency#3397
Merged
Conversation
A single `docker-agent run <oci-ref>` reads the same source twice (sandbox probe + team load); without caching each read pays a blocking registry digest round-trip even when the artifact is cached and unchanged. Memoize successful reads per process (1-minute TTL, singleflight via pkg/memoize) so the second call is free — warm OCI startup drops ~1.3s → ~0.65s. The cache key is the fully-qualified reference (registry host included): two refs differing only by registry are distinct trust boundaries and must never share cached bytes. Failures are never cached, and neither are degraded fallbacks (registry unreachable, stale local copy served) so recovery is retried on the next read. The corruption force-repull path is preserved, and callers get a clone of the cached bytes. Assisted-By: Claude
aheritier
added
area/config
gtardif
approved these changes
Jul 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A single
docker-agent run <oci-ref>invocation reads the same agent-config source twice: once in the sandbox-default probe (resolveSandboxDefault) and once during team load. EachociSource.Readperformed a blocking registry digest round-trip (~600 ms) even when the artifact was already in the local content store and unchanged. Warm startup for OCI refs was ~1.3 s, versus ~80 ms for local or built-in agents.ociSource.Readnow memoizes successful results per process using the existingpkg/memoizehelper (singleflight + TTL), with a 1-minute TTL. The cache key is the fully-qualified reference via a newremote.FullyQualifiedReferencehelper — two refs that differ only by registry host are distinct trust boundaries and never share cached bytes. Failures are never cached, and degraded fallbacks (registry unreachable → stale local copy served) are deliberately excluded from the cache via a sentinel error so that registry recovery is retried immediately on the next read. The corruption force-repull path is preserved unchanged; only a fully validated read populates the cache. Callers receive a clone of the cached bytes so no caller can mutate the shared entry.Warm
run <oci-ref>startup drops from ~1.3 s to ~0.65 s (one registry round-trip instead of two). Digest-pinned refs were already cache-first and are unaffected. New tests cover: memoization of successful reads (including equivalent ref forms sharing one entry), cache scoped to registry host, failed reads being retried rather than cached, and degraded fallbacks not being cached while validated reads are cached again after recovery. All tests are hermetic undergo test -count=2.