Skip to content

Add cache size limit for S3 uploads#23

Open
dmonroy wants to merge 3 commits into
mainfrom
darwin/dont-push-large-cache-directories
Open

Add cache size limit for S3 uploads#23
dmonroy wants to merge 3 commits into
mainfrom
darwin/dont-push-large-cache-directories

Conversation

@dmonroy

@dmonroy dmonroy commented Jan 7, 2026

Copy link
Copy Markdown

Problem: Build caches can grow very large. Uploading caches over 7GB to S3 wastes time and storage.

Solution: Check cache directory size before uploading. Skip the upload if it exceeds the limit.

  • Default limit: 7GB
  • Set APPPACK_MAX_CACHE_SIZE_GB=0 to disable the check
  • Set APPPACK_MAX_CACHE_SIZE_GB=10 for a 10GB limit
  • Invalid values (negative, non-numeric) fall back to 7GB with a warning

Changes:

  • build/build.go: Added dirSize(), getMaxCacheSizeGB(), updated archiveCache()
  • build/build_test.go: Tests for env var parsing and directory size calculation

Testing:

go test ./build/...

All tests pass.

Skip uploading build cache to S3 when it exceeds the size limit.
Configurable via APPPACK_MAX_CACHE_SIZE_GB environment variable
(default: 7GB). Set to 0 to disable the limit entirely.

Invalid or negative values fall back to the default with a warning.
Comment thread builder/build/build.go Outdated
Comment thread builder/build/build.go Outdated
dmonroy added 2 commits May 13, 2026 11:35
Addresses review feedback to use the logging system instead of
print statements. Also removes a duplicate print that was left
alongside the existing structured log call.
@dmonroy
dmonroy requested a review from ipmb July 17, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants