Skip to content

feat: Clean cache command#1394

Open
d3xter666 wants to merge 108 commits into
mainfrom
feat-clean-cache
Open

feat: Clean cache command#1394
d3xter666 wants to merge 108 commits into
mainfrom
feat-clean-cache

Conversation

@d3xter666

Copy link
Copy Markdown
Member

The command completely cleans the cache by removing the cache files as well as cleaning up the SQLite records.
It does not wipe out the SQLite DB file(s)

JIRA: CPOUI5FOUNDATION-891

@d3xter666 d3xter666 requested a review from a team May 22, 2026 12:22
@d3xter666 d3xter666 force-pushed the feat-clean-cache branch 2 times, most recently from 83f2262 to d52c4ad Compare May 26, 2026 06:29
@RandomByte RandomByte force-pushed the feat/incremental-build-4 branch from 7c59782 to 444977d Compare May 27, 2026 15:40
Comment thread packages/project/lib/cache/CacheCleanup.js Outdated
Comment thread packages/project/lib/cache/CacheCleanup.js Outdated
Comment thread packages/project/lib/cache/CacheCleanup.js Outdated
Comment thread packages/cli/lib/cli/commands/cache.js
@RandomByte RandomByte force-pushed the feat/incremental-build-4 branch from c2dc7b8 to 1041695 Compare May 29, 2026 08:11
@d3xter666 d3xter666 force-pushed the feat-clean-cache branch 2 times, most recently from dc31834 to f5def12 Compare May 29, 2026 08:29
@RandomByte RandomByte force-pushed the feat/incremental-build-4 branch from 1041695 to 66296d5 Compare May 29, 2026 08:49
@d3xter666 d3xter666 force-pushed the feat-clean-cache branch 2 times, most recently from 77f7320 to aa280da Compare May 29, 2026 10:27
Comment thread packages/project/lib/build/cache/CacheCleanup.js Outdated
@d3xter666 d3xter666 requested a review from matz3 May 29, 2026 13:21
@d3xter666 d3xter666 force-pushed the feat-clean-cache branch 2 times, most recently from 7b17cc0 to 569ff71 Compare May 29, 2026 15:44
@d3xter666 d3xter666 requested a review from a team June 1, 2026 07:46
@d3xter666 d3xter666 changed the base branch from feat/incremental-build-4 to main June 1, 2026 10:10
Comment thread packages/cli/lib/cli/commands/cache.js Outdated
Comment thread packages/cli/lib/cli/commands/cache.js Outdated
Comment thread packages/project/lib/build/cache/CacheManager.js Outdated
Comment thread packages/project/lib/ui5Framework/cache.js Outdated
Comment thread packages/project/lib/ui5Framework/cache.js Outdated
Comment thread packages/project/lib/ui5Framework/cache.js Outdated
Comment thread packages/project/lib/ui5Framework/cache.js Outdated
Comment thread packages/project/lib/ui5Framework/cache.js Outdated
Comment thread packages/project/lib/build/cache/CacheManager.js Outdated
@matz3

matz3 commented Jun 2, 2026

Copy link
Copy Markdown
Member

I think the usability of the command could be improved.

  • No information about the actual filesystem paths of the cache and relevant dirs (not in CLI description, not in --help, not in command output
  • No information about the relation to ui5DataDir config / UI5_DATA_DIR env var
  • No output when command is executed, and it might take a while to calculate the size, so users might assume the command is stuck if it does not print anything for a minute or longer.
  • Output lists two different sizes 197.0 MB vs 196.8 MB:
The following items from cache will be removed:
  • buildCache/v0_7 (197.0 MB)

Total: 197.0 MB


✓ Removed buildCache/v0_7 (196.8 MB)

Success: Cleaned 1 entry, freed 196.8 MB
  • (minor) Inconsistency in logged paths (framework/ vs framework):
The following items from cache will be removed:
  • framework/ (711.4 MB)

Total: 711.4 MB

Do you want to continue? (y/N) y

✓ Removed framework (711.4 MB)

@d3xter666

Copy link
Copy Markdown
Member Author

I have tried to improve the situation with the infmration for the execution of the cache clean command and have addressed all your concenrns.

Let me know if there's something more you'd expect.

Regarding the size report, there are some considerations we need to be aware of:

  • Getting framework cache dir size might require time. Even if you do that on your machine, it will take time. Even the du command on Mac uses cache in order to be quick, but the first run is not that fast. The optimal solution: Give information about the number of files that will be deleted
  • The purge of the DB cache is just the opposite! Selecting COUNT(*) of all rows might take significant amount of time to get the results. On the other hand, the size of the DB can be quite fast.

I have tried to somehow get advantage of these findings and provide the optimal solution- show metrics for what is fast and provide generic messages. Any other solutions will require certain compromises.

@d3xter666 d3xter666 requested review from a team and RandomByte June 2, 2026 19:06
@d3xter666

Copy link
Copy Markdown
Member Author

Edit

Now the UX of this command is revised!

The real issue is enormous cache!

Collecting full information about what would be deleted is a haevy work and we simply cannot do it real time. In the end, for the end user it's important what would be deleted!
Given that, my proposal would be a summary information that does not need to collect all the files or their sizes. Instead, just put an overview:

Confirmation

Checking cache at /my/data/dir/.ui5 …

The following cached data will be removed:

  • UI5 Framework packages   /my/data/dir/.ui5/framework   (1 project, 18 libraries, 212 versions)
  • Build cache (DB)         /my/data/dir/.ui5/buildCache/v0_7   (100.0 KB)

Do you want to continue? (y/N)

During cache cleanup

If we now decide to accept purging of the hughe cache, it will take some time cleaning it up. We cannot do anythinbg about it, but simply wait for the fuiles to be deleted.
For that purpose, I have created an real time "monitor" of what is being deleted:

 ⠏ UI5 Framework packages   …/sap.m/1.52.14/src/sap/m/BarRenderer.js   2.86 s

Final summary

✓ Removed UI5 Framework packages   (/my/data/dir/.ui5/framework · 1 project, 18 libraries, 212 versions)
✓ Removed Build cache (DB)   (/my/data/dir/.ui5/buildCache/v0_7 · 100.0 MB)

Success: Cleaned UI5 Framework packages and Build cache (DB)

Hopefully, this is good enough as UX and fast enough, so that developers get feedback what is actually happenning

@RandomByte

Copy link
Copy Markdown
Member

That's better. On my system it's very fast now:

Checking cache at /Users/me/.ui5 …

The following cached data will be removed:

  • UI5 Framework packages   /Users/me/.ui5/framework   (2 projects, 155 libraries, 1189 versions)
  • Build cache (DB)         /Users/me/.ui5/buildCache/v0_7   (70.7 MB)

Do you want to continue? (y/N) 

But I think the term projects is misleading here. Usually every UI5 app or library is a project. I would drop that entirely and rather say 1.189 versions of 155 libraries (ideally with a decimal separator?)

I don't really think we need a progress indicator for the deletion process. It is expected to take a while, I would rather not add code just for that.

@d3xter666 d3xter666 force-pushed the feat-clean-cache branch 3 times, most recently from 3f0d21c to 4d39800 Compare June 15, 2026 12:44

@matz3 matz3 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested it manually and it works as expected now in the relevant cases that typically appear (I did not test any rare race-condition).
The docs (once updated) and the CLI command (options, help, output) should be reviewed by UA. Some terms are not consistent (e.g. Orphaned framework data vs UI5 Framework packages).

So far, I did not review any of the tests.

Comment thread internal/documentation/docs/pages/Troubleshooting.md Outdated
Comment thread packages/project/lib/graph/ProjectGraph.js Outdated
Comment thread internal/documentation/docs/pages/Troubleshooting.md Outdated
@d3xter666 d3xter666 requested review from KlattG and matz3 June 29, 2026 20:15
@d3xter666

Copy link
Copy Markdown
Member Author

Comment thread internal/documentation/docs/pages/Troubleshooting.md Outdated
Comment thread internal/documentation/docs/pages/Troubleshooting.md Outdated
Comment thread internal/documentation/docs/pages/Troubleshooting.md Outdated
Comment thread internal/documentation/docs/pages/Troubleshooting.md Outdated
Comment thread internal/documentation/docs/pages/Troubleshooting.md Outdated
Comment thread packages/cli/lib/cli/commands/cache.js Outdated
Comment thread packages/cli/lib/cli/commands/cache.js Outdated
Comment thread packages/cli/lib/cli/commands/cache.js Outdated
Comment thread packages/cli/lib/cli/commands/cache.js Outdated
Comment thread packages/cli/lib/cli/commands/cache.js Outdated
Comment thread internal/documentation/docs/pages/Troubleshooting.md Outdated
d3xter666 and others added 10 commits June 30, 2026 17:06
Co-authored-by: Günter Klatt <guenter.klatt@sap.com>
Co-authored-by: Günter Klatt <guenter.klatt@sap.com>
Co-authored-by: Günter Klatt <guenter.klatt@sap.com>
Co-authored-by: Günter Klatt <guenter.klatt@sap.com>
Co-authored-by: Günter Klatt <guenter.klatt@sap.com>
Co-authored-by: Günter Klatt <guenter.klatt@sap.com>
Co-authored-by: Günter Klatt <guenter.klatt@sap.com>
Co-authored-by: Günter Klatt <guenter.klatt@sap.com>
Co-authored-by: Günter Klatt <guenter.klatt@sap.com>
Co-authored-by: Günter Klatt <guenter.klatt@sap.com>
@d3xter666 d3xter666 requested a review from KlattG June 30, 2026 14:08

@KlattG KlattG left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@d3xter666 d3xter666 requested a review from a team June 30, 2026 14:26

::: info
If you have configured a custom data directory via `UI5_DATA_DIR` or `ui5 config set ui5DataDir`, replace `~/.ui5/` with that path. See [Changing UI5 CLI's Data Directory](#changing-ui5-cli-s-data-directory).
If you have configured a custom data directory via `UI5_DATA_DIR` or `ui5 config set ui5DataDir`, the `ui5 cache clean` command automatically cleans up that location instead of the default `~/.ui5/`. See [Changing UI5 CLI's Data Directory](#changing-ui5-cli-s-data-directory).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you have configured a custom data directory via `UI5_DATA_DIR` or `ui5 config set ui5DataDir`, the `ui5 cache clean` command automatically cleans up that location instead of the default `~/.ui5/`. See [Changing UI5 CLI's Data Directory](#changing-ui5-cli-s-data-directory).
If you have configured a custom data directory via `UI5_DATA_DIR` or `ui5 config set ui5DataDir`, the `ui5 cache clean` command will clean up that location instead of the default `~/.ui5/`. See [Changing UI5 CLI's Data Directory](#changing-ui5-cli-s-data-directory).

- **Build cache (DB)** — build data (`~/.ui5/buildCache/`)
- **Orphaned framework data** — incomplete framework directories left over from previously interrupted cleanup operations (`~/.ui5/.framework_to_delete_*/`)

Any missing framework dependencies are downloaded during the next UI5 CLI invocation.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Any missing framework dependencies are downloaded during the next UI5 CLI invocation.
Any required framework dependencies will be re-downloaded during the next UI5 CLI invocation.

@@ -0,0 +1,170 @@
import path from "node:path";

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

acquireLockSync is not called in the ProjectGraph constructor but rather in the async function _preventCacheClean - Is acquireLockSync still needed or a relict?

* @returns {string} Absolute path to the locks directory (<code>~/.ui5/locks/</code>)
*/
export function getLockDir(ui5DataDir) {
return path.join(ui5DataDir, "locks");

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previous UI5 CLI versions use path.join(ui5DataDir, "framework", "locks");. Won't this cause clashes now if multiple CLI versions are used in parallel?

If possible, I would prefer to keep using the framework/locks path for framework-dependency related locks

* If provided, these files are excluded from the scan.
* @returns {Promise<boolean>} True if any matching non-stale lockfiles are held
*/
export async function hasActiveLocks(ui5DataDir, {include, exclude} = {}) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: The include/exclude parameters are only ever used for the cleanup lock. I think this could be simplified by providing dedicated functions for the cleanup lock instead of this generic one.

But that's no must-fix for this iteration, just a bit confusing when reading the code


// Poll until any in-progress cache clean releases its lock, or we time out.
const POLL_INTERVAL_MS = 200;
const TIMEOUT_MS = 10000;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we increase this to 60 seconds for a cache clean to finish?

* Releases the process-coordination lock held by this graph.
* Call this when the graph is no longer needed to unblock <code>ui5 cache clean</code>.
*
* If not called explicitly, the <code>lockfile</code> package's <code>signal-exit</code>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO this public API description goes into too much detail about our internal implementation (which might change)

* packages are being downloaded, or while those packages are actively referenced
* within the graph during the build/serve lifecycle.
*
* This is necessary because the graph holds references to framework package paths

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can mention that this is called in ui5Framework to make sure the graph is already locked once framework resources (which are affected by the cache clean) are used

"(~/.ui5/framework/)\n" +
" Build cache (DB): Build data " +
"(~/.ui5/buildCache/)\n" +
" Orphaned framework data: Incomplete directories from previously interrupted cleanups\n" +

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: Is this relevant enough to mention it in the user facing docs?

return false;
}

for (const lockFileName of lockFiles) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use Promise.all for checking multiple locks concurrently?

@RandomByte

Copy link
Copy Markdown
Member

I think we should discuss this topic again in the team and reconsider the priority. Things got more complex than expected and maybe it's worth starting the concept work from scratch again sometime later this year.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants