refactor(computing-unit-managing-service): share CU listing helpers and batch k8s calls

[kunwp1]

What changes were proposed in this PR?

Refactor the per-user computing-unit listing so its status/metrics logic lives in one reusable place and hits Kubernetes in bulk. No API response changes; this is groundwork for the admin listing endpoint (stacked follow-up PR).

• Centralize listing logic in ComputingUnitHelpers (resolveOwnerInfo, partitionLiveUnits, reconcileVanishedKubernetesUnits, buildDashboardUnit, plus the isKubernetes / podPhasesFor / podMetricsFor guards), and remove the duplicated private status/metrics helpers from ComputingUnitManagingResource.
• Add bulk KubernetesClient.getAllPodPhases / getAllPodMetrics (one namespace-wide list() / top() each) and a shared containerUsage helper; listComputingUnits now resolves pod status/metrics in bulk instead of probing pod existence twice per unit and re-fetching top() per unit.
• Per-user reconciliation now runs over cuid-deduplicated units, reducing redundant DB updates.
• Add a package-private KubernetesClient test seam (setClientForTesting) so the fabric8 client can be stubbed in tests; unused in production.

Any related issues, documentation, discussions?

Part of #6476 (Admin Computing Units Dashboard); prepares the shared listing helpers reused by the admin endpoint in the stacked follow-up. Originates from discussion #6322.

How was this PR tested?

Added ComputingUnitManagingResourceSpec and extended ComputingUnitHelpersSpec / KubernetesClientSpec (embedded Postgres via MockTexeraDB; stubbed fabric8 client for the bulk k8s paths).

Ran sbt 'ComputingUnitManagingService/testOnly org.apache.texera.service.util.ComputingUnitHelpersSpec org.apache.texera.service.util.KubernetesClientSpec org.apache.texera.service.resource.ComputingUnitManagingResourceSpec' (34 tests, all green) and sbt 'ComputingUnitManagingService/scalafmtCheck; ComputingUnitManagingService/Test/scalafmtCheck' (clean).

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code, Claude Opus 4.8

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @mengw15, @PG1204, @Ma77Ball
  You can notify them by mentioning @mengw15, @PG1204, @Ma77Ball in a comment.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.61%. Comparing base (e029e0d) to head (195cf16).
⚠️ Report is 187 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #6853      +/-   ##
============================================
+ Coverage     69.77%   70.61%   +0.83%     
- Complexity     3379     3505     +126     
============================================
  Files          1142     1145       +3     
  Lines         44828    44976     +148     
  Branches       4948     4963      +15     
============================================
+ Hits          31277    31758     +481     
+ Misses        11895    11518     -377     
- Partials       1656     1700      +44     

Flag	Coverage Δ		*Carryforward flag
access-control-service	70.00% <ø> (ø)
agent-service	76.76% <ø> (ø)		Carriedforward from adcd377
amber	68.02% <ø> (+1.45%)	⬆️
computing-unit-managing-service	43.10% <100.00%> (+33.13%)	⬆️
config-service	66.66% <ø> (+14.35%)	⬆️
file-service	67.21% <ø> (+3.23%)	⬆️
frontend	68.12% <ø> (ø)		Carriedforward from adcd377
notebook-migration-service	78.94% <ø> (ø)
pyamber	91.13% <ø> (ø)		Carriedforward from adcd377
workflow-compiling-service	55.14% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[kunwp1]

/request-review @mengw15

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 7 worse · ⚪ 8 noise (<±5%) · 0 without baseline

Compared against main 17cdba4 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	422	0.258	23,112/31,380/31,380 us	🔴 +11.2% / 🔴 +96.0%
🔴	bs=100 sw=10 sl=64	928	0.567	106,952/127,744/127,744 us	🔴 +5.9% / 🔴 +20.1%
🔴	bs=1000 sw=10 sl=64	1,080	0.659	927,233/1,081,678/1,081,678 us	🔴 +13.6% / 🔴 +6.1%

Baseline details

Latest main 17cdba4 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	422 tuples/sec	473 tuples/sec	767.66 tuples/sec	-10.8%	-45.0%
bs=10 sw=10 sl=64	MB/s	0.258 MB/s	0.289 MB/s	0.469 MB/s	-10.7%	-44.9%
bs=10 sw=10 sl=64	p50	23,112 us	20,791 us	12,623 us	+11.2%	+83.1%
bs=10 sw=10 sl=64	p95	31,380 us	30,254 us	16,011 us	+3.7%	+96.0%
bs=10 sw=10 sl=64	p99	31,380 us	30,254 us	18,856 us	+3.7%	+66.4%
bs=100 sw=10 sl=64	throughput	928 tuples/sec	953 tuples/sec	998.44 tuples/sec	-2.6%	-7.1%
bs=100 sw=10 sl=64	MB/s	0.567 MB/s	0.582 MB/s	0.609 MB/s	-2.6%	-7.0%
bs=100 sw=10 sl=64	p50	106,952 us	103,979 us	100,463 us	+2.9%	+6.5%
bs=100 sw=10 sl=64	p95	127,744 us	120,625 us	106,400 us	+5.9%	+20.1%
bs=100 sw=10 sl=64	p99	127,744 us	120,625 us	117,712 us	+5.9%	+8.5%
bs=1000 sw=10 sl=64	throughput	1,080 tuples/sec	1,110 tuples/sec	1,034 tuples/sec	-2.7%	+4.4%
bs=1000 sw=10 sl=64	MB/s	0.659 MB/s	0.678 MB/s	0.631 MB/s	-2.8%	+4.4%
bs=1000 sw=10 sl=64	p50	927,233 us	899,406 us	973,294 us	+3.1%	-4.7%
bs=1000 sw=10 sl=64	p95	1,081,678 us	952,431 us	1,019,213 us	+13.6%	+6.1%
bs=1000 sw=10 sl=64	p99	1,081,678 us	952,431 us	1,049,896 us	+13.6%	+3.0%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,473.42,200,128000,422,0.258,23112.27,31380.36,31380.36
1,100,10,64,20,2154.48,2000,1280000,928,0.567,106952.04,127744.37,127744.37
2,1000,10,64,20,18517.16,20000,12800000,1080,0.659,927232.83,1081677.77,1081677.77

[Copilot]

Pull request overview

Refactors the computing-unit managing service’s per-user CU listing to reuse shared helper logic and reduce Kubernetes API round-trips by fetching pod phase/metrics in bulk, while keeping the API response shape unchanged. This lays groundwork for the upcoming admin computing-unit listing endpoint.

Changes:

• Centralized CU dashboard-row construction and owner/status/metrics resolution in ComputingUnitHelpers, including bulk-friendly variants and vanished-pod reconciliation.
• Added bulk Kubernetes queries in KubernetesClient (getAllPodPhases, getAllPodMetrics) plus a test seam (setClientForTesting) to enable stubbing fabric8 without a cluster.
• Added/expanded unit tests for the new helper and bulk K8s paths, and introduced a resource-level spec for per-user endpoints using embedded Postgres.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
computing-unit-managing-service/src/main/scala/org/apache/texera/service/util/KubernetesClient.scala	Adds bulk namespace-wide pod phase/metrics fetchers and a package-private test seam for the fabric8 client.
computing-unit-managing-service/src/main/scala/org/apache/texera/service/util/ComputingUnitHelpers.scala	Introduces shared CU listing helpers: owner info resolution, bulk status/metrics, live/vanished partitioning, reconciliation, and dashboard-row builder.
computing-unit-managing-service/src/main/scala/org/apache/texera/service/resource/ComputingUnitManagingResource.scala	Refactors per-user listing to dedupe by cuid and use bulk pod phase/metrics maps plus shared helper methods.
computing-unit-managing-service/src/test/scala/org/apache/texera/service/util/KubernetesClientSpec.scala	Adds Mockito-based fabric8 stubs and tests covering bulk K8s wrappers and helper guard behavior.
computing-unit-managing-service/src/test/scala/org/apache/texera/service/util/ComputingUnitHelpersSpec.scala	Expands helper coverage for bulk variants, reconciliation behavior, and DB-backed owner-info resolution.
computing-unit-managing-service/src/test/scala/org/apache/texera/service/resource/ComputingUnitManagingResourceSpec.scala	Adds embedded-DB resource-level tests for per-user endpoints using local units (no K8s calls).
build.sbt	Wires test-scope dependencies for embedded DB reuse and forces serial test execution for this module due to JVM-wide singletons.

Comments suppressed due to low confidence (2)

computing-unit-managing-service/src/test/scala/org/apache/texera/service/util/KubernetesClientSpec.scala:124

• afterAll() should call super.afterAll(), ideally in a finally block, to ensure other traits' teardown runs even if resetting the KubernetesClient fails.

  override protected def afterAll(): Unit = {
    // Restore a real client so subsequent suites don't reuse the mock.
    KubernetesClient.setClientForTesting(new KubernetesClientBuilder().build())
  }

computing-unit-managing-service/src/test/scala/org/apache/texera/service/resource/ComputingUnitManagingResourceSpec.scala:73

• afterAll() should call super.afterAll(), preferably in a finally block, to ensure proper teardown if additional traits are mixed in later.

  override protected def afterAll(): Unit = shutdownDB()

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.