[AgentX] Test vLLM DeepSeek-V4 GB300 aggregate TP / 测试 vLLM DeepSeek-V4 GB300 聚合式 TP#2284
[AgentX] Test vLLM DeepSeek-V4 GB300 aggregate TP / 测试 vLLM DeepSeek-V4 GB300 聚合式 TP#2284ivanium wants to merge 15 commits into
Conversation
Backport per-node Dynamo health counting and provide explicit GB300 NUMA mappings for aggregate workers. 中文：回移按节点启动模式下的 Dynamo 健康检查计数修复，并为 GB300 聚合式工作进程显式配置 NUMA 映射。
中文：将性能变更日志中的 PR 链接更新为 #2269。
Lower gpu-memory-utilization from 0.95 to 0.94 after TP4 aggregate c1 and c8 both ran out of memory during FP4 MoE execution. 中文：将 gpu-memory-utilization 从 0.95 降至 0.94，为 GB300 TP4 聚合式 worker 预留显存余量；此前 c1 和 c8 均在执行 FP4 MoE 时发生显存不足。
Lower TP4 and TP8 gpu-memory-utilization to 0.92, enable the DeepSeek-V4 tool-call and reasoning parsers, and limit the changelog trigger to aggregate TP configurations. 中文：将 TP4 和 TP8 的 gpu-memory-utilization 降至 0.92，启用 DeepSeek-V4 工具调用与推理解析器，并将变更日志触发范围限制为聚合式 TP 配置。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Claude finished @ivanium's task in 3m 43s —— View job Review complete
SummaryNo blocking issues, two warnings posted inline:
Everything else checks out: the perf-changelog entry is present and correctly appended at the end of the file; the container image is a public Docker Hub reference; the master-config agg entry mirrors the existing sglang agg convention ( Note: I could not run |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| INFMAX_CONTAINER_WORKSPACE: /infmax-workspace | ||
| RESULT_DIR: /logs/agentic | ||
| PORT: "8000" | ||
| IS_MULTINODE: "false" |
There was a problem hiding this comment.
🟡 WARNING: IS_MULTINODE: "false" on a recipe that spans two nodes (agg_nodes: 2, gpus_per_agg: 8).
Why it matters: This env var is stamped into meta_env.json (benchmarks/benchmark_lib.sh:997) and is_multinode is part of the benchmark identity key in utils/validate_reusable_sweep_artifacts.py (lines 49/140/318). Every other node-spanning recipe in this tree (all disagg-gb300-*/disagg-gb200-* vLLM and SGLang agentic recipes) sets "true"; only the single-node agg recipes set "false". This looks like a copy-paste from the TP4 recipe and will mislabel the TP8 results as single-node, which can break artifact-reuse matching and result grouping.
Fix:
| IS_MULTINODE: "false" | |
| IS_MULTINODE: "true" |
| # Bound how many multinode agentic conc points share one server allocation. | ||
| MAX_MULTINODE_AGENTIC_CONCURRENCIES_PER_ALLOCATION = 4 | ||
| # One task/Slurm allocation per concurrency gives each benchmark point a | ||
| # fresh server deployment, matching single-node agentic sweep isolation. | ||
| MAX_MULTINODE_AGENTIC_CONCURRENCIES_PER_ALLOCATION = 1 |
There was a problem hiding this comment.
🟡 WARNING: Changing MAX_MULTINODE_AGENTIC_CONCURRENCIES_PER_ALLOCATION from 4 to 1 is a global change, not scoped to the new GB300 configs.
Why it matters: Every existing multinode agentic config (e.g. the GB200 dynamo-vllm 2p1d/3p2d disagg sweeps) now gets one Slurm allocation + full multi-node server deployment per concurrency point instead of batching up to 4 points per allocation — e.g. a conc-list of [16, 32, 64, 128, 256] goes from 2 allocations to 5. For large disagg deployments each extra allocation repeats queueing + model load, which materially increases sweep wall-clock and cluster load. The PR description says it includes "only the shared launcher and sweep-generation support required by these TP runs", but this constant affects all multinode agentic sweeps. If per-concurrency isolation is only needed for the new GB300 agg runs, consider scoping it (e.g. keying off the config) rather than changing the global default; if the global change is intended, please state that explicitly in the PR description/changelog since it changes job-matrix shape for existing configs.
📊 Line Count Report
- Total Lines: 1336
- Base Lines: 1334
- Change: 📈 +2 lines
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29708502087 |
Limit the GB300 Dynamo-vLLM AgentX aggregate validation to TP4 at concurrency 4 and TP8 at concurrency 1. Update the changelog description to match. 中文：将 GB300 Dynamo-vLLM AgentX 聚合验证限制为 TP4 并发 4 和 TP8 并发 1，并同步更新变更日志说明。
中文：将上游 main 合并到 GB300 TP 测试分支并解决冲突。
中文：移除 TP 测试分支中已由上游 main 覆盖的冗余矩阵逻辑改动。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29789687620 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29789786221 |
1 participant
Summary
gpu-memory-utilizationto0.92for both TP4 and TP8; do not enable expandable segments.Root cause
Previous TP4 and TP8 aggregate runs exhausted dynamic CUDA workspace headroom at
gpu-memory-utilization: 0.95. The observed failures occurred in different runtime kernels, with only about 0.5–1.8 GiB physically free. Using0.92reserves approximately 8.3 GiB/GPU relative to0.95.Validation
git diff --checkpasses.中文说明
gpu-memory-utilization均降至0.92，不启用 expandable segments。根因
此前 TP4 和 TP8 聚合式任务在
gpu-memory-utilization: 0.95下没有为动态 CUDA 工作区预留足够显存。故障出现在不同的运行时内核中，发生 OOM 时物理可用显存仅约 0.5–1.8 GiB。将配置降至0.92后，相比0.95每张 GPU 可额外预留约 8.3 GiB 显存。验证
git diff --check通过。