Skip to content

Integrate Vortex with official SGLang 0.5.12 plugins#2

Draft
Rockyeast wants to merge 10 commits into
Infini-AI-Lab:v0.8from
Rockyeast:sglang-plugin-v0.8
Draft

Integrate Vortex with official SGLang 0.5.12 plugins#2
Rockyeast wants to merge 10 commits into
Infini-AI-Lab:v0.8from
Rockyeast:sglang-plugin-v0.8

Conversation

@Rockyeast

@Rockyeast Rockyeast commented Jul 22, 2026

Copy link
Copy Markdown

Summary

  • replace the vendored SGLang 0.5.9 source-patch path with the standard sglang.srt.plugins entry point for official sglang==0.5.12.post1
  • register scoped hooks for ServerArgs, runtime integration, KV-pool sizing and initialization, fused KV writes, and PD receive handling
  • preserve the Vortex planner's batch_size <= 1024 contract when SGLang resolves the request-pool capacity
  • validate the SGLang/Vortex runtime contract and add the narrowly scoped Transformers 5.6.0 FlashAttention compatibility fix
  • align planner dtypes, per-layer context propagation, and SGLang 0.5.12 KV-pool interfaces
  • make generated indexer/cache modules namespaced, content-addressed, and atomically published
  • fix missing Transpose graph registration and update installation/launch documentation

Motivation

Vortex previously depended on a repository-local, modified SGLang 0.5.9 checkout and import-time monkey patches. Environments that install official SGLang could parse Vortex configuration without activating the complete sparse-attention runtime, while newer SGLang pool and scheduler interfaces no longer matched the vendored integration.

This change makes Vortex a normal SGLang plugin, keeps the compatibility surface inside vortex_torch, and fails explicitly when the expected runtime contract is incomplete.

Validation

  • python3 -m unittest discover -v -s tests -p 'test_*.py' (5 passed)
  • python3 -m compileall -q vortex_torch tests with a temporary bytecode cache
  • shell syntax checks for install_vortex.sh, install_vortex_glm.sh, and examples/misc/server_launch.sh
  • TOML parsing and git diff --check
  • Modal L40S runtime matrix with SGLang 0.5.12.post1: eager generation, CUDA Graph, overlap scheduling, CUDA Graph + overlap, DP=2, and checkpoint-backed per-layer Parameter
  • AstraFlow RaaS launch smoke through --vortex-config to successful token generation

The GPU validation is single-node and does not claim multi-node coverage, other GPU architectures, trained-compressor accuracy, or production latency/throughput guarantees.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant