fix(completions): reject empty PromptStrings in streaming to avoid index-out-of-range panic#11028
Conversation
…dex-out-of-range panic The streaming branch of CompletionEndpoint only guarded len(config.PromptStrings) > 1 before unconditionally reading config.PromptStrings[0]. A completion request whose prompt field is an empty array, an array of non-strings, or omitted leaves PromptStrings with length 0, so PromptStrings[0] panics with index out of range and crashes the handler goroutine. Guard for exactly one prompt string instead, returning a clean error for the 0-length case as well as the pre-existing multi-prompt case. Signed-off-by: Tai An <antai12232931@outlook.com>
Reject streaming completion requests whose prompt does not resolve to exactly one string (omitted prompt, empty array, or a multi-element array) with an HTTP 400 before writing any SSE headers, instead of returning a plain error that Echo surfaces as a 500. Extract the guard into validateStreamingPromptStrings and cover the three reported payloads with a regression test. Fixes mudler#11021 Signed-off-by: Tai An <antai12232931@outlook.com>
localai-org-maint-bot
left a comment
There was a problem hiding this comment.
The length guard prevents the panic, but returning a plain errors.New makes Echo respond with HTTP 500 for all three reported malformed inputs. I reproduced this against the PR head with e2e cases for an omitted prompt, prompt: [], and prompt: [1,2,3]; each still returned 500, while issue #11021 requires a 400 validation response. Please return echo.NewHTTPError(http.StatusBadRequest, "streaming completions require exactly one prompt string") before setting the SSE response headers, and add regression coverage for those three payloads. With that change, the focused e2e regression passes 3/3 locally. I prepared and verified the minor patch locally, but cannot push a new commit under your DCO sign-off; you need to apply/sign it.
|
Thanks for the detailed reproduction. Fixed in ba82146:
🤖 Generated with Claude Code |
localai-org-maint-bot
left a comment
There was a problem hiding this comment.
@mudler this is good to merge. The author addressed the requested validation path exactly: malformed streaming prompts now return an Echo HTTP 400 before SSE headers are written, with Ginkgo regression coverage for nil, empty, multiple, and single prompts. Fresh local verification: go test ./core/http/endpoints/openai -run TestOpenAI -count=1 passes.
What / why
CompletionEndpoint's streaming branch (core/http/endpoints/openai/completion.go) only guardedlen(config.PromptStrings) > 1before unconditionally readingconfig.PromptStrings[0]:A completion request to
/v1/completions(streaming) whosepromptfield is an empty array, an array of non-strings, or omitted entirely leavesconfig.PromptStringswith length 0. The> 1guard passes, andPromptStrings[0]then panics withindex out of range [0] with length 0, crashing the handler goroutine.Fix
Guard for exactly one prompt string.
len != 1now returns a clean error for the empty case as well as the pre-existing multi-prompt case — a minimal, behavior-preserving change (a well-formed single-prompt streaming request is unaffected).Fixes #11021.
🤖 Generated with Claude Code