Skip to content

fix(completions): reject empty PromptStrings in streaming to avoid index-out-of-range panic#11028

Merged
mudler merged 2 commits into
mudler:masterfrom
Anai-Guo:fix-streaming-empty-promptstrings
Jul 22, 2026
Merged

fix(completions): reject empty PromptStrings in streaming to avoid index-out-of-range panic#11028
mudler merged 2 commits into
mudler:masterfrom
Anai-Guo:fix-streaming-empty-promptstrings

Conversation

@Anai-Guo

Copy link
Copy Markdown
Contributor

What / why

CompletionEndpoint's streaming branch (core/http/endpoints/openai/completion.go) only guarded len(config.PromptStrings) > 1 before unconditionally reading config.PromptStrings[0]:

if len(config.PromptStrings) > 1 {
    return errors.New("cannot handle more than 1 `PromptStrings` when Streaming")
}
...
predInput := config.PromptStrings[0]   // panics when len == 0

A completion request to /v1/completions (streaming) whose prompt field is an empty array, an array of non-strings, or omitted entirely leaves config.PromptStrings with length 0. The > 1 guard passes, and PromptStrings[0] then panics with index out of range [0] with length 0, crashing the handler goroutine.

Fix

Guard for exactly one prompt string. len != 1 now returns a clean error for the empty case as well as the pre-existing multi-prompt case — a minimal, behavior-preserving change (a well-formed single-prompt streaming request is unaffected).

if len(config.PromptStrings) != 1 {
    return errors.New("streaming completions require exactly 1 `PromptStrings`")
}

Fixes #11021.

🤖 Generated with Claude Code

Anai-Guo added 2 commits July 21, 2026 12:12
…dex-out-of-range panic

The streaming branch of CompletionEndpoint only guarded len(config.PromptStrings) > 1
before unconditionally reading config.PromptStrings[0]. A completion request whose
prompt field is an empty array, an array of non-strings, or omitted leaves
PromptStrings with length 0, so PromptStrings[0] panics with index out of range and
crashes the handler goroutine.

Guard for exactly one prompt string instead, returning a clean error for the 0-length
case as well as the pre-existing multi-prompt case.

Signed-off-by: Tai An <antai12232931@outlook.com>
Reject streaming completion requests whose prompt does not resolve to
exactly one string (omitted prompt, empty array, or a multi-element
array) with an HTTP 400 before writing any SSE headers, instead of
returning a plain error that Echo surfaces as a 500. Extract the guard
into validateStreamingPromptStrings and cover the three reported
payloads with a regression test.

Fixes mudler#11021

Signed-off-by: Tai An <antai12232931@outlook.com>

@localai-org-maint-bot localai-org-maint-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The length guard prevents the panic, but returning a plain errors.New makes Echo respond with HTTP 500 for all three reported malformed inputs. I reproduced this against the PR head with e2e cases for an omitted prompt, prompt: [], and prompt: [1,2,3]; each still returned 500, while issue #11021 requires a 400 validation response. Please return echo.NewHTTPError(http.StatusBadRequest, "streaming completions require exactly one prompt string") before setting the SSE response headers, and add regression coverage for those three payloads. With that change, the focused e2e regression passes 3/3 locally. I prepared and verified the minor patch locally, but cannot push a new commit under your DCO sign-off; you need to apply/sign it.

@Anai-Guo

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed reproduction. Fixed in ba82146:

  • The guard now returns echo.NewHTTPError(http.StatusBadRequest, "streaming completions require exactly one prompt string") before any SSE headers are written, so an omitted prompt, prompt: [], and prompt: [1,2,3] all resolve to a 400 validation response instead of a 500 on a half-opened event stream.
  • Extracted the check into validateStreamingPromptStrings and added completion_stream_validation_test.go with table-driven regression coverage for the three reported payloads plus the happy path (exactly one prompt string).

🤖 Generated with Claude Code

@localai-org-maint-bot localai-org-maint-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mudler this is good to merge. The author addressed the requested validation path exactly: malformed streaming prompts now return an Echo HTTP 400 before SSE headers are written, with Ginkgo regression coverage for nil, empty, multiple, and single prompts. Fresh local verification: go test ./core/http/endpoints/openai -run TestOpenAI -count=1 passes.

@mudler
mudler merged commit d702070 into mudler:master Jul 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug 7: Malformed prompt field panics the streaming completions handler

3 participants