fix(cicd): chunk bot messages on character boundaries#5882
Open
anxkhn wants to merge 1 commit into
Open
Conversation
_chunk_up_api_message sliced the UTF-8 encoded message at fixed byte offsets and decoded each slice with errors="ignore". A multibyte character that straddled a chunk boundary had its bytes split across two slices, so the "ignore" handler silently dropped it from the bot's PR comment or check-run output. Accumulate characters into a chunk while the running UTF-8 byte length stays within MAX_BYTE_LENGTH instead of slicing raw bytes, so no character is ever bisected. ASCII-only messages chunk identically to before, and an empty message still yields no chunks. Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GithubController._chunk_up_api_messagesplits a message into chunks that fitGitHub's per-field byte limit (
MAX_BYTE_LENGTH, 65535). It did this by UTF-8encoding the whole message, slicing the resulting bytes at fixed
MAX_BYTE_LENGTHoffsets, and decoding each slice witherrors="ignore":When a multibyte character (accented letter, CJK, emoji, etc.) straddles a chunk
boundary, its UTF-8 bytes are split across two slices.
decode("utf-8", "ignore")then discards the incomplete byte sequence on both sides, so that character is
silently dropped from the bot's PR comment or check-run output.
This change accumulates characters into a chunk while the running UTF-8 byte
length stays within
MAX_BYTE_LENGTH, so a boundary is only ever placed betweenwhole characters and no character is bisected. Behavior for the common case is
unchanged: ASCII-only messages chunk into the exact same pieces as before, and an
empty message still yields no chunks. Both existing callers rely only on the
number of chunks (comment edit unpacks
body, *truncated; check-run outputunpacks
summary, text, *truncated), and that contract is preserved.Test Plan
test_chunk_up_api_message_preserves_multibyte_chars, which builds amessage just over
MAX_BYTE_LENGTHwith a multibyte character (é) placed onthe byte boundary between two chunks, and asserts the chunks reassemble into the
original message and each chunk stays within
MAX_BYTE_LENGTHbytes. This testfails on the previous byte-slicing implementation (the boundary character is
dropped) and passes with the fix.
pytest tests/integrations/github/cicd/ -m "not slow and not docker"-> 55passed, 28 deselected.
make style(ruff check + ruff format) clean on the changed files.Checklist
make styleand fixed any issuesmake fast-test)git commit -s) per the DCO