cranelift-jit: handle out-of-range x86_64 calls and symbol addresses#13975
Open
glebpom wants to merge 2 commits into
Open
cranelift-jit: handle out-of-range x86_64 calls and symbol addresses#13975glebpom wants to merge 2 commits into
glebpom wants to merge 2 commits into
Conversation
glebpom
marked this pull request as ready for review
July 25, 2026 05:08
github-actions
Bot
added
cranelift
glebpom
marked this pull request as draft
July 25, 2026 18:03
glebpom
changed the title
glebpom
marked this pull request as ready for review
July 25, 2026 19:08
bjorn3
reviewed
Jul 25, 2026
| /// Function references can be shared by `call` and `func_addr` | ||
| /// instructions. Keep the original reference for direct calls, which can | ||
| /// use veneers when necessary, and give `func_addr` its own far reference. | ||
| fn use_far_address_relocations(func: &mut ir::Function) { |
Contributor
There was a problem hiding this comment.
This feels like a hack to me. cranelift-jit shouldn't modify the function. A better solution is probably to just add support for PLT relocations and then tell Cranelift to emit PLT relocations.
`CompiledBlob::perform_relocations` computed the 32-bit displacement of
`X86CallPCRel4` relocations with `i32::try_from(..).unwrap()`, so a
`call` whose target ended up further than ±2 GiB away from the call site
panicked:
panicked at cranelift/jit/src/compiled_blob.rs:142:80:
called `Result::unwrap()` on an `Err` value: TryFromIntError(NegOverflow)
Nothing guarantees that two allocations of a `JITMemoryProvider` are
within ±2 GiB of each other: mmap is free to place separate code
allocations arbitrarily far apart in a 64-bit address space, so
module-local calls between separately defined functions can exceed the
`rel32` range. AArch64 already handles this by routing out-of-range
`Arm64Call` relocations through a veneer at the end of the function; do
the same for x86_64 by redirecting out-of-range `X86CallPCRel4`
relocations to a `jmp qword ptr [rip]` + absolute-address veneer,
reserving the exact worst-case veneer size per branch relocation.
Redirecting a control transfer through a veneer is only sound if the
relocation is applied to the displacement of a `call`/`jmp` instruction.
The x64 backend used `X86CallPCRel4` also for the RIP-relative `lea`
emitted by `Inst::LoadExtName` with `RelocDistance::Near`, which
materializes a symbol's address and must not be redirected. Switch that
to `X86PCRel4` (keeping the existing behavior of failing loudly when the
symbol is out of range), document the invariant on
`Reloc::X86CallPCRel4`, and teach x64's `LabelUse::from_reloc` about
`X86PCRel4` so that `TextSectionBuilder::resolve_reloc` can still
resolve such relocations against in-module functions. For object files
this also means the `lea` now gets a PC-relative relocation rather than
a branch relocation, matching its semantics; verify the relocations
emitted for address materialization across ELF, Mach-O, and COFF.
Also fix an off-by-one in the AArch64 veneer bounds assert:
`veneer_idx <= veneer_count` tolerated writing one veneer past the
reserved space. This is unreachable today because one veneer slot is
reserved per call relocation, but as a sanity check it was wrong.
The new JIT tests use a custom memory provider that deterministically
places allocations on opposite sides of a virtual reservation larger
than 2 GiB. They exercise a compiler-emitted call, a tail jump, both
displacement directions, and mixed near/far targets with densely packed
veneers; verify veneer shape, absolute targets, and allocation sizing;
and execute the JIT-ed code to check that the calls actually reach
their callees.
Routing out-of-range calls through veneers is not enough: JIT code can also materialize the address of a function or data object with `func_addr` or `symbol_value`. For colocated symbols the x64 backend emits a RIP-relative `lea` whose `X86PCRel4` relocation likewise assumes the symbol is within ±2 GiB of the code, and a branch veneer cannot fix an address-producing instruction: it would produce the address of the veneer rather than the requested symbol. When a `JITMemoryProvider` placed the symbol out of range, `finalize_definitions` panicked. As suggested by bjorn3, enable is_pic on x86_64 so that address materialization emits GOT-relative loads, and resolve them in the JIT relocation pass: - Reserve an 8-byte GOT entry for each `X86GOTPCRel4` relocation after the veneers at the end of the blob's allocation, store the absolute symbol address there, and point the load's displacement at the entry, which is always in range. - Relax the load to a direct `lea` when the symbol does turn out to be within displacement range, mirroring the `R_X86_64_REX_GOTPCRELX` relaxation performed by linkers, so near symbol accesses don't pay for the indirection. - Resolve `X86CallPLTRel4`, emitted for the ELF TLS model's `__tls_get_addr` call, like `X86CallPCRel4`: direct when in range and through a veneer otherwise. `JITBuilder` now defaults to is_pic=true on x86_64 hosts, and `JITModule` rejects is_pic=true elsewhere. Direct calls are unaffected: they keep using `X86CallPCRel4` with veneers for out-of-range targets, and the GOT entry holds the symbol's real address, preserving pointer identity with the finalized definitions. The new tests cover a read-only data object and a function allocated more than ±2 GiB from the code, verifying the emitted relocations, the GOT entry contents, pointer identity, and execution of the generated code, as well as the mov-to-lea relaxation for near symbols. Enable is_pic in the test ISA helper on x86_64, resolving an old FIXME from the era when the JIT last supported PIC.
glebpom
force-pushed
the
jit-x64-far-call-veneers
branch
from
July 25, 2026 22:14
2334e13 to
6d9998a
Compare
glebpom
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Handle JIT functions and data allocated outside x86_64's signed 32-bit PC-relative range.
This change:
leas when the symbol turns out to be in range.Fixes #4000.
Motivation
A
JITMemoryProvidermay allocate generated code, data objects, and functions independently. It does not guarantee that these allocations will be within ±2 GiB of one another.Both relevant x86_64 relocation forms use signed 32-bit PC-relative displacements:
X86CallPCRel4performs a direct call or jump.X86PCRel4produces the address of a symbol.If the target is outside that range,
finalize_definitionspreviously panicked when converting the displacement toi32.The problem was observed in Nervix while compiling Roto UDFs. The CI run intermittently failed with both
TryFromIntError(NegOverflow)andTryFromIntError(PosOverflow):https://github.com/nervix-io/nervix/actions/runs/30129378927/job/89600292950?pr=57#step:17:56674
After adding call veneers, a follow-up run showed that Roto could still fail while materializing addresses of separately allocated data:
https://github.com/nervix-io/nervix/actions/runs/30147477548/job/89651977365?pr=57
Implementation
The work is organized as two commits.
1. Calls and tail jumps: veneers
X86CallPCRel4relocation.jmp qword ptr [rip]veneer containing the absolute target address, the same wayArm64Callis already handled.Redirecting through a veneer is only sound for control transfers, so
X86CallPCRel4is now reserved forcall/jmpdisplacements and this invariant is documented on the relocation. The RIP-relativeleaemitted byLoadExtNamefor near symbols now usesX86PCRel4instead;LabelUse::from_relocand the text-section builder learned to resolve it, and object files now get a PC-relative rather than a branch relocation for thelea, matching its semantics.2. Symbol addresses: a per-blob GOT
A branch veneer cannot fix an address-producing instruction: it would produce the address of the veneer rather than the requested symbol. Instead, as suggested by @bjorn3 in review, the JIT now uses GOT-based address materialization:
JITBuilderdefaults tois_pic=trueon x86_64 (andJITModulerejects PIC on other architectures), soLoadExtNameemitsmovq sym@GOTPCREL(%rip)loads withX86GOTPCRel4relocations.X86GOTPCRel4relocation at the end of the blob's allocation ([code][veneers][GOT entries]), stores the absolute symbol address there at finalize time, and points the load's displacement at the entry, which is always in range.lea(a one-bytemov→leaopcode rewrite), mirroring theR_X86_64_REX_GOTPCRELXrelaxation performed by linkers, so near symbol accesses don't pay for the indirection.X86CallPLTRel4, emitted for the ELF TLS model's__tls_get_addrcall, resolves likeX86CallPCRel4: direct when in range, through a veneer otherwise.Direct calls are unaffected: they keep their colocated references and use veneers when needed. GOT entries hold the symbol's real address, preserving pointer identity with the finalized definitions.
Testing
The JIT integration tests use a custom memory provider that deterministically places allocations on opposite sides of a virtual address range larger than 2 GiB.
Call and veneer coverage:
Address-materialization coverage:
X86GOTPCRel4relocation, the GOT entry contents, pointer identity, and execution.callandfunc_addr, verifying the call remainsX86CallPCRel4while the address goes through the GOT.mov→learelaxation for a near symbol, verifying the rewritten instruction points directly at the data.LoadExtNamenear-symbol relocation kind,MachTextSectionBuilderresolution ofX86PCRel4, and non-branch address relocations for ELF, Mach-O, and COFF objects.The test ISA helper now enables
is_picon x86_64, resolving an old FIXME from when the JIT last supported PIC;libcall_functiontherefore exercises GOT resolution against real host symbols.Commands run:
cargo test -p cranelift-codegen --libcargo test -p cranelift-jitcargo test -p cranelift-objectcargo test -p cranelift-tools --test filetests -- --nocapturecargo clippy -p cranelift-codegen -p cranelift-jit -p cranelift-object --tests -- -D warningscargo fmt --all -- --checkNotes
AArch64 direct calls already use veneers, and its non-colocated materialization uses an inline literal, so it works at any distance. Its colocated
adrp+addsequence retains a pre-existing ±2 GiB assumption in theAarch64AdrPrelPgHi21handler; supportingis_picwith the AArch64 GOT relocations would be the analogous fix and is left as a follow-up.AI-assisted development
Claude Code assisted with the call-veneer implementation and with the GOT-based address materialization following the review suggestion. OpenAI Codex assisted with auditing the fix, recovering and evaluating stashed work, strengthening the tests, and porting the changes to the current main branch.
All AI-assisted code and text were reviewed by the human contributor, who understands the changes and accepts full responsibility for the contribution.
`