perf(v1): skip mask/where/const for dense variables in to_linexpr#836
Merged
Conversation
Variable.to_linexpr under v1 unconditionally built an `absent` mask, a `where(~absent)` copy and an explicit all-zero `const` array, even for a fully-dense variable that carries no absent slots. On the isolated `var * array` (match) path those are all no-ops but still allocate full-size temporaries — the CodSpeed test_op[var_mul_array_match] memory regression (212 -> 306 KB) and the small-model test_build peaks. Gate the mask/where/const on `has_absence`. Dense variables now build the same lean coeffs+vars expression as legacy; masked variables still carry NaN at absent slots for §6 propagation. Isolated var*array peak (memray): v1 5000-elem 133.7 -> 78.1 KiB, 200k 5275 -> 3125 KiB (now below legacy). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TODO: your own intent/motivation here.
Note
The following content was generated by AI.
What
Variable.to_linexprunder v1 unconditionally built anabsentmask, awhere(~absent)copy, and an explicit full-size all-zeroconstarray —even for a fully-dense variable with no absent slots. On the isolated
var * array(exact-match) path these are all no-ops but still allocatefull-size temporaries.
This is the source of the remaining CodSpeed memory regression after #804/#816:
test_op[var_mul_array_match]and the small-model
test_buildpeaks (knapsack-100 +19%, milp-10 +17%,sparse_network-10 +16%, sos-10 +13%). Large models were already at parity —
the all-zero
constis pruned at matrix assembly (#816), so built modelsare byte-identical; the regression only shows on the isolated expression object.
Fix
Gate the mask /
where/constonhas_absence = (labels == -1).any().Dense variables now build the same lean
coeffs+varsexpression as legacy;masked variables still carry NaN at absent slots for §6 propagation.
Measurements
Isolated
var * arraypeak (memray, warmed):v1 now sits below legacy on this path.
Tests
Full suite passes under both semantics: 7813 passed, 605 skipped
(
pytest test/). Dense/masked equivalence verified: maskedcoeffsstill
[0, nan, 2, nan].Follow-up (not in this PR)
The v1 branch still runs
reindex_like(fill_value=nan)even when thecoefficient already matches labels exactly (
first_mismatched_dimreturnedNone). Skipping that no-op copy — mirroring
_align_constant'sneeds_data_reindex=Falsepath from #804 — would trim further.