Micro-optimizations for the statistics module#152618
Merged
Merged
Conversation
Baseline: % ./python.exe -m timeit -s 'from statistics import _sum as sm, NormalDist as ND' -s 'data=ND().samples(1000 )' 'sm(data)' 1000 loops, best of 5: 293 usec per loop Patched: % ./python.exe -m timeit -s 'from statistics import _sum as sm, NormalDist as ND' -s 'data=ND().samples(1000)' 'sm(data)' 1000 loops, best of 5: 282 usec per loop
Reduction in strength from `x ** 2` to `x * x'. The latter is faster and more accurate. Baseline: % ./python.exe -m timeit -s 'from statistics import _kernel_specs as ks' -s 'pdf = ks["quartic"]["pdf"]' 'pdf(0.1234)' 5000000 loops, best of 5: 73 nsec per loop Patched: % ./python.exe -m timeit -s 'from statistics import _kernel_specs as ks' -s 'pdf = ks["quartic"]["pdf"]' 'pdf(0.1234)' 5000000 loops, best of 5: 50.2 nsec per loop
eendebakpt
reviewed
Jun 29, 2026
| @register('quartic', 'biweight') | ||
| def quartic_kernel(): | ||
| pdf = lambda t: 15/16 * (1.0 - t * t) ** 2 | ||
| pdf = lambda t: 15/16 * (u := 1.0 - t * t) * u |
Contributor
There was a problem hiding this comment.
This does not improve readability. Do you have a bench ark for the gain of this change?
Contributor
Author
There was a problem hiding this comment.
Yes. The timings were in the individual commit messages. I just moved them to the top message in Conversation.
It is not a beautiful edit, but it isn't terrible either. Note, besides giving a nice speed-up, the reduction in strength from x ** 2 to x * x also improves accuracy.
Contributor
There was a problem hiding this comment.
@rhettinger If it improves accuracy, it changes behavior. Can you add a news entry and a test?
(since you already merged the PR, this should be a followup PR)
eendebakpt
added a commit
to eendebakpt/cpython
that referenced
this pull request
Jun 29, 2026
…patch Builds on PR python#152618 (quartic walrus square; drop bound-method caching in _sum/_ss). groupby() yields type-homogeneous groups, so resolve typ.as_integer_ratio once per group instead of calling _exact_ratio() on every value. The dispatch lives in a new _exact_ratios(typ, values) generator so the accumulation loops stay unchanged; both _sum and _ss use it. Rare types without as_integer_ratio fall back to _exact_ratio (preserving its TypeError and Integral-ABC path); NAN/INF handled inline. ~4% faster than the PR on _sum, all 400 test_statistics pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
eendebakpt
added a commit
to eendebakpt/cpython
that referenced
this pull request
Jun 29, 2026
…patch Builds on PR python#152618 (quartic walrus square; drop bound-method caching in _sum/_ss). groupby() yields type-homogeneous groups, so resolve typ.as_integer_ratio once per group instead of calling _exact_ratio() on every value. The dispatch lives in a new _exact_ratios(typ, values) generator used by both _sum and _ss, so their accumulation loops are unchanged. _exact_ratios delegates all conversion semantics back to _exact_ratio (whole-group fallback for types without as_integer_ratio; per-element for float/Decimal NAN/INF), so no logic is duplicated. ~4% faster than the PR on _sum; all 400 test_statistics pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remove boundmethod optimizations that are no longer performant:
Replace
x ** 2withx * x: