From be00f5aaa163d18a36bfa399346370e03322bbe2 Mon Sep 17 00:00:00 2001 From: Kristofer Karlsson Date: Thu, 18 Jun 2026 10:08:16 +0200 Subject: [PATCH 01/10] Documentation/technical: add paint-down-to-common doc Add a technical document describing the paint_down_to_common() algorithm used for merge-base computation, covering the paint walk, generation number regions, and termination conditions. Signed-off-by: Kristofer Karlsson --- Documentation/Makefile | 1 + Documentation/technical/meson.build | 1 + .../technical/paint-down-to-common.adoc | 177 ++++++++++++++++++ commit-reach.c | 6 +- 4 files changed, 184 insertions(+), 1 deletion(-) create mode 100644 Documentation/technical/paint-down-to-common.adoc diff --git a/Documentation/Makefile b/Documentation/Makefile index 2699f0b24af192..f8dea4b3953250 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -129,6 +129,7 @@ TECH_DOCS += technical/long-running-process-protocol TECH_DOCS += technical/multi-pack-index TECH_DOCS += technical/packfile-uri TECH_DOCS += technical/pack-heuristics +TECH_DOCS += technical/paint-down-to-common TECH_DOCS += technical/parallel-checkout TECH_DOCS += technical/partial-clone TECH_DOCS += technical/platform-support diff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build index ec07088c57617f..9ce11d5e484d9c 100644 --- a/Documentation/technical/meson.build +++ b/Documentation/technical/meson.build @@ -18,6 +18,7 @@ articles = [ 'multi-pack-index.adoc', 'packfile-uri.adoc', 'pack-heuristics.adoc', + 'paint-down-to-common.adoc', 'parallel-checkout.adoc', 'partial-clone.adoc', 'platform-support.adoc', diff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc new file mode 100644 index 00000000000000..ff015c5c8f1d7d --- /dev/null +++ b/Documentation/technical/paint-down-to-common.adoc @@ -0,0 +1,177 @@ +Merge-Base Computation and paint_down_to_common() +================================================== + +The function `paint_down_to_common()` in `commit-reach.c` computes merge +bases by walking the commit graph backwards from two sets of tips and +finding where their ancestry meets. + +Use cases +--------- + +Computing merge bases is used in two different ways: + + 1. *Finding all merge bases* (`merge-base --all`, `merge-tree`, + `merge`, `rebase`). A merge base is a common ancestor that is + not itself an ancestor of another common ancestor. + + 2. *Ancestry checks* (`in_merge_bases`, used by `merge-base + --is-ancestor`, `branch -d`, `fetch`). These ask: "is commit A + an ancestor of commit B?" If a common ancestor equals one of the + inputs, that input is necessarily the only merge base -- no other + common ancestor can be both as recent and not an ancestor of it. + +Both use cases share the same algorithm and implementation. + +Algorithm +--------- + +Given a commit `one` and a set of commits `twos[]`, the walk paints +commits with two colors: + + - PARENT1: reachable from `one` + - PARENT2: reachable from any commit in `twos[]` + +The walk uses a priority queue ordered by generation number +(highest first), breaking ties by commit date. Each step dequeues +the highest-priority commit (this is when we say a commit is +"visited") and propagates its paint flags to its parents, enqueuing +them if they gained new flags. When a commit receives both PARENT1 +and PARENT2, it is a merge-base candidate. A candidate gains the +STALE flag so its ancestors propagate staleness -- any deeper common +ancestor is necessarily redundant. + +NOTE: When the commit-graph uses only topological levels (generation +number v1) and the caller passes `min_generation = 0`, a legacy +fallback replaces the generation-ordered comparator with a pure +commit-date comparator. This breaks the ordering invariants +described below -- see <>. + +[[generation-regions]] +INFINITY and finite generation regions +-------------------------------------- + +The properties in this section assume generation-number ordering (the +default comparator). They do NOT hold when the date-ordering fallback +is active -- see <>. + +The commit-graph stores a generation number for each commit. Commits +not in the commit-graph have generation `GENERATION_NUMBER_INFINITY`. The +graph is closed under reachability: if a commit is in the graph, all +its ancestors are too. This partitions the commit graph into two regions: + +.... + +---------------------------------------+ + | INFINITY region | + | generation = INFINITY | + | queue order: heuristic (commit date) | + +---------------------------------------+ + | + v + +---------------------------------------+ + | Finite region | + | generation = finite | + | queue order: topological | + +---------------------------------------+ +.... + +When the commit-graph is enabled, the INFINITY region is typically +very small -- it only contains commits added since the last +commit-graph refresh. + +All reachable INFINITY-generation commits are visited before any +finite-generation commit, because INFINITY is larger than any finite +value. Once the walk crosses into the finite region, it stays there. + +In the finite region, generation ordering guarantees topological +traversal: children are always visited before their parents. This +means that paint on already-visited commits is final -- no future +traversal step can add paint to them. + +In the INFINITY region, commit-date ordering can violate this: a +parent with a later date can be visited before a child with an earlier +date. Paint flags are therefore NOT final at visit time, and a +commit visited with only one side's paint may later gain the other. + +Paint flags are only added, never removed. Since each flag can be set +at most once per commit, the number of times a commit can be +re-enqueued is bounded by the number of flag transitions. + +Termination +----------- + +The walk uses a `nonstale_queue` wrapper around `prio_queue` that +tracks `max_nonstale`: the lowest-priority non-stale commit enqueued +so far. Once that commit is dequeued, every remaining entry is known +to be STALE and the loop terminates. Specifically, the main loop +ends when one of the following conditions holds: + + 1. The queue is empty. + 2. `max_nonstale` has been dequeued, meaning the queue only contains + STALE entries. + 3. Generation cutoff: the dequeued commit's generation is below + a caller-supplied `min_generation` threshold. + 4. Single result: the caller only needs one merge base, one has + been found, and the walk has entered the finite-generation + region. + +Stale entry condition +~~~~~~~~~~~~~~~~~~~~~ +Once all queued entries are stale, no new merge-base candidates can +be discovered -- that requires at least one non-stale commit from +each side meeting. Continuing the walk could still invalidate +existing candidates by proving one is an ancestor of another, but +`remove_redundant()` handles that as a post-processing step, so it +is safe to exit early. + +Generation cutoff +~~~~~~~~~~~~~~~~~ +Some callers (notably `remove_redundant()`) supply a `min_generation` +threshold -- the minimum generation of the input commits. No merge +base can have a generation below this threshold, so the walk +terminates as soon as it dequeues such a commit. + +Single result +~~~~~~~~~~~~~ +When only one merge base is needed, the walk is in the +finite-generation region, and the queue uses generation ordering, +the first candidate found is necessarily the highest-generation +common ancestor. No remaining commit in the queue can be a +descendant of this candidate (generation ordering guarantees +children are visited first), so it cannot be redundant and the walk +can stop immediately. + +This optimization is NOT safe when the date-ordering fallback is +active, because commit-date order can visit a deeper ancestor +before a shallower one -- see <>. + +[[date-ordering-fallback]] +Date-ordering fallback +---------------------- + +When `min_generation` is zero and the commit-graph does not contain +corrected commit dates (generation number v1, which stores only +topological levels), `paint_down_to_common()` replaces the default +generation-ordered comparator with `compare_commits_by_commit_date`. + +This was introduced as a performance heuristic: topological levels +are coarser than commit dates, so date ordering can reach merge +bases in fewer steps when timestamps are well-behaved. However, +commit dates are not required to be monotonic -- a parent can have +a later date than its child (clock skew, rebases, etc.) -- so the +queue may visit commits out of topological order. + +This disables optimizations that depend on generation ordering: + + 1. *Single result*: the first merge-base candidate found may not + be the shallowest, because a deeper ancestor with a higher + commit date can be dequeued first. + + 2. *Side-exhaustion* (see subsequent commits): one paint side can + appear to drain from the queue while commits from that side are + still waiting with lower dates, causing premature termination. + +Related documentation +--------------------- + + - `Documentation/technical/commit-graph.adoc` -- generation numbers + and the reachability closure property. diff --git a/commit-reach.c b/commit-reach.c index 5df471a313cf6b..a9483759e06c59 100644 --- a/commit-reach.c +++ b/commit-reach.c @@ -96,7 +96,11 @@ static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue) return commit; } -/* all input commits in one and twos[] must have been parsed! */ +/* + * See Documentation/technical/paint-down-to-common.adoc + * + * All input commits in one and twos[] must have been parsed! + */ static int paint_down_to_common(struct repository *r, struct commit *one, int n, struct commit **twos, From 143f5e0cc183bff1bbc5e9cfa65f456f615f4ad0 Mon Sep 17 00:00:00 2001 From: Kristofer Karlsson Date: Fri, 26 Jun 2026 11:13:42 +0200 Subject: [PATCH 02/10] test-lib-functions: improve diagnostic output for trace2 data assertions test_trace2_data is a bare grep that silently exits on failure. Add a more informative variant that verifies the event appears exactly once and reports what went wrong: key not found, multiple entries, or value mismatch. Diagnostics go to FD 4 like test_grep. Before (value mismatch): $ test_trace2_data status count/changed 999 --- t/test-lib-functions.sh | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh index 809c6621241944..3521efe5d732af 100644 --- a/t/test-lib-functions.sh +++ b/t/test-lib-functions.sh @@ -1996,6 +1996,42 @@ test_trace2_data () { grep -e '"category":"'"$1"'","key":"'"$2"'","value":"'"$3"'"' } +# Check that the given trace2 data event has the expected value and +# appears exactly once. Produces a diagnostic on failure. +# +# test_trace2_data_singular [