asterism branch T-2093/bugfix-settling-pass-budget commits 7 files 11 touched lines +843 / -93 settling pass 7.3 s → 1.69–1.70 s

Pre-push review: T-2093/bugfix-settling-pass-budget

Seven commits over origin/main (merge base 4cab443, head 299ffa6), PR #66. The duplicate reconciler's settling pass detaches a chunk's doomed Entry rows from Site.entries in one rewrite per Site; Req 10.1's 2 s budget is met (1.688–1.701 s) and asserted plainly.

At a glance

  • Fix is correct and minimal: 63 lines in one production file; Req 2.9, 1.5, 2.1 and Q86 preserved; verified by inspection and by a scratch probe of the rollback/replay path including a set that aborts at replay.

  • Budget claim checks out: the raw log holds three runs at 1.688 / 1.701 / 1.701 s medians, 7 known issues each, EXIT=0; every number quoted in the docs ties to it.

  • Removing the known issue is consistent with Q33: nothing is raised; the floor that goes was looser than the budget beside it.

  • Docs need a pass: the agent note's 'deletes-then-saves in one direction' is false, 'linear in the array' overstates, the 250 × 8 ms arithmetic does not reach 6.4 s, a Decision 32 bullet is inverted, and there is no CHANGELOG entry.

  • Test gap, not a bug: no test pins the detach mechanism or the abort-at-replay scenario; the probe used here is a ready-made test.

Verdict

Ready to push

Ready to push. The production change is small, matches Decision 32, and holds under the two paths the review was asked to scrutinise: a rolled-back chunk's Site.entries rewrite is restored before the per-set replay, and a set that changes concurrently during the failing save aborts at replay with its rows still attached — both confirmed by a scratch probe, not only by reasoning. Identity matching is sound within one context, and the 7.3 s → 1.7 s measurement is the proof it removes rows. Dropping the known issue with ~15% median headroom follows the Q33 precedent exactly (no bound is raised; a floor looser than the budget asserts nothing), with the caveat that CONTROLLED=1 asserts the worst of 10 samples and needs a quiet host. Nothing blocks a merge. Before merging, the doc corrections are cheap and worth doing: the agent note's rollback sentence and 'linear' claim, the 250 × 8 ms arithmetic, the inverted Decision 32 bullet, and the CHANGELOG entry. The test gaps (detach-before-save, abort-at-replay, multi-Site chunk) are follow-ups.

Review findings

16 raised · 0 fixed · 16 skipped

Jump to findings →

Tests

Pass rate: 100% (2328 of 2328)

New tests: 2

Diff coverage: 99% (117 of 118 added lines)

Jump to tests →

Commits

Three-level explanation

What Changed

Asterism keeps a list of every reading-app capture (an Entry) and, for each website, a Site record that holds the list of all its entries. When the app finds duplicate entries it deletes the extra copies in the background. That clean-up step (the "settling pass") was taking about 7 seconds when the rules said it should take 2.

The cause was hidden inside the database framework (SwiftData). Every time one entry was deleted, the framework quietly walked the site's whole list of entries — 5,000 items — to take that one entry out. Deleting 250 entries meant 250 walks of a 5,000-item list.

The fix removes all the doomed entries from the site's list in one walk before deleting them, so the framework has nothing left to do per row. The pass now takes about 1.7 seconds.

Why It Matters

The cost of deleting a duplicate used to grow with the size of the library. A reader with a big library would have seen the background clean-up get slower and slower as they captured more. Now it stays flat.

Key Concepts

  • Inverse relationshipEntry.site points at a site, and Site.entries is the same link seen from the other side. The framework keeps the two in sync automatically, and that syncing is what cost 8 ms per deleted row.
  • Chunk and replay — deletions are saved in batches ("chunks"). If a batch fails to save, everything in it is undone and each set is retried on its own. The fix had to keep working in that retry path too.
  • Known issue vs plain assertion — the test that measures this pass used to say "we know it's over budget, don't fail". Now it just asserts the 2 s budget.

Changes Overview

One production file: DuplicateReconciler.swift. stage(_:rows:...) gains an inout doomedEntries: [Entry] parameter and no longer calls context.delete for Entry plans — it appends the loser rows to the accumulator on the return true path only. commitDeletions stages the whole chunk, then calls a new private static func delete(entries:context:), which groups the doomed rows by their Site (keyed on ObjectIdentifier), does one site.entries?.removeAll { doomed.contains(ObjectIdentifier($0)) } per Site, then context.deletes each row. The per-set replay after a failed chunk does the same with a per-plan accumulator.

Tests: two new DuplicateReconcilerTests pin that a collapse detaches only its own rows and that an aborted plan contributes nothing to the detach; the existing failed-chunk replay test gains a Site.entries consistency assertion. M4DuplicateScalePerformanceTests drops the withKnownIssue wrapper and the 11 s ceiling and asserts the 2 s budget plainly. SiteInverseReachTests (the guard that keeps every read off Site.entries) gains a single sanctioned snippet, keyed on the call, with a check that the snippet still matches.

Implementation Approach

Profiling (a temporary phase accumulator, deleted before commit) attributed 88% of the 7.3 s pass to the single context.save() in commitDeletions, and micro-benchmarks isolated it to Entry.site teardown: 8.02 ms per row against a 5,000-element inverse versus 0.20 ms for the five-element Work.entries. Removing the rows from the array in one pass costs one walk per Site per chunk instead of one per row: 200 rows measured 1.83 s → 0.13 s.

Identity matching is load-bearing: an Entry.id is the logical record's UUID and a duplicate group's rows share it, so a UUID set would detach survivor rows too. Deferring Entry deletions to the end of a chunk's staging is safe because every Work plan in the chunk has already re-pointed its Entries by then (Req 1.5 ordering), and an aborted plan appends nothing.

Trade-offs

  • Batch delete (ModelContext.delete(model:where:)) would skip the maintenance entirely but produces no change tracking for CloudKit mirroring — rejected.
  • Clearing entry.site = nil per row only moves the cost out of save() — measured, rejected.
  • The guard test now carries one exemption in a rule whose value was having none. Decision 32 records that as the cost.
  • A replayed set after a failed chunk now pays a whole-array rewrite for one or two rows, roughly 1–2× the old per-set cost — an exceptional path Q86 already prices per set.

Technical Deep Dive

site.entries?.removeAll(where:) through the @Model-synthesised accessor is one get (materialising the full inverse — the cold fault is why measurements.md §4's 0.24 s detach phase is 6× the §3 warm-store 0.038 s) plus one set, in which SwiftData diffs old vs new and nils each removed row's site. The doomed rows come from DeletionRows' predicated fetch, so they are materialised objects; the array is only rewritten, never deleted through — which keeps this outside the crash mode the agent note's first rule records (snapshotting a deleted future-backed row on rollback).

Rollback/replay (Req 2.9, Q86). The review's central question was whether, after context.rollback(), the Site's cached entries array stays stale (the codebase documents stale @Model accessors after rollback, and DeletionRows.refault never re-fetches Site). If it did, the first replay save would write back an array missing the whole chunk's losers, and a set aborted later in the same replay would be left with site == nil. This was probed empirically in a scratch copy of the package (not committed): (A) chunk fails, no concurrent change — after the first replay save, zero rows are detached; (B) the failing save is made to coincide with a concurrent write to one set's loser through a second context — that set aborts at replay (collapsedMembers == 2, one settling key), its rows remain present with site != nil, and Site.entries equals the surviving set. So rollback does restore the array and the replay detach is per-plan as designed. The PR has no test for scenario B; the probe is a ready-made one.

Identity across fetches. Within one ModelContext, SwiftData uniques registered models by persistent identifier, so the inverse array and the predicated fetch hand back the same instances and ObjectIdentifier matching holds. The only evidence in the PR that the match actually removes anything is the M4 perf number (7.3 s → 1.7 s): both new unit tests pass with a no-op removeAll because the .nullify delete rule yields the same end state.

Architecture Impact

The cost of a collapse no longer scales with the number of Entries on the hostname. The Work-before-Entry plan ordering is unchanged and, within a chunk, is now less load-bearing than before (no Entry row is deleted until every Work plan has staged) — Decision 32's Consequences bullet says the opposite. The general rule captured in swiftdata-relationships.md is sound in substance but over-states linearity ("linear in the array"; cloudkit-mirroring Q27 measured the mutation superlinear) and mis-describes this path as never rolling back.

Potential Issues

  • Per-row arithmetic in the docs does not close: 250 Entry losers × 8 ms ≈ 2 s, not the 6.4 s measured in-pass. The attribution rests on the before/after save times (6.42 s → 0.41 s), which is fine, but the text implies a multiplication that does not hold.
  • CONTROLLED=1 asserts p95, and at settlingSampleCount = 10 that is the single slowest sample: 1.92–3.41 s recorded on a contended host against a 2 s bound. A quiet-host controlled run is the only one that can be expected green.
  • Rollback after the rewrite over thousands of unmaterialised rows is exercised only on a 6-entry fixture; the M4 suite never fails a save.
  • The sanctioned-snippet check is a substring match on the whole line; a second copy or a piggy-backed traversal is caught only by the hit-count mismatch, whose message says "drop the entry if the call is gone".

Completeness Assessment

Fully implemented: the fix, the Req 10.1 plain assertion, the guard exemption, Decision 32 / Decision 27 superseded, bugfix report and measurements, agent notes, known-issue counts. Partially: test coverage of the mechanism (detach before save, abort-at-replay, multi-Site chunk, site == nil arm are untested). Missing: CHANGELOG entry, specs/OVERVIEW.md refresh, corrections to the agent-note prose.

Important changes — detailed

DuplicateReconciler: one Site.entries rewrite per Site per chunk

Packages/AsterismCore/Sources/AsterismCore/DuplicateReconciler.swift

Why it matters. The whole fix. SwiftData's per-row inverse maintenance was 88% of the settling pass; this pays the array walk once per Site instead of once per deleted row.

What to look at. DuplicateReconciler.swift:932-966 delete(entries:context:)

Takeaway. When deleting many rows that share one large to-many parent, remove them from the parent's array in one pass first, then delete. Match by object identity when application ids are shared across rows.
Rationale. Measured: 200 rows 1.83 s deleted one at a time vs 0.13 s after one removeAll; entry.site = nil per row costs the same 8 ms. Batch delete rejected because CloudKit mirroring needs change tracking (Decision 32).

stage: hand doomed rows back instead of deleting them

Packages/AsterismCore/Sources/AsterismCore/DuplicateReconciler.swift

Why it matters. Deferring Entry deletions to the end of a chunk's staging is what makes the single rewrite possible; correctness depends on an aborted plan appending nothing and on Work plans having re-pointed before any Entry is deleted.

What to look at. DuplicateReconciler.swift:1002-1033 (.entry arm), 886-928 (chunk loop and replay)

Takeaway. An inout accumulator threaded through a for-where clause works but a return value ([Entry]? with nil = aborted) would read cleaner and drop the per-plan local in the replay.
Rationale. The doc comment on commitDeletions states the ordering argument explicitly; Decision 32's Consequences bullet inverts it (it says the rule is now more load-bearing; within a chunk it is less).

Replay after a failed chunk: per-plan detach

Packages/AsterismCore/Sources/AsterismCore/DuplicateReconciler.swift

Why it matters. Q86's per-set rollback guarantee lives here. If rollback left Site.entries stale, the first replay save could detach rows of sets that later abort.

What to look at. DuplicateReconciler.swift:911-926

Takeaway. Probed empirically (scratch copy, not committed): rollback restores the array; a set that changes concurrently during the failing save aborts at replay with its rows still on the Site. The PR carries no test for that scenario — the probe is a ready-made one.
Rationale. Author reasoning: 'a failing chunk rolls back and replays its sets one at a time, each with its own detach' (Decision 32). The assumption that rollback restores the Site's array is not stated anywhere; the review verified it. (inferred — not stated by the author)

M4DuplicateScalePerformanceTests: Req 10.1 asserted plainly

Packages/AsterismCore/Tests/AsterismCoreTests/M4DuplicateScalePerformanceTests.swift

Why it matters. Removes the withKnownIssue wrapper and the 11 s ceiling; the 2 s budget is now a hard failure with ~15% median headroom (1.688-1.701 s over three contended runs).

What to look at. M4DuplicateScalePerformanceTests.swift:36-40, 71-89, 160

Takeaway. Q33 precedent: a floor looser than the budget beside it asserts nothing, so it goes when the breach closes. The 'never weaken a budget' rule is about raising bounds; nothing here is raised.
Rationale. Stated in the file comment and Decision 32. CONTROLLED=1 asserts p95, which at n=10 is the worst sample (1.92-3.41 s recorded) — a controlled run needs a quiet host.

SiteInverseReachTests: one sanctioned traversal

Packages/AsterismCore/Tests/AsterismCoreTests/SiteInverseReachTests.swift

Why it matters. The guard that keeps every read off Site.entries now has an exemption; how narrow it really is determines whether the guard still guards.

What to look at. SiteInverseReachTests.swift:32-57, 90-107

Takeaway. Keyed on the call text and asserted to still match, so it cannot go stale silently. But it is a substring match on the line: a second copy or a piggy-backed traversal is caught only by the hit-count mismatch, whose message says 'drop the entry'.
Rationale. Stated: 'keyed on the exact call rather than on the file, so any other traversal in DuplicateReconciler.swift still fails this test'.

docs/agent-notes/swiftdata-relationships.md: the general rule

docs/agent-notes/swiftdata-relationships.md

Why it matters. Cross-feature knowledge; a future session will act on it. It is right in substance and wrong in two details.

What to look at. swiftdata-relationships.md:77-127

Takeaway. The rule (detach in one pass, match by identity) is sound. 'Linear in the array' is not what was measured (Q27 found the mutation superlinear; 5 -> 0.20 ms, 5,000 -> 8.02 ms), and 'this path deletes-then-saves in one direction' is false — commitDeletion rolls back on a failed save. The title still says 'two rules'.
Rationale. Author's reconciliation with the first rule; the accurate one is that the deleted rows come from a predicated fetch and only the array is rewritten.

Key decisions

Detach via one array rewrite per Site, not batch delete or per-row nil.

Decision 32. Batch delete produces no change tracking for CloudKit mirroring; entry.site = nil per row measured the same 8.5 ms; dropping the inverse is a schema change CloudKit forbids; bounding sets per pass leaves the per-row cost in place.

Match doomed rows by ObjectIdentifier, never by id.

Decision 32, code comment, both new tests. A duplicate group's rows share the logical record's UUID, so an id set would detach survivors' rows.

Delete the known issue and the 11 s floor; assert 2 s plainly.

Q33 of drop-superseded-columns: a floor looser than the budget asserts nothing once the breach closes. The budget itself is untouched.

Exempt exactly one call in SiteInverseReachTests, keyed on the call text.

Decision 32 Consequences. The alternative — exempting the file — would let a future read slip through.

Defer a chunk's Entry deletions to the end of staging.

commitDeletions doc comment: every Work plan has re-pointed before any Entry row is deleted, so the Req 1.5 ordering is preserved.

Record the band on a contended host rather than wait for a quiet one.

report.md says so openly. Medians agree to 13 ms; the p95 outlier in run 2 (3.41 s) is attributed to contention. A quiet RUNS=1 would strengthen the record.

(inferred — not stated by the author.)

Review findings

SeverityAreaFindingResolution
minorswiftdata-relationships.md:119-123The reconciliation with the first rule says 'this path deletes-then-saves in one direction'. It does not: commitDeletion calls context.rollback() on a failed save and replays. The accurate statement is that the deleted rows come from a predicated fetch and only the array is rewritten.Not applied (review-only run). Reword the sentence.
minorswiftdata-relationships.md:1,3,83Title says 'two rules' and intro says 'Both were learned in multi-site-works'; there are three sections now. 'Linear in the array' overstates the measurement: 5 -> 0.20 ms, 5,000 -> 8.02 ms, and cloudkit-mirroring Q27 measured the mutation superlinear.Not applied. Update the title/intro; say 'grows with the size of the array, paid once per deleted row'.
minortesting.md:285, swiftdata-relationships.md:119, Decision 27 status, DuplicateReconciler.swift:940-942'8.0 ms per deleted row' paired with '300 rows' reads as an account of the 6.4 s save, but only the 250 Entry losers sit in Site.entries and 250 x 8 ms is ~2 s. The attribution stands on the before/after save times (6.42 s -> 0.41 s), not on that product.Not applied. Say '250 Entry rows' and note the micro-benchmark under-predicts the in-pass cost.
minordecision_log.md Decision 32, Consequences/Negative bullet 2'the ordering rule ... is now load-bearing at one more point' is inverted: within a chunk no Entry row is deleted until every Work plan has staged, so the hazard the rule guards cannot occur there. The commitDeletions comment has it right.Not applied. Replace with the real negative: doomed rows stay live across the chunk's staging, so a later plan can still write to a row about to be deleted.
minorDuplicateReconcilerTests.swift (replay path)No test covers a set that aborts at replay after a failed chunk. aFailedDeletionChunkReplaysPerSet replays every set successfully, so a stale Site.entries after rollback would be invisible to it. Verified by a scratch probe that the path is correct (rollback restores the array; the aborted set's rows stay attached).Not applied. Add the probe as a test: a save strategy that throws on the first deleting save and mutates one loser through a second context; assert that set's rows are present with site != nil and Site.entries matches.
minorDuplicateReconcilerTests.swift:657-731Both new tests pass whether or not removeAll removes anything (the .nullify delete rule yields the same end state). Nothing on the pre-commit bar pins that the detach happens or that SwiftData nils entry.site on removal; the only evidence is the opt-in M4 number.Not applied. Make delete(entries:context:) internal and assert, before save, that doomed rows have site == nil, Site.entries lacks them, and a survivor sharing the id keeps its site. Also cover a multi-Site chunk and a doomed row with site == nil.
minorSiteInverseReachTests.swift:50-57, 89-93isSanctioned is a substring match on the whole line. A second site.entries?.removeAll in the file, or a line with the snippet plus another traversal, is never an offence; it trips only the hit-count check, whose message says 'drop the entry if the call is gone'.Not applied. Anchor the snippet at the regex match and count hits per snippet so extras report as offences.
minorM4DuplicateScalePerformanceTests.swift:99-103CONTROLLED=1 asserts p95; at settlingSampleCount = 10 that is samples[9], the single slowest run. Recorded p95 1.92 / 3.41 / 1.95 s against 2 s. The settlingSampleCount comment does not say the controlled statistic is the max.Not applied. Say so in the comment, or raise the sample count if controlled runs are expected green.
minorCHANGELOG.mdBugfixes in this repo get an entry under [Unreleased] / Fixed with the ticket and a pointer to the report (T-2293 at line 11 is the model). T-2093 has none.Not applied. Add the entry.
minorspecs/OVERVIEW.md:13, specs/duplicate-reconciliation/tasks.md:290The Duplicate Reconciliation row still says the settling pass is breached and unattributed (T-2093). tasks.md's 10.1 line still reads BREACHED.Not applied. Regenerate OVERVIEW.md (specs-overview skill); add 'resolved by T-2093, Decision 32' to the tasks.md line.
minormeasurements.md section 4The 0.236-0.243 s 'detaching the chunk from its Site' phase is 6x the section 3 micro-benchmark (0.038 s). The difference is the cold fault of Site.entries in a fresh context; unstated, it looks like a regression to chase.Not applied. One sentence stating the phase includes the cold fault.
minorCLAUDE.md:61'See specs/work-and-reading-status/verification-run.md section 4 for the current numbers' now points at a settling-pass row (7.413 s, known issue) this PR supersedes.Not applied. Add specs/bugfixes/settling-pass-budget/report.md as the current settling-pass record.
nitdecision_log.md Decision 32, Impact'Every other bulk deletion path in the repository is reader-scoped' holds for production code; the test-only M4PerformanceFixture.reseedM4DuplicateSets deletes ~1,350 Entry rows on the same Site per settling sample and would benefit from the helper (roughly 11 s per sample of untimed re-seed).Not applied. Qualify as 'every other production path', or reuse the helper in the fixture.
nittesting.md:292'Zero would mean the filter or the opt-in gate...' now follows the last bullet with no blank line, so it renders as a continuation of the settling-pass bullet and its antecedent is two paragraphs away.Not applied. Blank line, and move it up after '...seven now.'
nitDuplicateReconciler.swift:891-926, 1002-1008; DuplicateReconcilerTests.swift:881-883inout doomedEntries threaded through a for-where clause; the chunk and replay loops repeat stage -> delete -> commitDeletion; the test comment says 'a survivor off its Site' where only a loser of a later-aborted set could be affected.Not applied. Optional: return [Entry]? from stage; reword the comment.
nitreport.md:218-220, measurements.md:23-27,115, M4DuplicateScalePerformanceTests.swift:39Req 5.4 range is run 1 only (three-run range 0.170-0.204 s); the Share column mixes generations (88% vs 87%); '7.26-7.48 s' splices the task-24 low with the instrumented high, whose table tops at 7.451 s.Not applied. Quote ranges consistently.

Tests

Source: local run at 2026-09-05T23:25:30+10:00 · snapshot 299ffa643d8ebaac05717c51981fe0102a521ea0

Baseline: none

Execution: passed · JUnit: 1 file · Coverage: 1 file · Baseline: absent

Coverage scope: every test in the repository

Totals: 2328 passed · 0 failed · 35 skipped · 0 errored · 0 flaky

New and removed tests

Derived by declaration name, from the diff (no baseline run).

Diff coverage

FileAdded linesCoveredDiff coverage
Packages/AsterismCore/Sources/AsterismCore/DuplicateReconciler.swift6324100%
Packages/AsterismCore/Tests/AsterismCoreTests/DuplicateReconcilerTests.swift9777100%
Packages/AsterismCore/Tests/AsterismCoreTests/M4DuplicateScalePerformanceTests.swift2400%
Packages/AsterismCore/Tests/AsterismCoreTests/SiteInverseReachTests.swift4316100%
docs/agent-notes/swiftdata-relationships.md52no coverage data
docs/agent-notes/testing.md21no coverage data
specs/bugfixes/settling-pass-budget/report.md256no coverage data
specs/bugfixes/settling-pass-budget/measurements.md125no coverage data
specs/duplicate-reconciliation/decision_log.md147no coverage data
specs/duplicate-reconciliation/implementation.md14no coverage data
CLAUDE.md1no coverage data

Aggregate diff coverage: 99% (117 of 118 measurable added lines).

Overall coverage

Head 93.7% (78236 of 83488 lines)

4 of 11 changed files matched coverage data.

Blast radius

Files that import a changed file on the left, changed files in the centre, files a changed file imports on the right. Snapshot working-tree against base 4cab44318f4d42a779ed4d9bc2c920cce7fcdedc.

addedmodifieddeletedrenamedunchangedcollapsed package group or +N more⚑N test files with an edge to the node

Per-file diffs

Click to expand.

Packages/AsterismCore/Sources/AsterismCore/DuplicateReconciler.swift Modified +63 / -7
diff --git a/Packages/AsterismCore/Sources/AsterismCore/DuplicateReconciler.swift b/Packages/AsterismCore/Sources/AsterismCore/DuplicateReconciler.swiftindex ef6d897..e43c754 100644--- a/Packages/AsterismCore/Sources/AsterismCore/DuplicateReconciler.swift+++ b/Packages/AsterismCore/Sources/AsterismCore/DuplicateReconciler.swift@@ -858,6 +858,15 @@ enum DuplicateReconciler {     /// would be a second statement of the same rule, and the one place it could     /// drift is the two lines in `run`. Chunking preserves it — chunks are cut     /// out of `plans` in order and never reorder within one.+    ///+    /// **The Entry deletions of a chunk are deferred to the end of its staging**+    /// (T-2093), so they can be detached from their Site together — see+    /// `delete(entries:context:)` for why that is worth doing. It does not touch+    /// the ordering rule above: every Work plan of a chunk is staged, and has+    /// therefore already re-pointed everything it was going to re-point, before+    /// any Entry row of that chunk is deleted. A staged plan's rows are held+    /// aside rather than kept: an aborted plan appends nothing, exactly as it+    /// previously deleted nothing.     static func commitDeletions(         _ plans: [DuplicateDeletionPlan],         canonicalWorkIDs: [UUID: UUID],@@ -879,14 +888,16 @@ enum DuplicateReconciler {             // replay only runs after a rollback has invalidated these rows.             var distinctPairs = try context.fetch(FetchDescriptor<WorkDistinctPair>())             var staged: [DuplicateDeletionPlan] = []+            var doomedEntries: [Entry] = []             for plan in chunk             where try stage(                 plan, rows: &rows, canonicalWorkIDs: canonicalWorkIDs, types: types,-                distinctPairs: distinctPairs, context: context)+                distinctPairs: distinctPairs, doomedEntries: &doomedEntries, context: context)             {                 staged.append(plan)             }             guard !staged.isEmpty else { continue }+            delete(entries: doomedEntries, context: context)             if try commitDeletion(                 context: context, saveStrategy: saveStrategy,                 refault: { try rows.refault(context: context) })@@ -900,11 +911,13 @@ enum DuplicateReconciler {             // failing one — the cost of the chunking, and the reason the replay             // exists rather than the whole chunk being abandoned.             distinctPairs = try context.fetch(FetchDescriptor<WorkDistinctPair>())-            for plan in staged-            where try stage(-                plan, rows: &rows, canonicalWorkIDs: canonicalWorkIDs, types: types,-                distinctPairs: distinctPairs, context: context)-            {+            for plan in staged {+                var replayed: [Entry] = []+                guard try stage(+                    plan, rows: &rows, canonicalWorkIDs: canonicalWorkIDs, types: types,+                    distinctPairs: distinctPairs, doomedEntries: &replayed, context: context)+                else { continue }+                delete(entries: replayed, context: context)                 if try commitDeletion(                     context: context, saveStrategy: saveStrategy,                     refault: { try rows.refault(context: context) })@@ -916,6 +929,42 @@ enum DuplicateReconciler {         return committed     } +    /// Deletes Entry rows, detaching them from their Site in **one rewrite of+    /// that inverse per Site** rather than one removal per row (T-2093).+    ///+    /// `Site.entries` is the inverse of `Entry.site`, and it holds every Entry+    /// captured on the hostname — 5,000+ rows on the M4 fixture, and the reason+    /// the relationship is `internal` and never traversed (Q17). SwiftData+    /// maintains it on every deletion, and the maintenance is linear in the+    /// array: **8.0 ms per row over a 5,000-row Site**, whether it is paid at+    /// `context.delete` time or by assigning `entry.site = nil`. That is a cost+    /// per deleted row proportional to the size of the library, and it was 88%+    /// of the Req 10.1 settling pass — 6.4 s of a 7.3 s pass sat inside the one+    /// `save` this phase performs, and the staging around it cost 0.09 s.+    ///+    /// Removing the whole doomed set from the array in a single pass pays that+    /// linear walk **once per Site per chunk** instead of once per row: 200 rows+    /// measured 1.83 s deleted one at a time and 0.13 s this way, and+    /// SwiftData nils each removed row's `site` for us, so the rows reach+    /// `context.delete` already detached.+    ///+    /// Rows are matched by object identity rather than by `id`: an Entry's `id`+    /// is the *logical record's* UUID and a duplicate group shares it, so an id+    /// set would detach rows this call was not asked to delete.+    private static func delete(entries: [Entry], context: ModelContext) {+        guard !entries.isEmpty else { return }+        let doomed = Set(entries.map(ObjectIdentifier.init))+        var sitesByIdentity: [ObjectIdentifier: Site] = [:]+        for entry in entries {+            guard let site = entry.site else { continue }+            sitesByIdentity[ObjectIdentifier(site)] = site+        }+        for site in sitesByIdentity.values {+            site.entries?.removeAll { doomed.contains(ObjectIdentifier($0)) }+        }+        for entry in entries { context.delete(entry) }+    }+     /// Plans grouped into commit chunks, cut on the number of *rows* a chunk     /// deletes rather than the number of sets — the same measure     /// `LibraryRepository.chunks` uses, and the reason a set never straddles a@@ -944,12 +993,19 @@ enum DuplicateReconciler {     /// Returns whether the plan was staged: a fingerprint that no longer matches     /// is Req 2.9's abort, and it is decided per set whether the save that     /// follows covers one set or a chunk of them.+    ///+    /// - Parameter doomedEntries: the Entry rows the plan wants deleted, handed+    ///   back rather than deleted here so the caller can detach a whole chunk of+    ///   them from their Site in one rewrite of that inverse (T-2093, see+    ///   `delete(entries:context:)`). Only a plan that returns `true` appends to+    ///   it, so an aborted plan still leaves its rows untouched.     private static func stage(         _ plan: DuplicateDeletionPlan,         rows: inout DeletionRows,         canonicalWorkIDs: [UUID: UUID],         types: WorkTypeDirectory,         distinctPairs: [WorkDistinctPair],+        doomedEntries: inout [Entry],         context: ModelContext     ) throws -> Bool {         switch plan.key.recordType {@@ -973,7 +1029,7 @@ enum DuplicateReconciler {                 in: touched.compactMap(\.work),                 timestamp: touched.map(\.modifiedAt).max() ?? .distantPast)             for id in plan.loserIDs {-                for row in rows.entries[id] ?? [] { context.delete(row) }+                doomedEntries += rows.entries[id] ?? []             }             return true         case .work:
Packages/AsterismCore/Tests/AsterismCoreTests/DuplicateReconcilerTests.swift Modified +97 / -0
diff --git a/Packages/AsterismCore/Tests/AsterismCoreTests/DuplicateReconcilerTests.swift b/Packages/AsterismCore/Tests/AsterismCoreTests/DuplicateReconcilerTests.swiftindex 92051c2..9ea11a8 100644--- a/Packages/AsterismCore/Tests/AsterismCoreTests/DuplicateReconcilerTests.swift+++ b/Packages/AsterismCore/Tests/AsterismCoreTests/DuplicateReconcilerTests.swift@@ -642,6 +642,94 @@ struct DuplicateReconcilerTests {         #expect(try store.entryFacts().count == 1)     } +    // MARK: - T-2093: the deletion phase detaches a chunk from its Site at once++    /// The Site's side of the collapse, which nothing asserted while the+    /// deletion phase relied on `context.delete` alone to maintain it.+    ///+    /// `DuplicateReconciler.delete(entries:context:)` now removes a whole+    /// chunk's doomed rows from `Site.entries` in one rewrite of that array,+    /// because SwiftData's per-row maintenance of it is linear in the array and+    /// was 88% of the Req 10.1 settling pass. Matching by object identity is+    /// what keeps that rewrite honest — a duplicate group's rows *share* an+    /// `id`, so removing by UUID would take a survivor's other rows out of the+    /// Site along with the losers'.+    @Test("A collapse detaches its own rows from the Site and no others")+    func aCollapseDetachesOnlyItsOwnRows() throws {+        let store = try DuplicateStore()+        let site = store.addSite()+        // Three collapsing sets and two bystanders, all on one Site.+        for index in 0..<3 {+            store.addEntry(+                id: DuplicateStore.rankedID(index * 2), key: "chapter-\(index)",+                capturedAt: TimeInterval(index), site: site)+            store.addEntry(+                id: DuplicateStore.rankedID(index * 2 + 1), key: "chapter-\(index)",+                capturedAt: TimeInterval(index) + 50, site: site)+        }+        let bystanderIDs = [DuplicateStore.rankedID(90), DuplicateStore.rankedID(91)]+        for (offset, id) in bystanderIDs.enumerated() {+            store.addEntry(+                id: id, key: "solo-\(offset)", capturedAt: TimeInterval(900 + offset), site: site)+        }+        try store.commit()++        _ = try store.reconcile()+        let settled = try store.reconcile()+        #expect(settled.collapsedMembers == 3)++        try store.read { context in+            let entries = try context.fetch(FetchDescriptor<Entry>())+            #expect(entries.count == 5, "three losers went, the survivors and bystanders stayed")+            #expect(+                entries.allSatisfy { $0.site != nil },+                "a surviving row must still be on its Site")+            let hosted = try #require(context.fetch(FetchDescriptor<Site>()).first?.entries)+            #expect(+                Set(hosted.map(\.id)) == Set(entries.map(\.id)),+                "Site.entries must hold exactly the rows that survived")+            #expect(Set(bystanderIDs).isSubset(of: Set(hosted.map(\.id))))+        }+    }++    /// The other half of the same rule: a plan the commit-time re-verification+    /// aborts contributes nothing to the chunk's detach, so its rows are left+    /// exactly as they were found — on the Site, undeleted — while the healthy+    /// set beside it in the same chunk still collapses.+    @Test("An aborted plan leaves its rows attached while the chunk collapses around it")+    func anAbortedPlanLeavesItsRowsAttached() throws {+        let store = try DuplicateStore()+        let site = store.addSite()+        let changedID = DuplicateStore.rankedID(2)+        store.addEntry(+            id: DuplicateStore.rankedID(1), key: "chapter-1", capturedAt: 0, site: site)+        store.addEntry(id: changedID, key: "chapter-1", capturedAt: 10, site: site)+        store.addEntry(+            id: DuplicateStore.rankedID(3), key: "chapter-2", capturedAt: 20, site: site)+        store.addEntry(+            id: DuplicateStore.rankedID(4), key: "chapter-2", capturedAt: 30, site: site)+        try store.commit()+        _ = try store.reconcile()++        // One set changes between the derivation and the deleting transaction.+        let outcome = try store.reconcile { context in+            let descriptor = FetchDescriptor<Entry>(predicate: #Predicate { $0.id == changedID })+            for row in try context.fetch(descriptor) {+                row.lastSharedAt = DuplicateStore.epoch.addingTimeInterval(9_000)+            }+        }++        #expect(outcome.collapsedMembers == 1, "the untouched set still collapsed")+        try store.read { context in+            let entries = try context.fetch(FetchDescriptor<Entry>())+            #expect(entries.count == 3)+            #expect(entries.contains { $0.id == changedID }, "the aborted set kept both rows")+            #expect(entries.allSatisfy { $0.site != nil })+            let hosted = try #require(context.fetch(FetchDescriptor<Site>()).first?.entries)+            #expect(Set(hosted.map(\.id)) == Set(entries.map(\.id)))+        }+    }+     // MARK: - Req 2.9: the deleting commit re-verifies      @Test("An arrival between derivation and deletion aborts the deleting commit")@@ -790,6 +878,15 @@ struct DuplicateReconcilerTests {         #expect(settled.settlingSetKeys.isEmpty, "no set was left behind by the failed chunk")         #expect(strategy.hasFired, "the chunk save must actually have failed")         #expect(try store.entryFacts().count == 3)+        // T-2093: the chunk's detach from `Site.entries` rolled back with the+        // rest of its work, and the replay's per-plan detach put the store back+        // in a consistent state rather than leaving a survivor off its Site.+        try store.read { context in+            let entries = try context.fetch(FetchDescriptor<Entry>())+            #expect(entries.allSatisfy { $0.site != nil })+            let hosted = try #require(context.fetch(FetchDescriptor<Site>()).first?.entries)+            #expect(Set(hosted.map(\.id)) == Set(entries.map(\.id)))+        }     }      // MARK: - Req 2.5: every commit boundary is a library the app can open
Packages/AsterismCore/Tests/AsterismCoreTests/M4DuplicateScalePerformanceTests.swift Modified +24 / -65
diff --git a/Packages/AsterismCore/Tests/AsterismCoreTests/M4DuplicateScalePerformanceTests.swift b/Packages/AsterismCore/Tests/AsterismCoreTests/M4DuplicateScalePerformanceTests.swiftindex edc9df4..ed95b31 100644--- a/Packages/AsterismCore/Tests/AsterismCoreTests/M4DuplicateScalePerformanceTests.swift+++ b/Packages/AsterismCore/Tests/AsterismCoreTests/M4DuplicateScalePerformanceTests.swift@@ -33,27 +33,11 @@ import Testing     "M4 duplicate-reconciliation scale budgets", .serialized,     .enabled(if: ProcessInfo.processInfo.environment["ASTERISM_RUN_PHYSICAL_PERFORMANCE"] == "1")) struct M4DuplicateScalePerformanceTests {-    /// Req 10.1's own budget. **Breached, and asserted inside `withKnownIssue`-    /// for it** — see `requirement101KnownIssue`.+    /// Req 10.1's own budget. **Met, and asserted plainly**, since T-2093 stopped+    /// the deletion phase paying `Site.entries` inverse maintenance once per+    /// deleted row: 7.26-7.48 s over three milestones' worth of runs, and+    /// 1.61 s once a chunk's Entry rows are detached from their Site together.     private let settlingPassBudget = Duration.seconds(2)-    /// The regression floor under the breach, asserted *outside* the known-issue-    /// block, in the shape `M4ToleratedScalePerformanceTests` established for-    /// Req 5.5. `withKnownIssue` swallows 9.1 s and 91 s alike, which is exactly-    /// the "assert nothing and record the number" property the house discipline-    /// rejects; this refuses a run that has drifted far enough to be a *new*-    /// problem rather than the recorded one. It sits above the measured-    /// **7.264–7.365 s** (medians over three host release runs after-    /// Decision 29's chunking; it was 8.861–9.080 s before) with room for noise-    /// and well under a doubling. Moving it up to make a run pass would give the-    /// test back the property it exists to remove.-    ///-    /// `multi-site-works` moved it to **9.215–9.433 s** (one run, 2026-08-26),-    /// because two whole-library passes ran in front of every reconcile and this-    /// loop runs two reconciles per sample. `drop-superseded-columns` deleted-    /// both and it is back at **7.347–7.411 s**, at the band this ceiling was-    /// drawn over. The ceiling stays where it is: it is the number a *new*-    /// problem has to clear, and it was never moved to accommodate the rise.-    private let settlingPassCeiling = Duration.seconds(11)     /// **The observation pass is back inside its budget and asserted plainly.**     ///     /// It measured 0.93–0.95 s against 2 s before `multi-site-works`, 2.65–2.79 s@@ -84,47 +68,25 @@ struct M4DuplicateScalePerformanceTests {     /// of magnitude.     private let readPathCeiling = Duration.seconds(3) -    /// **Req 10.1 does not hold, and this suite records that rather than hiding-    /// it or deleting the budget.**-    ///-    /// Measured on an M1 Max in release over the 5,000-Entry fixture seeded with-    /// 250 Entry sets, 50 Work sets and 10 rule groups: the settling pass —-    /// Req 10.1's second pass, the one that performs the deletions — measures-    /// **7.264–7.365 s against a 2 s budget** (medians over three runs), with a-    /// min-to-max spread of ≤ 1.04×. It is a measurement, not a hiccup. The-    /// *observation* pass beside it, which does all the writing, is 0.93–0.95 s-    /// and inside budget.-    ///-    /// **`multi-site-works` and `drop-superseded-columns` moved it and moved it-    /// back**, to 9.215–9.433 s and then to 7.347–7.411 s, without ever touching the-    /// budget or the ceiling. The breach itself is unchanged and still-    /// unattributed (Q7 of `drop-superseded-columns`, T-2093).-    ///-    /// **The first cost model was wrong, and the measurement that replaced it-    /// says so.** Task 22 attributed the breach to `commitDeletions` running one-    /// `saveStrategy.save(context)` per set — 300 collapses, 300 saves, "roughly-    /// 30 ms each". Decision 29 chunked those saves, and the pass came down from-    /// 8.861–9.080 s to 7.264–7.365 s: the transaction count was worth ~1.6 s,-    /// not ~7 s. Whatever the remaining ~7 s is, it is **not** the number of-    /// transactions, and the next attempt should profile rather than reason from-    /// the shape of the code — which is what the first attempt did.-    ///-    /// Req 10.1's 2 s over 300 sets allows 6.7 ms per set; the measurement is-    /// ~24 ms. Closing that is a design decision — raise the budget, bound the-    /// sets per pass, or find the real cost — and a breach recorded inside a-    /// measurement task does not authorise taking it. Routed the way-    /// `cloudkit-mirroring` routed its two (Q55): recorded, loud, and left for-    /// the design owner. See Decision 27.-    ///-    /// `isIntermittent` is deliberately **not** set: this is 3.7× its budget,-    /// not 11% over it, and no quiet run is going to dip under 2 s.-    private static let requirement101KnownIssue: Comment = """-        Req 10.1 (2 s) is exceeded on the host at 7.26-7.37 s, down from \-        8.86-9.08 s once Decision 29 chunked the deletion saves. The remaining \-        cost is not the transaction count and is unattributed. Host-only \-        measurement. See the comment above this test, Decision 27, and \-        implementation.md.-        """+    // `requirement101KnownIssue` and `settlingPassCeiling` stood here. The+    // budget held from `duplicate-reconciliation` task 22 to T-2093 at 3.7x its+    // 2 s bound, recorded as a known issue with an 11 s regression ceiling+    // asserted outside it; both are gone with the breach, in the shape Q33 of+    // `drop-superseded-columns` used when the observation pass came back inside+    // budget — a 2 s bound asserted plainly is a tighter statement than the+    // floor that stood under it, so keeping the floor would only be a second,+    // weaker assertion about the same path.+    //+    // What the breach was, recorded because two cost models were proposed for it+    // and the first one was wrong by 4x: 6.4 s of the 7.3 s pass was the single+    // `context.save()` in `commitDeletions`, and the staging around it — the+    // fingerprint re-verification, the character re-pointing, the membership+    // collapse — cost 0.09 s all together. Task 22 attributed the pass to the+    // *number of transactions*; Decision 29 chunked them to one and the pass+    // fell by 1.6 s of 7. What was actually in that save was `Site.entries`+    // inverse maintenance, 8.0 ms per deleted row over a 5,000-row Site,+    // measured at `DuplicateReconciler.delete(entries:context:)`, which is the+    // repair. See Decision 32 and `specs/bugfixes/settling-pass-budget/`.      /// Ten samples, not twenty. Every sample of the settling pass needs its own     /// generation of duplicate rows — 1,350 rows deleted and re-seeded — plus a@@ -195,10 +157,7 @@ struct M4DuplicateScalePerformanceTests {         }          let measured = PerformanceDistribution(settlingSamples)-        withKnownIssue(Self.requirement101KnownIssue) {-            expectWithinBudget("duplicate-settling-pass", measured, settlingPassBudget)-        }-        expectWithinCeiling("duplicate-settling-pass", measured, settlingPassCeiling)+        expectWithinBudget("duplicate-settling-pass", measured, settlingPassBudget)          // Recorded beside it: a pass that met 2 s only by having deferred half         // its work to the pass before it would satisfy the letter of Req 10.1
Packages/AsterismCore/Tests/AsterismCoreTests/SiteInverseReachTests.swift Modified +43 / -0
diff --git a/Packages/AsterismCore/Tests/AsterismCoreTests/SiteInverseReachTests.swift b/Packages/AsterismCore/Tests/AsterismCoreTests/SiteInverseReachTests.swiftindex 9bc8a3f..cbaa0a7 100644--- a/Packages/AsterismCore/Tests/AsterismCoreTests/SiteInverseReachTests.swift+++ b/Packages/AsterismCore/Tests/AsterismCoreTests/SiteInverseReachTests.swift@@ -19,6 +19,9 @@ import Testing /// not. A receiver that mentions a site without being one (`compositeX.entries`) /// would be a false positive; none exists, and renaming out of the way is a /// cheaper answer than a real parser.+///+/// It carries exactly one exemption, keyed on the call rather than the file —+/// see `sanctioned` for what it is and the measurement behind it. @Suite("The Site inverses stay unreached") struct SiteInverseReachTests { @@ -26,6 +29,33 @@ struct SiteInverseReachTests {     /// the names may legitimately appear.     private static let declarationFiles: Set<String> = ["Models.swift"] +    /// The traversals this rule deliberately allows, as (file, snippet) pairs.+    ///+    /// **The rule is about what a traversal costs, and one write path pays that+    /// cost whether it traverses or not** (T-2093). Deleting an Entry makes+    /// SwiftData maintain `Site.entries` for that row, and the maintenance is+    /// linear in the array: 8.0 ms per deleted row over a 5,000-row Site,+    /// whether it is paid at `context.delete` time or by assigning+    /// `entry.site = nil`. The duplicate reconciler's settling pass deletes+    /// hundreds of rows in one commit, so it paid that walk hundreds of times —+    /// 6.4 s of a 7.3 s pass, against a 2 s budget — for a fan-out it was going+    /// to fault anyway. Removing the whole doomed set in **one** rewrite of the+    /// array pays it once: the same 200 rows measured 1.83 s the implicit way+    /// and 0.13 s this way.+    ///+    /// Narrow on purpose. The entry is keyed on the exact call rather than on+    /// the file, so any *other* traversal in `DuplicateReconciler.swift` — a+    /// read, above all — still fails this test. Adding to this list is a+    /// deliberate act with a measurement behind it, which is the property the+    /// guard exists to keep.+    private static let sanctioned: [(file: String, snippet: String)] = [+        (file: "DuplicateReconciler.swift", snippet: "site.entries?.removeAll")+    ]++    private static func isSanctioned(file: String, code: String) -> Bool {+        sanctioned.contains { $0.file == file && code.contains($0.snippet) }+    }+     private static var sourcesDirectory: URL {         URL(fileURLWithPath: #filePath)          // …/Tests/AsterismCoreTests/<this file>             .deletingLastPathComponent()          // …/Tests/AsterismCoreTests@@ -45,6 +75,7 @@ struct SiteInverseReachTests {             FileManager.default.enumerator(at: sources, includingPropertiesForKeys: nil))          var scannedFiles = 0+        var sanctionedHits = 0         var offences: [String] = []         for case let url as URL in enumerator where url.pathExtension == "swift" {             guard !Self.declarationFiles.contains(url.lastPathComponent) else { continue }@@ -56,12 +87,24 @@ struct SiteInverseReachTests {                     guard let chainRange = Range(match.range(at: 1), in: code) else { continue }                     let chain = String(code[chainRange])                     guard chain.lowercased().contains("site") else { continue }+                    guard !Self.isSanctioned(file: url.lastPathComponent, code: code) else {+                        sanctionedHits += 1+                        continue+                    }                     offences.append("\(url.lastPathComponent):\(number): \(code.trimmingCharacters(in: .whitespaces))")                 }             }         }          #expect(scannedFiles > 20, "the scan found almost no sources — check the path")+        // A sanctioned entry that matches nothing is a stale exemption, and a+        // stale exemption is a hole nobody is watching.+        #expect(+            sanctionedHits == Self.sanctioned.count,+            """+            every sanctioned traversal must still exist — \(sanctionedHits) of \+            \(Self.sanctioned.count) matched; drop the entry if the call is gone+            """)         #expect(offences.isEmpty, """             A Site inverse is being traversed. `Site.entries` faults every Entry \             for a hostname — roughly 125× the fan-out of `Work.entries` — which \
docs/agent-notes/swiftdata-relationships.md Modified +52 / -0
diff --git a/docs/agent-notes/swiftdata-relationships.md b/docs/agent-notes/swiftdata-relationships.mdindex c41bba2..4d1350b 100644--- a/docs/agent-notes/swiftdata-relationships.md+++ b/docs/agent-notes/swiftdata-relationships.md@@ -73,3 +73,55 @@ Two caveats worth carrying forward:   nothing and the traversal looks free; the cost only shows on a cold context   over a large table, which is precisely the production capture path and not the   usual unit test.++## Deleting a row costs a walk of every to-many inverse it sits in++**Rule: when you delete many rows that share one large to-many parent, remove+them from that parent's array in one pass first, then delete them.** (T-2093.)++SwiftData maintains a deleted row's inverse arrays for you, and the maintenance+is **linear in the array**, once per row. Measured on the M4 fixture (release,+M1 Max), deleting `Entry` rows whose `site` points at a `Site` holding 5,000+entries:++| What | Per deleted row |+|---|---|+| `context.delete(entry)` then one `save()` | 9.1–9.8 ms |+| clearing `entry.work` (a five-element inverse) | 0.20 ms |+| clearing `entry.site` (a five-thousand-element inverse) | 8.02 ms |++So the cost of deleting a row is proportional to the size of the *library*, not+to anything about the row. It does not matter where you pay it: `context.delete`+and `entry.site = nil` cost the same, the assignment just pays it earlier. It is+not a per-transaction cost either — it is the same 9 ms per row whether 50 rows+go in one save or 200 do.++The repair is one rewrite of the array instead of N removals from it:++```swift+// 200 rows: 1.83 s+for entry in doomed { context.delete(entry) }+try context.save()++// the same 200 rows: 0.13 s+let ids = Set(doomed.map(ObjectIdentifier.init))+site.entries?.removeAll { ids.contains(ObjectIdentifier($0)) }   // one walk+for entry in doomed { context.delete(entry) }                    // already detached+try context.save()+```++SwiftData nils each removed row's to-one for you, so the rows reach+`context.delete` detached. **Match by object identity, not by `id`**: in this+codebase an `Entry.id` is the *logical record's* UUID and a duplicate group's+rows share it, so an id set would detach rows you did not mean to delete.++This was 88% of the Req 10.1 settling pass — 6.4 s of a 7.3 s pass sat in the+single `save()` that deleted 300 rows — and two earlier cost models for that+breach both missed it by reasoning from the shape of the code. It is also the+counterexample to reading the first rule above as "never touch an inverse+array": the rule there is about *rolling back* through future-backed rows, and+this path deletes-then-saves in one direction.++`DuplicateReconciler.delete(entries:context:)` is the shipping instance, and+`SiteInverseReachTests` sanctions it by name — the guard that otherwise keeps+every read off `Site.entries` (Q17).
docs/agent-notes/testing.md Modified +21 / -10
diff --git a/docs/agent-notes/testing.md b/docs/agent-notes/testing.mdindex 7c26cfd..5e78b51 100644--- a/docs/agent-notes/testing.md+++ b/docs/agent-notes/testing.md@@ -262,22 +262,33 @@ for i in 1 2 3; do make test-performance-m4 RUNS=1 > /tmp/m4-run$i.log 2>&1 || t ```  **Read the exit status, not the count of `recorded a known issue` lines.**-**Eight is the steady state since `drop-superseded-columns`** — four before-`multi-site-works` (or five on a noisy run), nine after it, eight now. The eight-are Req 10.1's settling pass, Req 5.4's three capture-projection arms, Req 5.5's+**Seven is the steady state since T-2093** — four before `multi-site-works` (or+five on a noisy run), nine after it, eight after `drop-superseded-columns`,+seven now. The seven are Req 5.4's three capture-projection arms, Req 5.5's three diagnosis re-derivations, and the full-tier no-op reconcile. Every one has a regression ceiling asserted **outside** the known-issue block, so a run that-drifts further still fails. **V10 confirmed the same eight** — see+drifts further still fails. V10 confirmed the eight it inherited — see `specs/work-and-reading-status/implementation.md` §4, where the three new `Work` columns cost the capture-projection and diagnosis arms 3–6% and reached no ceiling. -The one that retired is Req 10.1's *observation* pass: V9 deleted-`V8PopulationPass` and gated `MembershipReconciler.heal` on the diagnosis, the-full-tier no-op reconcile fell 36× (1.07 s → 0.030 s), and the observation pass-followed it back inside its 2 s budget. See-`specs/drop-superseded-columns/verification-run.md`, and-`specs/multi-site-works/verification-run.md` §4 and §7 for the previous shape.+Two have retired, both of them Req 10.1's, and both by attribution rather than+by moving a bound:++- the **observation** pass, at V9: `drop-superseded-columns` deleted+  `V8PopulationPass` and gated `MembershipReconciler.heal` on the diagnosis, the+  full-tier no-op reconcile fell 36× (1.07 s → 0.030 s), and the observation+  pass followed it back inside its 2 s budget. See+  `specs/drop-superseded-columns/verification-run.md`, and+  `specs/multi-site-works/verification-run.md` §4 and §7 for the previous shape.+- the **settling** pass, at T-2093: 88% of it was SwiftData maintaining+  `Site.entries` while the deletion phase deleted 300 rows — 8.0 ms per deleted+  row over a 5,000-row Site, so a cost proportional to the size of the library.+  Detaching a chunk's doomed rows from their Site in one rewrite of that array+  brought the pass from 7.3 s to ~1.6 s and the 11 s floor under the known issue+  went with it. See `specs/bugfixes/settling-pass-budget/` and Decision 32 of+  `specs/duplicate-reconciliation/decision_log.md`; the general rule is in+  `docs/agent-notes/swiftdata-relationships.md`. Zero would mean the filter or the opt-in gate stopped the suites from running at all, which is the failure mode the Makefile's no-xcbeautify comment exists for.
specs/bugfixes/settling-pass-budget/report.md Added +256 / -0
diff --git a/specs/bugfixes/settling-pass-budget/report.md b/specs/bugfixes/settling-pass-budget/report.mdnew file mode 100644index 0000000..57506ec--- /dev/null+++ b/specs/bugfixes/settling-pass-budget/report.md@@ -0,0 +1,256 @@+# Bugfix Report: Req 10.1 settling pass, 7.3 s against a 2 s budget++**Date:** 2026-09-05+**Status:** Fixed+**Ticket:** T-2093++## Description of the Issue++`make test-performance-m4` recorded the duplicate reconciler's **settling pass**+— Req 10.1's second pass, the one that performs the deletions — at+**7.26–7.48 s against a 2 s budget**, roughly 3.7x. It had been recorded as a+known issue since `duplicate-reconciliation` task 22 (Decision 27), survived one+attempt at a fix (Decision 29, which bought 1.6 s of the ~7), and about **7 s of+it was unattributed**.++**Reproduction steps:**++1. `make test-performance-m4` (host only, release, ~21 minutes).+2. Read the `ASTERISM-PERF duplicate-settling-pass` line.+3. It reports a median around 7.3 s, and the suite records a known issue against+   the 2 s budget beside it.++The fixture is the 5,000-Entry composed fixture seeded with 250 silently+resolvable Entry sets, 50 Work sets and 10 rule identity groups — 300+collapsible sets, 300 rows deleted.++**Impact:** low in absolute terms, and worth stating precisely because it shaped+the fix. This is a background pass; nothing is lost, nothing is at risk, and an+interrupted pass resumes safely. The M4b runbook saw **zero** duplicates across+8,144 synced entries, so the fixture is synthetic and deliberately pessimistic.+What made it worth fixing is the *shape* rather than the number: the cost per+deleted row is proportional to the number of Entries on the hostname, so the+pass got more expensive as a real library grew.++## Investigation Summary++The ticket's own instruction was to **profile before changing anything**, on the+grounds that two cost models had already been proposed for this pass and the+first was wrong by 4x. That is what was done.++- **Symptoms examined:** the settling pass at 7.3 s beside an *observation* pass+  over the same fixture at 1.0 s. The two passes differ in one phase —+  `commitCollapses` — so the delta was already localised to the deletion half+  before anything was instrumented.+- **Instrumentation:** a temporary phase accumulator (`ReconcilePhaseProfile`)+  with a span around every phase of `reconcileAfterSync` and of+  `DuplicateReconciler.commitDeletions`, driven by a temporary suite over the+  real fixture in release. Both were deleted before the commit;+  `measurements.md` records what they measured and how to rebuild them.+- **Hypotheses tested:** the two Decision 27 recorded, plus the transaction+  count it had already falsified.+  - *Transaction count* — already dead, and the profile shows why:+    `bulkOperationBatchSize` is 500 and the pass deletes 300 rows, so Decision+    29 had already reduced this to **one** save. That one save cost 6.42 s.+  - *`repointEntries` over the 50 collapsing Works* — **ruled out**: 0.2 ms for+    all fifty.+  - *`Site.entries` inverse maintenance* — **confirmed**, then isolated in a+    micro-benchmark that varied the batch size, the relationship cleared, and+    whether the clearing happened before or during the delete.++## Discovered Root Cause++**88% of the settling pass was SwiftData maintaining `Site.entries` while the+deletion phase deleted 300 rows.**++| Phase of a 7.28 s pass | Cost | Share |+|---|---|---|+| `commitDeletions`' single `context.save()` | 6.42 s | **88%** |+| `DuplicateScan.run` | 0.65 s | 9% |+| staging all 300 plans | 0.085 s | 1.1% |++Inside that save, per deleted `Entry` row over a `Site` holding 5,000 entries:++| What | Per row |+|---|---|+| `context.delete` + save | 9.1–9.8 ms |+| clearing `Entry.work` (five-element inverse) | 0.20 ms |+| clearing `Entry.site` (five-thousand-element inverse) | **8.02 ms** |+| deleting a `Work` whose Entries were already re-pointed | 0.41 ms |++**Defect type:** performance defect — an O(deleted rows x library size)+relationship-maintenance cost on a path that deletes rows in bulk.++**Why it occurred:** `Entry.site` is a to-one whose inverse, `Site.entries`,+holds every Entry captured on the hostname. SwiftData removes a deleted row from+that array itself, and the removal walks the array. The cost is the same+wherever it is paid — `context.delete(entry)` and `entry.site = nil` measure+alike — and it is linear in the number of deleted rows, so nothing about the+batching or the transaction count could move it. Q17 already knew this fan-out+was expensive and kept `Site.entries` internal and unread; what nobody had+noticed is that a *write* path was walking it several hundred times without ever+naming it.++**Contributing factors:** the two earlier cost models were both derived by+reading the code rather than by measuring it, and both landed on the phase that+was *visible* in the source — the loop, then the saves — rather than on the+implicit work SwiftData does inside `save()`.++## Resolution for the Issue++**Changes made:**++- `Packages/AsterismCore/Sources/AsterismCore/DuplicateReconciler.swift` —+  `commitDeletions` holds a chunk's doomed Entry rows aside while it stages,+  then hands them to a new `delete(entries:context:)`, which removes them all+  from `Site.entries` in **one rewrite of that array per Site** before deleting+  them. `stage` hands the rows back through an `inout` parameter instead of+  deleting them itself; the per-set replay path after a failed chunk does the+  same for its one plan.+- `Packages/AsterismCore/Tests/AsterismCoreTests/M4DuplicateScalePerformanceTests.swift`+  — the Req 10.1 budget is asserted **plainly**. The `withKnownIssue` wrapper+  and the 11 s regression floor outside it are deleted, in the shape Q33 of+  `drop-superseded-columns` used when the observation pass came back inside+  budget: a 2 s bound asserted plainly is a tighter statement than the floor+  under it.+- `Packages/AsterismCore/Tests/AsterismCoreTests/SiteInverseReachTests.swift` —+  a narrow sanctioned entry for that one call, keyed on the call rather than the+  file, and asserted to still match so the exemption cannot go stale.++**Approach rationale:** the pass was going to fault `Site.entries` either way —+that is what made each deletion cost 8 ms. Doing it once instead of once per row+is not new work, it is the same work paid once. Nothing about the design moved:+the same rows are deleted, the per-set fingerprint re-verification is untouched+and still per-set, Q86's per-set rollback still holds where it is observable,+and Req 2.1's write-before-delete ordering is unchanged.++**Alternatives considered:**++- **Batch delete (`ModelContext.delete(model:where:)`)** — bypasses the object+  graph, so the maintenance never happens. Rejected: both configurations mirror+  to CloudKit, and a store-level batch delete produces no change tracking for+  the mirroring to export, so the deletions would never reach another device. It+  also skips the delete rules the collapse depends on.+- **Bound the collapses per pass** (Decision 27's third option) — rejected again+  for the reason it was rejected the first time: it buys the budget by doing+  less work per pass and leaves the per-row cost where it is.+- **Clear `entry.site` before `context.delete`** — the obvious spelling of+  "detach first". Rejected by measurement: it costs the same 8.5 ms per row and+  only moves the cost out of `save()`.+- **Raise the budget** — the option Decision 27 left open. Not needed.++Full reasoning, with the rejected alternatives and the consequences, is+**Decision 32** in `specs/duplicate-reconciliation/decision_log.md`.++## Regression Test++**The budget assertion is the regression test.** Req 10.1's 2 s bound in+`M4DuplicateScalePerformanceTests.settlingPassOverSeededDuplicates` failed at+7.3 s before the fix and passes at ~1.6 s after it, and it is now asserted with+nothing wrapped around it. Run it with `make test-performance-m4` (host only,+safe, ~21 minutes).++Two correctness tests guard the risk the *fix* introduces — that a single array+rewrite could detach a row it was not asked to:++**Test file:** `Packages/AsterismCore/Tests/AsterismCoreTests/DuplicateReconcilerTests.swift`++| Test | What it verifies |+|---|---|+| `A collapse detaches its own rows from the Site and no others` | after three sets collapse beside two bystanders, `Site.entries` holds exactly the surviving rows and every survivor still has a `site` |+| `An aborted plan leaves its rows attached while the chunk collapses around it` | a plan the commit-time re-verification aborts contributes nothing to the chunk's detach — its rows stay on the Site and undeleted — while the healthy set in the same chunk still collapses |++Both would have passed before the fix (`context.delete` maintained the inverse+correctly, only slowly), which is the point: they pin that the faster path is+still the correct one. Run with+`make test-core CORE_TEST=DuplicateReconcilerTests`.++## Affected Files++| File | Change |+|------|--------|+| `Packages/AsterismCore/Sources/AsterismCore/DuplicateReconciler.swift` | deferred Entry deletion per chunk; new `delete(entries:context:)` doing one inverse rewrite per Site |+| `Packages/AsterismCore/Tests/AsterismCoreTests/M4DuplicateScalePerformanceTests.swift` | Req 10.1 asserted plainly; known issue and 11 s floor removed |+| `Packages/AsterismCore/Tests/AsterismCoreTests/DuplicateReconcilerTests.swift` | two tests for the Site's side of a collapse |+| `Packages/AsterismCore/Tests/AsterismCoreTests/SiteInverseReachTests.swift` | narrow sanctioned traversal, asserted to still exist |+| `specs/duplicate-reconciliation/decision_log.md` | Decision 32; Decision 27 marked superseded |+| `specs/duplicate-reconciliation/implementation.md` | the hypotheses paragraph marked resolved |+| `docs/agent-notes/swiftdata-relationships.md` | the general rule, with the numbers |+| `CLAUDE.md`, `docs/agent-notes/testing.md` | the known-issue count drops from eight to seven |++## Verification++**Automated:**++- [x] `make test-core` — `Test run with 2363 tests in 230 suites passed after 86.955 seconds.`+- [x] `swift build --package-path Packages/AsterismCore` — no warnings.+- [x] `make test-performance-m4 RUNS=3` — see the band below.++**Measured:** `specs/bugfixes/settling-pass-budget/measurements.md` carries the+attribution, the micro-benchmarks and the before/after phase tables.++### The recorded band++Host: M1 Max, release, `make test-performance-m4 RUNS=3`, ten samples per run.+Medians per run; the band is the spread of the three medians. **The+host was not quiet** — another worktree of this repository was compiling for part of the run,+which is the condition `docs/agent-notes/testing.md` warns raises numbers — so+these are a pessimistic reading rather than a best case.++| Measurement | Run 1 | Run 2 | Run 3 | Band | Before | Budget |+|---|---|---|---|---|---|---|+| `duplicate-settling-pass` | 1.688 s | 1.701 s | 1.701 s | **1.688–1.701 s** | 7.264–7.365 s (task 24), 7.347–7.411 s (`drop-superseded-columns`), 7.28–7.48 s (this host, instrumented) | 2 s — **inside** |+| `duplicate-observation-pass` | 1.056 s | 1.076 s | 1.071 s | 1.056–1.076 s | 1.008–1.025 s | 2 s — inside |++The three medians span **13 ms**, which is a tighter agreement than this suite+usually manages and is the strongest part of the reading. Within-run spread was+1.21x and 1.22x in runs 1 and 3; run 2 threw one outlier on each pass (p95+3.406 s on the settling pass, 2.595 s on the observation pass, spreads 2.19x and+2.54x) against a median that moved by 13 ms. That is the contended host, and it+is the same single-sample non-reproducibility CLAUDE.md documents — a run of+unchanged code measured 0.7805 / 1.2789 / 0.7389 s under+`library-integrity-tolerance`. The medians are what the suite asserts.++All three runs reported `Test run with 32 tests in 5 suites passed after …+seconds with 7 known issues` — 1136.240 s, 1237.290 s and 1085.479 s — and the+target exited **0**. **Seven, down from eight**, and the one that left is+Req 10.1's settling pass. The remaining seven are the three Req 5.4+capture-projection arms (0.177–0.204 s, elevated by the contention, under their+250 ms ceiling), the three Req 5.5 diagnosis re-derivations (0.285–0.300 s) and+the full-tier no-op reconcile (0.0305 s). **No regression ceiling was breached+on any untouched arm.**++The instrumented quiet-host reading of the same pass, for comparison, was+1.610–1.611 s (measurements.md §4). Either way Req 10.1 holds with roughly 15–20%+of headroom on the median, which is the number the suite asserts off-`CONTROLLED`.+`CONTROLLED=1` asserts the p95 and still wants a quiet machine: a p95 of 1.92 s+was recorded on the contended run, which is under 4% headroom on that arm — far+thinner than the observation pass's ~50% — so a contended `CONTROLLED=1` run+will flake here before it flakes anywhere else. That is host contention, not a+regression; the plain assertion is kept on the Q33 precedent.++## Prevention++**Recommendations to avoid similar bugs:**++- **Profile before proposing a cost model.** Two models were proposed for this+  pass from reading the code, and both named the phase that was visible in the+  source rather than the work the framework does inside `save()`. The+  instrumentation that settled it took well under an hour to write.+- **Treat a to-many inverse as a cost centre on every write path, not only on+  reads.** `docs/agent-notes/swiftdata-relationships.md` now carries the rule+  and the numbers: deleting a row walks every to-many inverse it sits in, once+  per row, and the repair is one rewrite of the array before the deletions.+- **An architectural guard that lists no exemptions is easier to trust than one+  that lists many.** The sanctioned entry added here is keyed on the exact call+  and fails if the call disappears; keep it that way rather than widening it to+  a file.++## Related++- Decision 27 (the breach) and **Decision 32** (this fix) in+  `specs/duplicate-reconciliation/decision_log.md`+- Decision 29 (the chunking, and the first falsified cost model)+- `specs/duplicate-reconciliation/implementation.md` — the bands before this+- `cloudkit-mirroring` Q27, which measured this superlinear shape first, and Q17,+  which is why `Site.entries` is internal
specs/bugfixes/settling-pass-budget/measurements.md Added +125 / -0
diff --git a/specs/bugfixes/settling-pass-budget/measurements.md b/specs/bugfixes/settling-pass-budget/measurements.mdnew file mode 100644index 0000000..e541d17--- /dev/null+++ b/specs/bugfixes/settling-pass-budget/measurements.md@@ -0,0 +1,125 @@+# Measurements: settling-pass-budget (T-2093)++Host: M1 Max, quiet machine, `AsterismCore` package, **release**+(`-c release -Xswiftc -DASTERISM_PERFORMANCE_TESTING`), `--no-parallel`.+Every number here is host-only and comparable to another run of the same command+on the same machine and to nothing else (Decision 10 of+`library-integrity-tolerance`).++The instrumentation these tables come from was a temporary phase accumulator+(`ReconcilePhaseProfile`) and a temporary suite (`T2093SettlingProfileTests`),+both deleted before the commit. They are described here in enough detail to+rebuild: a `span(_:_:)` wrapper writing `(name, Duration, count)` into a+dictionary, one span per phase of `reconcileAfterSync` and of+`DuplicateReconciler.commitDeletions`, enabled around the timed pass only.++## 1. Where the settling pass spent its 7.3 s (before)++Fixture: the 5,000-Entry composed fixture in `.duplicateSets` — 250 silently+resolvable Entry sets, 50 Work sets of two Works with five Entries each, and 10+rule identity groups. Four generations; the two below are representative and the+other two agree to within 3%.++| Phase | Generation 0 | Generation 3 | Share |+|---|---|---|---|+| **whole settling pass** | **7.281 s** | **7.451 s** | 100% |+| `commitCollapses` | 6.538 s | 6.598 s | 89% |+| ⤷ `commitDeletions`' one `context.save()` | **6.420 s** | **6.480 s** | **88%** |+| ⤷ `DeletionRows` fetch (500 Entry + 100 Work ids) | 0.031 s | 0.032 s | 0.4% |+| ⤷ staging all 300 plans | 0.085 s | 0.083 s | 1.1% |+| `DuplicateScan.run` | 0.645 s | 0.662 s | 9% |+| `MembershipReconciler.run` | 0.029 s | 0.118 s | 1.6% |+| `DuplicateReconciler.run` (the write phase) | 0.062 s | 0.067 s | 0.9% |+| `SiteReconciler.run`, `WorkTypeReconciler.run`, work lists | < 0.003 s | < 0.003 s | ~0% |++Staging, broken out (the phases Decision 27's two hypotheses named):++| Phase | Calls | Total |+|---|---|---|+| `CharacterCitationRepointing.repoint` | 250 | 0.045–0.054 s |+| `workFingerprint` re-verification | 50 | 0.013 s |+| `entryFingerprint` re-verification | 250 | 0.011 s |+| `collapseMemberships` | 50 | 0.012 s |+| `repointEntries` | 50 | 0.0002 s |+| `context.delete` of the Work rows | 50 | 0.0001 s |+| `context.delete` of the Entry rows | 250 | 0.0008 s |++**So the second hypothesis is dead**: the `repointEntries` fault over the 50+collapsing Works is 0.2 ms for all fifty. And the transaction count is not it+either — `bulkOperationBatchSize` is 500 and the pass deletes 300 rows, so+Decision 29's chunking had already reduced this to **one save**. One save cost+6.4 s.++## 2. What is inside that save++Same fixture family, plain 5,000-Entry composed fixture, one `ModelContext`,+timing `context.save()` around N deleted `Entry` rows:++| Arm | n | Cost | Per row |+|---|---|---|---|+| `context.delete` then one save | 50 | 0.490 s | 9.8 ms |+| `context.delete` then one save | 50 | 0.483 s | 9.7 ms |+| `context.delete` then one save | 200 | 1.827 s | 9.1 ms |+| `context.delete` + save, per row | 50 | 1.185 s | 23.7 ms |+| clear `site` **and** `work`, then delete + one save | 200 | 1.697 s (1.628 s of it in the clearing, 0.067 s in the save) | 8.5 ms |+| clear `work` only | 100 | 0.019 s | **0.20 ms** |+| clear `site` only | 100 | 0.802 s | **8.02 ms** |++Reading:++- The cost is **linear in the number of deleted rows** (9.8 / 9.7 / 9.1 ms per+  row at n = 50, 50, 200), so it is not a per-transaction cost and not+  superlinear in the batch.+- It is **relationship teardown, not the delete**: clearing the two references+  first moves the whole cost out of `save()` and into the assignments.+- It is **`Entry.site`, not `Entry.work`**: 8.02 ms against 0.20 ms, a factor of+  40. `Work.entries` holds five rows; `Site.entries` holds every Entry captured+  on the hostname — 5,000 here. The cost per deleted row is proportional to the+  size of the library.++The Work half is not the problem:++| Arm | n | Cost | Per row |+|---|---|---|---|+| delete Works whose Entries were already re-pointed | 50 | 0.021 s | 0.41 ms |++## 3. One rewrite of the inverse instead of N removals++Same store, same 200 rows, removing them from `Site.entries` in one pass and+then deleting them:++| Arm | n | Rewrite | Delete + save | Total |+|---|---|---|---|---|+| `site.entries?.removeAll { doomed.contains(…) }` | 200 | 0.038 s | 0.097 s | **0.135 s** |+| `site.entries = site.entries?.filter { … }` | 200 | 0.036 s | 0.096 s | **0.132 s** |+| (baseline) per-row `context.delete` + one save | 200 | — | 1.827 s | 1.827 s |++**13.5×.** Both spellings cost the same; `removeAll(where:)` is the one that+shipped. SwiftData nils each removed row's `site` for us — the arms asserted+`stillAttached == 0` after the rewrite — so the rows reach `context.delete`+already detached.++## 4. The settling pass afterwards (same instrumentation, same host)++| Phase | Generation 2 | Generation 3 |+|---|---|---|+| **whole settling pass** | **1.610 s** | **1.611 s** |+| `DuplicateScan.run` | 0.657 s | 0.657 s |+| `commitDeletions`' one `context.save()` | 0.408 s | 0.399 s |+| detaching the chunk from its Site | 0.236 s | 0.243 s |+| `MembershipReconciler.run` | 0.121 s | 0.127 s |+| staging all 300 plans | 0.086 s | 0.085 s |+| `DuplicateReconciler.run` | 0.060 s | 0.059 s |+| `DeletionRows` fetch | 0.032 s | 0.032 s |++**7.28–7.48 s → 1.61 s**, inside Req 10.1's 2 s budget with room. The observation+pass beside it is unchanged at 1.02 s: nothing on the write path was touched.++The largest remaining phase is `DuplicateScan.run` at 0.66 s — the detection+walk, which the observation pass pays too and which no requirement here bounds+separately.++## 5. The recorded band++See §1 of `report.md` for the three-run `make test-performance-m4 RUNS=3` band+that replaces the numbers above as the suite's own record.
specs/duplicate-reconciliation/decision_log.md Modified +147 / -5
diff --git a/specs/duplicate-reconciliation/decision_log.md b/specs/duplicate-reconciliation/decision_log.mdindex 08ce2ea..35b13c7 100644--- a/specs/duplicate-reconciliation/decision_log.md+++ b/specs/duplicate-reconciliation/decision_log.md@@ -2013,11 +2013,16 @@ measurement tables in `specs/duplicate-reconciliation/implementation.md`. ## Decision 27: Req 10.1's 2 s budget is breached and accepted as a known issue  **Date**: 2026-08-02 (band and cost model corrected 2026-08-03)-**Status**: accepted — recorded breach, tracked as **T-2093** (medium). **Band-improved by Decision 29 and the breach stands**: 7.264–7.365 s against 2 s,-down from 8.861–9.080 s. T-2093 carries the negative result — the ~7 s that-remains is not the transaction count — and the two hypotheses that are not-evidence.+**Status**: **superseded by Decision 32** (2026-09-05). T-2093 profiled the pass:+88% of it was `Site.entries` inverse maintenance inside the deletion phase's+single `save`, at 8.0 ms per deleted row — the first of the two hypotheses this+entry recorded, while the second (`repointEntries` over the 50 collapsing Works)+measured 0.2 ms for all fifty. Detaching a chunk from its Site in one rewrite of+that array brought the pass from 7.28–7.48 s to 1.61 s, so the budget is met,+asserted plainly, and this known issue is gone.++Kept below as written — including the amendment that follows — because how the+first two cost models were wrong is the useful part of the record.  > **Amended after task 24's re-measurement.** Two things below are now known to > be wrong and are corrected here rather than rewritten away, because the way@@ -2530,3 +2535,140 @@ non-deterministic representative — which Req 2.4 forbids outright. (internal rather than private), and `V4LibraryValidator.validate(site:)`.  ---++## Decision 32: The settling pass detaches a chunk from its Site in one rewrite of the inverse++**Date**: 2026-09-05+**Status**: accepted — closes **T-2093**, and supersedes Decision 27's known+issue. The Req 10.1 budget is met and asserted plainly again.++### Context++Decision 27 recorded Req 10.1's 2 s budget as breached at 7.26–7.48 s and left+the cause unattributed. It also recorded that the *first* cost model was wrong:+task 22 attributed the whole ~9 s to `commitDeletions` running one save per set,+Decision 29 chunked those saves, and the pass fell by 1.6 s of 7 rather than by+7. T-2093 carried the negative result — whatever the remaining ~7 s was, it was+not the transaction count — and two hypotheses that were explicitly not+findings: `Site.entries` inverse maintenance, and the `repointEntries` fault+over the 50 collapsing Works.++The ticket's own instruction was to profile before changing anything. A+temporary phase accumulator around `reconcileAfterSync` and+`DuplicateReconciler.commitDeletions` says:++| Phase | Cost | Share of a 7.28 s pass |+|---|---|---|+| `commitDeletions`' **one** `context.save()` | 6.42 s | 88% |+| `DuplicateScan.run` | 0.65 s | 9% |+| staging all 300 plans (verify, re-point, collapse memberships) | 0.085 s | 1.1% |+| `repointEntries` over the 50 collapsing Works | 0.0002 s | 0.003% |++One save, because `bulkOperationBatchSize` is 500 and the pass deletes 300 rows+— Decision 29 had already reduced this to a single transaction. So the second+hypothesis is dead and the first one is the whole pass.++Inside that save, over the plain 5,000-Entry fixture:++| What | Per deleted Entry row |+|---|---|+| `context.delete` + save | 9.1–9.8 ms |+| clearing `Entry.work` (a five-element inverse) | 0.20 ms |+| clearing `Entry.site` (a five-thousand-element inverse) | **8.02 ms** |+| deleting a Work whose Entries were already re-pointed | 0.41 ms |++The cost is linear in the number of deleted rows and independent of how the+removal is spelled — `context.delete` and `entry.site = nil` cost the same, the+second one just pays it earlier. It is SwiftData maintaining `Site.entries`,+which holds every Entry captured on the hostname. That makes the cost of+deleting a row **proportional to the size of the library**, which is exactly the+superlinear shape `cloudkit-mirroring` Q27 measured and the reason Q17 keeps the+relationship internal and unread.++### Decision++`DuplicateReconciler.commitDeletions` holds a chunk's doomed Entry rows aside+while it stages, then removes them all from `Site.entries` in **one rewrite of+that array per Site** before deleting them —+`DuplicateReconciler.delete(entries:context:)`. Rows are matched by object+identity, never by `id`. `SiteInverseReachTests` gains a narrow sanctioned entry+for that one call, keyed on the call rather than on the file.++Decision 27's known issue and the 11 s regression floor under it are deleted;+the 2 s budget is asserted plainly, in the shape Q33 of+`drop-superseded-columns` used when the observation pass came back inside+budget.++### Rationale++The pass was going to fault `Site.entries` either way — that is what made each+deletion cost 8 ms. Doing it once instead of once per row is not a new cost, it+is the same cost paid once: 200 rows measured 1.83 s deleted one at a time and+0.13 s this way, and the whole settling pass went from 7.28–7.48 s to 1.61 s.++Nothing about the design moved. The rows deleted are the same rows, the per-set+fingerprint re-verification is untouched and still per-set, Q86's per-set+rollback still holds where it is observable (a failing chunk rolls back and+replays its sets one at a time, each with its own detach), and Req 2.1's+write-before-delete ordering is unchanged. Every Work plan of a chunk is staged+— and has therefore already re-pointed everything it was going to re-point —+before any Entry row of that chunk is deleted, so the plan-order rule+`commitDeletions` records is preserved rather than newly relied upon.++Matching by object identity rather than `id` is load-bearing: an Entry's `id` is+the *logical record's* UUID and a duplicate group's rows share it, so an id-set+rewrite would take a survivor's other rows out of the Site along with the+losers'.++### Alternatives Considered++- **Batch delete (`ModelContext.delete(model:where:)`)**: bypasses the object+  graph entirely, so the inverse maintenance never happens - Rejected: both+  configurations mirror to CloudKit, and a store-level batch delete produces no+  change tracking for the mirroring to export. The rows would come back from+  another device, or never leave it. It also skips the delete rules the collapse+  depends on.+- **Bound the collapses per pass** (Decision 27's third option): buy the budget+  by doing less work per pass - Rejected again, for the reason it was rejected+  the first time: a library with 300 sets then takes many more passes to settle,+  and the per-row cost is untouched.+- **Clear `entry.site` before `context.delete`**: the obvious spelling of+  "detach first" - Rejected by measurement: it costs the same 8.5 ms per row,+  it only moves the cost out of `save()` and into the assignment.+- **Drop the `Site.entries` inverse**: the relationship is internal, never+  traversed, and exists only because CloudKit requires every relationship to+  have one - Rejected: CloudKit requires it, and a schema change is not+  available to a bugfix.+- **Raise the budget**: the option Decision 27 explicitly left to the design+  owner - Not needed. The budget holds.++### Consequences++**Positive:**+- Req 10.1 holds: 7.28–7.48 s → 1.61 s against a 2 s budget, and the suite's+  known-issue count drops from eight to seven.+- The cost of a collapse stops scaling with the size of the library. The+  previous shape charged O(deleted rows × Entries on the hostname).+- The attribution is now measured rather than modelled, after two cost models+  that were not.++**Negative:**+- One sanctioned traversal of `Site.entries` now exists, in a rule whose whole+  value was that it had none. It is keyed on the exact call and the guard fails+  if the call disappears, so the exemption cannot go stale unnoticed — but it is+  a hole, and a future read added to that file has to be caught by review of the+  list rather than by the list itself.+- The chunk's Entry deletions are deferred to the end of its staging, so the+  ordering rule between Work plans and Entry plans is now load-bearing at one+  more point than it was. It is recorded at `commitDeletions` and it still holds+  for the same reason: `run` appends the Work phase's deletions first.++### Impact++`DuplicateReconciler.commitDeletions`, `.stage` and the new+`.delete(entries:context:)`; the Req 10.1 assertions in+`M4DuplicateScalePerformanceTests`; the sanctioned list in+`SiteInverseReachTests`. Every other bulk deletion path in the repository is+reader-scoped and bounded by one Work or one set, so none of them was changed.++---
specs/duplicate-reconciliation/implementation.md Modified +14 / -5
diff --git a/specs/duplicate-reconciliation/implementation.md b/specs/duplicate-reconciliation/implementation.mdindex 4aae6aa..8e557a6 100644--- a/specs/duplicate-reconciliation/implementation.md+++ b/specs/duplicate-reconciliation/implementation.md@@ -377,11 +377,20 @@ were considered and neither is evidence: inverse-array maintenance on findings, and carried into **T-2093** so the next attempt starts from the negative result rather than re-deriving it. -Decision 27 therefore stands, with its band updated and its cost model-corrected. The 2 s budget is unchanged and still asserted inside-`withKnownIssue`; the regression floor outside it comes down from 14 s to 11 s,-which preserves the ~1.5× margin over the measured band that the old constant-had.+As of this task (2026-08-03), Decision 27 therefore stood, with its band+updated and its cost model corrected: the 2 s budget was unchanged and still+asserted inside `withKnownIssue`, and the regression floor outside it came down+from 14 s to 11 s, which preserved the ~1.5× margin over the measured band that+the old constant had. That state lasted until T-2093, below.++> **Resolved by T-2093 (2026-09-05).** The first hypothesis was right and the+> second was not. 88% of the pass was `Site.entries` inverse maintenance inside+> the deletion phase's single `save` — 8.0 ms per deleted row, proportional to+> the number of Entries on the hostname — while `repointEntries` over all 50+> collapsing Works cost 0.2 ms. Detaching a chunk's doomed rows from their Site+> in **one** rewrite of that array brought the settling pass to **1.61 s**,+> inside the 2 s budget, so the known issue and the 11 s floor under it are both+> gone. See Decision 32 and `specs/bugfixes/settling-pass-budget/`.  ### Decision 28's regression is fixed 
CLAUDE.md Modified +1 / -1
diff --git a/CLAUDE.md b/CLAUDE.mdindex 69824ea..fd1647d 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -58,7 +58,7 @@ invocations where a target exists. - `make test-quick` — unit-test bundle only (simulator), preceded by `build-mac`: a macOS compile failure fails it (Req 9.1). The Mac build is never installed or launched. `SKIP_MAC=1` drops that dependency loudly and owes a clean `make build-mac` before the push. - `make test` / `make test-ui` — full suites (simulator, iPhone); they skip the iPad-only suites by name - `make test-ui-ipad` — the wide-layout and wide-layout-accessibility suites on `IPAD_SIMULATOR` (simulator, safe)-- `make test-performance-m4` — M4 Core budgets, host only, no device, safe to run. **~21 minutes** (1,093 s of test time measured 2026-08-28, 1,120 s over 28 tests on 2026-08-30 after `character-ranking` added its own, and **1,120.6 s over 32 tests on 2026-09-05** after T-1910 added four Work-bearing preview arms — all quiet-host runs, all `EXIT=0` with the same eight known issues, so the 21 minutes still holds and the four added arms cost no measurable wall time. A *loaded* run of those same 32 tests took 1,380 s and breached a regression ceiling on an untouched arm; that is host contention, not a band. Add a ~190 s release build to any of them): most of it is the worst-case single-hostname consolidation in `M4ScalePerformanceTests` (5 samples, each paying its own ~40 s divert before a ~40 s measurement) and the Req 10.1 settling pass (10 samples, each re-seeding 1,350 duplicate rows plus an untimed observation pass). The V4→V5 migration measurement is **gone** — `retire-migration-chain` deleted the pass it timed along with the suite. **The target exits 0 on a quiet host**, with the accepted breaches reported as `withKnownIssue` known issues rather than failures — **eight** since `drop-superseded-columns` (four before `multi-site-works`, nine after it). Four are long-standing: Req 10.1's settling pass and Req 5.5's three diagnosis re-derivations. Three are Req 5.4's capture-projection arms (0.093–0.102 s pre-V8 → 0.160–0.170 s at V9 → **0.169–0.176 s at V10**, the one on a path the reader waits on; the three new `Work` columns and the wider `orderComponents` cost them 3–6%, still well inside a 250 ms ceiling). The eighth is the **full**-tier no-op reconcile, and V9 recovered most of it: 1.07 s → **0.0296–0.0302 s** once `V8PopulationPass` was deleted with the columns and `MembershipReconciler.heal` was gated on the diagnosis, which is still 3.0× a 10 ms ceiling drawn before the library had a membership table; V10 left it there (0.0301 s). Req 10.1's *observation* pass **retired** with that fall (2.69 s → 1.01 s, back inside its 2 s budget, and 1.02 s at V10). Every one has a regression ceiling asserted *outside* its known-issue block, so a run that drifts further still fails; `RUNS=3` completes all three runs. See `specs/work-and-reading-status/verification-run.md` §4 for the current numbers, `specs/drop-superseded-columns/verification-run.md` and `specs/multi-site-works/verification-run.md` §4 and §7 for the previous ones, and `docs/agent-notes/testing.md` for recording a band.+- `make test-performance-m4` — M4 Core budgets, host only, no device, safe to run. **~21 minutes** (1,093 s of test time measured 2026-08-28, 1,120 s over 28 tests on 2026-08-30 after `character-ranking` added its own, and **1,120.6 s over 32 tests on 2026-09-05** after T-1910 added four Work-bearing preview arms — all quiet-host runs, all `EXIT=0` with the same eight known issues, so the 21 minutes still holds and the four added arms cost no measurable wall time. T-2093 took the settling pass from ~7.3 s to ~1.7 s per sample; three contended runs on 2026-09-05 measured 1,136 s, 1,237 s and 1,085 s over the same 32 tests, all `EXIT=0` with **seven** known issues. A *loaded* run of those same 32 tests took 1,380 s and breached a regression ceiling on an untouched arm; that is host contention, not a band. Add a ~190 s release build to any of them): most of it is the worst-case single-hostname consolidation in `M4ScalePerformanceTests` (5 samples, each paying its own ~40 s divert before a ~40 s measurement) and the Req 10.1 settling pass (10 samples, each re-seeding 1,350 duplicate rows plus an untimed observation pass). The V4→V5 migration measurement is **gone** — `retire-migration-chain` deleted the pass it timed along with the suite. **The target exits 0 on a quiet host**, with the accepted breaches reported as `withKnownIssue` known issues rather than failures — **seven** since T-2093 (four before `multi-site-works`, nine after it, eight after `drop-superseded-columns`). Three are long-standing: Req 5.5's three diagnosis re-derivations. Three are Req 5.4's capture-projection arms (0.093–0.102 s pre-V8 → 0.160–0.170 s at V9 → **0.169–0.176 s at V10**, the one on a path the reader waits on; the three new `Work` columns and the wider `orderComponents` cost them 3–6%, still well inside a 250 ms ceiling). The seventh is the **full**-tier no-op reconcile, and V9 recovered most of it: 1.07 s → **0.0296–0.0302 s** once `V8PopulationPass` was deleted with the columns and `MembershipReconciler.heal` was gated on the diagnosis, which is still 3.0× a 10 ms ceiling drawn before the library had a membership table; V10 left it there (0.0301 s). Req 10.1's *observation* pass **retired** with that fall (2.69 s → 1.01 s, back inside its 2 s budget, and 1.02 s at V10), and its **settling** pass retired at T-2093: 88% of that pass was SwiftData maintaining `Site.entries` while the deletion phase deleted 300 rows — 8.0 ms per deleted row over a 5,000-row Site — and detaching a chunk's doomed rows from their Site in one rewrite of that array took it from 7.3 s to ~1.6 s, inside its 2 s budget, with the 11 s floor under the known issue gone too (`specs/bugfixes/settling-pass-budget/`, Decision 32). Every one has a regression ceiling asserted *outside* its known-issue block, so a run that drifts further still fails; `RUNS=3` completes all three runs. See `specs/work-and-reading-status/verification-run.md` §4 for the current numbers, `specs/drop-superseded-columns/verification-run.md` and `specs/multi-site-works/verification-run.md` §4 and §7 for the previous ones, and `docs/agent-notes/testing.md` for recording a band. - `make test-performance-chunks` — host-only calibration sweep of the shared bulk chunk constant (import commits and the reconciler re-pin). No device, safe to run, but gated on `ASTERISM_RUN_CHUNK_SWEEP=1` and **~20 minutes per run**, so it is deliberately *not* part of `make test-performance-m4`. It asserts nothing — a calibration is reported, not budgeted. Re-run it when the bulk write paths change (Q53 and the task 25 section of `specs/cloudkit-mirroring/implementation.md`). - `make test-performance-m4-recent` — **physical device, see above** 

Things to double-check

Rollback over a large, partly unmaterialised Site.entries.

The rewrite-then-rollback path is exercised only on a 6-entry fixture where every row was fetched. On a real library the array holds thousands of future-backed rows. The crash mode in the agent note's first rule is specific to deleted future-backed rows, and the doomed rows here come from a predicated fetch, so it should not apply — but nothing runs it at scale.

Contended-host band as the record.

All three runs were taken while another worktree compiled. Medians agree to 13 ms so the band is trustworthy for the median assertion; the p95 arm (CONTROLLED=1, the max of 10 samples) recorded 1.92 / 3.41 / 1.95 s and will only pass on a quiet host. A quiet RUNS=1 would remove the caveat.

The perf number is the only proof the detach runs.

Both new unit tests are end-state checks that pass with a no-op removeAll. If a future SwiftData change made the array's instances differ from the fetch's, the pass would silently go back to 7 s with every unit test green.