asterism branch feature/url-locator-generalisation commits 10 files 36 touched lines +3540 / -335 findings 18 raised / 12 fixed / 6 skipped

Pre-push review: url-locator-generalisation

Ten unpushed commits implementing specs/url-locator-generalisation/: an unanchored PathAnchor side so one taught URL rule works for every story on a site, the removal of the fabricating rule accessor, an archive-reachability export policy, and a repair regression driven through the editor's own gestures. Reviewed by four parallel agents (reuse, quality, efficiency, spec); every finding fixed or logged.

At a glance

  • Phase 1 (cbb84e5): URLRulePattern.definition throws instead of fabricating .work(.query("identity")); explicit failable setDefinition; unreadable-rule state threaded through capture, teaching, recalculation, merge, validator, and export.
  • Phase 2 (326434f): PathAnchor.unanchored with byte-identical existing encodings; both-unanchored refused at validation; teaching defaults the right side open; editor state extracted into a testable value type; plain-language rule descriptions; flagged rows exempt from the commit preview's cap.
  • Phase 3 (b5570d5): the repair regression drives the editor's real gesture path for all four sites — no identity key changes (tth's 40 version-2 keys byte-identical), not a no-op, and the fanfiction.net key change reported before commit.
  • Review fixes (2a69f27): validator misattribution edge, typed export refusal, merge-basis invariant guard, optional previous-key, preview derivation caching, grammar, the missing import-path test, a stale agent note, and decision-log rows Q21–Q24.

Verdict

Ready to push

All 29 spec requirements are verified satisfied (two by documented decision, the rest by tests), the review's major findings are fixed in 2a69f27, and both gates — make test-core and make test-quick — are green with no compiler warnings. The skipped findings are deliberate (test-file constraint, or cost outweighing benefit) and each carries its justification below. The four manual re-teaches in prerequisites.md remain the user's, in the app, after this ships.

Review findings

18 raised · 12 fixed · 6 skipped

Jump to findings →

Commits

Three-level explanation

What Changed

Asterism identifies which story a captured page belongs to using taught URL rules. A rule used to have to name both neighbours of the URL piece it wanted — and on most sites the right-hand neighbour is the story's own name, so every rule only matched the story it was taught from. Now a rule side can be left open ("the component after fiction, whatever follows it"), and teaching produces that by default. Separately, a rule whose stored bytes can't be read used to be silently replaced with a made-up rule that matched nothing — erasing story identities and even reaching backup files. It now fails honestly: capture keeps working, the health check names the rule, re-teaching repairs it, and a backup either omits it or refuses to write a file.

Why It Matters

The reader repairs their pinned sites by re-teaching — no migration — and the repair provably changes no story identity. A silent-corruption path into backups is closed.

Key Concepts

Anchor: one side of the bracket a rule uses to find a path component — start, end, a literal neighbour, or now unanchored. Identity key: the string grouping captures into one story; repairs must not change any. Teaching: tapping components of a real URL to show the app a site's structure.

Changes Overview

Three phases, ordered by Decision 5 — the integrity fix lands before the feature that makes undecodable bytes ordinary. Phase 1 replaces the fabricating accessor with a throwing getter plus failable setDefinition, adds ComposedURLRuleState (none/readable/unreadable, carrying id+version because the version projection must offer version + 1, never a colliding 1), gives the validator a typed unreadableURLRule case, and bounds export by archive reachability (Q14): omit an unreadable row nothing cites, refuse — writing no file — over one that records cite. Phase 2 adds PathAnchor.unanchored (synthesized Codable, existing bytes pinned, the positional _0 key tripwired per Q11), a validation guard against both-sides-unanchored, the unanchored-right teaching default, and moves the editor's gestures into URLEditorState so Req 3.7's template retention is real and testable. Phase 3 proves the repairs through that gesture path.

Implementation Approach

  • State over sentinel wherever a third state exists; the enum carries what its consumers need.
  • Per-consumer try? decisions per the design's consumer table rather than a blanket throw — a propagated throw would fail capture and close the repair route.
  • Testability by extraction: the gestures were private View members; the regression must drive them or it passes while the defect ships.

Trade-offs

Wholesale archive refusal on older builds instead of a format bump (Decision 2); m.fanfiction.net re-taught work-only, losing an inexpressible sequence locator (Q8); merge dead-ends at commit on an affected hostname until re-teaching (Q15); no anchoring control ships until a site needs one.

Technical Deep Dive

  • Wire tripwire: PathAnchor.literal's payload key is the positional _0; labelling it orphans every archive. Pinned by test (Q11). unanchored encodes as {"unanchored":{}}; older builds refuse such archives wholesale before the checksum (Decision 2, Req 4.8).
  • Validation vs selection: both-unanchored resolves on a single-component path, so the refusal is a validate guard — enforced at construction and at import via the codec's reference validation; the import half gained its test in review.
  • Export edge: the reachability partition counts an id unreadable only when no row decodes; a duplicate-UUID group with one corrupt row can still reach the mapper (representative chosen by ordering, not decodability) — now a named unrepresentableValue refusal, failing closed.
  • Merge invariant: WorkMergeBasis refuses the contradictory currentRule-plus-ruleUnreadable state in its throwing init; the = false default otherwise let a forgetful builder re-open the identity-clearing fold Q15 prevents.
  • Preview derivations cached per outcome via didSet (ordering safe: frozenBasis always assigned first); previousIdentityKey is String?, killing an empty-string sentinel that collided with real version-1 keys.
  • Fixture truth (Q20): seeding capture→teach→capture, through projectCapture/commitCapturecapture(_:) applies no rules, and teaching a fully-seeded site manufactures a phantom Work collision.

Architecture Impact

ComposedTeachingBasis gained a source-compatible third state behind a computed currentURLRule. The editor View is render-and-dispatch only; the state machine is shared ground with optional-chapter-sequence (Q12). Every repaired site forfeits Work-URL candidates (WorkURLPlanner requires right == .end) — the builder's .end-when-last preference is the sole remaining route.

Potential Issues

The archive regression pins a reconstructed corpus (Q16) — replace it if the real export ever enters the repo. The title-rule export side keeps refuse-always (Q21), a fail-closed dead end if an undecodable historical title row ever appears. Per-Entry rule decodes in the validator replay remain (microseconds; memoization skipped as not worth threading a cache through two private signatures).

Important changes — detailed

Models.swift: throwing definition accessor replaces the fabrication

Packages/AsterismCore/Sources/AsterismCore/Models.swift

Why it matters. The root data-integrity fix: a decode failure used to yield a fabricated rule that failed every extraction, cleared Work identities, and reached archives with a valid checksum. The setter's empty-Data fallback could manufacture the corrupt bytes with no second device involved.

What to look at. URLRulePattern.definition (get throws) and setDefinition(_:) throws, Models.swift ~480-510

Takeaway. When an accessor cannot fail visibly, it will fail invisibly — and an invented default that validates cleanly is worse than a crash, because every diagnostic downstream blames something else.
Rationale. Mirrors TitlePattern.definition; Swift property setters cannot throw, hence the named method. A failed encode leaves stored bytes untouched (Req 4.6), which is also what keeps the rule recoverable.

ComposedTeachingProjection: three-state ComposedURLRuleState

Packages/AsterismCore/Sources/AsterismCore/ComposedTeachingProjection.swift

Why it matters. Folding 'unreadable' into 'no rule' makes urlVersionProjection propose version 1 for a site retaining versions 1-4 — authoring a library the validator refuses. The state carries id and version because they are load-bearing, not diagnostic.

What to look at. ComposedURLRuleState, URLRuleDecodeFailure, urlVersionProjection's .unreadable arm

Takeaway. A nil that means two different things will eventually be read as the wrong one; when a third state exists, spend the enum.
Rationale. The error travels as its description (URLRuleDecodeFailure) because the basis is Sendable and Equatable — compared for staleness between approval and commit — and nothing branches on the error type (Q23).

URLIdentityTypes/Parsing: PathAnchor.unanchored and its guard

Packages/AsterismCore/Sources/AsterismCore/URLIdentityTypes.swift

Why it matters. The feature itself: an unanchored side never excludes an index, so one taught rule stops pinning itself to the story it was taught from. The validation guard is the only thing refusing both-sides-unanchored — selection would accept it on a single-component path.

What to look at. URLIdentityTypes.swift PathAnchor + validate; URLIdentityParsing.swift match arms

Takeaway. Check what selection semantics actually reject before assuming a degenerate case self-rejects; here the dangerous case resolves successfully on exactly one path shape.
Rationale. Fully synthesized Codable keeps every existing case byte-identical (Req 4.2) and makes archive/mirror compatibility free; the positional _0 payload key is pinned by test because renaming it orphans every archive (Q11).

ComposedURLEditorState: gestures extracted from the View

Asterism/Asterism/Views/ComposedURLEditorState.swift

Why it matters. Req 3.7 (repairing tthfanfic must not re-key its 40 captures) was unachievable through the shipped editor — the only gesture applying the corrected locator also dropped the taught template. The retention rule lives here, and the regression drives these exact transitions.

What to look at. URLEditorState: seed/select/beginSplit/toggleSplitToken/useWholeComponent/clear/rule(in:)

Takeaway. A test that hand-builds the value a UI should produce passes while the UI ships the defect; extract the state machine and test the gestures.
Rationale. Retain the template across a locator change on the same component — compared by index, since a path can repeat a value — and clear it on every gesture that supersedes it (Q12, Q17).

BackupV4Exporter: archive-reachability partition

Packages/AsterismCore/Sources/AsterismCore/BackupV4Exporter.swift

Why it matters. Scoping the decode check to current rules cannot work — the mapper needs a typed definition for every row it writes, and a bad historical blob could never be cleared by re-teaching. Reachability is the boundary: omit what nothing cites, refuse (no file) over what records cite.

What to look at. partitionUnreadableURLRules, citedURLRuleIDs, mapV4URLRuleRecord (now a named refusal)

Takeaway. When deciding what may be dropped from a serialized artifact, the question is reachability from the records being written, not any status flag on the row.
Rationale. Q14; the refusal case exists because dropping a cited row would orphan its citers' provenance. The title side deliberately keeps refuse-always (Q21).

URLRepairThroughEditorTests: the gesture-path repair regression

Asterism/AsterismTests/URLRepairThroughEditorTests.swift

Why it matters. The proof the feature exists for: each pinned site is repaired via seed-from-stored-rule, chip tap, real commit — asserting every capture resolves, no identity key changes, the repair versions rather than no-ops, and the one deliberate key change is reported before commit.

What to look at. Two tests over real V4 stores in temp directories; pre-repair state pinned so 'no key changed' cannot pass vacuously

Takeaway. Pin the before-state in the same test that asserts the after-state, or the assertion can be satisfied by a fixture that never had the property.
Rationale. Seeding order is capture-teach-capture through projectCapture/commitCapture (Q20) — the two fixture defects found en route are recorded in the decision log and docs/agent-notes/testing.md.

V4LibraryValidator: unreadableURLRule, and the review's misattribution catch

Packages/AsterismCore/Sources/AsterismCore/V4LibraryValidator.swift

Why it matters. The diagnosis now names the rule instead of quarantining the captures citing it. The review found the work-identity replay branch still folded a decode failure into try?, re-blaming the Entry — shadowed by loop ordering today, but a latent contradiction of the design's intent.

What to look at. V4ValidationError.unreadableURLRule; per-Site decode check ~:547; replay branches ~:845-885

Takeaway. When two sibling branches handle the same failure differently, one of them is wrong even if ordering currently hides it.
Rationale. A case, not a reason string — reason is free-form and nothing can branch on it. Arriving as .siteTuple keeps the diagnosis clearable by re-teaching (Q13), which is the repair.

Key decisions

Fix the fabrication before shipping the unanchored side (Decision 5). One release, but strict task ordering: an unanchored side is the first change that makes undecodable bytes plausible rather than exotic, so the fabricating accessor had to be gone first. Not a deployment boundary — the library is single-device (Q9).
No backup format bump for the new anchor case (Decision 2). An archive containing an unanchored locator is refused wholesale by older builds (decodingFailed, before the checksum) — accepted rather than spending a format generation on an additive change. Unmarked archives stay byte-identical; mirroring is free because the store carries the definition as an opaque blob (Req 4.10).
m.fanfiction.net is re-taught work-only, losing its chapter sequence (Q8). Its sequence component sits between a story id and a chapter slug — both vary, and the component is neither first nor last, so no expressible anchor reaches it. Atomic .workAndSequence application means a half-repair yields nothing; its single capture is a throwaway.
Export reachability, not isCurrent (Q14); title side keeps refuse-always (Q21). The mapper needs a typed definition for every row it writes, so a current-only pre-flight still throws on historical rows that re-teaching can never clear. The title side reconstructs from columns, has never shown the failure, and fails closed — so generalising the partition was deliberately skipped.
Merge dead-ends at commit on an affected hostname (Q15). The planner retains the merged identity (via ruleUnreadable), but commitMerge's pre-existing rollback on any hostname diagnosis returns .invalidated while the unreadable-rule diagnosis stands. No mutation occurs and re-teaching repairs both — accepted rather than special-cased.
Editor state extracted to make the gesture path testable (Q17). The gestures were private View members; Req 3.7's test must drive them or it passes while the defect ships. The types are nonisolated because the app target defaults to MainActor isolation.
The preview keeps a six-row cap for ordinary rows only (Q22). The design's traceability line said "drops the cap", but unresolved and re-keyed rows always render past it — Req 3.4 is about not hiding the rows that matter, and a 173-capture site's unchanged rows are not those.
Fixtures reconstruct the private archive (Q16/Q19) and seed capture-teach-capture (Q20). The real 2026-08-07 export is not in the repo. The corpus rebuilds its documented shape behind the existing #if DEBUG || ASTERISM_PERFORMANCE_TESTING seam; the seeding order avoids a phantom Work collision that a teach-once fixture manufactures.

Review findings

SeverityAreaFindingResolution
majorV4LibraryValidator.swift:845 (work replay branch)try? URLRuleApplicator.apply(rule.definition, ...) folded a decode failure into 'stored URL extraction does not equal retained-rule replay' — re-blaming the Entry, the exact misattribution Decision 5 removes. The sibling sequence branch handled it explicitly.Decode hoisted with the same catch-and-throw-unreadableRule as the sequence branch.
majorComposedTeachingViewModel preview derivationsunresolvedURLCaptures / identityKeyChanges / previewRows / identityKeyChangeNotice were uncached computed properties, each an O(entries) walk with a dictionary rebuild; one render pass of the preview section evaluated them ~5 times.A PreviewReport derived once per previewOutcome via didSet; the public properties read the cache. frozenBasis is always assigned first, so ordering is safe.
majorReq 1.3 import half (testing gap)Both-sides-unanchored was tested at construction only; no test fed an archive containing one through the codec's reference validation, where refusal must hold because selection would accept it on a single-component path.New fixture + two BackupV4CodecTests: both-unanchored refused on import, single-sided imports cleanly.
majordocs/agent-notes/rule-wire-format.mdThe fabrication-hazard section quoted the removed getter as current code and said 'the export path does not check at all' in present tense — a future session would re-diagnose a fixed bug.Rewritten as historical with pointers to the throwing getter, the validator case, and the Q14 export policy; the compatibility table row updated.
minorBackupV4Exporter.swift mapV4URLRuleRecordA duplicate-UUID rule group with one readable and one corrupt row counts as readable, and the site projection may pick the corrupt row as representative — escaping as a raw DecodingError while the doc comment claimed the try was an unreachable backstop.Decode wrapped into a named BackupV4ExportError.unrepresentableValue; comment rewritten to state the reachable case. Fails closed either way.
minorProjectionContract.swift WorkMergeBasiscurrentRule non-nil plus ruleUnreadable true was representable and meaningless, and the false default meant a forgetful basis builder silently re-opens the identity-clearing fold Q15 exists to prevent.Throwing init refuses the contradiction (new WorkMergePlanningError.contradictoryRuleState).
minorComposedEntryBasis/Projection previousIdentityKeyEmpty-string sentinel made the empty string mean 'unknown', and the defaulted previousKeyVersion of 1 collided with real version-1 keys.previousIdentityKey is String? (nil default); the production basis maps a blank stored key to nil; the consumer guards on the optional.
minorComposedTeachingViewModel.identityKeyChangeNoticePluralisation produced '1 capture move to a conservative identity key' — and the singular case is exactly the real m.fanfiction.net repair.Verb pluralised with the phrase ('1 capture moves' / 'captures move'), matching the repo's convention.
minorSpec divergences unloggedTitle-side export policy stays refuse-always while the design promised one shared history policy; the preview keeps a cap the design said was dropped; URLRuleDecodeFailure deviates from the design's any-Error sketch; Req 4.5's omission record is a log line.Q21-Q24 added; Decision 2 gained the Req 4.10 sentence.
nitBackupV4Exporter.swift mapV4SiteRecordomittingURLRules defaulted to empty with no caller using the default — a future caller omitting it re-opens the membership/record mismatch the parameter closes.Default removed; the compiler now asks.
nitComposedURLDetailsEditor componentTextOne-line private forwarder to the static it wrapped.Call sites use the static directly.
nitCHANGELOG.md phase-3 entryClaimed one five-site 173-capture library is repaired in tests; the repair tests seed four single-site libraries — the corpus figure belongs to the selection regression.Softened to each site's real shape.
minorV4LibraryValidator per-Entry decodeThe replay loop decodes the cited rule once per Entry (173 entries citing one rule = 173 decodes of identical bytes).Skipped: microseconds against a budget whose cost lives elsewhere, at the price of threading a cache through two private static signatures. Documented at the site.
minorComposedTeachingView preview renderingUnresolved captures render both past the previewRows cap and in the itemised unresolved list, in a non-lazy VStack.Skipped: the two appearances serve different roles (before/after row vs named failure, Req 3.4), n is bounded by the site's captures, and the recompute cost was removed by the report cache.
minorURLRepairThroughEditorTests FixedRepairClockDuplicates the existing public FixedRepositoryClock (and needs @unchecked Sendable where the original doesn't).Skipped: test-file-only cleanup, barred by the review's no-test-modification constraint. Worth folding into the next change that touches the file.
minorBackupV4Exporter citedURLRuleIDs vs citerHostnamesTwo walks over entries and their rule citations within ~15 lines of each other.Skipped: merging couples the partition to a hostname-keyed map built for a different purpose; the citation-target filter is clearer kept explicit, and the second walk only runs when unreadable rows exist.
minorURLEditorState.rule(in:)A query that silently mutates retainedTemplate when a live split derives — the one non-gesture mutation in the state machine.Skipped: behaviour is correct and documented at the site; renaming ripples into the tests that drive the type.
nitdescribe(_:) placement / sortedKeys byte-identity / preview test sleepProse formatters live on the ViewModel rather than the presentation namespace; byte-identity tests pin structure via sortedKeys rather than literal bytes; one preview test uses a fixed 50ms sleep where a poll helper exists.Skipped: consistency nits with precedent on both sides, and the sleep is a test-file change barred by constraint.

Per-file diffs

Click to expand.

Asterism/Asterism/ViewModels/ComposedTeachingViewModel.swift Modified +168 / -11
Asterism/Asterism/Views/ComposedTeachingView.swift Modified +50 / -2
Asterism/Asterism/Views/ComposedURLDetailsEditor.swift Modified +44 / -172
Asterism/Asterism/Views/ComposedURLEditorState.swift Added +291 / -0
Asterism/AsterismTests/ComposedTeachingPreviewTests.swift Added +185 / -0
Asterism/AsterismTests/ComposedURLEditorStateTests.swift Added +274 / -0
Asterism/AsterismTests/ComposedURLRuleDescriptionTests.swift Added +197 / -0
Asterism/AsterismTests/URLRepairThroughEditorTests.swift Added +407 / -0
CHANGELOG.md Modified +3 / -0
Packages/AsterismCore/Sources/AsterismCore/BackupV4Exporter.swift Modified +103 / -9
Packages/AsterismCore/Sources/AsterismCore/ComposedTeachingProjection.swift Modified +93 / -9
Packages/AsterismCore/Sources/AsterismCore/LibraryRepository+Capture.swift Modified +7 / -2
Packages/AsterismCore/Sources/AsterismCore/LibraryRepository+ComposedTeaching.swift Modified +34 / -8
Packages/AsterismCore/Sources/AsterismCore/LibraryRepository+ReparseCapture.swift Modified +17 / -3
Packages/AsterismCore/Sources/AsterismCore/LibraryRepository+URLIdentity.swift Modified +9 / -2
Packages/AsterismCore/Sources/AsterismCore/LibraryRepository+WorkMerge.swift Modified +40 / -21
Packages/AsterismCore/Sources/AsterismCore/Models.swift Modified +34 / -6
Packages/AsterismCore/Sources/AsterismCore/ProjectionContract.swift Modified +18 / -1
Packages/AsterismCore/Sources/AsterismCore/URLArchiveCorpusFixture.swift Added +168 / -0
Packages/AsterismCore/Sources/AsterismCore/URLIdentityParsing.swift Modified +5 / -0
Packages/AsterismCore/Sources/AsterismCore/URLIdentityTypes.swift Modified +21 / -1
Packages/AsterismCore/Sources/AsterismCore/V4LibraryValidator.swift Modified +42 / -3
Packages/AsterismCore/Sources/AsterismCore/WorkMergePlanner.swift Modified +7 / -0
Packages/AsterismCore/Tests/AsterismCoreTests/BackupExportDegradedRefusalTests.swift Modified +96 / -0
Packages/AsterismCore/Tests/AsterismCoreTests/BackupV4CodecTests.swift Modified +20 / -0
Packages/AsterismCore/Tests/AsterismCoreTests/BackupV4Fixtures.swift Modified +33 / -0
Packages/AsterismCore/Tests/AsterismCoreTests/ComposedPreviewKeyChangeTests.swift Added +87 / -0
Packages/AsterismCore/Tests/AsterismCoreTests/URLUnanchoredPathAnchorTests.swift Added +172 / -0
Packages/AsterismCore/Tests/AsterismCoreTests/URLUnanchoredSelectionTests.swift Added +249 / -0
Packages/AsterismCore/Tests/AsterismCoreTests/UnreadableURLRuleTests.swift Added +559 / -0
Packages/AsterismCore/Tests/AsterismCoreTests/V5RelationshipPassTests.swift Modified +1 / -1
docs/agent-notes/rule-wire-format.md Modified +36 / -47
docs/agent-notes/testing.md Modified +20 / -0
specs/OVERVIEW.md Modified +2 / -2
specs/url-locator-generalisation/decision_log.md Modified +14 / -1
specs/url-locator-generalisation/tasks.md Modified +34 / -34

Things to double-check

The four manual re-teaches are yours.

prerequisites.md lists re-teaching www.tthfanfic.org, tapas.io, www.royalroad.com, and m.fanfiction.net in the app after this ships. Nothing on this branch touched a device. The fanfiction.net re-teach will convert its one capture's key — the preview will say so before you commit.

The archive regression pins documentation, not the archive.

The corpus is reconstructed from the documented shape of the private 2026-08-07 export (Q16). If the real archive ever enters the repo, replace URLArchiveCorpusFixture with it.

Merge is unavailable on a hostname with an unreadable rule.

Until re-taught, commitMerge returns .invalidated there (Q15). Expected, but worth remembering if a merge ever seems mysteriously stuck.

Repaired sites forfeit Work-URL candidates.

WorkURLPlanner requires right == .end; an unanchored right does not establish a terminal component. The .end-when-genuinely-last teaching preference is the sole remaining route to a Work URL.