Asterism

Merge preparation: accepted source-free note fix (T-2462)

Verdict

Ready for review / merge decision

Installed classifier5e683fcf accepted on phone. Owner explicitly waived further UI tests. Core3828pass/55skip and fast456pass match853Swift files. Signed Mac/iOS builds passed. Earlier native interruption and UI failures remain recorded. GitHub CLEAN/MERGEABLE with no CI checks or submitted reviews; no merge performed.

Review findings

2 raised · 2 fixed · 0 skipped

Jump to findings →

Tests

Pass rate: 100% (4284 of 4284)

New tests: 4

Diff coverage: n/a

Jump to tests →

Three-level explanation

A manually written note or chapter can have no web address. Those entries no longer trigger a missing-site warning. Broken references still warn, and the displayed count matches the list.

The scan, full validator and affected-record projector share the authored source-free scalar predicate. A predicate-only identifier query distinguishes malformed attached Site references. The physical Entry total remains independent from the missing-Site buckets.

The foreground walk reads source scalar fields without per-entry relationship faults. A single identifiers-only fetch costs O(k) space for inconsistent blank-host relationships. The full validator still checks the Site relationship, authored data, UUID identity, citations and Work inverse; refresh retains tuple diagnoses. The projector applies the full relationship condition.

Important changes — detailed

Separate optional provenance from missing references

Packages/AsterismCore/Sources/AsterismCore/AuthoredEntryTarget.swift

Why it matters. Valid authored notes may have no captured-page Site.

What to look at. hasSiteFreeAuthoredSource

Rationale. URL-optional note contract; user confirmation.

Preserve damaged references and library totals

Packages/AsterismCore/Sources/AsterismCore/LibraryDiagnostics.swift

Why it matters. Avoid suppressing malformed Site relationships or shrinking library shape.

What to look at. LibraryToleranceScan.scan

Rationale. A blank URL alone does not prove a healthy authored record.

Keep affected counts and rows consistent

Packages/AsterismCore/Sources/AsterismCore/LibraryDiagnosticRecords.swift

Why it matters. Healthy notes must not appear beside genuinely broken empty-host captures.

What to look at. siteMissing projection filter

Rationale. Use the same source and relationship conditions as validation.

Review findings

SeverityAreaFindingResolution
majorUnexpected Site relationshipScalar-only exemption hid an authored Entry with a mismatched nonnil Site.Identifiers-only predicate preserves scan finding, projection checks relationship, regression added.
majorPhysical entry countFiltering hostname buckets reduced the library-shape denominator.Independent per-row count and reopen regression preserve total. Original failing test run retained.

Tests

Source: local run at 2026-10-15T08:32:21.645240+00:00 · snapshot 5e683fcf17ebf5cbf29a11828d839460251d6d4c

Baseline: none

Execution: passed (partial results) · JUnit: 2 files · Coverage: none · Baseline: absent

Coverage scope: as the project configures it

Totals: 4284 passed · 0 failed · 55 skipped · 0 errored · 0 flaky

New and removed tests

Derived by declaration name, from the diff (no baseline run).

Blast radius

Files that import a changed file on the left, changed files in the centre, files a changed file imports on the right. Snapshot working-tree against base 7f0f41c8.

addedmodifieddeletedrenamedunchangedcollapsed package group or +N more⚑N test files with an edge to the node

Per-file diffs

8 files: 5 code · 3 docs. Click to expand.

Code (5)

Packages/AsterismCore/Sources/AsterismCore/AuthoredEntryTarget.swift Modified
diff --git a/Packages/AsterismCore/Sources/AsterismCore/AuthoredEntryTarget.swift b/Packages/AsterismCore/Sources/AsterismCore/AuthoredEntryTarget.swiftindex b5e72087..bb2c7838 100644--- a/Packages/AsterismCore/Sources/AsterismCore/AuthoredEntryTarget.swift+++ b/Packages/AsterismCore/Sources/AsterismCore/AuthoredEntryTarget.swift@@ -61,6 +61,14 @@ public struct EntryMergeResolution: Codable, Equatable, Sendable { }  extension Entry {+    /// Authored records need no captured-page Site when all source scalars are+    /// absent. Keep this scalar-only: the foreground scan must not fault Site+    /// relationships. Full validation still checks relationships and target data.+    internal var hasSiteFreeAuthoredSource: Bool {+        authoredTargetData != nil && rawURLString.isEmpty+            && hostname.isEmpty && canonicalURLString == nil+    }+     public var authoredTarget: AuthoredEntryTarget? {         get { authoredTargetData.flatMap { try? AuthoredEntryTarget.decode($0) } }     }
Packages/AsterismCore/Sources/AsterismCore/LibraryDiagnosticRecords.swift Modified
diff --git a/Packages/AsterismCore/Sources/AsterismCore/LibraryDiagnosticRecords.swift b/Packages/AsterismCore/Sources/AsterismCore/LibraryDiagnosticRecords.swiftindex 6aa9f9a6..e9857956 100644--- a/Packages/AsterismCore/Sources/AsterismCore/LibraryDiagnosticRecords.swift+++ b/Packages/AsterismCore/Sources/AsterismCore/LibraryDiagnosticRecords.swift@@ -65,6 +65,7 @@ enum LibraryDiagnosticRecords {             case .siteMissing(let hostname, _, _):                 let entries = try context.fetch(FetchDescriptor<Entry>(                     predicate: #Predicate { $0.hostname == hostname }))+                    .filter { !$0.hasSiteFreeAuthoredSource || $0.site != nil }                     .sorted { ($0.id.uuidString, $0.captureTitle, $0.rawURLString, $0.firstCapturedAt)                         < ($1.id.uuidString, $1.captureTitle, $1.rawURLString, $1.firstCapturedAt) }                 var variants: [UUID: Int] = [:]
Packages/AsterismCore/Sources/AsterismCore/LibraryDiagnostics.swift Modified
diff --git a/Packages/AsterismCore/Sources/AsterismCore/LibraryDiagnostics.swift b/Packages/AsterismCore/Sources/AsterismCore/LibraryDiagnostics.swiftindex 59812853..9c50f6b1 100644--- a/Packages/AsterismCore/Sources/AsterismCore/LibraryDiagnostics.swift+++ b/Packages/AsterismCore/Sources/AsterismCore/LibraryDiagnostics.swift@@ -611,8 +611,17 @@ public enum LibraryToleranceScan {         var entriesWithoutCitationBlob = 0         var patternsWithoutDefinitionBlob = 0 +        // A source-free authored row with an unexpected Site relationship is+        // still damaged. Ask the store for those identities once; testing+        // `entry.site` inside the foreground walk would fault every relationship.+        let unexpectedSiteEntries = Set(try context.fetchIdentifiers(FetchDescriptor<Entry>(+            predicate: #Predicate { $0.hostname == "" && $0.site != nil })))+        var entryCount = 0         try context.enumerate(FetchDescriptor<Entry>(), batchSize: batchSize) { entry in-            entriesByHostname[entry.hostname, default: 0] += 1+            entryCount += 1+            if !entry.hasSiteFreeAuthoredSource || unexpectedSiteEntries.contains(entry.persistentModelID) {+                entriesByHostname[entry.hostname, default: 0] += 1+            }             entryIdentities.record(entry.id)             if entry.citationsAreAbsent { entriesWithoutCitationBlob += 1 }             if let key = DuplicateScan.entryBucketKey(@@ -745,7 +754,7 @@ public enum LibraryToleranceScan {             diagnoses: diagnoses,             shape: LibraryShape(                 siteCount: siteRowsByHostname.values.reduce(0, +),-                entryCount: entriesByHostname.values.reduce(0, +),+                entryCount: entryCount,                 // Distinct Work ids, **not** the per-hostname sum: a Work on two                 // sites is one Work, and summing memberships would count it                 // twice in the ratio `suggestsDamage` reads (Q21).
Packages/AsterismCore/Sources/AsterismCore/LibraryValidator.swift Modified
diff --git a/Packages/AsterismCore/Sources/AsterismCore/LibraryValidator.swift b/Packages/AsterismCore/Sources/AsterismCore/LibraryValidator.swiftindex 5d71b2bf..24a21458 100644--- a/Packages/AsterismCore/Sources/AsterismCore/LibraryValidator.swift+++ b/Packages/AsterismCore/Sources/AsterismCore/LibraryValidator.swift@@ -580,8 +580,7 @@ public enum LibraryValidator {         }         for group in entryRows {             guard let entry = entries[group.key]?.first else { continue }-            let siteFreeAuthored = entry.authoredTargetData != nil && entry.rawURLString.isEmpty-                && entry.hostname.isEmpty && entry.canonicalURLString == nil && entry.site == nil+            let siteFreeAuthored = entry.hasSiteFreeAuthoredSource && entry.site == nil             guard siteFreeAuthored || siteHostnames.contains(entry.hostname) else {                 guard strictness == .tolerant else {                     throw unresolved("Entry", entry.id.uuidString, "Site \(entry.hostname)")
Packages/AsterismCore/Tests/AsterismCoreTests/LibraryToleranceScanTests.swift Modified
diff --git a/Packages/AsterismCore/Tests/AsterismCoreTests/LibraryToleranceScanTests.swift b/Packages/AsterismCore/Tests/AsterismCoreTests/LibraryToleranceScanTests.swiftindex 3e9155ee..6bccaf00 100644--- a/Packages/AsterismCore/Tests/AsterismCoreTests/LibraryToleranceScanTests.swift+++ b/Packages/AsterismCore/Tests/AsterismCoreTests/LibraryToleranceScanTests.swift@@ -530,3 +530,97 @@ extension LibraryToleranceScanTests {         #expect(try store.contents() == before && !store.context.hasChanges)     } }+++extension LibraryToleranceScanTests {+    @Test("Authored source-free notes and chapters do not need a Site on refresh or reopen",+          arguments: [AuthoredEntryTarget.Kind.note, .chapter])+    func sourceFreeAuthoredEntries(kind: AuthoredEntryTarget.Kind) throws {+        let store = try ToleranceScanStore()+        let entry = Entry(captureTitle: "Manual", captureTitleSource: .host,+            rawURLString: "", hostname: "", entryIdentityKey: "", timestamp: ToleranceScanStore.epoch)+        try entry.authorTarget(.init(kind: kind, title: "Manual", designation: kind == .chapter ? "12" : nil))+        store.context.insert(entry)+        try store.save()+        let reopened = try store.reopen()+        for context in [store.context, ModelContext(store.container), reopened.context] {+            let scan = try LibraryToleranceScan.scan(context: context)+            #expect(scan.shape.entryCount == 1)+            #expect(scan.diagnoses.isEmpty)+            let full = try LibraryValidator.validate(context: context)+            #expect(full.affectedRecordCount == 0)+            #expect(try LibraryDiagnosticRecords.project(scan.diagnoses, context: context).isEmpty)+        }+    }++    @Test("Source-free exemption preserves missing-site damage and count/list agreement")+    func sourceFreeNotesDoNotMaskBrokenSources() throws {+        let store = try ToleranceScanStore()+        let valid = Entry(captureTitle: "Valid", captureTitleSource: .host,+            rawURLString: "", hostname: "", entryIdentityKey: "", timestamp: ToleranceScanStore.epoch)+        try valid.authorTarget(.init(kind: .note))+        store.context.insert(valid)+        // Empty URL alone must not hide a legacy capture with no authored identity.+        let broken = store.insertEntry(hostname: "")+        broken.rawURLString = ""; broken.conservativeIdentityKey = ""+        let sourced = store.insertEntry(hostname: "missing.example")+        try sourced.authorTarget(.init(kind: .note))+        _ = store.insertWork(hostname: "missing.example")+        let canonicalOnly = Entry(captureTitle: "Canonical history", captureTitleSource: .host,+            rawURLString: "", hostname: "", entryIdentityKey: "", timestamp: ToleranceScanStore.epoch)+        try canonicalOnly.authorTarget(.init(kind: .note))+        canonicalOnly.canonicalURLString = "https://missing.example/history"+        store.context.insert(canonicalOnly)+        try store.save()+        let before = try store.contents()+        let scan = try LibraryToleranceScan.scan(context: store.context)+        let diagnostics = LibraryDiagnostics.union(tupleDiagnoses: [:], toleratedStates: scan.diagnoses, shape: scan.shape)+        let rows = try LibraryDiagnosticRecords.project(scan.diagnoses, context: store.context).values.flatMap { $0 }+        #expect(diagnostics.affectedRecordCount == 4)+        #expect(rows.count == diagnostics.affectedRecordCount)+        #expect(!rows.contains { $0.recordID == valid.id.uuidString })+        #expect(rows.contains { $0.recordID == broken.id.uuidString })+        #expect(rows.contains { $0.recordID == sourced.id.uuidString })+        #expect(rows.contains { $0.recordID == canonicalOnly.id.uuidString })+        #expect(try store.contents() == before && !store.context.hasChanges)+    }+}+++extension LibraryToleranceScanTests {+    @Test("Unexpected Site relationships on otherwise source-free notes remain unresolved")+    func sourceFreeNoteWithBrokenSiteRelationship() throws {+        let store = try ToleranceScanStore()+        let site = store.insertSite(hostname: "other.example")+        let entry = Entry(captureTitle: "Damaged", captureTitleSource: .host,+            rawURLString: "", hostname: "", entryIdentityKey: "", timestamp: ToleranceScanStore.epoch)+        try entry.authorTarget(.init(kind: .note))+        entry.site = site+        store.context.insert(entry)+        try store.save()+        let full = try LibraryValidator.validate(context: store.context)+        let scan = try LibraryToleranceScan.scan(context: store.context)+        #expect(scan.diagnoses == [.siteMissing(hostname: "", entryCount: 1, workCount: 0)])+        #expect(full.diagnoses == scan.diagnoses)+        let rows = try LibraryDiagnosticRecords.project(full.diagnoses, context: store.context).values.flatMap { $0 }+        #expect(rows.count == full.affectedRecordCount)+        #expect(rows.contains { $0.recordID == entry.id.uuidString })+    }++    @Test("Source-free notes still undergo authored data and identity validation", arguments: [true, false])+    func sourceFreeAuthoredDamageRemainsDiagnosed(unreadable: Bool) throws {+        let store = try ToleranceScanStore()+        let entry = Entry(captureTitle: "Damaged", captureTitleSource: .host,+            rawURLString: "", hostname: "", entryIdentityKey: "", timestamp: ToleranceScanStore.epoch)+        try entry.authorTarget(.init(kind: .note))+        if unreadable { entry.authoredTargetData = Data([0xff]) }+        else { entry.entryIdentityKey = "wrong" }+        store.context.insert(entry)+        try store.save()+        let full = try LibraryValidator.validate(context: store.context)+        #expect(full.tupleDiagnoses[""] != nil)+        let refreshed = LibraryDiagnostics.union(tupleDiagnoses: full.tupleDiagnoses,+            toleratedStates: try LibraryToleranceScan.scan(context: store.context).diagnoses)+        #expect(refreshed.affectedRecordCount == 1)+    }+}

Docs (3)

CHANGELOG.md Modified
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex d8c860e7..f618ad55 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -161,6 +161,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).  ### Fixed +- **Source-free notes (T-2462):** stop reporting valid authored notes and chapters+  as missing-site records on foreground refresh; retain damaged source references+  and keep affected counts and record lists consistent.+ - **Unresolved records (T-2457):** list affected works, captures and sites in Check Library with names, identities and capture context. Open available works or captures through their usual details; explain unavailable records and read failures without changing the unresolved count or repairing data.  - **Character-extraction UI journeys (T-2439):** scroll record pills fully clear of the tab bar before tapping, verify work-detail navigation, and recognize the wide Today pane by its stable identifier. Preserve alias/fact/citation assertions and failure evidence.
specs/bugfixes/source-free-note-diagnostics/report.md Modified
diff --git a/specs/bugfixes/source-free-note-diagnostics/report.md b/specs/bugfixes/source-free-note-diagnostics/report.mdnew file mode 100644index 00000000..87ee5a12--- /dev/null+++ b/specs/bugfixes/source-free-note-diagnostics/report.md@@ -0,0 +1,44 @@+# Bugfix: Source-free notes reported as unresolved (T-2462)++## Problem and evidence++The user identified PR110's unresolved entries as legitimate notes without source URLs (Sentinel_e00b5a1793b881919abd56be491bd415). UUID-authored notes and chapters may have no capture URL, hostname, canonical URL or Site. Source associations are optional provenance, not identity. No private library was inspected or edited.++## Investigation++The foreground `LibraryToleranceScan` unconditionally buckets every Entry by hostname and emits `siteMissing` for each absent Site, including the empty hostname. Full `LibraryValidator` already exempts source-free authored entries before validating their tuple. `refreshDiagnostics` replaces tolerated findings with the scan output, so even successful full validation cannot clear the warning across foreground refresh. PR110's affected-record projection lists all entries under the diagnosed hostname, which would include healthy notes beside a genuinely broken empty-host capture after a count-only fix.++Why chain: warning counts missing-Site diagnoses; scan produces them for empty hosts; scan predates authored URL-free records; the validator exemption was added without updating the cheap scan; projection mirrored the scan's prior hostname-only assumption.++## Scope and plan++Continue draft PR110, retaining T-2457 visibility work. T-2462 tracks the explicitly expanded classification correction. Share the existing scalar source-free predicate across full validation, scan and affected-list selection. Keep full validator relationship, authored blob/UUID, citations and Work inverse checks. Do not omit all empty-URL entries or alter repairs, schema, persisted data or missing-Work diagnoses. The scan keeps its per-entry walk scalar-only. One predicate identifiers-only query finds empty-host rows with unexpected Site relationships, so those damaged rows are not exempted; no per-entry Site relationship is faulted. Library shape uses an independent physical entry count so healthy source-free entries remain in the total.++## Regression plan++Verify generic notes and authored chapters across current/fresh/reopened contexts; preserve legacy empty-source damage, sourced authored notes missing a Site and missing-Site Work membership; assert count/list identity agreement and no writes. Run affected Core and manual-note repository suites with bounded execution. No broad UI development run, merge or device install.++## Results++Initial test compilation failed with two test-only errors (zero execution); corrected before the reproduction. The pre-fix reproduction then failed both declarations with 14 expected assertions. First green attempt failed only six library-shape assertions, revealing the count denominator needed an independent entry counter. Fixed and retained that failure. A four-suite focused run passed all 59 declarations; later final-source checks are recorded below. Reviews caught and fixed the unexpected-Site edge, with regressions for that case, malformed authored data, invalid UUID identity and canonical-only provenance.++Prior PR110 broad-phone failures remain preserved and are not a green acceptance verdict. No new broad UI run, live-data edits, repair, merge or installation is part of T-2462.+++## Final-source verification and limits++- Unfiltered serial Core `make test-core`: exit 0, 3,828 passed XML cases and 55 skipped opt-in cases; zero assertion failures and compiler warnings. Raw log has four additional passing run-summary lines, so the initial progress count of 3,832 was corrected. 246 Core Data external temporary-file diagnostic lines occurred during synthetic fixtures; these are retained, not treated as clean runtime output. Source hashes match.+- `make test-fast`: exit 0, 152 declarations / 456 passed XML cases, zero skips/failures, exact source hashes retained.+- `make test-quick`: signed Mac app and embedded extension compile completed. At the parent-directed capacity pause the owned Make/Xcode group was interrupted (receipt exit -2, console TEST INTERRUPTED), zero test execution observed; the process group drained. Simulator compilation had warnings in untouched OptionalSequenceTeachingMessagesTests/ToReadMatchModelTests and toolchain/metadata diagnostics. No clean native/warnings verdict is claimed. No further heavy verification may launch until a new capacity decision.+- Four independent source reviews completed; unexpected-Site finding fixed and re-reviewed, no remaining source correctness findings. `git diff --check` passed. Review HTML uses the bundled renderer; historical broad-phone failures remain open.++The classification fix is suitable for draft review, with native and full rendered acceptance outstanding on its changed Core source. Old exact-source results are not reused across this classifier edit. No T-2462 build was installed; the phone still has PR110 at 7f0f41c8 until a separately requested installation.+++## Accepted for merge preparation (2026-10-15)++The user tested the installed classifier at `5e683fcf17ebf5cbf29a11828d839460251d6d4c` and confirmed: “Seems to have fixed the issue. It's only background, we don't need UI tests to run. So get it ready for merge please” (reply to Sentinel_492d01ad281481918274d734c8c3762e). This explicitly waives further UI tests for this change and authorizes preparing PR110, not merging it.++The signed Devsterism phone installation completed at 2026-10-15T08:25:08Z with the existing bundle, App Group and CloudKit identifiers. No uninstall, reset or live data edits occurred. Reused Core (3,828 passed XML cases / 55 skipped) and fast (456 passed XML cases) results match all 853 recorded Swift files exactly. Signed Mac app/extension and physical iOS builds passed. Four source reviews and their corrective follow-up remain valid; this preparation changes documentation only. No additional tests or builds were run.++Current base `main` is `0e3c91bdba1e0e1017fa6cf2ef0df57dbbd274f6`, already the branch ancestor; no rebase is needed. GitHub reports CLEAN/MERGEABLE, zero review threads/comments, no submitted reviews and no status checks. No CI-green or external-approval claim is made. Earlier failed/incomplete UI and interrupted native results remain historical limits, not reclassified as passes. The owner accepted the phone behavior and waived UI verification; current-source unfiltered native execution remains incomplete, with affected package behavior and signed app builds verified. PR110 can be marked ready for review/merge decision. No merge performed.
specs/bugfixes/unresolved-record-visibility/report.md Modified
diff --git a/specs/bugfixes/unresolved-record-visibility/report.md b/specs/bugfixes/unresolved-record-visibility/report.mdindex 7f7abd6b..ed0c5102 100644--- a/specs/bugfixes/unresolved-record-visibility/report.md+++ b/specs/bugfixes/unresolved-record-visibility/report.md@@ -1,7 +1,7 @@ # Bugfix Report: Unresolved Record Visibility  **Date:** 2026-10-14-**Status:** Implemented and source-reviewed in draft PR110 (T-2457). Broad phone acceptance failed/incomplete; no merge or installation.+**Status:** PR110 implemented and source-reviewed (T-2457/T-2462); installed classifier accepted on phone, with further UI tests explicitly waived by the owner on 2026-10-15. Preparing ready-for-review status; historical broad-phone failures remain recorded. No merge.  ## Description of the Issue @@ -105,3 +105,13 @@ Review artifact: `/Users/arjen/CodeReviews/2026-10/2026-10-14-github-asterism-br Draft PR: https://github.com/ArjenSchwarz/asterism/pull/110 (`bugfix/unresolved-record-visibility` → `main`). The implementation commit is `964aa3020daec0e6455276ed7600492350c7c9c7`, following the preserved investigation commit `17d1763d`. PR and remote branch were verified at that implementation head before this documentation-only handoff update. The final remote head is verified again in `/tmp/unresolved-records/final-handoff.json`.  Both dedicated simulators are Shutdown; no owned Xcodebuild/UI runner remains. A post-drain host read showed59% memory free, with load averages67.39/60.59/112.91; this proves resource release, not a quiet-host guarantee. Transit T-2457 creation is confirmed; repeated scoped full reads returned READ_TIMEOUT, including a final no-comments read, so no status mutation was attempted with a stale revision. The ticket may still display Idea despite the draft; this administrative limit is explicit.+++## User verification and expanded classification scope (2026-10-15)++PR110 at `7f0f41c8` was installed on the explicitly requested Development phone. The user identified the affected entries as legitimate notes without source URLs and explicitly requested classification correction (Sentinel_e00b5a1793b881919abd56be491bd415). T-2462 and `../source-free-note-diagnostics/report.md` track that follow-up within draft PR110. This supersedes the original visibility-only limitation for the classification fix; it does not change the historical results above. No follow-up install or merge is authorized.+++## Owner acceptance and UI waiver++The installed classifier at `5e683fcf` resolved the reported issue according to the user. The user explicitly waived UI testing for this background change and requested merge preparation. See `../source-free-note-diagnostics/report.md` for the acceptance quote, exact-source evidence reuse and current base/review/CI inspection. Existing failure verdicts above remain unchanged; no new tests/builds are authorized or needed for this documentation-only preparation.

Things to double-check

Native capacity pause

Owned group37124 interrupted and verified drained. Core Data temporary-file diagnostics and native compile warnings retained. No simulator runtime or final rendered green verdict.

Acceptance and waiver

User: Seems to have fixed the issue. It is only background; UI tests need not run. Get it ready for merge. This authorizes ready-for-review preparation, not merging.