prism branch T-1713/bugfix-…-search-badges commits 1 files 3 touched lines +153 / -28 tests 91/91 targeted green lint 0 violations

Pre-push review: T-1713 hidden-comment footnote search badges

PR #334 — Fix T-1713: footnote references hidden inside HTML comments produced stale search badge state. The feeder scanned the raw inline source for [^id] while the native count path scanned a comment-visibility-gated source; the fix extracts a single shared scan-source computation so the two can never drift again.

At a glance

  • Root cause: duplicated scan logic — SearchStateFeeder.referencedFootnoteIdentifiers scanned RAW inline source for [^id], while MarkdownBlock.combinedSearchableText scanned a comment-stripped, visibility-gated source. A docstring claimed parity; no shared code enforced it.
  • Fix shape: new MarkdownBlock.footnoteScanComponents(forInlineText:context:) returning FootnoteScanComponents (baseWithoutMarkers, surfacedCommentBodies, joined scanSource); both call sites route through it.
  • Reuse check clean: the reference regex was already single-sourced (footnoteReferenceRegex aliases FootnoteData.referencePattern), no other call site needs the new helper, and the feeder's inline-source traversal matches the native path case-for-case.
  • Parity traced: unresolvable [^id] can never surface a badge (definition(for:) guard + count > 0 filter), and badge ordinal ordering is structurally guaranteed by the shared scanSource.
  • Perf impact bounded: the feeder's extra regex passes run once per 200 ms-debounced query settle, only for footnote-bearing documents.
  • Minor suggestions (not blocking): one of the three new tests does not discriminate pre/post-fix behaviour, and a mixed-visibility case (visible + hidden ref in one paragraph) is untested.

Verdict

Ready to push

All three review passes (reuse, quality, efficiency) found no critical or major issues. The extraction is a faithful, behaviour-preserving refactor of the native path plus a genuine correctness fix for the feeder path; the parity argument was traced end-to-end and holds (resolvability gating, ordinal ordering, inline-source enumeration all match). Targeted verification: 91/91 test-case executions green across the five search/footnote suites, SwiftLint clean. Four minor findings were raised and are reported below as suggestions — none blocks the push.

Review findings

4 raised · 0 fixed · 4 skipped

Jump to findings →

Commits

Three-level explanation

What Changed

Prism documents can contain footnotes ([^1], shown as small tappable badges) and hidden HTML comments (<!-- like this -->). When you search a document, footnote badges light up if the footnote's text contains the search term. Two separate pieces of code were involved: one counts how many matches exist, and another tells the web page which badges to light up.

Why It Matters

The counter correctly ignored footnote references hidden inside comments, but the badge-lighting code read the raw text and lit badges anyway. So a search could report "0 results" while a badge still glowed — confusing and inconsistent. The fix makes both pieces of code read from one shared computation, so they always agree.

Key Concepts

  • Footnote reference: [^1] in the text points at a definition elsewhere; Prism renders it as a badge.
  • HTML comment: <!-- … --> content authors hide; Prism has a toggle to show or hide it.
  • Parity: when two systems must agree, sharing one implementation beats promising agreement in a comment.

Changes Overview

Three files: prism/Models/MarkdownBlock.swift (extraction), prism/Services/SearchStateFeeder.swift (re-route), prismTests/WebRendering/WebSearchBridgeTests.swift (three regression tests).

Implementation Approach

combinedSearchableText (native count path) stripped <!--…--> spans and, only when showHTMLComments is on, appended comment bodies that HTMLCommentParser.parseBlock accepts. SearchStateFeeder.referencedFootnoteIdentifiers re-implemented the [^id] scan on the raw source without that gate — the drift T-1713 describes. The fix extracts the gated computation into MarkdownBlock.footnoteScanComponents(forInlineText:context:), returning a FootnoteScanComponents value (baseWithoutMarkers, surfacedCommentBodies, computed scanSource), and routes both call sites through it. The unified gate has three behaviours: refs in stripped comments contribute nothing when comments are off; refs in surfaced bodies contribute when on; refs in rejected shapes (conditional comments etc.) never contribute.

Trade-offs

The feeder now pays 1–2 extra regex passes per inline source, but only per 200 ms-debounced query settle and only for footnote-bearing documents. Sharing one precomputed per-block components list across the two independently-triggered call graphs (coordinator recompute vs SwiftUI .onChange push) would avoid that, at real invalidation-correctness risk — rightly out of scope for a bugfix PR.

Technical Deep Dive

The extraction is byte-for-byte faithful to the old combinedSearchableText body — including the pre-existing double execution of inlineHTMLCommentRegex (strip + match) when the toggle is on, and the always-true numberOfRanges >= 2 guard (the regex has exactly one capture group). scanSource joins base and bodies with a space; since FootnoteData.referencePattern (\[\^([a-zA-Z0-9_-]+)\]) requires contiguous characters, cross-boundary token merges are impossible. Comment bodies are appended after the base regardless of physical position — inherited behaviour, and now safe by construction: the feeder's within-block ordinal walk assumes the same enumeration order combinedSearchableText uses, which the shared scanSource guarantees.

Architecture Impact

Resolvability was already gated identically on both sides (!footnoteData.isEmpty + definition(for:), plus the feeder's count > 0 filter on matchedFootnoteIds), and the reference regex was already single-sourced — so this fix closes the last drift axis: WHERE the pattern is applied. The helper lives on MarkdownBlock because it needs the private inlineHTMLCommentRegex; internal visibility, no API surface change.

Potential Issues

  • visibleCommentFootnoteRefSurfacesBadge passes against pre-fix code too (the raw scan also found the surfaced ref) — it is parity coverage, not a regression pin; the other two tests do fail pre-fix.
  • No test covers a paragraph with one visible and one hidden ref together, which would also pin the ordering nuance.
  • Per-settle duplicate computation of components across the two call graphs is a known, bounded trade-off; revisit only if profiling shows it on very large footnote-heavy documents.

Important changes — detailed

MarkdownBlock: extract footnoteScanComponents as the single scan-source truth

prism/Models/MarkdownBlock.swift

Why it matters. This is the fix's core: the comment-visibility gate that decides WHERE a [^id] reference may count now exists exactly once. combinedSearchableText expands footnote definitions from it, and the feeder derives badge ids from it, so badge state can no longer drift from the native match count.

What to look at. MarkdownBlock.swift — FootnoteScanComponents struct and footnoteScanComponents(forInlineText:context:), roughly lines 948-1005; combinedSearchableText rewritten on top of it

Takeaway. When two subsystems must agree and a docstring promises they do, extract the shared computation instead — a comment claiming parity is a bug factory; a shared function is a guarantee.
Rationale. Stated in the commit message: the root cause was duplicated scan logic missing the comment-visibility gate the canonical copy had; the extraction makes the parity structural rather than promised.

SearchStateFeeder: route referencedFootnoteIdentifiers through the shared scan source

prism/Services/SearchStateFeeder.swift

Why it matters. This is where the bug lived: the raw-source scan saw [^id] refs inside hidden comments (comments off) and inside rejected comment shapes (comments on), marking badges the native model counted zero matches for — violating the search/DOM parity contract (Req 6.1/7.2).

What to look at. SearchStateFeeder.swift:200-221 — referencedFootnoteIdentifiers now scans MarkdownBlock.footnoteScanComponents(forInlineText:context:).scanSource per inline source

Takeaway. Downstream gating already protected against unresolvable ids (definition(for:) guard + count > 0 filter); the drift was purely in the scan source. Fixing the one divergent input is enough — no payload-shape change needed.
Rationale. Stated in the commit message and the updated doc comment: each inline source routes through the shared helper so hidden-comment refs contribute nothing when comments are off, and rejected shapes never contribute.

WebSearchBridgeTests: three regression tests pin the comment-gate behaviours

prismTests/WebRendering/WebSearchBridgeTests.swift

Why it matters. Two of the three tests (hidden-comment ref with comments off; ref inside a rejected conditional-comment shape with comments on) fail against pre-fix code and pin the exact T-1713 regression. The third (surfaced ref with comments on) passes pre-fix too — it is positive parity coverage, not a discriminating pin.

What to look at. WebSearchBridgeTests.swift:125-197 — hiddenCommentFootnoteRefProducesNoState, visibleCommentFootnoteRefSurfacesBadge, rejectedCommentShapeFootnoteRefProducesNoState

Takeaway. Assert both halves of a parity contract in one test: the native count (SearchService.countMatches) and the emitted payload (SearchStateFeeder.buildStates) checked against each other, not against independently hardcoded expectations.
Open question. Rationale not stated by the author and not inferable from the diff.

Key decisions

Extract a shared components value rather than passing precomputed text between subsystems.

The alternative — having SearchCoordinator compute scan sources once and hand them to SearchStateFeeder — would entangle two independently-triggered call graphs (coordinator recompute vs the SwiftUI .onChange push in DocumentScrollContent) and add invalidation-correctness risk. The extraction keeps both call sites free to run independently while structurally guaranteeing they gate identically.

(inferred — not stated by the author.)
Accept duplicate per-block component computation per search settle instead of caching.

During one debounced query settle, combinedSearchableText (count path) and referencedFootnoteIdentifiers (payload path) each call footnoteScanComponents for the same inline text. The duplicated cost is bounded to the comment-regex portion, gated on !footnoteData.isEmpty, and runs once per 200 ms settle — caching would be overkill for a bugfix PR.

(inferred — not stated by the author.)
House the helper on MarkdownBlock rather than a dedicated type.

The computation needs MarkdownBlock's private inlineHTMLCommentRegex; a standalone helper (mirroring HTMLCommentStripping) would also have worked but would force widening that regex's visibility. Both the struct and the function are internal — no API-surface change.

(inferred — not stated by the author.)
Carry over the double comment-regex execution unchanged.

footnoteScanComponents runs inlineHTMLCommentRegex twice when the toggle is on (strip via stringByReplacingMatches, then extract via matches). This is a byte-for-byte relocation of the pre-existing combinedSearchableText body — collapsing it into a single strip+capture pass is a separate change, correctly kept out of a bugfix diff.

(inferred — not stated by the author.)

Review findings

SeverityAreaFindingResolution
minorWebSearchBridgeTests.visibleCommentFootnoteRefSurfacesBadgeThe test passes against pre-fix code too: the old raw-source scan also found the surfaced [^1] when comments are on, so this test verifies real behaviour but does not discriminate the T-1713 fix. Tests 1 and 3 do fail pre-fix and carry the regression pin.Reported for the author. Keep the test as positive parity coverage; optionally note that in its comment or augment it with a case only the fix gets right. Report-only review — test files not modified.
minorTest coverage — mixed visibilityNo test covers a paragraph with one visible [^1] and one hidden [^2] together (e.g. "See note.[^1] <!-- hidden [^2] -->" with comments off). That case would pin both that the visible ref still counts and that the hidden ref is excluded, and would exercise the bodies-appended-after-base ordering nuance.Reported for the author as a suggested follow-up test (plus its comments-on counterpart). Report-only review — test files not modified.
minorSearchStateFeeder hot pathThe feeder path now performs the comment-strip regex (plus comment-extract when the toggle is on) per inline source, where it previously did a single reference-regex pass. Bounded: runs once per 200 ms-debounced query settle, gated on !footnoteData.isEmpty, over typically short inline strings.Accepted as a correctness-for-cost trade-off; no action. Revisit only if profiling shows it on very large footnote-heavy documents.
minorCross-subsystem duplicate computationPer search settle, SearchCoordinator.recomputeMatchCounts and SearchStateFeeder.buildStates each independently compute footnoteScanComponents for the same blocks. Duplicated cost is limited to the comment-regex portion (stripMarkdownFormatting and definition expansion are not duplicated).Accepted; sharing a precomputed per-block components list across the two independently-triggered call graphs would add invalidation risk disproportionate to the saving. Suggested shape recorded in the decisions section if profiling ever warrants it.

Per-file diffs

Click to expand.

prism/Models/MarkdownBlock.swift Modified +69 / -25
diff --git a/prism/Models/MarkdownBlock.swift b/prism/Models/MarkdownBlock.swiftindex 16c7c11..d03f6b6 100644--- a/prism/Models/MarkdownBlock.swift+++ b/prism/Models/MarkdownBlock.swift@@ -915,29 +915,14 @@ nonisolated enum MarkdownBlock: Identifiable, Equatable, Sendable {         forInlineText text: String,         context: SearchContext     ) -> String {-        let nsText = text as NSString-        let fullRange = NSRange(location: 0, length: nsText.length)--        // Strip comment markers from the base so search never matches-        // on raw `<!--…-->`. Same input is used to extract inner text-        // below when the toggle is ON.-        let baseWithoutMarkers = inlineHTMLCommentRegex.stringByReplacingMatches(-            in: text, range: fullRange, withTemplate: ""-        )-        var result = stripMarkdownFormatting(baseWithoutMarkers)--        // Bodies of comment spans the renderer actually surfaces. Collected-        // once so the search append and the footnote-reference scan below-        // agree on which comment content is "visible".-        var surfacedBodies: [String] = []-        if context.showHTMLComments {-            let matches = inlineHTMLCommentRegex.matches(in: text, range: fullRange)-            for match in matches where match.numberOfRanges >= 2 {-                let span = nsText.substring(with: match.range)-                guard let body = HTMLCommentParser.parseBlock(span) else { continue }-                surfacedBodies.append(body)-                result += " " + body-            }+        let components = footnoteScanComponents(forInlineText: text, context: context)+        var result = stripMarkdownFormatting(components.baseWithoutMarkers)++        // Append the bodies of comment spans the renderer actually surfaces+        // (toggle ON only) so comment content is searchable exactly when it+        // is visible.+        for body in components.surfacedCommentBodies {+            result += " " + body         }          // Append footnote definition content for any resolvable `[^id]`@@ -948,9 +933,8 @@ nonisolated enum MarkdownBlock: Identifiable, Equatable, Sendable {         // references inside rendered comment ranges contribute consistently         // with the render — but references inside rejected (never-rendered)         // comment shapes still contribute nothing.-        let footnoteScanSource = ([baseWithoutMarkers] + surfacedBodies).joined(separator: " ")         if !context.footnoteData.isEmpty {-            for match in footnoteScanSource.matches(of: footnoteReferenceRegex) {+            for match in components.scanSource.matches(of: footnoteReferenceRegex) {                 let identifier = String(match.1)                 if let definition = context.footnoteData.definition(for: identifier) {                     result += " " + stripMarkdownFormatting(definition.content)@@ -961,6 +945,66 @@ nonisolated enum MarkdownBlock: Identifiable, Equatable, Sendable {         return result     } +    // MARK: - Footnote scan source (shared with SearchStateFeeder)++    /// The comment-visibility-gated pieces of an inline source that are+    /// eligible for footnote-reference scanning.+    ///+    /// Single source of truth for WHERE a `[^id]` reference may count:+    /// ``combinedSearchableText(forInlineText:context:)`` expands footnote+    /// definitions from it, and `SearchStateFeeder` derives+    /// `matchedFootnoteIds` from it — so the badges the web search payload+    /// marks can never drift from the references the native search model+    /// counts (T-1713).+    struct FootnoteScanComponents {+        /// The inline source with every `<!--…-->` span removed. References+        /// hidden inside a comment never appear here (T-1365).+        let baseWithoutMarkers: String+        /// Bodies of comment spans the renderer actually surfaces: populated+        /// only when `showHTMLComments` is ON, and only for shapes+        /// `HTMLCommentParser.parseBlock` accepts — rejected shapes+        /// (conditional comments, note-infrastructure tags, empty bodies)+        /// are never rendered, so they contribute nothing.+        let surfacedCommentBodies: [String]++        /// The joined text to scan for `[^id]` references.+        var scanSource: String {+            ([baseWithoutMarkers] + surfacedCommentBodies).joined(separator: " ")+        }+    }++    /// Splits an inline source into the comment-gated pieces eligible for+    /// footnote-reference scanning, honouring `context.showHTMLComments`.+    static func footnoteScanComponents(+        forInlineText text: String,+        context: SearchContext+    ) -> FootnoteScanComponents {+        let nsText = text as NSString+        let fullRange = NSRange(location: 0, length: nsText.length)++        // Strip comment markers from the base so scans never see content+        // inside a `<!--…-->` range. Same input is used to extract inner+        // text below when the toggle is ON.+        let baseWithoutMarkers = inlineHTMLCommentRegex.stringByReplacingMatches(+            in: text, range: fullRange, withTemplate: ""+        )++        var surfacedBodies: [String] = []+        if context.showHTMLComments {+            let matches = inlineHTMLCommentRegex.matches(in: text, range: fullRange)+            for match in matches where match.numberOfRanges >= 2 {+                let span = nsText.substring(with: match.range)+                guard let body = HTMLCommentParser.parseBlock(span) else { continue }+                surfacedBodies.append(body)+            }+        }++        return FootnoteScanComponents(+            baseWithoutMarkers: baseWithoutMarkers,+            surfacedCommentBodies: surfacedBodies+        )+    }+     // MARK: - Compatibility shims      /// Legacy plain-text search builder.
prism/Services/SearchStateFeeder.swift Modified +10 / -3
diff --git a/prism/Services/SearchStateFeeder.swift b/prism/Services/SearchStateFeeder.swiftindex 077fe3c..c497492 100644--- a/prism/Services/SearchStateFeeder.swift+++ b/prism/Services/SearchStateFeeder.swift@@ -198,15 +198,22 @@ enum SearchStateFeeder {     }      /// The footnote identifiers referenced within a block, in source order, honouring-    /// the comment-visibility gate the same way `combinedSearchableText` does (a-    /// reference inside a hidden comment contributes nothing when comments are off).+    /// the comment-visibility gate the same way `combinedSearchableText` does: each+    /// inline source is routed through the shared+    /// `MarkdownBlock.footnoteScanComponents(forInlineText:context:)`, so a reference+    /// inside a hidden `<!--…-->` range contributes nothing when comments are off,+    /// and — when comments are on — only references in surfaced comment bodies+    /// count, never ones in rejected (never-rendered) comment shapes (T-1713).     private static func referencedFootnoteIdentifiers(         in block: MarkdownBlock,         context: SearchContext     ) -> [String] {         var identifiers: [String] = []         for source in inlineSources(of: block) {-            for match in source.matches(of: FootnoteData.referencePattern) {+            let scanSource = MarkdownBlock+                .footnoteScanComponents(forInlineText: source, context: context)+                .scanSource+            for match in scanSource.matches(of: FootnoteData.referencePattern) {                 identifiers.append(String(match.1))             }         }
prismTests/WebRendering/WebSearchBridgeTests.swift Modified +74 / -0
diff --git a/prismTests/WebRendering/WebSearchBridgeTests.swift b/prismTests/WebRendering/WebSearchBridgeTests.swiftindex 360e519..f387548 100644--- a/prismTests/WebRendering/WebSearchBridgeTests.swift+++ b/prismTests/WebRendering/WebSearchBridgeTests.swift@@ -122,6 +122,80 @@ struct WebSearchBridgeTests {         #expect(state?.matchedFootnoteIds == ["1"])     } +    // MARK: - T-1713: comment-gated footnote refs and badge-state parity++    @Test("A footnote ref hidden inside an HTML comment produces no badge state when comments are off")+    func hiddenCommentFootnoteRefProducesNoState() {+        // "alpha" matches only footnote 1's content, and the only [^1] reference+        // sits inside a `<!-- … -->` comment. With showHTMLComments == false the+        // native search model strips the comment before expanding footnotes, so it+        // counts zero matches — the web payload must agree and emit NO state.+        // Before the T-1713 fix, the feeder scanned the RAW source, saw the hidden+        // [^1], and emitted a stale matchedFootnoteIds: ["1"] state.+        let blocks: [MarkdownBlock] = [+            .paragraph(markdown: "Visible text. <!-- hidden [^1] -->"),+        ]+        let context = footnoteContext() // showHTMLComments: false+        let counts = blocks.map { SearchService.countMatches(query: "alpha", in: $0, context: context) }+        #expect(counts[0] == 0)++        let states = SearchStateFeeder.buildStates(+            query: "alpha",+            blocks: blocks,+            matchCountsPerBlock: counts,+            currentGlobalMatchIndex: nil,+            context: context+        )+        #expect(states.isEmpty)+    }++    @Test("A footnote ref inside a surfaced HTML comment contributes badge state when comments are on")+    func visibleCommentFootnoteRefSurfacesBadge() throws {+        // Same paragraph, but with showHTMLComments == true the comment body is+        // surfaced, its [^1] expands footnote 1's content natively (1 match for+        // "alpha"), and the feeder must mark the badge — as a footnote match, not+        // a text match.+        let blocks: [MarkdownBlock] = [+            .paragraph(markdown: "Visible text. <!-- hidden [^1] -->"),+        ]+        let context = SearchContext(showHTMLComments: true, footnoteData: footnoteData())+        let counts = blocks.map { SearchService.countMatches(query: "alpha", in: $0, context: context) }+        #expect(counts[0] == 1)++        let states = SearchStateFeeder.buildStates(+            query: "alpha",+            blocks: blocks,+            matchCountsPerBlock: counts,+            currentGlobalMatchIndex: nil,+            context: context+        )+        let state = try #require(states.first)+        #expect(state.textMatchCount == 0)+        #expect(state.matchedFootnoteIds == ["1"])+    }++    @Test("A footnote ref inside a rejected comment shape produces no badge state even when comments are on")+    func rejectedCommentShapeFootnoteRefProducesNoState() {+        // Conditional comments are never surfaced by HTMLCommentParser, so a [^1]+        // inside one must not expand its definition — the native model counts zero+        // matches even with the toggle ON, and the payload must carry no state.+        let blocks: [MarkdownBlock] = [+            .paragraph(markdown: "Visible text. <!--[if IE]> [^1] <![endif]-->"),+        ]+        let context = SearchContext(showHTMLComments: true, footnoteData: footnoteData())+        let counts = blocks.map { SearchService.countMatches(query: "alpha", in: $0, context: context) }+        #expect(counts[0] == 0)++        let states = SearchStateFeeder.buildStates(+            query: "alpha",+            blocks: blocks,+            matchCountsPerBlock: counts,+            currentGlobalMatchIndex: nil,+            context: context+        )+        #expect(states.isEmpty)+    }+     // MARK: - current-match addressing (text ordinal vs footnote badge id)      @Test("Current match in the text portion is addressed by text ordinal")

Things to double-check

Comment-body ordering vs badge ordinals.

scanSource appends surfaced comment bodies after the base regardless of their physical position in the source (a <!-- [^2] --> then [^1] paragraph yields order ["1","2"]). This is inherited from combinedSearchableText, and the feeder's within-block ordinal walk now shares the same enumeration by construction — but if a future change makes badge ordinals position-sensitive, this is the spot to revisit.

Verification basis.

This review ran the five targeted suites the commit names (WebSearchBridgeTests, WebSearchParityTests, MarkdownBlockSearchableTextTests, FootnoteSearchTests, SearchServiceTests): 91/91 test-case executions green via Tools/check-test-results.sh, plus make lint clean. The commit message additionally reports 903/903 across a wider sweep and a clean make build-macos; the full suite was not re-run here per review constraints.