Second review of PR #395. The first returned Needs fixes: an exponential HTMLCommentParser.structuralRegex on the same block input, a residual outer quadratic in stripCommentsInsideLinkLabels, four stale forward-references, and no decision-log entry. Commit 4b8c79d0 addresses every one of them, and I verified the two replacements independently — 500,000 differential fuzz rounds against the retired regexes, zero divergence. One claim is still not true: the same function the CHANGELOG names as linearised still front-ends the comment scan with a quadratic \[([^\]]*)\], and neither new growth guard can see it.
parseBlock output; stripCommentsInsideLinkLabels output) against verbatim reconstructions of the retired regexes — zero divergence. The alphabet included every whitespace class the brief asked about: NBSP, U+2028, U+2029, U+000B, U+0085, U+3000, ZWSP, U+FEFF.\s / \S question resolves in the code's favour. ICU's \S is by definition the complement of \s, so taking the gap test from the same engine makes drift structurally impossible — and the doc comment's specific claim that \s matches U+000B, U+0085 and U+00A0 is true (I probed 18 scalars; ZWSP, U+FEFF and U+180E are correctly not whitespace under either).parseBlock on "<!--a--> "×k + "x": 217 B was 3.5 s, now 0.011 ms; 180 KB now 4.7 ms. stripCommentsInsideLinkLabels on comment-bearing labels: 480 KB was 1.9 s, now 25.4 ms, 2.0x per doubling.linkLabelRegex (MarkdownBlockParser.swift:117) is quadratic on a run of [ with no ] — 8.4 s at 32,000 brackets, 6.2 s on a 96 KB paragraph of ordinary prose containing unmatched [. It is the first pass of the very function the CHANGELOG calls linear, and G13/G15 are blind to it because both fixtures close their bracket.CommentMatch: Equatable, Sendable, removing(_:from:) so the toggle-ON path scans once instead of twice, G13 refixtured to reach the mutating branch with the old no-op fixture preserved as G15, and Decision 22 written to the project's format with the single-owner alternative recorded.retiredParseBlock reproduces origin/main's function field-for-field — both .dotMatchesLineSeparators options, the CDATA/DOCTYPE guard, both reject regexes, the separator append and both trims. The only textual difference is one inlined local.Needs fixes
Everything the first review raised is fixed, and the fixes are right. I re-derived both replacements from scratch rather than trusting the diff: a 200,000-round differential fuzz of the new isCommentOnly classifier against the retired ^(?:\s*<!--(?:(?!-->).)*?-->\s*)+$ plus <!--((?:(?!-->).)*?)--> pair (accept/reject verdict and both span lists), a 300,000-round fuzz of the whole parseBlock output, and a 300,000-round fuzz of stripCommentsInsideLinkLabels old-vs-new — zero divergence in any of them, over an alphabet carrying NBSP, U+2028, U+000B, U+0085, ZWSP, U+FEFF, emoji and combining marks. The whitespace question the review was asked to settle resolves cleanly: ICU's \S is the exact complement of the \s the retired pattern used, so the gap test cannot drift, and the doc comment's specific claim (\s matches U+000B, U+0085 and U+00A0) is correct — I probed all eighteen candidate scalars. The exponential is gone: 217 bytes went from 3.5 s to 0.011 ms, and 180 KB now measures 4.7 ms. The link-label rebuild is gone: 480 KB from 1.9 s to 25 ms, 2.0x per doubling. make lint 0/555, make build-macos Build Succeeded with no new warnings, make verify-test-isolation OK. All four stale references are corrected, and Decision 22 is a well-formed Enhanced Nygard entry that records the alternative the review asked for.
What holds the push is the same class of finding as last time, one layer further in. MarkdownBlockParser.stripCommentsInsideLinkLabels is the step the CHANGELOG names as "removing comments from link labels", and its first pass — linkLabelRegex = \[([^\]]*)\] at MarkdownBlockParser.swift:117, unchanged by this PR — is quadratic on a run of [ with no ]. Measured on the production pattern: 2,000 → 32 ms, 4,000 → 130 ms, 8,000 → 517 ms, 16,000 → 2.1 s, 32,000 → 8.4 s, a clean 4x per doubling; and on ordinary prose shaped "see [ note. " repeated, a 96 KB paragraph takes 6.2 s. This runs on every formatted paragraph (:605, :705, :794, :807, :870) — the same URL-reachable path the ticket's threat model names. Neither G13 nor G15 can see it, because both fixtures close their bracket. So the CHANGELOG's "Every one of these steps now reads the document once, from left to right, and grows in step with its length rather than with its square", the decision log's "G10–G16 pin the linear behaviour of every affected call site", and the commit message's "the CHANGELOG entry … is now exactly true" are all still overclaims. Pre-existing, not a regression — but the sentence is what this PR ships.
Everything else is small: two factual slips inside Decision 22 that were copied into production source, a doc line claiming "the last lazy-wildcard pattern in the app is gone" when seven survive in MarkdownBlock.swift (one of them cubic, at 680 ms on 1.6 KB), and an undocumented same-string precondition on removing(_:from:).
cb5c75e3 Fix T-2147: linearise HTMLCommentStripping's quadratic <!--…--> scan 8ecc70ae Fix T-2147: widen comment-scan tests with non-ASCII goldens and fuzz 4b8c79d0 Fix T-2147: retire the exponential structural regex and the per-label rebuild An HTML comment looks like <!-- this is a comment -->. Prism has to find them in several places, and it used to find them with regular expressions — small pattern-matching programs. The first two commits on this branch replaced one such pattern with a simple left-to-right walk. This third commit replaces two more.
The first review measured the code and found the fix had not reached the worst offender. Deciding whether a block of HTML is nothing but comments used a pattern that had to try every possible way of dividing up the spaces between the comments. Every extra comment doubled the number of divisions to try. A document of 217 bytes — a handful of perfectly ordinary comments and one other character — took three and a half seconds. At 235 bytes it took twenty-two. There was no upper limit.
The second problem was smaller but on the same path: when a comment sat inside a link's label, the code rebuilt the entire document from scratch to remove it, once per label. A 480 KB paragraph took nearly two seconds.
Both are replaced by the same left-to-right walk the earlier commits introduced. A block is comments-only if it contains at least one comment and everything outside the comments is blank space — one pass answers both. Link labels are now assembled in a single pass, copying each untouched stretch once, instead of rebuilding the document per label.
The 217-byte document now takes 0.011 milliseconds instead of 3.5 seconds. The 480 KB paragraph takes 25 milliseconds instead of 1,900.
The description Prism ships to users says all of these steps now read the document once. One of them does not. Before it can look for comments inside a link label, the code first has to find the labels — and that step still uses a pattern with the same failure mode. Give it a paragraph containing a lot of [ characters and no ], and it slows down with the square of the length: eight seconds for 32 KB. That is not a new problem, and this branch did not create it, but the sentence saying it is fixed is new.
Two more NSRegularExpression constants leave HTMLCommentParser: the structural validator ^(?:\s*<!--(?:(?!-->).)*?-->\s*)+$ and the extractor <!--((?:(?!-->).)*?)-->. Both are replaced by a classifier over the HTMLCommentStripping.matches(in:) scan the earlier commits added:
matches(in:) reports leftmost-first, non-overlapping spans, each ending at the first --> after its opener.isCommentOnly walks those spans and tests every gap between them with a single anchorless \S match — the block is comment-only iff there is at least one span and no gap holds a non-whitespace character.Taking the gap test from the regex engine's own \S rather than a hand-rolled CharacterSet is the load-bearing choice: \S is defined as the complement of \s, so the whitespace definition is identical by construction to the \s* runs it replaces. A CharacterSet would have been free to drift — Foundation's .whitespacesAndNewlines and ICU's \s genuinely disagree (U+0085 is in both, U+180E is in neither, and the two sets are not derived from one another).
Separately, stripCommentsInsideLinkLabels stops rebuilding the whole document per mutating label. The old loop walked matches right-to-left calling replacingCharacters on the accumulating result; the new one walks left-to-right appending [copied, labelStart) verbatim and then the stripped label, with copied > 0 as the "nothing changed" sentinel.
Derive the complement from the engine, don't restate it. The whole class of "my whitespace set and the regex's whitespace set disagree" bugs disappears when the replacement asks the same engine the same question.
Keep the retired implementation as a test-only oracle — the whole function, not the pattern. HTMLCommentParserStructuralEquivalenceTests reproduces parseBlock in full: both regex options, the CDATA/DOCTYPE guard, both reject passes, the separator append, both trims. Reproducing only the regexes would have compared a fragment and left the surrounding logic unpinned.
When a growth guard is added, check its fixture reaches the branch. G13 originally used "[" + "<!--"×n + "]" — a label whose strip is a no-op, so the mutating branch never ran and the guard measured nothing. It now uses "[a<!--c-->](u) "×n, and the old fixture is kept as G15 for the branch it does cover. That split is the right response to "this guard was blind", better than replacing it.
The classifier now costs an O(n) scan and a CommentMatch array before it can reject a non-comment block, where ^\s*<!-- rejected in O(1). No complexity regression, but a constant-factor cost paid on every raw-HTML block in every document.
The retired validator is ^(?:\s*<!--(?:(?!-->).)*?-->\s*)+$ with .dotMatchesLineSeparators, so (?:(?!-->).)*? is [\s\S]*? plus a per-position negative lookahead. That lookahead is what makes the equivalence structural rather than incidental: a comment body can never contain --> at any offset, so a span beginning at an opener cannot extend past the first -->. The factorisation is therefore forced and unique, and backtracking can only vary (a) the number of iterations and (b) how each inter-comment gap is split between one iteration's trailing \s* and the next one's leading \s*. Neither varies the spans. Hence: the validator accepts iff the canonical leftmost factorisation covers the input with all-\s gaps — which is exactly what isCommentOnly tests, and which is exactly the span list matches(in:) computes. Backwards is the same argument: \s* can consume any run every character of which matches \s, and \S is the complement, so an accepted gap is always consumable.
The one place I expected a divergence and did not find one is ICU's $, which matches at end-of-input and before a final line terminator. It cannot separate the two implementations, because the only character it can leave unconsumed is the terminator itself, and every line terminator is in \s.
Empirically: 200,000 rounds comparing accept/reject and full/inner span lists, then 300,000 rounds comparing complete parseBlock return values (note and conditional rejects included), over an alphabet of <!--, -->, --!>, <!-->, <!--->, <![CDATA[, <!DOCTYPE, [if], [endif], comment:ab12, /comment:ff, tab, LF, CRLF, NBSP, U+2028, U+000B, U+0085, ZWSP, U+FEFF, emoji and e+U+0301. 4,376 accepting inputs exercised. Zero mismatches.
ICU \s, probed scalar by scalar through the production regex: TAB, LF, VT (U+000B), FF, CR, SP, NEL (U+0085), NBSP (U+00A0), U+1680, U+2000, U+2028, U+2029, U+202F, U+205F, U+3000 are in; ZWSP (U+200B), U+FEFF and U+180E are out. The doc comment at HTMLCommentParser.swift:33-38 names exactly U+000B, U+0085 and U+00A0 as the interesting cases, and is correct on all three. Retired and new agree on every one.
stripCommentsInsideLinkLabels is still two passes, and only the second was linearised. linkLabelRegex = \[([^\]]*)\] runs first. On input with [ and no ], position p matches \[, [^\]]* consumes to end-of-input, \] fails, and backtracking is pointless because every character it gives back is itself in [^\]] — so ICU pays O(n − p) and then retries at p+1. Summed: O(n²). Measured on the production pattern (fastest of one run each, -O): 2,000 → 32.2 ms; 4,000 → 130.4; 8,000 → 516.7; 16,000 → 2,074.9; 32,000 → 8,367.1. Escaped as \[ (what Paragraph.format() may emit) it is the same curve at half the density: 32 KB → 4.1 s. On prose-shaped input "see [ note. "×n, 96 KB → 6.2 s.
The fix is the same insight the comment scan uses and would be four lines: if the forward search for ] exhausts the input from position p, no q > p can find one either, so stop. As it stands the function is linear in the number of labels and quadratic in the number of unclosed brackets, and both growth guards fixture a closing bracket.
HTMLCommentParser.swift:86 now asserts "the last lazy-wildcard pattern in the app is gone". MarkdownBlock.swift:1130-1136 holds seven of them, and linkRegex (\[(.+?)\]\(.+?\), used by stripMarkdownFormatting on the searchable-text path) is not quadratic but cubic: 200 B → 1.6 ms, 400 B → 10.9, 800 B → 86.5, 1,600 B → 680.2 — 8x per doubling on "[a]("×n. Nothing to do with T-2147, but the sentence should not be shipped as written.
removing(_:from:) documents that matches must be non-overlapping and increasing; violating that yields a negative-length NSRange and an uncatchable NSRangeException, which is loud. The precondition it does not document — that the matches were computed from this same text — is the one that fails quietly, producing a plausible-looking wrongly-stripped string. One production call site today, taking its matches from the adjacent line.
HTMLCommentParser.swift
Why it matters. The headline fix of this commit, and the one the first review blocked on. structuralRegex was exponential, not quadratic: a 235-byte document of ordinary well-formed comments measured 22 s. It ran on whole raw-HTML blocks ten lines before the already-linearised strip, so the ticket's own threat model reached it. It is now isCommentOnly over matches(in:) plus one anchorless \S per gap; extractRegex goes with it, because the same span list serves both.
What to look at. HTMLCommentParser.swift:29-42 (nonWhitespaceRegex), :64-110 (isCommentOnly / isWhitespace), :119-124 (parseBlock's new preamble)
MarkdownBlockParser.swift
Why it matters. The outer factor the first two commits left standing. The old loop called replacingCharacters on the whole accumulating result once per mutating label — O(document x comment-bearing labels), 1.9 s on a 480 KB paragraph. Now 25.4 ms, 2.0x per doubling. I fuzzed old against new over 300,000 inputs (nested and adjacent labels, label at offset 0, empty label, no-op strip, emoji): identical output every time.
What to look at. MarkdownBlockParser.swift:127-157
MarkdownBlock.swift
Why it matters. New API HTMLCommentStripping.removing(_:from:) lets the toggle-ON branch derive the stripped base from the match list it already needs, instead of walking the text to strip and again to locate. The toggle-OFF branch deliberately keeps the array-free stripHTMLComments. A reviewer should confirm the two produce byte-identical output — they do; both run the same forward scan, and I fuzzed removing(matches(in:s), from: s) against stripHTMLComments(s) and against the retired regex replacement over 300,000 inputs.
What to look at. MarkdownBlock.swift:1069-1090; HTMLCommentStripping.swift:126-147
HTMLCommentStrippingTests.swift
Why it matters. The first review found G13 blind: its fixture produced a label whose strip is a no-op, so `strippedLabel != label` was false and the guard measured a branch that never ran. The response is the right one — G13 moves to "[a<!--c-->](u) " x n (which does rewrite), the old fixture survives as G15 for the per-label scan it genuinely pins, and G16 is added for the exponential shape. All three assert a growth RATIO, not a millisecond budget.
What to look at. HTMLCommentStrippingTests.swift:253-329
CHANGELOG.md
Why it matters. This is what holds the push. The entry says "Every one of these steps now reads the document once, from left to right, and grows in step with its length rather than with its square", and names "removing comments from link labels" as one of them. That function's FIRST pass, linkLabelRegex = \[([^\]]*)\] at MarkdownBlockParser.swift:117, is quadratic on a run of [ with no ] — 8.4 s at 32,000 brackets, 6.2 s on a 96 KB paragraph of ordinary prose. Decision 22's "G10-G16 pin the linear behaviour of every affected call site" and the commit message's "the CHANGELOG entry is now exactly true" carry the same overclaim.
What to look at. CHANGELOG.md:25; decision_log.md:746; MarkdownBlockParser.swift:117
Decision 22 records the narrow alternative — ^(?:\s*<!--(?:(?!-->).)*?-->)+\s*$, a one-character-class move that removes the blow-up — and rejects it because it would leave a third lazy-wildcard pattern in the file with no oracle, and because the scan formulation deletes the extractor as well. That reasoning holds: the file's own counter-example is that structuralRegex already used the non-swallowing body and was still the worst performer.
Recorded in Decision 22 and in the doc comment at HTMLCommentParser.swift:29-38. This is the single most important line in the change and the reason the classifier is exactly equivalent rather than approximately so — \S is defined as the complement of \s, so the definition cannot drift from the \s* runs it replaces. I verified the specific scalars the comment names.
The new suite reproduces origin/main's parseBlock field-for-field, including both .dotMatchesLineSeparators options, the CDATA/DOCTYPE prefix guard, both reject regexes, the separator append and both trims — I diffed it against git show origin/main: and found one inlined local as the only difference. Reproducing the patterns alone would have compared a fragment and left the surrounding logic unpinned. The residual risk is that the copy is maintained by inspection only, with nothing keeping it in step.
The blind fixture is preserved as G15 with a comment saying which factor it pins. Keeping a guard that was merely mislabelled — rather than deleting it — is the right call: the per-label scan it covers is real coverage, it just was not the coverage the title claimed.
(inferred — not stated by the author.)Justified in the test comment on the grounds that the restored rebuild measures 11.7x at 8,000 but only 10.1x at 4,000, so the extra 0.3 s buys 2x clearance over the 8x ceiling. Reasonable, and unverifiable from the file as committed.
| Severity | Area | Finding | Resolution |
|---|---|---|---|
| major | MarkdownBlockParser.swift:117 + CHANGELOG.md:25 + decision_log.md:746 — a quadratic remains inside a step now described as linear | stripCommentsInsideLinkLabels is two passes and this commit linearised the second. The first, linkLabelRegex = \[([^\]]*)\] (unchanged by this PR), is quadratic on a run of [ with no ]: at each [ the negated class consumes to end-of-input, \] fails, and every character backtracking gives back is itself in [^\]] — so the position costs O(n-p) and the scan retries at p+1. MEASURED on the exact production pattern, -O, fastest of one: 2,000 -> 32.2 ms; 4,000 -> 130.4 ms; 8,000 -> 516.7 ms; 16,000 -> 2,074.9 ms; 32,000 -> 8,367.1 ms. Clean 4x per doubling. Escaped as \[ (what Paragraph.format() may emit): 32 KB -> 4.1 s. On ORDINARY PROSE shaped "see [ note. " x n, a 96 KB paragraph takes 6.2 s. The function runs on every formatted paragraph, blockquote, list item and table cell (MarkdownBlockParser.swift:605, :705, :794, :807, :870) — the same URL-reachable path the ticket's threat model names. Neither new growth guard can see it: G13's fixture is "[a<!--c-->](u) " x n and G15's is "[" + "<!--" x n + "]", and both close their bracket. So three artefacts overclaim: CHANGELOG.md:25 ("Every one of these steps now reads the document once, from left to right, and grows in step with its length rather than with its square"), decision_log.md:746 ("G10-G16 pin the linear behaviour of every affected call site"), and the commit message of 4b8c79d0 ("the CHANGELOG entry is now exactly true"). Independently found by a second reviewer on this pass. | Editorial review — no fix applied. Pre-existing, not a regression. Two acceptable resolutions: (a) narrow the three claims to the passes actually fixed and file the residual as its own ticket, or (b) fix it here with the same insight the comment scan already uses — if the forward search for ] exhausts the input from position p, no q > p can find one either, so stop. Option (b) is roughly four lines and would let G13/G15 gain a fixture with an unclosed bracket, which is the coverage neither currently has. Note that a possessive/atomic rewrite ([^\]]*+) does NOT fix it: it removes the backtracking but not the per-start-position rescan, so the sum is still O(n^2). |
| minor | decision_log.md:753 and HTMLImageParser.swift:106-108 — wrong file count, stated twice | "HTMLImageParser.stripHTMLComments can no longer be narrowed back to private: three production sites in two other files call it." The three callers live in three other files, not two: prism/Services/HTMLCommentStripping.swift:59, prism/Services/MarkdownBlockParser.swift:147, and prism/Models/MarkdownBlock.swift:1090. The same wrong count was copied verbatim into the production doc comment at HTMLImageParser.swift:106-108, so it now ships in the source as well as the log. | Editorial review — no fix applied. Change "two other files" to "three other files" in both places. |
| minor | HTMLCommentParser.swift:86 — "the last lazy-wildcard pattern in the app is gone" is false | The doc comment closes: "So one scan now serves both, and the last lazy-wildcard pattern in the app is gone." Seven survive in prism/Models/MarkdownBlock.swift:1130-1136 (stripMarkdownFormatting's boldStarRegex, boldUnderscoreRegex, italicStarRegex, italicUnderscoreRegex, inlineCodeRegex, linkRegex, strikethroughRegex), and linkRegex = \[(.+?)\]\(.+?\) is not quadratic but CUBIC: measured on "[a](" x n, 200 B -> 1.64 ms, 400 B -> 10.92 ms, 800 B -> 86.54 ms, 1,600 B -> 680.19 ms — 8x per doubling, so a 1.6 KB paragraph costs 0.68 s and a 6.4 KB one roughly 44 s. That path (stripMarkdownFormatting over combinedSearchableText) is the "text Prism searches" the CHANGELOG entry names, though it is not a comment-scanning step and is squarely outside this ticket. Decision 22's own wording is correctly scoped ("No lazy-wildcard <!--…--> pattern over whole-block input survives"); the source comment and the commit message are not. | Editorial review — no fix applied. Narrow the sentence to the <!--…--> family as Decision 22 already does, and consider filing the MarkdownBlock.swift:1130-1136 family — linkRegex in particular — as a separate ticket. |
| minor | decision_log.md:714 / :716 — the call-site arithmetic does not add up | Line 714 says the retired inlineCommentRegex was "used by four call sites" (correct: HTMLCommentStripping.swift:59/63, HTMLCommentExport.swift:49, MarkdownBlock.swift:991, MarkdownBlockParser.swift:127 on origin/main). Line 716 then enumerates "Two want a string ... three want the match RANGES", which is five, because HTMLCommentParser.parseBlock is folded into the list — but parseBlock was never an inlineCommentRegex call site; it had its own two regexes. The paragraph reads as though one of the four was double-counted. | Editorial review — no fix applied. Either say "four call sites, plus HTMLCommentParser.parseBlock once its own two patterns are folded in" or split the enumeration. |
| minor | decision_log.md:731 and HTMLCommentParser.swift:75-77 vs the commit message — two sets of numbers for one fixture | For "<!--a--> " x k + "x", the decision log and the production doc comment say 145 B 13 ms, 181 B 273 ms, 217 B 3.5 s, 235 B 22 s end-to-end; the commit message for 4b8c79d0 says 145 B 13 ms, 181 B 210 ms, 217 B 3.4 s, 235 B 13.4 s. The 22 s figure is the first review's end-to-end measurement through swift-markdown while 13.4 s is regex-only, so both may be true — but they are presented as the same series in three places without saying which is which. | Editorial review — no fix applied. Pick one series and label it (regex-only vs end-to-end through the parser). |
| nit | HTMLCommentStripping.swift:134-135 — the precondition that fails silently is the one not documented | removing(_:from:) documents "matches must be non-overlapping and in increasing order". Violating that is LOUD: match.range.location - copied goes negative and NSString.substring(with:) raises an uncatchable NSRangeException. The precondition that is NOT stated — that the matches were computed from this same text — is the one that fails quietly, yielding a plausible-looking but wrongly-stripped string. There is one production call site (MarkdownBlock.swift:1083) and it takes its matches from the adjacent line, so nothing is wrong today. | Editorial review — no fix applied. Add "...and must have been computed from this same `text`" to the doc comment. |
| nit | HTMLCommentParser.swift:119-124 — the reject path lost its O(1) fast exit | The retired structuralRegex was anchored at ^\s*<!--, so a non-comment raw-HTML block (<div>…</div>, a <details> body, a table) was rejected in O(1). parseBlock now runs matches(in:) over the WHOLE block and allocates a CommentMatch per comment before isCommentOnly can reject on the first gap. Linear, so no complexity regression, but a constant-factor cost paid on every raw-HTML block in every document, and an allocation proportional to the comment count on blocks that will be rejected anyway. Note the obvious guard is not free: trimmed.hasPrefix("<!--") uses Foundation's .whitespacesAndNewlines, which is NOT ICU's \s (they disagree on U+180E among others), so it would need the same \S treatment to stay equivalent. | Editorial review — no fix applied, and none clearly warranted — measured 4.7 ms on a 180 KB block. Worth knowing before someone reaches for hasPrefix as an optimisation. |
| nit | HTMLCommentStrippingTests.swift:463 — .serialized on a suite with no timing test | HTMLCommentParserStructuralEquivalenceTests is marked .serialized. The sibling growth suite's rationale for that attribute is that its tests are timing measurements; this one has none. It costs wall-clock time for a 25,000-round fuzz whose oracle is an exponential regex. | Editorial review — no fix applied. Drop .serialized unless there is a reason not stated. |
| nit | HTMLCommentStrippingTests.swift:594-596 — three reject shapes reach the fuzz only through goldens | The parser fuzz alphabet cannot generate <![CDATA[ or <!DOCTYPE (no [, C, D or T units), the /comment: form (no / unit), or [endif] — each is covered by a single golden instead. More notably, no test in the file or in HTMLCommentParserTests.swift uses an uppercase [IF or [ENDIF, so conditionalRegex's .caseInsensitive option is exercised by nothing: removing it would leave the suite green. | Editorial review — no fix applied. One uppercase golden would close the .caseInsensitive gap; the rest are adequately covered by the existing goldens. |
| nit | HTMLCommentStrippingTests.swift:466-516 — the oracle is a hand copy with nothing keeping it in sync | retiredParseBlock reproduces origin/main's function field-for-field — I diffed it and the only difference is an inlined bodyNSLength local. It is correct today. It is also correct only by inspection: nothing detects a future edit to the real parseBlock's surrounding logic that the oracle does not mirror, and the fuzz would then be comparing the new function against a stale idea of the old one. The same is true of the strip oracle, and of ImageParserCommentStrippingTests before it, so this is the project's established shape rather than a new risk. | Editorial review — no fix applied. Noting it so the next person to edit parseBlock knows the oracle exists and must move with it. |
| nit | HTMLCommentStrippingTests.swift:199 — an arm that cannot be taken | match.numberOfRanges >= 2 ? … : NSRange(location: NSNotFound, length: 0) in the strip oracle. The retired pattern's capture group is not optional, so numberOfRanges was invariably 2 and the NSNotFound arm is unreachable. Harmless; the equivalent guard at :499 is deliberate faithfulness to the retired production code and should stay. | Editorial review — no fix applied. |
Click to expand.
diff --git a/prism/Services/HTMLCommentParser.swift b/prism/Services/HTMLCommentParser.swiftindex 632e4f51..94f4c913 100644--- a/prism/Services/HTMLCommentParser.swift+++ b/prism/Services/HTMLCommentParser.swift@@ -26,27 +26,18 @@ import Foundation /// Decision 20); /// - empty / whitespace-only comments after trimming (Req 2.6). enum HTMLCommentParser {- /// Structural validator: matches an input that consists entirely of one- /// or more `<!--…-->` comments separated by optional whitespace.+ /// Matches any character ICU's `\s` does not. Used to test the gaps between+ /// comments in ``isCommentOnly(_:matches:)``. ///- /// The body uses `(?:(?!-->).)*?` so non-greedy expansion cannot swallow- /// `-->` runs and turn `<!--a--> x <!--b-->` into a single comment with- /// body `a--> x <!--b`.- nonisolated private static let structuralRegex: NSRegularExpression = {+ /// Taking the test from the regex engine's own `\S` keeps the whitespace+ /// definition byte-identical to the `\s*` runs in the retired structural+ /// pattern (below), which a hand-rolled `CharacterSet` would be free to drift+ /// from — `\s` here matches U+000B, U+0085 and U+00A0, which is not what+ /// every whitespace definition in Foundation does. Anchorless `\S` over one+ /// gap is a single linear forward scan with nothing to backtrack.+ nonisolated private static let nonWhitespaceRegex: NSRegularExpression = { // swiftlint:disable:next force_try- try! NSRegularExpression(- pattern: #"^(?:\s*<!--(?:(?!-->).)*?-->\s*)+$"#,- options: .dotMatchesLineSeparators- )- }()-- /// Inner-content extractor.- nonisolated private static let extractRegex: NSRegularExpression = {- // swiftlint:disable:next force_try- try! NSRegularExpression(- pattern: #"<!--((?:(?!-->).)*?)-->"#,- options: .dotMatchesLineSeparators- )+ try! NSRegularExpression(pattern: #"\S"#) }() /// Body shape of a note-infrastructure tag — anything that looks like@@ -71,27 +62,69 @@ enum HTMLCommentParser { ) }() + /// Linear replacement for the retired structural validator+ /// `^(?:\s*<!--(?:(?!-->).)*?-->\s*)+$` and its companion extractor+ /// `<!--((?:(?!-->).)*?)-->` (T-2147).+ ///+ /// The retired validator was EXPONENTIAL, not merely quadratic: the whitespace+ /// between two comments is assignable either to iteration *i*'s trailing `\s*` or+ /// to iteration *i+1*'s leading `\s*`, so once the anchored `$` fails the engine+ /// enumerates all 2^k assignments. Measured on `"<!--a--> " * k + "x"`: 145 bytes+ /// 13 ms, 181 bytes 273 ms, 217 bytes 3.5 s, and 22 s end-to-end at 235 bytes.+ /// It ran on whole raw-HTML blocks in `MarkdownBlockParser`'s `.html` fallback ten+ /// lines BEFORE the strip pass T-2147 had already linearised, so the ticket's own+ /// threat model — a run of comment openers in a document opened by URL — still+ /// reached it. Removing the gaps made the same input linear, which is what+ /// isolates the ambiguous whitespace rather than the lazy body as the trigger.+ ///+ /// The scan answers the same question: the block is comment-only iff it holds at+ /// least one `<!--…-->` and every character outside those spans is whitespace.+ /// ``HTMLCommentStripping/matches(in:)`` reports leftmost-first, non-overlapping+ /// spans each ending at the first `-->` after its opener — exactly the+ /// factorisation the retired validator's non-swallowing `(?:(?!-->).)*?` body+ /// forced, and exactly what the retired extractor reported on any input the+ /// validator accepted. So one scan now serves both, and the last lazy-wildcard+ /// pattern in the app is gone.+ nonisolated private static func isCommentOnly(+ _ rawHTML: String,+ nsInput: NSString,+ matches: [HTMLCommentStripping.CommentMatch]+ ) -> Bool {+ guard !matches.isEmpty else { return false }++ var gapStart = 0+ for match in matches {+ let gap = NSRange(location: gapStart, length: match.range.location - gapStart)+ guard isWhitespace(rawHTML, gap) else { return false }+ gapStart = match.range.location + match.range.length+ }+ return isWhitespace(rawHTML, NSRange(location: gapStart, length: nsInput.length - gapStart))+ }++ nonisolated private static func isWhitespace(_ text: String, _ range: NSRange) -> Bool {+ guard range.length > 0 else { return true }+ return nonWhitespaceRegex.firstMatch(in: text, range: range) == nil+ }+ /// Returns the joined inner text of a comment-only block, or `nil` for /// any input that should not be classified as a comment-only block. nonisolated static func parseBlock(_ rawHTML: String) -> String? { // Defensive prefix rejects: CDATA and DOCTYPE are not comments, but- // they superficially resemble comment syntax. The structural regex+ // they superficially resemble comment syntax. The structural check // would already reject them; this is an explicit guard for clarity. let trimmed = rawHTML.trimmingCharacters(in: .whitespacesAndNewlines) if trimmed.hasPrefix("<![CDATA[") || trimmed.hasPrefix("<!DOCTYPE") { return nil } - // Structural validation: the input must be ENTIRELY comment-only.+ // Structural validation: the input must be ENTIRELY comment-only. One+ // linear scan supplies both the verdict and the spans (T-2147). let nsInput = rawHTML as NSString- let fullRange = NSRange(location: 0, length: nsInput.length)- guard structuralRegex.firstMatch(in: rawHTML, range: fullRange) != nil else {+ let matches = HTMLCommentStripping.matches(in: rawHTML)+ guard isCommentOnly(rawHTML, nsInput: nsInput, matches: matches) else { return nil } - let matches = extractRegex.matches(in: rawHTML, range: fullRange)- guard !matches.isEmpty else { return nil }- // Build the joined string. Each `<!--…-->` contributes its body // (trimmed of outer whitespace per comment for clean rendering); // the run of source characters BETWEEN successive comments@@ -101,9 +134,7 @@ enum HTMLCommentParser { var lastEnd = matches[0].range.location for (i, match) in matches.enumerated() {- guard match.numberOfRanges >= 2 else { continue }- let bodyRange = match.range(at: 1)- let body = nsInput.substring(with: bodyRange)+ let body = nsInput.substring(with: match.innerRange) let bodyNSLength = (body as NSString).length let bodyFullRange = NSRange(location: 0, length: bodyNSLength)
diff --git a/prism/Services/HTMLCommentStripping.swift b/prism/Services/HTMLCommentStripping.swiftindex 54ff207e..ac6c357a 100644--- a/prism/Services/HTMLCommentStripping.swift+++ b/prism/Services/HTMLCommentStripping.swift@@ -19,22 +19,15 @@ import Foundation /// runs FIRST, because a comment may itself contain `[^x]` tokens that /// otherwise look like footnote references. enum HTMLCommentStripping {- /// Precompiled regex matching an HTML comment (including multi-line- /// bodies), with the inner text captured as group 1. Shared with- /// ``MarkdownBlock`` and ``HTMLCommentExport`` so all `<!--…-->`-shape- /// scans use one source of truth and pay the regex-compile cost once.- ///- /// KNOWN QUADRATIC (T-2147): the lazy `[\s\S]*?` has the accumulate-and-- /// rescan shape T-1951 removed from `HTMLImageParser` — on a run of `<!--`- /// openers with no `-->`, every opener scans to the end of the input before- /// failing, and `strip`'s `firstMatch` guard alone pays that cost. `strip`- /// runs on whole raw-HTML blocks (MarkdownBlockParser's `.html` fallback),- /// so this is reachable from a URL-opened document. The linear replacement- /// pattern to follow lives at `HTMLImageParser.stripHTMLComments`.- nonisolated static let inlineCommentRegex: NSRegularExpression = {- // swiftlint:disable:next force_try- try! NSRegularExpression(pattern: #"<!--([\s\S]*?)-->"#)- }()+ /// One matched `<!--…-->` span, in the same left-to-right, non-overlapping order+ /// `NSRegularExpression.matches(in:)` reported for the retired `<!--([\s\S]*?)-->`+ /// pattern.+ struct CommentMatch: Equatable, Sendable {+ /// Range of the full `<!--…-->` span, including the delimiters.+ let range: NSRange+ /// Range of the captured inner text — the characters between the delimiters.+ let innerRange: NSRange+ } /// Precompiled collapse regex (2+ horizontal-whitespace runs → one space). nonisolated private static let collapseRegex: NSRegularExpression = {@@ -42,6 +35,9 @@ enum HTMLCommentStripping { try! NSRegularExpression(pattern: #"[ \t]{2,}"#) }() + nonisolated private static let commentOpen: [unichar] = Array("<!--".utf16)+ nonisolated private static let commentClose: [unichar] = Array("-->".utf16)+ /// Returns `text` with all HTML comments removed, adjacent runs of /// whitespace collapsed to a single space, and outer whitespace trimmed. ///@@ -50,21 +46,22 @@ enum HTMLCommentStripping { /// removed comment, so they run solely when a comment was actually stripped /// (T-1364). This keeps the helper a true no-op for comment-free raw HTML, /// where whitespace (e.g. inside `<pre>`) is significant.+ ///+ /// Delegates the removal itself to ``HTMLImageParser/stripHTMLComments(_:)``+ /// (T-2147): that scan already reproduces this exact `<!--([\s\S]*?)-->`+ /// shape in linear time — differentially fuzzed against the identical+ /// retired regex in `ImageParserCommentStrippingTests` — and "was anything+ /// stripped" is simply `removed != text`, since a removal can only shorten+ /// the string. `strip` used to pay for that answer with its own quadratic+ /// `firstMatch` scan before even reaching the (also quadratic)+ /// `stringByReplacingMatches` call. nonisolated static func strip(_ text: String) -> String {- let nsText = text as NSString- let fullRange = NSRange(location: 0, length: nsText.length)+ let removed = HTMLImageParser.stripHTMLComments(text) // No comment present → return the input untouched. Collapse/trim only // make sense as cleanup after a comment is excised.- guard inlineCommentRegex.firstMatch(in: text, range: fullRange) != nil else {- return text- }+ guard removed != text else { return text } - let removed = inlineCommentRegex.stringByReplacingMatches(- in: text,- range: fullRange,- withTemplate: ""- ) let nsRemoved = removed as NSString let collapsed = collapseRegex.stringByReplacingMatches( in: removed,@@ -73,4 +70,79 @@ enum HTMLCommentStripping { ) return collapsed.trimmingCharacters(in: .whitespacesAndNewlines) }++ /// Linear-time replacement for `NSRegularExpression.matches(in:)` against+ /// `<!--([\s\S]*?)-->` (T-2147). Used by call sites that need the matched+ /// ranges themselves — not just the stripped string — such as+ /// ``HTMLCommentExport`` (to substitute each comment with export text) and+ /// ``MarkdownBlock`` (to recover each surfaced comment's span for+ /// ``HTMLCommentParser``).+ ///+ /// Built from the same forward-only scan `HTMLImageParser.stripHTMLComments`+ /// established for the strip-only shape: a match at a `<!--` necessarily+ /// ends at the FIRST `-->` at or after it, because the lazy wildcard stops+ /// at the earliest position where `-->` matches, and matches are+ /// non-overlapping and leftmost. So each opener needs one forward lookup,+ /// the lookups arrive in increasing order, and once no `-->` remains ahead+ /// no later opener can match either — one pass answers everything.+ nonisolated static func matches(in text: String) -> [CommentMatch] {+ let nsText = text as NSString+ let length = nsText.length++ func matchesLiteral(_ literal: [unichar], at position: Int) -> Bool {+ guard position + literal.count <= length else { return false }+ for (offset, unit) in literal.enumerated()+ where nsText.character(at: position + offset) != unit {+ return false+ }+ return true+ }++ var results: [CommentMatch] = []+ var index = 0+ while index < length {+ guard matchesLiteral(commentOpen, at: index) else {+ index += 1+ continue+ }+ // The first `-->` at or after the opener's end. If none exists, neither this+ // opener nor any later one can close, so the scan is done.+ let innerStart = index + commentOpen.count+ var close = innerStart+ while close < length, !matchesLiteral(commentClose, at: close) { close += 1 }+ guard close < length else { break }+ let matchEnd = close + commentClose.count+ results.append(CommentMatch(+ range: NSRange(location: index, length: matchEnd - index),+ innerRange: NSRange(location: innerStart, length: close - innerStart)+ ))+ index = matchEnd+ }+ return results+ }++ /// Returns `text` with the spans in `matches` excised, and nothing else changed.+ ///+ /// For `matches(in: text)` this produces exactly what+ /// ``HTMLImageParser/stripHTMLComments(_:)`` returns — pinned by the goldens and the+ /// differential fuzz — so a caller that already needs the match list can derive the+ /// stripped string from it instead of walking the text a second time+ /// (``MarkdownBlock`` does, when the comment toggle is ON). Callers that need only+ /// the stripped string should keep using `stripHTMLComments`, which allocates no+ /// match array.+ ///+ /// `matches` must be non-overlapping and in increasing order, which is what+ /// ``matches(in:)`` reports.+ nonisolated static func removing(_ matches: [CommentMatch], from text: String) -> String {+ guard !matches.isEmpty else { return text }+ let nsText = text as NSString+ var output = ""+ var copied = 0+ for match in matches {+ output += nsText.substring(with: NSRange(location: copied, length: match.range.location - copied))+ copied = match.range.location + match.range.length+ }+ output += nsText.substring(from: copied)+ return output+ } }
diff --git a/prism/Services/MarkdownBlockParser.swift b/prism/Services/MarkdownBlockParser.swiftindex c3630675..34401e2e 100644--- a/prism/Services/MarkdownBlockParser.swift+++ b/prism/Services/MarkdownBlockParser.swift@@ -105,12 +105,12 @@ enum MarkdownBlockParser: Sendable { /// already treats them as part of the URL, and a valid URL cannot /// legitimately contain an HTML comment. ///- /// Implementation: regex over `\[([^\]]*)\]` with a nested replace of- /// `<!--[\s\S]*?-->` against the captured label. The pattern matches- /// reference-form (`[label][ref]`), inline-form (`[label](url)`), and- /// shortcut-form (`[label]`) link labels.+ /// Implementation: regex over `\[([^\]]*)\]` with the linear comment scan+ /// (``HTMLImageParser/stripHTMLComments(_:)``) applied to each captured label.+ /// The pattern matches reference-form (`[label][ref]`), inline-form+ /// (`[label](url)`), and shortcut-form (`[label]`) link labels. /// Precompiled bracketed-label matcher used by ``stripCommentsInsideLinkLabels``.- /// Matches `[...]` runs; the comment regex is then anchored inside the+ /// Matches `[...]` runs; the comment scan is then applied inside the /// captured label only. nonisolated private static let linkLabelRegex: NSRegularExpression = { // swiftlint:disable:next force_try@@ -124,38 +124,38 @@ enum MarkdownBlockParser: Sendable { // `\[((?:[^\[\]]|\[[^\]]*\])*)\]` would be stricter but unnecessary // — the comment regex is anchored inside the captured label only. - let commentRegex = HTMLCommentStripping.inlineCommentRegex- let nsInput = markdown as NSString let matches = linkLabelRegex.matches(in: markdown, range: NSRange(location: 0, length: nsInput.length)) - // Apply replacements right-to-left so earlier ranges stay valid.- var result = markdown- for match in matches.reversed() {+ // Build the result in ONE left-to-right pass, appending the untouched run before+ // each mutating label and then its stripped form (T-2147). This used to walk the+ // matches right-to-left and call `replacingCharacters` on the whole document per+ // mutating match, which rebuilt the entire string every time: O(document length ×+ // comment-bearing labels), measured at 4x per doubling and 1.9 s on a 480 KB+ // paragraph. Linearising the per-label scan alone left that outer factor standing.+ // The `[...]` matches are non-overlapping and reported in increasing order, and+ // each label range sits inside its own match, so `copied` never runs past one.+ var output = ""+ var copied = 0+ for match in matches { guard match.numberOfRanges >= 2 else { continue } let labelRange = match.range(at: 1) guard labelRange.location != NSNotFound else { continue } - let nsResult = result as NSString- // Guard against any drift caused by simultaneous text mutation:- // the right-to-left order means previous mutations were all to the- // RIGHT of `match`, so the original NSRange is still valid against- // the current `result`.- guard labelRange.location + labelRange.length <= nsResult.length else { continue }-- let label = nsResult.substring(with: labelRange)- let strippedLabel = commentRegex.stringByReplacingMatches(- in: label,- range: NSRange(location: 0, length: (label as NSString).length),- withTemplate: ""- )+ let label = nsInput.substring(with: labelRange)+ // Linear scan (T-2147) — see `HTMLImageParser.stripHTMLComments`.+ let strippedLabel = HTMLImageParser.stripHTMLComments(label)+ guard strippedLabel != label else { continue } - if strippedLabel != label {- result = nsResult.replacingCharacters(in: labelRange, with: strippedLabel)- }+ output += nsInput.substring(with: NSRange(location: copied, length: labelRange.location - copied))+ output += strippedLabel+ copied = labelRange.location + labelRange.length } - return result+ // No label carried a comment → return the input instance untouched.+ guard copied > 0 else { return markdown }+ output += nsInput.substring(from: copied)+ return output } /// Protects blank lines inside `<details>` blocks from terminating the HTML block.
diff --git a/prism/Models/MarkdownBlock.swift b/prism/Models/MarkdownBlock.swiftindex bb8472ea..b48f8128 100644--- a/prism/Models/MarkdownBlock.swift+++ b/prism/Models/MarkdownBlock.swift@@ -984,13 +984,6 @@ nonisolated enum MarkdownBlock: Identifiable, Equatable, Sendable { } } - /// Reference to the shared HTML comment regex. Centralised in- /// ``HTMLCommentStripping/inlineCommentRegex`` so the strip-and-extract- /// paths cannot drift.- private static var inlineHTMLCommentRegex: NSRegularExpression {- HTMLCommentStripping.inlineCommentRegex- }- /// Returns markdown-formatted inline text post-processed for search: /// markdown stripped, HTML comment markers stripped from the base, /// comment inner text appended when the context's toggle is ON, and@@ -1074,23 +1067,27 @@ nonisolated enum MarkdownBlock: Identifiable, Equatable, Sendable { context: SearchContext ) -> FootnoteScanComponents { let nsText = text as NSString- let fullRange = NSRange(location: 0, length: nsText.length)-- // Strip comment markers from the base so scans never see content- // inside a `<!--…-->` range. Same input is used to extract inner- // text below when the toggle is ON.- let baseWithoutMarkers = inlineHTMLCommentRegex.stringByReplacingMatches(- in: text, range: fullRange, withTemplate: ""- ) + // Strip comment markers from the base so scans never see content inside a+ // `<!--…-->` range. Linear scan (T-2147) — see `HTMLImageParser.stripHTMLComments`.+ //+ // With the toggle ON the spans are needed too, so ONE pass supplies both: the+ // match list is taken first and the stripped base derived from it, rather than+ // walking the text once to strip and again to locate. With the toggle OFF the+ // match list would be allocated and never read, so that branch keeps the+ // array-free strip.+ let baseWithoutMarkers: String var surfacedBodies: [String] = [] if context.showHTMLComments {- let matches = inlineHTMLCommentRegex.matches(in: text, range: fullRange)- for match in matches where match.numberOfRanges >= 2 {+ let matches = HTMLCommentStripping.matches(in: text)+ baseWithoutMarkers = HTMLCommentStripping.removing(matches, from: text)+ for match in matches { let span = nsText.substring(with: match.range) guard let body = HTMLCommentParser.parseBlock(span) else { continue } surfacedBodies.append(body) }+ } else {+ baseWithoutMarkers = HTMLImageParser.stripHTMLComments(text) } return FootnoteScanComponents(
diff --git a/prism/Services/HTMLCommentExport.swift b/prism/Services/HTMLCommentExport.swiftindex dadf0328..87f4f25a 100644--- a/prism/Services/HTMLCommentExport.swift+++ b/prism/Services/HTMLCommentExport.swift@@ -45,8 +45,7 @@ enum HTMLCommentExport { /// `InlineNotesExporter` round-trip anchor format survives. static func flatten(_ markdown: String, visible: Bool) -> String { let nsInput = markdown as NSString- let fullRange = NSRange(location: 0, length: nsInput.length)- let matches = HTMLCommentStripping.inlineCommentRegex.matches(in: markdown, range: fullRange)+ let matches = HTMLCommentStripping.matches(in: markdown) guard !matches.isEmpty else { return markdown } @@ -62,12 +61,7 @@ enum HTMLCommentExport { result += before } - let bodyText: String- if match.numberOfRanges >= 2 {- bodyText = nsInput.substring(with: match.range(at: 1))- } else {- bodyText = ""- }+ let bodyText = nsInput.substring(with: match.innerRange) if isNoteInfrastructureBody(bodyText) { // Preserve the literal `<!-- comment:HASH -->` (or `/comment:HASH`)
diff --git a/prism/Services/HTMLImageParser.swift b/prism/Services/HTMLImageParser.swiftindex 6b30b7a4..5149d3e8 100644--- a/prism/Services/HTMLImageParser.swift+++ b/prism/Services/HTMLImageParser.swift@@ -90,9 +90,12 @@ enum HTMLImageParser: Sendable { /// a URL-opened document. /// /// The scan reproduces the pattern exactly, established by the differential fuzz and- /// goldens in `ImageParserCommentStrippingTests`. (The separate `HTMLCommentStripping`- /// helper still carries the same regex shape on other call paths — tracked as T-2147,- /// not silently widened into this fix.) Reading the pattern: a match at a `<!--`+ /// goldens in `ImageParserCommentStrippingTests`. (T-2147 has since landed, so this is+ /// no longer only the image parser's scan: it is the shared removal for+ /// `HTMLCommentStripping.strip` and `MarkdownBlockParser.stripCommentsInsideLinkLabels`+ /// as well, with the range-reporting twin at `HTMLCommentStripping.matches(in:)` —+ /// see Decision 22 in `specs/render-html-comments/decision_log.md` for why the two+ /// entry points are split across two types.) Reading the pattern: a match at a `<!--` /// necessarily ends at the FIRST `-->` after it, because the lazy wildcard stops at the /// earliest position where `-->` matches, and matches are non-overlapping and leftmost. /// So each opener needs one forward lookup, the lookups arrive in increasing order, and@@ -100,7 +103,9 @@ enum HTMLImageParser: Sendable { /// everything. /// /// Internal rather than private so the differential fuzz can compare it against the- /// retired regex directly, instead of through `parse`'s output.+ /// retired regex directly, instead of through `parse`'s output — and, since T-2147,+ /// because three production sites in two other files call it, so it cannot be narrowed+ /// back to private by routing the fuzz through `parse`. nonisolated static func stripHTMLComments(_ html: String) -> String { let nsHTML = html as NSString let length = nsHTML.length
diff --git a/prismTests/HTMLCommentStrippingTests.swift b/prismTests/HTMLCommentStrippingTests.swiftindex 1d0d8a0e..3f43b998 100644--- a/prismTests/HTMLCommentStrippingTests.swift+++ b/prismTests/HTMLCommentStrippingTests.swift@@ -164,3 +164,448 @@ struct HTMLCommentStrippingTests { #expect(result == "[foobar](url)") } }++// MARK: - Growth + equivalence guards (T-2147)++// The reachable surfaces named in the ticket: `HTMLCommentStripping.strip` (called on whole+// raw-HTML blocks at MarkdownBlockParser's `.html` fallback — DoS-shaped, reachable from a+// URL-opened document), `HTMLCommentStripping.matches(in:)` (the new linear replacement for+// `NSRegularExpression.matches(in:)`, used by `HTMLCommentExport.flatten` and+// `MarkdownBlock.footnoteScanComponents`), and `MarkdownBlockParser.stripCommentsInsideLinkLabels`.+// All shared the same accumulate-and-rescan shape T-1951 removed from `HTMLImageParser`: the lazy+// `[\s\S]*?` re-scanned to the end of the whole input from every unmatched `<!--` opener before+// failing.+//+// `strip` and `stripCommentsInsideLinkLabels` now delegate to the already-linear, already-fuzzed+// `HTMLImageParser.stripHTMLComments` (see `ImageParserCommentStrippingTests` in+// `RawHTMLImageScanGrowthTests.swift`), so the growth guards below exercise those call sites+// rather than re-proving the underlying scan. `matches(in:)` is new production code and gets its+// own growth + differential-fuzz treatment, mirroring `RawHTMLImageScanGrowthTests`. See that+// file's header for why growth is asserted as a RATIO, not an absolute budget.+@Suite("HTML comment scanning — growth and equivalence (T-2147)", .serialized)+struct HTMLCommentStrippingGrowthTests {++ // The retired implementation, kept verbatim as the differential oracle for `matches(in:)`.+ // Identical shape to `<!--[\s\S]*?-->` (the pattern `HTMLImageParser`'s oracle uses) with an+ // added capture group, which changes nothing about which characters match — only what a+ // caller can additionally read back out.+ // swiftlint:disable:next force_try+ private static let retiredRegex = try! NSRegularExpression(pattern: #"<!--([\s\S]*?)-->"#)++ private static func retiredMatches(_ text: String) -> [(range: NSRange, inner: NSRange)] {+ let nsText = text as NSString+ let fullRange = NSRange(location: 0, length: nsText.length)+ return retiredRegex.matches(in: text, range: fullRange).map { match in+ (match.range, match.numberOfRanges >= 2 ? match.range(at: 1) : NSRange(location: NSNotFound, length: 0))+ }+ }++ // Test-only oracle for `strip`: the same removal the retired regex performed, followed by+ // the same collapse-then-trim post-processing `strip` still does today. Only the removal+ // mechanism changed in T-2147 (regex scan → linear scan), so reproducing the regex removal+ // here and reusing the identical collapse/trim shape isolates the comparison to that+ // mechanism rather than re-deriving the whole function.+ // swiftlint:disable:next force_try+ private static let collapseRegexOracle = try! NSRegularExpression(pattern: #"[ \t]{2,}"#)++ private static func retiredStrip(_ text: String) -> String {+ let nsText = text as NSString+ let fullRange = NSRange(location: 0, length: nsText.length)+ let removed = retiredRegex.stringByReplacingMatches(in: text, range: fullRange, withTemplate: "")+ guard removed != text else { return text }+ let nsRemoved = removed as NSString+ let collapsed = collapseRegexOracle.stringByReplacingMatches(+ in: removed,+ range: NSRange(location: 0, length: nsRemoved.length),+ withTemplate: " "+ )+ return collapsed.trimmingCharacters(in: .whitespacesAndNewlines)+ }++ // MARK: Growth++ @Test("G10: HTMLCommentStripping.strip stays linear on unclosed openers")+ func stripEntryPointScalesLinearly() {+ // The DoS-shaped call site: `strip` runs unconditionally on whole raw-HTML blocks+ // from `MarkdownBlockParser`'s `.html` fallback.+ GrowthRatioGuard.expectLinearGrowth(shape: "HTMLCommentStripping.strip of unclosed <!-- openers",+ baseCount: 4_000) { count in+ _ = HTMLCommentStripping.strip(String(repeating: "<!--", count: count))+ }+ }++ @Test("G11: HTMLCommentStripping.matches(in:) stays linear on unclosed openers")+ func matchesScalesLinearly() {+ GrowthRatioGuard.expectLinearGrowth(shape: "HTMLCommentStripping.matches(in:) of unclosed <!-- openers",+ baseCount: 4_000) { count in+ _ = HTMLCommentStripping.matches(in: String(repeating: "<!--", count: count))+ }+ }++ @Test("G12: HTMLCommentExport.flatten stays linear on unclosed openers")+ func exportFlattenScalesLinearly() {+ GrowthRatioGuard.expectLinearGrowth(shape: "HTMLCommentExport.flatten of unclosed <!-- openers",+ baseCount: 4_000) { count in+ _ = HTMLCommentExport.flatten(String(repeating: "<!--", count: count), visible: true)+ }+ }++ @Test("G13: stripCommentsInsideLinkLabels stays linear when every label is REWRITTEN")+ func linkLabelStrippingScalesLinearlyWhenRewriting() {+ // The fixture must make `strippedLabel != label` TRUE, or the rewriting branch never+ // runs and the guard measures nothing. G13 originally used a run of unclosed openers+ // inside one label — whose strip is a no-op — and so could not see the outer quadratic+ // this call site carried until T-2147: it rebuilt the WHOLE document with+ // `replacingCharacters` once per mutating label, i.e. O(document × comment-bearing+ // labels). That shape measured 4.2-5.0x per doubling and 1.9 s on a 480 KB paragraph+ // (PR #395 pre-push review). The no-op fixture is kept as G15 below, which pins the+ // per-label scan instead.+ //+ // baseCount is 8,000 rather than the 4,000 the sibling guards use because the ratio is+ // the evidence: restoring the rebuild measures 11.7x here but only 10.1x at 4,000, and+ // a mutant that clears the 8x ceiling by 2x is a guard worth the extra 0.3 s.+ GrowthRatioGuard.expectLinearGrowth(+ shape: "stripCommentsInsideLinkLabels of comment-bearing labels (rewriting branch)",+ baseCount: 8_000+ ) { count in+ _ = MarkdownBlockParser.stripCommentsInsideLinkLabels(+ String(repeating: "[a<!--c-->](u) ", count: count)+ )+ }+ }++ @Test("G14: MarkdownBlock.footnoteScanComponents stays linear on unclosed openers")+ func footnoteScanComponentsScalesLinearly() {+ // Exercises MarkdownBlock.swift's own consumer of `matches(in:)`, gated ON so the+ // matches path (not just the strip path) runs.+ GrowthRatioGuard.expectLinearGrowth(+ shape: "MarkdownBlock.footnoteScanComponents of unclosed <!-- openers",+ baseCount: 4_000+ ) { count in+ let context = SearchContext(showHTMLComments: true, footnoteData: .empty)+ _ = MarkdownBlock.footnoteScanComponents(+ forInlineText: String(repeating: "<!--", count: count),+ context: context+ )+ }+ }++ @Test("G15: stripCommentsInsideLinkLabels stays linear on unclosed openers inside a label")+ func linkLabelStrippingScalesLinearly() {+ // The no-op half of the call site: the label's strip changes nothing, so this pins the+ // per-label SCAN (the inner factor T-2147 linearised) and deliberately not the rewrite+ // path, which G13 covers.+ GrowthRatioGuard.expectLinearGrowth(+ shape: "stripCommentsInsideLinkLabels of unclosed <!-- openers inside a label",+ baseCount: 4_000+ ) { count in+ let label = "[" + String(repeating: "<!--", count: count) + "]"+ _ = MarkdownBlockParser.stripCommentsInsideLinkLabels(label)+ }+ }++ @Test("G16: HTMLCommentParser.parseBlock stays linear on whitespace-separated comments")+ func parseBlockScalesLinearly() {+ // The exact shape that made the RETIRED structural validator+ // `^(?:\s*<!--(?:(?!-->).)*?-->\s*)+$` exponential: k well-formed comments separated by+ // whitespace, then one character that cannot be part of any comment. Each gap is+ // assignable to either neighbouring `\s*`, so once the anchored `$` fails the engine+ // enumerates 2^k assignments. Measured on the retired regex: 145 B (k=16) 13 ms,+ // 181 B (k=20) 273 ms, 217 B (k=24) 3.5 s, and 22 s end-to-end at 235 bytes — for a+ // document of a few hundred BYTES, reachable from a URL-opened document because this+ // ran on whole raw-HTML blocks ten lines before the strip pass.+ //+ // The counts here are sized for the linear implementation. A mutant that restores the+ // regex does not fail this guard so much as never finish it: at baseCount 1,000 the+ // retired pattern would need ~3.7^492 seconds. Mutation was therefore verified by+ // temporarily lowering baseCount to 4 (large = 16), where the restored regex is still+ // finite and the guard failed on the ratio; see the PR comment.+ GrowthRatioGuard.expectLinearGrowth(+ shape: "HTMLCommentParser.parseBlock of whitespace-separated comments + a rejecting tail",+ baseCount: 1_000+ ) { count in+ _ = HTMLCommentParser.parseBlock(String(repeating: "<!--a--> ", count: count) + "x")+ }+ }++ // MARK: Equivalence — goldens++ private static let asciiGoldenCases = [+ "<!-- a comment -->",+ "before <!-- a --> after",+ "<!-- line\nbreak -->",+ "<!---->",+ "<!--->",+ "<!----->",+ "<!-- a --> b -->",+ "<!-- x <!-- y -->",+ "<img <!-- c --> src=x>",+ "a<!---->b<!---->c",+ "<!-- never closed",+ "no opener -->",+ "",+ "no comments here"+ ]++ /// Golden fixtures added for the T-2147 review note: the scan is UTF-16-indexed, so it is+ /// most at risk on input where an index-by-code-unit walk could plausibly diverge from a+ /// scalar- or grapheme-aware one — non-ASCII BMP text, standalone/composed combining marks,+ /// and supplementary-plane emoji (including multi-codepoint ZWJ sequences) — placed inside a+ /// comment, straddling its delimiters with no separating character, and inside an+ /// unterminated opener.+ private static let nonASCIIGoldenCases = [+ "<!-- café münchen -->", // non-ASCII BMP text inside+ "wörld<!-- x -->wörld", // non-ASCII BMP text around+ "e\u{0301}<!-- x -->e\u{0301}", // combining mark (base+combining) around+ "<!-- e\u{0301} combining -->", // combining mark inside+ "\u{0301}<!--\u{0301}-->\u{0301}", // standalone combining marks, incl. touching delimiters+ "😀<!-- x -->😀", // surrogate-pair emoji around+ "<!-- 😀 emoji -->", // surrogate-pair emoji inside+ "<!--😀-->", // emoji touching both delimiters, no separator+ "😀<!--😀-->😀", // emoji touching delimiters on every side+ "👨👩👧👦<!-- family -->👨👩👧👦", // ZWJ emoji sequence around+ "<!-- 👨👩👧👦 -->", // ZWJ emoji sequence inside+ "text <!-- 文 café -->more", // mixed CJK + accented text inside+ "<!-- 😀 unterminated", // unterminated opener, emoji content, no closer+ "😀<!-- unterminated", // unterminated opener preceded by emoji+ "<!--e\u{0301}", // unterminated opener, combining mark touching delimiter+ "<!--👨👩👧👦" // unterminated opener, ZWJ sequence touching delimiter+ ]++ @Test("matches(in:) reproduces the retired regex's ranges exactly")+ func matchesGoldenShapes() {+ for input in Self.asciiGoldenCases + Self.nonASCIIGoldenCases {+ let expected = Self.retiredMatches(input)+ let actual = HTMLCommentStripping.matches(in: input)+ #expect(actual.count == expected.count, Comment(rawValue:+ "\(input.debugDescription): expected \(expected.count) matches, got \(actual.count)"))+ for (actualMatch, expectedMatch) in zip(actual, expected) {+ #expect(actualMatch.range == expectedMatch.range, Comment(rawValue:+ "\(input.debugDescription): range mismatch"))+ #expect(actualMatch.innerRange == expectedMatch.inner, Comment(rawValue:+ "\(input.debugDescription): inner range mismatch"))+ }+ }+ }++ @Test("strip reproduces the retired regex's removal exactly, including non-ASCII goldens (T-2147 review note)")+ func stripGoldenShapes() {+ for input in Self.asciiGoldenCases + Self.nonASCIIGoldenCases {+ let expected = Self.retiredStrip(input)+ let actual = HTMLCommentStripping.strip(input)+ #expect(actual == expected, Comment(rawValue:+ "\(input.debugDescription): expected \(expected.debugDescription), got \(actual.debugDescription)"))+ }+ }++ // MARK: Equivalence — differential fuzz++ @Test("matches(in:) and strip reproduce the retired regex on random fragments")+ func fuzzRandomFragments() {+ // Same hostile alphabet shape as `ImageParserCommentStrippingTests`' fuzz: the units+ // that decide a match at high density, so a generated fragment is far more likely than+ // real markup to sit on a boundary case. Widened for the T-2147 review note with+ // non-ASCII units — a lone combining mark, a base+combining pair, BMP non-ASCII text, a+ // surrogate-pair emoji, a multi-codepoint ZWJ sequence, and a standalone ZWJ — so+ // generated fragments also land emoji/combining marks directly against delimiters.+ let alphabet = ["<!--", "-->", "<!", "--", "-", "<", ">", "!", "a", " ", "\n", "\t",+ "<!---->", "<!--->", "<img>", "text",+ "😀", "👨👩👧👦", "é", "e\u{0301}", "\u{0301}", "🏳️🌈", "文", "\u{200D}"]+ var rng = SeededRandomNumberGenerator(seed: 0x2147_1A)+ for _ in 0..<25_000 {+ let units = Int.random(in: 0...40, using: &rng)+ var input = ""+ for _ in 0..<units { input += alphabet.randomElement(using: &rng) ?? "a" }+ let expected = Self.retiredMatches(input)+ let actual = HTMLCommentStripping.matches(in: input)+ guard expected.count == actual.count else {+ Issue.record(Comment(rawValue: "match count diverged on \(input.debugDescription):"+ + " expected \(expected.count), got \(actual.count)"))+ return+ }+ for (expectedMatch, actualMatch) in zip(expected, actual) {+ guard expectedMatch.range == actualMatch.range, expectedMatch.inner == actualMatch.innerRange else {+ Issue.record(Comment(rawValue: "match range diverged on \(input.debugDescription)"))+ return+ }+ }+ let expectedStrip = Self.retiredStrip(input)+ let actualStrip = HTMLCommentStripping.strip(input)+ guard expectedStrip == actualStrip else {+ Issue.record(Comment(rawValue: "strip diverged on \(input.debugDescription):"+ + " expected \(expectedStrip.debugDescription), got \(actualStrip.debugDescription)"))+ return+ }+ // `removing(matches(in:), from:)` is the one-pass derivation `MarkdownBlock` uses+ // when the comment toggle is ON; it must agree with the two-pass removal it replaces.+ let derived = HTMLCommentStripping.removing(actual, from: input)+ guard derived == HTMLImageParser.stripHTMLComments(input) else {+ Issue.record(Comment(rawValue: "removing(matches:) diverged from stripHTMLComments"+ + " on \(input.debugDescription)"))+ return+ }+ }+ }+}++// MARK: - HTMLCommentParser structural classification (T-2147)++// `HTMLCommentParser.parseBlock` used to answer "is this block nothing but comments?" with+// `^(?:\s*<!--(?:(?!-->).)*?-->\s*)+$` and then re-find the bodies with+// `<!--((?:(?!-->).)*?)-->`. The first was EXPONENTIAL — see G16 for the measurements — and it+// ran on the same raw-HTML block ten lines before the strip pass T-2147 had already linearised,+// so the ticket's fix was incomplete without it. Both are now expressed over the one linear+// `HTMLCommentStripping.matches(in:)` scan plus a whitespace test on the gaps.+//+// Both retired regexes are kept here verbatim, wired into a faithful reproduction of the retired+// `parseBlock`, so the replacement is compared against the WHOLE function it replaced — not just+// against the classification — on goldens and on random fragments.+@Suite("HTML comment parser — structural classification equivalence (T-2147)", .serialized)+struct HTMLCommentParserStructuralEquivalenceTests {++ // swiftlint:disable:next force_try+ private static let retiredStructuralRegex = try! NSRegularExpression(+ pattern: #"^(?:\s*<!--(?:(?!-->).)*?-->\s*)+$"#,+ options: .dotMatchesLineSeparators+ )+ // swiftlint:disable:next force_try+ private static let retiredExtractRegex = try! NSRegularExpression(+ pattern: #"<!--((?:(?!-->).)*?)-->"#,+ options: .dotMatchesLineSeparators+ )+ // Unchanged by T-2147; reproduced so the oracle is the whole retired function.+ // swiftlint:disable:next force_try+ private static let retiredNoteRegex = try! NSRegularExpression(pattern: #"^\s*/?comment:[a-f0-9]+\s*$"#)+ // swiftlint:disable:next force_try+ private static let retiredConditionalRegex = try! NSRegularExpression(+ pattern: #"^\s*\[(?:if|endif)\b"#,+ options: .caseInsensitive+ )++ private static func retiredParseBlock(_ rawHTML: String) -> String? {+ let trimmed = rawHTML.trimmingCharacters(in: .whitespacesAndNewlines)+ if trimmed.hasPrefix("<![CDATA[") || trimmed.hasPrefix("<!DOCTYPE") { return nil }++ let nsInput = rawHTML as NSString+ let fullRange = NSRange(location: 0, length: nsInput.length)+ guard retiredStructuralRegex.firstMatch(in: rawHTML, range: fullRange) != nil else { return nil }++ let matches = retiredExtractRegex.matches(in: rawHTML, range: fullRange)+ guard !matches.isEmpty else { return nil }++ var joined = ""+ var lastEnd = matches[0].range.location+ for (i, match) in matches.enumerated() {+ guard match.numberOfRanges >= 2 else { continue }+ let body = nsInput.substring(with: match.range(at: 1))+ let bodyFullRange = NSRange(location: 0, length: (body as NSString).length)+ if retiredNoteRegex.firstMatch(in: body, range: bodyFullRange) != nil { return nil }+ if retiredConditionalRegex.firstMatch(in: body, range: bodyFullRange) != nil { return nil }+ if i > 0 {+ let sepLength = match.range.location - lastEnd+ if sepLength > 0 {+ joined += nsInput.substring(with: NSRange(location: lastEnd, length: sepLength))+ }+ }+ joined += body.trimmingCharacters(in: .whitespacesAndNewlines)+ lastEnd = match.range.location + match.range.length+ }++ let result = joined.trimmingCharacters(in: .whitespacesAndNewlines)+ return result.isEmpty ? nil : result+ }++ /// Whitespace runs are the whole point of the retired pattern's ambiguity, so the gap+ /// fixtures deliberately include the characters where a hand-rolled whitespace set could+ /// diverge from ICU's `\s`: U+000B, U+0085, U+00A0, U+2028, U+3000.+ private static let goldenCases = [+ // Accepted as comment-only.+ "<!-- a -->",+ " <!-- a --> ",+ "<!-- a --><!-- b -->",+ "<!-- a --> <!-- b -->",+ "<!-- a -->\n\n<!-- b -->",+ "<!-- a -->\u{0B}<!-- b -->",+ "<!-- a -->\u{85}<!-- b -->",+ "<!-- a -->\u{A0}<!-- b -->",+ "<!-- a -->\u{2028}<!-- b -->",+ "<!-- a -->\u{3000}<!-- b -->",+ "<!-- a -->\u{0B}",+ "<!-- a -->\u{85}",+ "<!-- multi\nline -->",+ "<!---->",+ "<!----->",+ "<!-- a --><!---->",+ "<!--a<!--b-->",+ "<!-- 😀 -->",+ "<!--😀-->",+ "<!-- e\u{0301} -->",+ // Rejected — mixed content, the shape that made the retired regex explode.+ "<!-- a --> x",+ "<!-- a --> <!-- b --> x",+ "x <!-- a -->",+ "<!-- a --> b <!-- c -->",+ "<p><!-- a --></p>",+ // Rejected — no closer, or no comment at all.+ "<!-- a",+ "<!--->",+ "<!--",+ "-->",+ "",+ " ",+ "plain text",+ // Rejected — not comments.+ "<![CDATA[ x ]]>",+ "<!DOCTYPE html>",+ // Rejected — note infrastructure and conditional comments.+ "<!-- comment:abc123 -->",+ "<!-- /comment:abc123 -->",+ "<!-- a --><!-- comment:deadbeef -->",+ "<!--[if IE]>x<![endif]-->",+ "<!-- [endif] -->",+ // Rejected — empty after trimming.+ "<!-- -->",+ "<!---->\n<!---->"+ ]++ @Test("parseBlock reproduces the retired structural regex + extractor exactly")+ func parseBlockGoldenShapes() {+ for input in Self.goldenCases {+ let expected = Self.retiredParseBlock(input)+ let actual = HTMLCommentParser.parseBlock(input)+ #expect(actual == expected, Comment(rawValue:+ "\(input.debugDescription): expected \(String(describing: expected)),"+ + " got \(String(describing: actual))"))+ }+ }++ @Test("parseBlock reproduces the retired regexes on random fragments")+ func fuzzRandomFragments() {+ // Hostile alphabet in the same spirit as the strip fuzz, plus the whitespace units the+ // structural pattern's `\s*` runs consume — those gaps are what the retired regex could+ // split two ways, so they are the units most likely to expose a classification+ // difference.+ //+ // Fragments are capped at 24 units, shorter than the strip fuzz's 40, because the+ // ORACLE here is the exponential regex: the code under test is linear, but a 40-unit+ // draw alternating `<!---->` with a space would make the ORACLE take hundreds of+ // milliseconds. At 24 units the worst conceivable draw is ~12 gaps, i.e. under a+ // millisecond.+ let alphabet = ["<!--", "-->", "<!---->", "<!--a-->", "<!--->", "<!", "--", "-", "<", ">",+ "!", "a", "x", " ", " ", "\n", "\t", "\u{0B}", "\u{85}", "\u{A0}",+ "\u{2028}", "\u{3000}", "<p>", "comment:ab12", "[if IE]", "😀", "e\u{0301}"]+ var rng = SeededRandomNumberGenerator(seed: 0x2147_5B)+ for _ in 0..<25_000 {+ let units = Int.random(in: 0...24, using: &rng)+ var input = ""+ for _ in 0..<units { input += alphabet.randomElement(using: &rng) ?? "a" }+ let expected = Self.retiredParseBlock(input)+ let actual = HTMLCommentParser.parseBlock(input)+ guard expected == actual else {+ Issue.record(Comment(rawValue: "parseBlock diverged on \(input.debugDescription):"+ + " expected \(String(describing: expected)), got \(String(describing: actual))"))+ return+ }+ }+ }+}
diff --git a/prismTests/RawHTMLImageScanGrowthTests.swift b/prismTests/RawHTMLImageScanGrowthTests.swiftindex 240c550f..2ade1652 100644--- a/prismTests/RawHTMLImageScanGrowthTests.swift+++ b/prismTests/RawHTMLImageScanGrowthTests.swift@@ -236,7 +236,8 @@ struct VoidElementNormalisationTests { // Named for its subject — `HTMLImageParser.stripHTMLComments` — because the suite name // `HTMLCommentStrippingTests` is taken by the tests for the separate `HTMLCommentStripping`-// helper, whose own regex still carries this shape (tracked as T-2147).+// helper, whose own regex carried this shape until T-2147 retired it (see+// `HTMLCommentStrippingGrowthTests`; `strip` now delegates its removal to the scan below). @Suite("Image-parser comment stripping — growth and equivalence (T-1951)", .serialized) struct ImageParserCommentStrippingTests {
diff --git a/specs/render-html-comments/decision_log.md b/specs/render-html-comments/decision_log.mdindex c512b9fe..378a5317 100644--- a/specs/render-html-comments/decision_log.md+++ b/specs/render-html-comments/decision_log.md@@ -704,6 +704,60 @@ Conditional comments are a different feature with different semantics; rendering --- +## Decision 22: Two Comment-Scan Entry Points Split Across Two Types++**Date**: 2026-08-23+**Status**: accepted++### Context++The feature originally recognised `<!--…-->` with one shared regex, `HTMLCommentStripping.inlineCommentRegex` (`<!--([\s\S]*?)-->`), used by four call sites, plus two more lazy-wildcard patterns of the same family inside `HTMLCommentParser` (a structural validator and an inner-text extractor). T-2147 found the lazy body quadratic on a run of `<!--` openers with no close: every opener re-scanned to the end of the input before failing. This is reachable from a document opened by URL, so it is DoS-shaped rather than merely slow.++T-1951 had already met the identical pattern one layer down and replaced it with a linear forward-only scan, `HTMLImageParser.stripHTMLComments`, backed by goldens and a differential fuzz against the retired regex. The question T-2147 had to answer was not "scan or regex" — that was settled — but where the replacement primitive should live, given that the four call sites do not all consume the same thing. Two want a string with the comments gone (`HTMLCommentStripping.strip`, `MarkdownBlockParser.stripCommentsInsideLinkLabels`); three want the match RANGES, which a stripped string cannot give back (`HTMLCommentExport.flatten` substitutes export text per comment, `MarkdownBlock.footnoteScanComponents` recovers each span, and `HTMLCommentParser.parseBlock` needs both the spans and the gaps between them).++### Decision++Split the replacement by what callers consume, across the two types that already exist:++- Removal-only sites call `HTMLImageParser.stripHTMLComments` — the T-1951 scan, reused as-is.+- Range-reporting sites call a new `HTMLCommentStripping.matches(in:) -> [CommentMatch]`, a second forward-only loop over the same shape, with `HTMLCommentStripping.removing(_:from:)` for the one caller that needs both from a single pass.++`HTMLCommentStripping.inlineCommentRegex` and both of `HTMLCommentParser`'s lazy-wildcard regexes are deleted. The parser's structural check is re-expressed over `matches(in:)` plus a whitespace test on the gaps between matches, using the regex engine's own `\S` so the whitespace definition cannot drift from the `\s*` runs it replaces.++### Rationale++Reusing `stripHTMLComments` for removal means the strip path inherits an existing differential-fuzz oracle instead of needing a second one maintained in parallel; the two patterns differ only by a capture group, which changes what a caller can read back and not which characters match. Giving the range consumers their own entry point avoids the two shapes that would otherwise be forced: a `strip`-only primitive would make every range consumer re-derive positions it just discarded, and a `matches`-only primitive would make the strip path allocate a `CommentMatch` per comment it never reads (~4.5 MB transient on a pathological comment-dense 1 MB block).++Folding `HTMLCommentParser` into the same scan was not originally in scope and turned out to be the more urgent half. Its structural validator, `^(?:\s*<!--(?:(?!-->).)*?-->\s*)+$`, is EXPONENTIAL rather than quadratic — the whitespace between two comments is assignable either to one iteration's trailing `\s*` or the next one's leading `\s*`, so once the anchored `$` fails the engine enumerates all 2^k assignments. It runs on the same raw-HTML block ten lines before the already-linearised `strip`, so leaving it would have made the fix's own claim false: measured on `"<!--a--> " × k + "x"`, 145 bytes took 13 ms, 217 bytes 3.5 s, and a 235-byte document 22 s end-to-end. The same scan answers the structural question in one pass and retires the extractor with it, so no lazy-wildcard pattern over whole-block input survives in the app.++### Alternatives Considered++- **Single owner: put the primitive on `HTMLCommentStripping` and have `HTMLImageParser.stripHTMLComments` forward to it.** One copy of the loop, one delimiter table, no layering inversion, and the type documented as the central comment authority would stay the owner. Rejected on risk-for-reward at this moment: `stripHTMLComments` is the scan the T-1951 fuzz and goldens are written against directly (it is `internal`, not `private`, precisely so the fuzz can reach it), so making it a forwarding wrapper moves the tested subject out from under its own oracle in the same change that is fixing a security-shaped defect elsewhere. The unification is a refactor with no behaviour to verify, and it should be made on its own, against both oracles, rather than smuggled into a bug fix.+- **Keep the regex and make it safe (possessive/atomic body, or `(?:(?!-->).)*?`).** Rejected: a rewritten regex needs its own equivalence proof, and the only easy oracle available is the retired pattern it is replacing — precisely the argument for reusing the scan that already has one. `HTMLCommentParser`'s structural regex is also the counter-example: it ALREADY used the non-swallowing body, and was the worst performer in the file.+- **Narrow the structural regex's whitespace instead (`^(?:\s*<!--(?:(?!-->).)*?-->)+\s*$`).** Disambiguating the gaps does remove the exponential blow-up, and is a one-character-class move. Rejected because it leaves a third lazy-wildcard pattern in the file with no oracle, and because the scan formulation deletes the extractor as well.+- **Build `stripHTMLComments` on top of `matches(in:)`.** Rejected on allocation: the strip path is the hot one and currently allocates no match array.++### Consequences++**Positive:**++- No lazy-wildcard `<!--…-->` pattern over whole-block input survives anywhere in the app; the exponential structural validator is gone with it.+- Both entry points are pinned against the retired patterns as test-only oracles — goldens (ASCII and non-ASCII) plus differential fuzzes — so equivalence is evidence rather than assertion.+- Growth guards G10–G16 pin the linear behaviour of every affected call site, including the exact `"<!--a--> " × k + "x"` shape that triggered the exponential case.+- `HTMLCommentParser` gets its spans and its verdict from one pass instead of two regexes.++**Negative:**++- Two near-identical scan loops now exist (`HTMLCommentStripping.matches(in:)` and `HTMLImageParser.stripHTMLComments`), each with its own delimiter table and its own copy of the paragraph explaining why the first `-->` wins. Semantic drift would be caught — both are fuzzed against the same pattern — but a one-sided IMPROVEMENT would not: a future change to the scan shape can be applied to one and silently not the other.+- A layering inversion: `HTMLCommentStripping`, documented as the central comment authority, depends on a type named for image parsing, and `MarkdownBlockParser` now reaches into the image parser for a non-image concern.+- `HTMLImageParser.stripHTMLComments` can no longer be narrowed back to `private`: three production sites in two other files call it.++### Impact++`prism/Services/HTMLCommentStripping.swift`, `prism/Services/HTMLCommentParser.swift`, `prism/Services/HTMLImageParser.swift` (doc only), `prism/Services/HTMLCommentExport.swift`, `prism/Services/MarkdownBlockParser.swift`, `prism/Models/MarkdownBlock.swift`. Behaviour is unchanged at every call site; only cost and mechanism change.++---+ ## Notes for Design Phase (non-blocking) The following items came out of external review but are implementation choices rather than decisions to lock in at requirements time. The design phase should address each:
diff --git a/specs/render-html-comments/implementation.md b/specs/render-html-comments/implementation.mdindex e24b03c1..9595e9b5 100644--- a/specs/render-html-comments/implementation.md+++ b/specs/render-html-comments/implementation.md@@ -54,7 +54,7 @@ HTML comments are how authors leave private side-notes in markdown ("TODO: revis - **Parse-time vs render-time split** — author content is parsed once into blocks; toggling the preference never re-runs the swift-markdown AST parse (Req 9.2). The toggle is read at render time by `HighlightedInlineText`, `HTMLCommentBlockView`, and `MarkdownBlock.searchableText(in:)`. - **Mirror the footnote system** — the footnote system already pairs a parser/preprocessor (`FootnotePreprocessor`), a stripping helper (`FootnoteStripping`), a Textual extension (`SyntaxExtension.footnoteReferences`), an accessibility modifier (`FootnoteAccessibilityModifier`), and a `[^id]` regex in `MarkdownBlock`. The HTML-comments system mirrors that shape one-for-one.-- **Centralised regex** — one `HTMLCommentStripping.inlineCommentRegex` powers `HTMLCommentStripping`, `HTMLCommentExport`, `MarkdownBlock.searchableText`, and `MarkdownBlockParser.stripCommentsInsideLinkLabels`. The parser's own four regexes (structural, extract, note-infrastructure, conditional) are hoisted to `static let` so they compile once.+- **Centralised scan** (was "centralised regex"; changed by T-2147, Decision 22) — the `<!--…-->` shape is recognised by one linear forward-only scan, exposed through two entry points split by what the caller consumes. Removal-only sites (`HTMLCommentStripping.strip`, `MarkdownBlockParser.stripCommentsInsideLinkLabels`, and `MarkdownBlock.footnoteScanComponents` with the toggle off) call `HTMLImageParser.stripHTMLComments`; range-reporting sites (`HTMLCommentExport.flatten`, `MarkdownBlock.footnoteScanComponents` with the toggle on, and `HTMLCommentParser.parseBlock`) call `HTMLCommentStripping.matches(in:)`. This was originally one `HTMLCommentStripping.inlineCommentRegex` — a lazy `<!--([\s\S]*?)-->` — which was quadratic on a run of unclosed openers. `HTMLCommentParser`'s structural and extract regexes are gone for the same reason (the structural one was exponential, not quadratic); its remaining two regexes (note-infrastructure, conditional) are anchored, run on a single comment body each, and stay hoisted to `static let` so they compile once. - **Compatibility shims for search** — `searchableText` and `searchableText(with:)` keep compiling for existing callers; the new `searchableText(in: SearchContext)` is the canonical method. ### Trade-offs@@ -90,7 +90,7 @@ HTML comments are how authors leave private side-notes in markdown ("TODO: revis - **Comments inside fenced/indented code blocks** — never reach `HighlightedInlineText`; the code path uses HighlightSwift directly. The Textual extension does not need a code-block guard. Inline code spans (backtick) ARE skipped via `PatternProcessor.isInsideCodeSpan(range:in:)` so `` `<!--x-->` `` survives literally in both states. - **Multi-line comments preserve internal whitespace** (Req 2.5). `HTMLCommentParser` joins comment bodies using source-separating whitespace (so blank lines between comments survive); outer whitespace is trimmed. - **Author content used as a format key** — `HTMLCommentBlockView` uses `String(localized: "Comment, \(rawText)")` (Swift catalog-aware interpolation) for the accessibility label, never `LocalizedStringKey("Comment, \(rawText)")` (which would interpolate at construction time and miss the catalog key). The block view text itself uses `Text(verbatim: rawText)` to avoid accidental catalog lookup on content like `%d items`.-- **Performance budgets** — `HTMLCommentParser`'s four regexes are `static let` so each document open compiles them once. `HTMLCommentStripping.strip` uses precompiled regexes via `stringByReplacingMatches`. A new XCTest performance bench (`HighlightedInlineTextRenderBench`) asserts sub-quadratic per-render scaling at N ∈ {0, 10, 100} inline comments per paragraph. The 500 KB parsing benchmark gains a comment-bearing variant pinning the existing budget.+- **Performance budgets** (revised by T-2147) — `HTMLCommentParser`'s remaining regexes are `static let` so each document open compiles them once. `HTMLCommentStripping.strip` removes comments with a linear forward-only scan (`HTMLImageParser.stripHTMLComments`); only the collapse-whitespace pass that follows a removal is still a precompiled regex via `stringByReplacingMatches`. Growth is pinned as a RATIO, not a millisecond budget, by `HTMLCommentStrippingGrowthTests` (G10–G16) — an absolute budget cannot separate a linear pass from a quadratic one and is flaky under concurrent test load. A new XCTest performance bench (`HighlightedInlineTextRenderBench`) asserts sub-quadratic per-render scaling at N ∈ {0, 10, 100} inline comments per paragraph. The 500 KB parsing benchmark gains a comment-bearing variant pinning the existing budget. ### Completeness Assessment
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex bb517458..2e626ad0 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -22,6 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- Documents containing HTML comments (`<!--…-->`) no longer stall while opening (T-2147). Several steps that look for comments — in a block of raw HTML, inside a link's label, and in the text Prism searches and exports — cost time in proportion to the *square* of what they were given, so a document that would otherwise open instantly could hold the app for tens of seconds. A run of comment openers with no closing `-->`, which is what a document being written, generated, or truncated mid-comment looks like, was the trigger: every opener read the whole rest of the document looking for a close before giving up. One step was worse than slow. Deciding whether a block of HTML is nothing but comments cost roughly four times as much for every two comments added, so a 145-byte document took 13 milliseconds, a 217-byte one 3.5 seconds, and a 235-byte one 22 seconds, with no upper bound beyond that — and it needed only a handful of ordinary, correctly closed comments followed by a single other character, not a malformed document at all. Removing comments from link labels had a second problem on top of the first: the entire document was rebuilt from scratch once per label carrying a comment, which cost 1.9 seconds for a 480 KB paragraph. Every one of these steps now reads the document once, from left to right, and grows in step with its length rather than with its square. Because a document can be opened from a URL, a file written this way could previously have kept a device busy for a long time on someone else's behalf. Nothing about how comments are displayed changes: each replacement was checked against the exact step it replaced, character for character, over tens of thousands of generated fragments as well as hand-written awkward cases. - An image that is tiny as a file but enormous as a picture can no longer exhaust memory or terminate Prism, whether it comes from the web, from a file beside the document, or written directly into the markdown (T-2132, T-2149, T-2151, T-1867). A picture is stored compressed, and a plain-coloured one compresses at about a thousand to one — so a 400 KB download can be a 20,000 by 20,000 image that needs about 400 MB the moment anything tries to display it, and four times that if it is in colour. Prism's limits were all written on the wrong side of that: a 50 MB cap on the download said nothing about the picture inside it, and the 2 MB cap on a local SVG was applied only after the whole file had already been read, so a very large one could freeze the app on its way to being refused. The limits that did exist covered only images fetched from the web; the same image referenced from a file next to your document, or embedded inline in the markdown, went straight to the renderer unchecked. Prism now reads the picture's dimensions from its header — a few bytes, before anything is decoded — and decides from that. An ordinary image is displayed as before. A very large one referenced from the web or from a file is scaled down to fit. One beyond any reasonable size is refused outright and shows the usual "Image failed to load" placeholder, rather than being handed to a decoder that would have to build the whole thing first. How large a picture is now also accounts for how much detail each dot of it carries: most pictures store one byte per colour, but some store two or four, and Prism previously assumed the smaller size for all of them and so under-counted the deep ones by half or three quarters. One consequence you may see: a very large deep-colour photograph that used to display at full size is now scaled down, because its true size was always above the limit and is now measured as such. Files are now read up to their limit instead of read whole and then measured — including the copy Prism keeps of a document you have not saved yet, which is restored when the app reopens. How much decoding happens at once is limited by how much memory those pictures actually need rather than by how many of them there are, so a page full of large images no longer overruns while appearing to stay within its bounds. Two things behave differently, both deliberately. An image whose file does not say how big it is, or what kind of dots it stores, now shows the "Image failed to load" placeholder instead of being displayed — there is no way to know what it would cost until it has already cost it. And an image written directly into the markdown is treated more strictly than the same image kept in a file beside the document: it is either small enough to display as it is or refused, never scaled down. That difference is about memory rather than effort. Scaling a picture that is written into the markdown means rebuilding it and writing the smaller version back into the page, where it then stays for as long as the document is open — which costs more memory, for longer, than not showing it. A picture in a file has somewhere else to keep its smaller version, so it can be scaled instead of refused. Animated images are unaffected in either case: they play as before, however many frames they have. - A verification scan that starts during the app's initial entitlement bootstrap can no longer publish a stale result while a newer scan is still in flight (T-2152). While `entitlementState` was still `.loading`, any scan's result was accepted regardless of whether a more recent scan — for example one started right after `AppStore.sync()` — was still reading the world; the older scan finishing first could briefly flip the paywall to locked (or unlocked) ahead of the newer, more current answer. An older result that arrives while a newer scan is still outstanding is now held back rather than published. If the newer scan goes on to answer, its fresher result is published and the held-back one is simply dropped; if instead it is cancelled without ever answering, the held-back result is released, so a cancelled scan cannot leave the paywall stranded on `.loading`. The trade is that the brief loading state now ends when the last overlapping scan answers rather than the first, so it can last marginally longer; every control it gates is disabled meanwhile, so nothing silently does nothing. - Saving a pasted document to a file no longer disturbs whatever document you opened next (T-2213). A save finishes in two parts: the file is written straight away, but the document only becomes that file once its notes have been moved across, and on a slow iCloud connection that second part can still be running after you have closed the document or opened another one. When it finished late, it acted on the document then on screen instead of the one it had saved: the pasted text of that other document was deleted from the place Prism keeps unsaved documents — so it could no longer be recovered after a relaunch — its entry in Recent Files was labelled with the wrong document's title, and an action you had queued behind its own Save prompt could run without you confirming it. A save that failed to move its notes also raised an alert naming a file you were no longer looking at. Each of these now belongs to the document that was actually saved, and the document on screen is left alone. Its Recent Files entry is labelled with its own title rather than the other document's. Where that other document had itself started saving in the meantime, the late save no longer takes over the shortcut that document had prepared for its own file, which can leave the saved file without a Recent Files entry of its own. The file is saved either way, and can be opened from the Files app.@@ -55,7 +56,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Signing into iCloud with a document open no longer costs you the notes you already had on it (T-1811). Opening a document while signed out leaves its notes unread — there is nowhere to read them from — and signing in afterwards made notes available again without ever going back for them, so the first note you added was saved as if it were the only one the document had ever had, replacing every note already stored for it. Signing in now reloads the open document's notes straight away, so they reappear without closing and reopening the file, and a note added in the moment before that finishes waits for it rather than racing it. Saving a note can no longer replace notes it has not read, on any path: if nothing has been loaded for a document, what is already stored is read first and the new note is added to it. A note is also always saved to the document you are actually reading — opening a different file in the moment after signing in leaves the first one alone, and the notes you add then belong to the file in front of you rather than to the one you left. - Signing into iCloud with a document open no longer costs you the notes you already had on it (T-1811). Opening a document while signed out leaves its notes unread — there is nowhere to read them from — and signing in afterwards made notes available again without ever going back for them, so the first note you added was saved as if it were the only one the document had ever had, replacing every note already stored for it. Signing in now reloads the open document's notes straight away, so they reappear without closing and reopening the file, and a note added in the moment before that finishes waits for it rather than racing it. Saving a note can no longer replace notes it has not read, on any path: if nothing has been loaded for a document, what is already stored is read first and the new note is added to it — and two notes added in the same moment now both survive, rather than the second saving over the first. A note is also always saved to the document you are actually reading — opening a different file in the moment after signing in leaves the first one alone, and the notes you add then belong to the file in front of you rather than to the one you left. - Documents containing images written in raw HTML no longer stall while opening (T-1951). Two steps of that work cost time in proportion to the *square* of what they were given, so a document that would otherwise open instantly could hold the app for tens of seconds. Half-written image tags — a run of `<img ` with no closing bracket, which is what a document being written, generated, or truncated mid-tag looks like — were the worst of it: 0.6 seconds for 8 KB of them, 35 seconds for 70 KB, and that step runs on every piece of raw HTML in a document before anything is drawn. A single image carrying a long list of alternative sizes cost 2.8 seconds for a 62 KB list. Both now take under a millisecond and a millisecond and a half respectively, and both grow in step with the length of the document rather than with its square. Because a document can be opened from a URL, a file written this way could previously have kept a device busy for a long time on someone else's behalf. Nothing about how such documents are displayed changes: each replacement was checked against the exact step it replaced over more than a million generated fragments, character for character. One thing deliberately left standing is a `>` written inside a quoted attribute value, as in `<img src="a>b">`, which still ends the tag early and is tracked separately (T-1976). With these two, every step of the raw-HTML image path that was known to slow down this way has now been fixed.-- Documents containing images written in raw HTML no longer stall while opening (T-1951). Three steps of that work cost time in proportion to the *square* of what they were given, so a document that would otherwise open instantly could hold the app for tens of seconds. Half-written image tags — a run of `<img ` with no closing bracket, which is what a document being written, generated, or truncated mid-tag looks like — were the worst of it: 0.6 seconds for 8 KB of them, 35 seconds for 70 KB, and that step runs on every piece of raw HTML in a document before anything is drawn. A single image carrying a long list of alternative sizes cost 2.8 seconds for a 62 KB list. And the removal of HTML comments, which runs even earlier on the same raw HTML, slowed the same way on a run of comment openers with no close — 2.3 seconds for 32 KB of them — a case found while this fix was being reviewed rather than by the original report. All three now take a few milliseconds at most, and grow in step with the length of the document rather than with its square. Because a document can be opened from a URL, a file written this way could previously have kept a device busy for a long time on someone else's behalf. Nothing about how such documents are displayed changes: each replacement was checked against the exact step it replaced over more than a million generated fragments, character for character — with one deliberate exception too archaic to meet in practice, an image tag spelled with the mediaeval long-s (`<ſource>`), which the old matching treated as `<source>` and the new, standards-following matching does not. One thing deliberately left standing is a `>` written inside a quoted attribute value, as in `<img src="a>b">`, which still ends the tag early and is tracked separately (T-1976). With these three, every step of the raw-HTML image path that was known to slow down this way has now been fixed; one same-shaped scan on the neighbouring path — the comment removal applied when raw HTML is displayed as-is rather than as an image — was found while fixing these and is tracked separately (T-2147).+- Documents containing images written in raw HTML no longer stall while opening (T-1951). Three steps of that work cost time in proportion to the *square* of what they were given, so a document that would otherwise open instantly could hold the app for tens of seconds. Half-written image tags — a run of `<img ` with no closing bracket, which is what a document being written, generated, or truncated mid-tag looks like — were the worst of it: 0.6 seconds for 8 KB of them, 35 seconds for 70 KB, and that step runs on every piece of raw HTML in a document before anything is drawn. A single image carrying a long list of alternative sizes cost 2.8 seconds for a 62 KB list. And the removal of HTML comments, which runs even earlier on the same raw HTML, slowed the same way on a run of comment openers with no close — 2.3 seconds for 32 KB of them — a case found while this fix was being reviewed rather than by the original report. All three now take a few milliseconds at most, and grow in step with the length of the document rather than with its square. Because a document can be opened from a URL, a file written this way could previously have kept a device busy for a long time on someone else's behalf. Nothing about how such documents are displayed changes: each replacement was checked against the exact step it replaced over more than a million generated fragments, character for character — with one deliberate exception too archaic to meet in practice, an image tag spelled with the mediaeval long-s (`<ſource>`), which the old matching treated as `<source>` and the new, standards-following matching does not. One thing deliberately left standing is a `>` written inside a quoted attribute value, as in `<img src="a>b">`, which still ends the tag early and is tracked separately (T-1976). With these three, every step of the raw-HTML image path that was known to slow down this way has now been fixed; one same-shaped scan on the neighbouring path — the comment removal applied when raw HTML is displayed as-is rather than as an image — was found while fixing these and is fixed under T-2147, above. - Choosing where to go while a document reloads now takes you there (T-1975). If a file changed on disk — or a URL document was refreshed — while you had it open, and during the moment the app spends preparing the new version you picked a table-of-contents entry, tapped a note, followed a link to a heading, or stepped to a search match, the reloaded document appeared at your saved reading position instead. What you chose was handed to the copy still on screen, which was about to be replaced, so nothing was left to say where you had asked to go and restoring your place won — and on a large document, where preparing the new version takes longest, that window is at its widest. The document on screen is now treated as superseded from the moment a reload starts rather than from the moment the new version is ready, so anything you choose in between is held for the version that is coming and takes precedence over your saved place, exactly as it already did when you chose a moment later. This holds when a file changes twice in quick succession, so a second reload beginning before the first has finished preparing still takes you where you asked rather than back to your saved place. Reloads you did not navigate during still return you to where you were reading, and once the reloaded document has taken you where you asked, the next reload restores your place normally. Scrolling while a reload prepares still counts too, however you do it — dragging, a trackpad or wheel, **Page Up** and **Page Down**, or **Scroll to Top** and **Scroll to Bottom** — because the document stays in front of you and stays scrollable the whole time: the place you scroll to is the place you are returned to. - Changing the reading font or text size no longer moves you somewhere else in the document (T-1965). Both settings already applied without reloading, but they reflow the whole document and nothing put you back afterwards: raising **Larger Text** to an accessibility size makes every block roughly three times as tall, so the text you were reading slid off the bottom of the screen and left you looking at something you had already been through. The app then recorded that new spot as where you were reading, so closing and reopening the document returned you to it as well. Your place is now kept across the change — including how far into a paragraph you were, so the same words stay in front of you rather than merely the same paragraph starting at the top — and a place you were never reading can no longer be saved while the document settles. Jumping somewhere while the change is settling wins: a table-of-contents entry, a link, a note, or a search match all take you where you asked, and the re-anchoring steps aside. Collapsing the section you were reading during the change leaves you at its heading rather than at content that is no longer shown. - A document that goes blank because its rendering process stopped now restores itself (T-1943). The app has always been able to recover from this — it reloads the document and puts back your theme, your reading position, your note markers, and any active search highlights — but nothing was ever watching for the rendering process to stop, so the recovery never actually ran. A large or image-heavy document whose renderer was shut down under memory pressure therefore showed an empty page, with no error and no way back except closing the file and opening it again. The app now watches for it and recovers on the spot. An ordinary failure to load — a link that goes nowhere, an image that cannot be fetched — is told apart from a stopped renderer, so it neither causes a needless reload nor stops the app watching for a real one afterwards. The recovery also covers its own failure: if the reload it starts cannot itself load the document, that counts as the recovery failing and is tried again, instead of leaving the page blank with nothing running. A reload that neither succeeds nor fails — one that simply never finishes — is covered too: it is given a generous time limit, well beyond what even a large document takes to appear, and is then treated as a failed recovery and tried again rather than leaving the page blank indefinitely. If reloading repeatedly fails to bring the document back, the app stops retrying rather than reloading over and over — and says so, with a banner offering to reload. Taking that reload also restores the document's ability to recover on its own again, so giving up is never permanent while the file stays open.
Three independent differential fuzzes, all against verbatim reconstructions of the origin/main regexes, all zero-divergence:
isCommentOnly + matches(in:) vs structuralRegex + extractRegex, comparing the accept/reject verdict and both span lists element by element. 5,641 accepting inputs.parseBlock return value, note and conditional rejects included. 4,376 accepting inputs.stripCommentsInsideLinkLabels old right-to-left replacingCharacters loop vs the new single pass.Alphabet: <!--, -->, --!>, <!-->, <!--->, <!--a-->, <![CDATA[, <!DOCTYPE, [, ], ](u), [a], [if], [endif], comment:ab12, /comment:ff, SP, TAB, LF, CRLF, NBSP, U+2028, U+000B, U+0085, ZWSP, U+FEFF, emoji, e+U+0301.
Probed through the production regex: \s matches TAB, LF, VT (U+000B), FF, CR, SP, NEL (U+0085), NBSP (U+00A0), U+1680, U+2000, U+2028, U+2029, U+202F, U+205F, U+3000; it does not match ZWSP (U+200B), U+FEFF or U+180E. CharacterSet.whitespacesAndNewlines differs (U+180E is a known divergence). Anywhere in this file that a hand-rolled set is substituted for \S — including any future "fast reject" using hasPrefix on a trimmingCharacters result — reintroduces exactly the drift the current code is designed to make impossible.
Nothing maps an offset in baseWithoutMarkers or scanSource back to the source. The only place badge occurrences are matched by UTF-16 source offset — SearchStateFeeder.badgeEligibility — iterates the raw source string, not footnoteScanComponents. InlineHTMLRenderer, DocumentSourceMap, FootnoteReferenceScanner and BlockHTMLEmitter call none of the changed helpers; the single overlap is InlineHTMLRenderer.swift:613 calling parseBlock, and the source-map cursor there is advanced by consumingVisibleText before the classifier runs and independently of its verdict.
make lint — 0 violations, 0 serious, 555 files.make build-macos — Build Succeeded, no new warnings (only the pre-existing "Stamp git commit hash" run-script note).make verify-test-isolation — OK, plus its own 43 unit tests.xcodebuild build-for-testing (macOS) — TEST BUILD SUCCEEDED.HTMLCommentParserTests, ImageParserCommentStrippingTests): total=57 passed=57 failed=0. Bundle 2 (HTMLCommentStrippingTests, HTMLCommentStrippingGrowthTests — G1–G16 — and HTMLCommentParserStructuralEquivalenceTests): total=41 passed=41 failed=0. Both confirmed by Tools/check-test-results.sh, so the bundles really executed what they claim.structs whose -only-testing: identifier is the TYPE name (HTMLCommentStrippingGrowthTests), not the @Suite display string. My first run filtered on the display-derived name, matched nothing, and reported TEST EXECUTE SUCCEEDED having silently skipped all three — exactly the failure mode check-test-results.sh exists to catch, and it caught it.origin/main at 8063d330.@MainActor and every test is synchronous, but nothing in them can reach WebKit (pure Foundation), which is why the isolation guard passes.inlineCommentRegex and :93 "precompiled regexes" — both rewritten. No reference to the deleted symbol survives outside the decision log's historical Context, where it belongs.Equatable, Sendable added.removing(_:from:).