Pre-push review (round 2): T-2243 test-result guard fail-open

diff --git a/Tools/check-test-results.sh b/Tools/check-test-results.shindex 811be9f3..64855efb 100755--- a/Tools/check-test-results.sh+++ b/Tools/check-test-results.sh@@ -28,6 +28,21 @@ #     PARTIAL bundle: one or more passing tests, zero recorded failures, and a #     top-level result of "unknown" rather than "Passed". Counts alone look like #     a clean pass (T-1993).+#   * `xcresulttool` CAN exit non-zero while still printing well-formed,+#     parseable JSON on stdout — this is a documented general property of the+#     tool, not something reproduced against a real bundle here. The two calls+#     below used to check only "is stdout non-empty", and a stub built to+#     model it (print the clean fixture JSON, exit 7) confirmed the gap: both+#     calls still produced "OK" and exit 0 (T-2243).+#   * The schema allows a per-test or per-attempt node to carry the+#     schema-valid but UNRESOLVED result "unknown", a `children` field that is+#     present but not an array (hiding whatever failure is nested inside it),+#     or a grouping node (e.g. Test Plan) carrying its own invalid/unresolved+#     result. All three used to be silently accepted or silently emptied+#     rather than treated as unreadable (T-2243) — guarded against on the+#     schema's authority, since no real bundle read here has ever shown any+#     of the three; see the fixed guards' own comments for what has and has+#     not been confirmed against real data. # # The zero-test guard below is the fix for the "launched nothing" shape, together # with its skipped-run variant (a run whose every selected test was skipped, which@@ -82,8 +97,14 @@ if [ ! -e "$BUNDLE" ]; then fi  SUMMARY=$(xcrun xcresulttool get test-results summary --path "$BUNDLE" 2>/dev/null)-if [ -z "$SUMMARY" ]; then-    echo "FAIL [$LABEL]: could not read a test summary from $BUNDLE" >&2+SUMMARY_STATUS=$?+# A non-empty SUMMARY does not mean the extraction succeeded: xcresulttool can+# exit non-zero while still printing valid, parseable JSON (a partial read+# that stops after emitting a complete-looking document). Checking emptiness+# alone let a stub that printed the clean fixture and exited 7 report OK+# (T-2243), so the exit status is required in addition to non-empty output.+if [ "$SUMMARY_STATUS" -ne 0 ] || [ -z "$SUMMARY" ]; then+    echo "FAIL [$LABEL]: could not read a test summary from $BUNDLE (xcresulttool exited $SUMMARY_STATUS)" >&2     exit 1 fi @@ -348,8 +369,11 @@ fi # Test Case -> Test Plan Configuration nesting is measured on real bundles from this # project, and is what `make test` and `make test-ui` produce on every run. TESTS_JSON=$(xcrun xcresulttool get test-results tests --path "$BUNDLE" 2>/dev/null)-if [ -z "$TESTS_JSON" ]; then-    echo "FAIL [$LABEL]: could not read the per-test tree from $BUNDLE." >&2+TESTS_STATUS=$?+# Same non-empty-is-not-enough reasoning as the summary read above: a non-zero+# exit means this extraction failed too, whatever it printed (T-2243).+if [ "$TESTS_STATUS" -ne 0 ] || [ -z "$TESTS_JSON" ]; then+    echo "FAIL [$LABEL]: could not read the per-test tree from $BUNDLE (xcresulttool exited $TESTS_STATUS)." >&2     echo "  The top-level counts said $TOTAL test(s) ran, but the detail this script" >&2     echo "  needs to rule out a retry-laundered failure is unreadable. Refusing to" >&2     echo "  report success on a run that cannot be fully verified." >&2@@ -383,9 +407,52 @@ def clean(text):     # so that a test name spelled with double spaces comes out as the author wrote it.     return "".join(" " if (ord(c) < 0x20 or ord(c) == 0x7f) else c for c in str(text)) -def kids(node):+def kids(node, label, report=True):+    # A `children` field that is present but not an array is malformed, not+    # empty: an object or scalar in its place can still be hiding a failed+    # nested attempt. The original version coerced any non-list value to []+    # and moved on silently, which is indistinguishable from "no children" —+    # exactly the shape that hid a failed repetition (T-2243). A missing+    # field (None) is not malformed; plenty of leaf nodes carry none. That is+    # a deliberate choice, not an oversight: `dict.get` cannot tell an+    # explicit JSON `null` from an absent key, both come back as this same+    # `None`, and there is nothing more informative to recover from a null+    # than from an absent key — so both read as "no children". Only a+    # present, non-list value (an object, a string, a number) is proof of a+    # shape this script does not understand.+    #+    # report=False is for collect_attempts(), which calls this on nodes it+    # does not own the label of — see its own comment for why silencing the+    # problem here (rather than passing a label) is the fix, not a suppression.     children = node.get("children")-    return children if isinstance(children, list) else []+    if children is None:+        return []+    if isinstance(children, list):+        return children+    if report:+        problems.append("node %s has a non-array children field, so its subtree cannot be read" % label)+    return []++def check_result(label, result):+    # The one place every node result is judged, whatever kind of node it is+    # on — Test Case, attempt, or a grouping node like Test Plan. `None` (no+    # result recorded) is not itself flagged here: not every node type+    # carries one, and a caller that requires a result on THIS node handles a+    # missing one with its own diagnosis. Returns True only for a resolved,+    # schema-valid outcome. "unknown" is schema-valid (it is a real member of+    # the TestResult enum) but never resolved, so it is rejected exactly like+    # an enum violation — an unknown Test Case or attempt result used to sail+    # through because it is a member of RESULTS, and an unknown result on a+    # grouping node was never even looked at (T-2243).+    if result is None:+        return False+    if result not in RESULTS:+        problems.append("node %s has result %r, which is not a TestResult" % (label, clean(result)))+        return False+    if result == "unknown":+        problems.append("node %s has result unknown, which is schema-valid but never resolved" % label)+        return False+    return True  def collect_attempts(node, acc, label):     # Attempts are gathered from the WHOLE subtree of a Test Case, not from its@@ -410,7 +477,22 @@ def collect_attempts(node, acc, label):         problems.append("a node under %s is not a JSON object" % label)         return     before = len(acc)-    for child in kids(node):+    # report=False: this recursion walks every descendant of the Test Case+    # under the Test Case OWN label, not each descendant own path label+    # (that label lives only in walk() `here`, which this function never+    # sees). walk() below independently recurses into every one of these same+    # descendants and calls kids() on each with its correct own label, so a+    # malformed `children` field anywhere in the subtree is already reported+    # once, correctly labelled, by that traversal. Reporting here too does+    # not add a missed case, it duplicates that one under the wrong label:+    # a Repetition own malformed `children` came out as belonging to the Test+    # Case itself. Round 3 fixed this only where the malformed field sits+    # directly on the Test Case (walk() and this loop shared one already-computed+    # `kids()` result there); one level deeper, the two calls were independent+    # and both reported. Silencing the report here, rather than threading each+    # descendant own label through, is what closes that gap for every depth,+    # not just the Test Case own children.+    for child in kids(node, label, report=False):         collect_attempts(child, acc, label)     if node.get("nodeType") in ATTEMPT_TYPES:         # Post-order on purpose: children are collected first, so an attempt node@@ -418,15 +500,25 @@ def collect_attempts(node, acc, label):         # satisfied by a nested attempt that does. Only when NOTHING in the subtree         # produced a result is the attempt unreadable — and an unreadable attempt is         # reported, never treated as "no failure here".+        #+        # This deliberately does NOT call check_result on the attempt node own+        # result. walk() already does that for every node in the tree,+        # attempts included (see the walk() comment on why excluding them there+        # was wrong) — calling it here too would report the SAME bad result+        # twice, under two different labels: this function uses the OWNING+        # Test Case label, while walk() reaches this same node on its own+        # path and reports it under the attempt node own label. So this is+        # purely a membership test for "may this result be folded into the+        # retry-detection accumulator" — "unknown" is schema-valid but never+        # resolved, so it is excluded here exactly as check_result would+        # reject it, without appending a second problem line for it.         result = node.get("result")-        if result in RESULTS:+        if result is None:+            if len(acc) == before:+                problems.append("an attempt node under %s records no result, and neither does "+                                "anything nested inside it" % label)+        elif result in RESULTS and result != "unknown":             acc.append(result)-        elif result is not None:-            problems.append("an attempt node under %s has result %r, which is not a TestResult"-                            % (label, clean(result)))-        elif len(acc) == before:-            problems.append("an attempt node under %s records no result, and neither does "-                            "anything nested inside it" % label)  def walk(node, path):     if not isinstance(node, dict):@@ -438,14 +530,59 @@ def walk(node, path):         problems.append("unrecognised nodeType %r in the tests tree" % clean(node_type))     here = path + [name] if (node_type and name) else path     label = "/".join(p for p in here if p) or "(unnamed)"+    children = kids(node, label)++    # Validate the result recorded directly on this node, whatever kind of node+    # it is — including an attempt node (Repetition / Test Case Run). Previously+    # only Test Case and attempt nodes were checked, so an invalid or unresolved+    # result on a grouping node (Test Plan, Test Suite, a bundle node, ...) was+    # ignored outright (T-2243). Attempt nodes used to be excluded here too, on+    # the theory that collect_attempts() validates them instead — but+    # collect_attempts is reached ONLY from the Test Case branch below, walking+    # that one Test Case own subtree. An attempt node sitting anywhere ELSE in+    # the tree — outside any Test Case, a shape no real bundle here has shown+    # but nothing in the schema forbids either — was never visited by+    # collect_attempts, and with the exclusion was never checked here either:+    # an orphaned attempt recording "unknown" or an out-of-enum result reported+    # OK (T-2243). check_result now runs on every node type without exception,+    # and collect_attempts no longer calls it a second time for the same node+    # (see the comment there for why) — this is the one and only place a bad+    # attempt result gets reported, whichever Test Case, if any, it sits under.+    #+    # Measured on 13 real bundles from this machine (3 clean passes of 49-50+    # tests, 7 with genuine failures including a 3759-test cascade, 3 zero-test+    # host-launch failures): every grouping node TYPE ACTUALLY PRESENT in them —+    # Test Plan, Unit test bundle, Test Suite, Test Plan Configuration — carries+    # Passed, Failed or Skipped, and the only nodes with no result at all are+    # Failure Message nodes, which check_result deliberately does not flag. No+    # real bundle carried a non-array children field, and none produced a+    # problem line from this check; the 3 clean passes still reported OK. That+    # survey never turned up an Attachment, Expression, Test Value, or Runtime+    # Warning node at all, so how this check behaves on those types is guarded+    # against, not confirmed — the same schema-derived-not-measured distinction+    # the retry scan attempt-node handling already carries below. Known+    # residual risk, no counterexample found in a real bundle but not disproved:+    # a grouping node recording "unknown" on a run that is otherwise complete.+    # Treating that as a FAIL is the intended direction — it is the same "never+    # reached a verdict" marker the top-level guard already refuses — so a first+    # such sighting should be read as a partial extraction, not as this check+    # crying wolf.+    result = node.get("result")+    check_result(label, result)      if node_type == "Test Case":         counts["cases"] += 1-        result = node.get("result")-        if result is not None and result not in RESULTS:-            problems.append("test case %s has result %r, which is not a TestResult"-                            % (label, clean(result)))-        if result == "Failed":+        if result is None:+            # check_result lets a missing result pass silently, because plenty+            # of node types legitimately carry none (Failure Message, an+            # attempt whose own verdict lives on a nested attempt instead). A+            # Test Case is not one of them: it is the leaf-most node carrying+            # test IDENTITY, and everything above only ever compared this+            # value against "Failed" — so a Test Case whose result was absent+            # or null was counted as a case and never flagged, sitting inside+            # a summary that otherwise looked like a clean pass (T-2243).+            problems.append("Test Case %s has no recorded result at all" % label)+        elif result == "Failed":             counts["failed_cases"] += 1             # A cascade artefact is a failed Test Case with NO recorded duration:             # the test never ran because the host was already dead when it was@@ -478,12 +615,12 @@ def walk(node, path):         # iteration genuinely recorded Failed while the case still ended green —         # which IS the laundering shape.         attempts = []-        for child in kids(node):+        for child in children:             collect_attempts(child, attempts, label)         if result != "Failed" and "Failed" in attempts:             retried.append(label) -    for child in kids(node):+    for child in children:         walk(child, here)  try:

diff --git a/Tools/Tests/test-check-test-results.sh b/Tools/Tests/test-check-test-results.shindex 1fd0a772..4bdc6185 100755--- a/Tools/Tests/test-check-test-results.sh+++ b/Tools/Tests/test-check-test-results.sh@@ -1,9 +1,9 @@ #!/bin/bash # # test-check-test-results.sh — regression tests for Tools/check-test-results.sh-# (T-2224 / T-1993).+# (T-2224 / T-1993 / T-2243). #-# Two shapes let a bad run report OK before this suite existed:+# Three kinds of shape let a bad run report OK before this suite existed: # #   * T-2224: a test can FAIL on its first attempt and PASS on a retry. The #     bundle's top-level counts record only the final attempt, so the failure@@ -14,8 +14,22 @@ #     readable PARTIAL bundle with passing-looking counts (zero recorded #     failures) while the bundle's own top-level `result` field is "unknown" #     rather than "Passed" — the run never reached a real verdict.+#   * T-2243: six more fail-open paths, all beneath the top-level summary.+#     Both `xcresulttool` command substitutions accepted any non-empty stdout+#     without checking the command's own exit status; "unknown" (a genuine+#     TestResult enum member, just never a RESOLVED one) was accepted at the+#     Test Case and attempt level exactly like a real outcome; a `children`+#     field that was present but not an array was silently read as "no+#     children" rather than as malformed; and result validation never looked+#     at grouping nodes (Test Plan, Test Suite, ...) at all. A follow-up review+#     of that same fix found two more, still in the same family: an attempt+#     node was validated only when it happened to sit inside some Test Case's+#     own subtree, so one recorded OUTSIDE that scope (a shape no real bundle+#     has shown, but nothing in the schema forbids) reported OK; and a Test+#     Case whose `result` was absent or null was counted as a case and never+#     flagged, because every check on it only ever compared against "Failed". #-# Both are invisible to a black-box "does it exit 0" smoke test on a real+# All of them are invisible to a black-box "does it exit 0" smoke test on a real # xcodebuild run: reproducing them for real requires flaky live tests or a # wedged host, which is exactly the kind of thing this suite must not depend # on to be repeatable. Instead each fixture under@@ -72,6 +86,12 @@ mkdir -p "$SANDBOX/bin" # --path <dir>`, and serves that directory's summary.json / tests.json. Any # other invocation exits 1, so a real xcrun elsewhere on PATH is never reached # by accident.+#+# A fixture that wants to model a NONZERO xcresulttool exit alongside valid+# stdout (T-2243 — the exit status of both calls used to be discarded, so a+# stub that printed clean JSON and exited 7 still reported OK) drops the+# desired exit code in summary.exit / tests.exit next to the JSON file. Its+# absence means "behave as before": exit with whatever `cat` itself returned. cat > "$SANDBOX/bin/xcrun" <<'EOF' #!/bin/bash subcmd="${4:-}"@@ -82,8 +102,18 @@ for arg in "$@"; do     prev="$arg" done case "${1:-}/${2:-}/${3:-}/$subcmd" in-    xcresulttool/get/test-results/summary) exec cat "$path/summary.json" ;;-    xcresulttool/get/test-results/tests)   exec cat "$path/tests.json" ;;+    xcresulttool/get/test-results/summary)+        cat "$path/summary.json"+        status=$?+        if [ -f "$path/summary.exit" ]; then exit "$(cat "$path/summary.exit")"; fi+        exit "$status"+        ;;+    xcresulttool/get/test-results/tests)+        cat "$path/tests.json"+        status=$?+        if [ -f "$path/tests.exit" ]; then exit "$(cat "$path/tests.exit")"; fi+        exit "$status"+        ;;     *) exit 1 ;; esac EOF@@ -293,7 +323,10 @@ expect_fail "retried-newest-first" \ expect_fail "retried-single-attempt" \     "a single recorded attempt of Failed under a non-Failed Test Case is caught" \     "passed only on retry"-expect_fail "attempt-unknown-result" \+# Fixture name is "invalid", not "unknown": its Repetition carries "Borked",+# which is outside the TestResult enum entirely — "unknown" (in-enum but+# unresolved) is covered separately below, under the T-2243 section.+expect_fail "attempt-invalid-result" \     "an attempt result outside the TestResult enum fails closed rather than counting as no failure" \     "which is not a TestResult" expect_fail "attempt-no-result" \@@ -358,6 +391,132 @@ else     fail "the retried test name was reformatted before being reported"$'\n'"$OUT" fi +echo+echo "--- T-2243: xcresulttool's own exit status must be checked ------------"+# Both command substitutions used to check only "is stdout non-empty", so a+# stub that printed the clean fixture JSON and exited non-zero for either call+# still produced OK. The fixtures' summary.json/tests.json are byte-identical+# to clean-pass; only the stub's exit status (summary.exit / tests.exit) makes+# them fail, which is the whole point — a non-empty, parseable read is not the+# same thing as a successful one.+#+# The two fixtures use DIFFERENT exit codes (7 and 9) and the needles check+# for each one specifically, not just the shared "xcresulttool exited" text —+# otherwise a bug that fired the wrong one of the two guards (summary's status+# checked in place of the tests read's, say) would still match either needle+# and this pair would not catch it.++expect_fail "xcresulttool-nonzero-summary" \+    "a nonzero exit from the summary extraction fails even though stdout parses cleanly" \+    "could not read a test summary from"+run "$FIXTURES/xcresulttool-nonzero-summary" "xcresulttool-nonzero-summary"+if saw "xcresulttool exited 7"; then+    pass "the summary guard reports its own exit code (7), not the tests guard's"+else+    fail "the summary guard did not report exit code 7"$'\n'"$OUT"+fi++expect_fail "xcresulttool-nonzero-tests" \+    "a nonzero exit from the per-test tree extraction fails even though stdout parses cleanly" \+    "could not read the per-test tree from"+run "$FIXTURES/xcresulttool-nonzero-tests" "xcresulttool-nonzero-tests"+if saw "xcresulttool exited 9"; then+    pass "the tests guard reports its own exit code (9), not the summary guard's"+else+    fail "the tests guard did not report exit code 9"$'\n'"$OUT"+fi++echo+echo "--- T-2243: 'unknown' is unresolved, not merely a valid enum member ---"+# "unknown" is a genuine member of the TestResult enum (it is the top-level+# run's own abnormal-result marker, per the T-1993 guard above), so a naive+# `result in RESULTS` check waves it through anywhere else in the tree too.+# Only `Passed`/`Failed`/`Skipped`/`Expected Failure` are RESOLVED outcomes;+# `unknown` at the Test Case or attempt level means the same "never reached a+# verdict" as it does at the top level, and must fail the same way.++expect_fail "unknown-test-case-result" \+    "a Test Case with result 'unknown' fails, even though the summary counts look like a clean pass" \+    "schema-valid but never resolved"+expect_fail "unknown-attempt-in-passed-case" \+    "a Passed Test Case whose attempt node resolved to 'unknown' still fails" \+    "schema-valid but never resolved"++echo+echo "--- T-2243: malformed children must not be read as empty children -----"+# A `children` field that is present but not an array used to be coerced to []+# and treated exactly like "no children" — indistinguishable from a leaf node,+# even when the malformed value is itself hiding a failed nested attempt.++expect_fail "malformed-children-object" \+    "a Test Case whose children is an object (not an array) fails, instead of reading as childless" \+    "non-array children field"+run "$FIXTURES/malformed-children-object" "malformed-children-object"+if [ "$(grep -c 'non-array children field' <<< "$OUT")" -eq 1 ]; then+    pass "the malformed-children problem is reported once, not once per pass over the node"+else+    fail "the malformed-children problem was reported more than once for the same node"$'\n'"$OUT"+fi++# The Test Case's OWN malformed children (above) is reported once because+# walk() and collect_attempts() share one already-computed kids() result for+# that node. One level deeper — a malformed `children` on a Repetition BELOW+# an otherwise well-formed Test Case — used to slip past that sharing:+# collect_attempts() recursed into the Repetition with kids() under the+# Test Case's OWN label (report round 3 missed), while walk() separately+# recursed into the same Repetition with its own correct label — so the same+# malformed field came out reported twice, once correctly labelled as the+# Repetition's and once wrongly as the Test Case's own.+expect_fail "malformed-children-object-nested" \+    "a Repetition nested under a Test Case whose own children is fine still fails on its own malformed children" \+    "Repetition 1 has a non-array children field"+run "$FIXTURES/malformed-children-object-nested" "malformed-children-object-nested"+if [ "$(grep -c 'non-array children field' <<< "$OUT")" -eq 1 ]; then+    pass "the nested malformed-children problem is reported exactly once"+else+    fail "the nested malformed-children problem was reported more than once"$'\n'"$OUT"+fi+if saw "testOne() has a non-array children field"; then+    fail "the nested malformed-children problem was mislabelled as belonging to the Test Case itself"$'\n'"$OUT"+else+    pass "the nested malformed-children problem is not mislabelled as the Test Case's own"+fi++echo+echo "--- T-2243: grouping nodes are validated too, not just Test Case ------"+# Result validation used to stop at Test Case and attempt nodes. A grouping+# node such as Test Plan or Test Suite can carry its own `result`, and an+# invalid or unresolved value there was ignored outright.++expect_fail "grouping-node-unknown-result" \+    "an unresolved 'unknown' result on the Test Plan root node fails, not just a Test Case's own result" \+    "schema-valid but never resolved"++echo+echo "--- T-2243 review: an attempt node is validated wherever it sits ------"+# collect_attempts() is reached only from the Test Case branch, walking that+# Test Case's own subtree. An attempt node (Repetition / Test Case Run) sitting+# OUTSIDE any Test Case used to be excluded from check_result on the theory+# that collect_attempts would validate it — it never would, since it is never+# reached. This models the orphan directly: a Repetition sits as a sibling of+# a Test Case under a Test Suite, not nested inside one.++expect_fail "orphan-attempt-node" \+    "an attempt node outside any Test Case still gets its own result validated" \+    "schema-valid but never resolved"++echo+echo "--- T-2243 review: a Test Case must record a verdict at all -----------"+# check_result lets a missing result pass silently — correct for most node+# types, since plenty legitimately carry none. A Test Case is not one of+# them, but the only place its result was ever inspected compared it against+# "Failed" alone, so a Test Case whose result was absent or null was counted+# as a case and never flagged.++expect_fail "test-case-no-result" \+    "a Test Case with no recorded result at all fails, even though the summary counts look clean" \+    "has no recorded result"+ echo echo "--- Clean shapes must still pass, unchanged ----------------------------" 

diff --git a/Tools/Tests/Fixtures/check-test-results/attempt-unknown-result/summary.json b/Tools/Tests/Fixtures/check-test-results/attempt-invalid-result/summary.jsonsimilarity index 100%rename from Tools/Tests/Fixtures/check-test-results/attempt-unknown-result/summary.jsonrename to Tools/Tests/Fixtures/check-test-results/attempt-invalid-result/summary.jsondiff --git a/Tools/Tests/Fixtures/check-test-results/attempt-unknown-result/tests.json b/Tools/Tests/Fixtures/check-test-results/attempt-invalid-result/tests.jsonsimilarity index 100%rename from Tools/Tests/Fixtures/check-test-results/attempt-unknown-result/tests.jsonrename to Tools/Tests/Fixtures/check-test-results/attempt-invalid-result/tests.jsondiff --git a/Tools/Tests/Fixtures/check-test-results/grouping-node-unknown-result/summary.json b/Tools/Tests/Fixtures/check-test-results/grouping-node-unknown-result/summary.jsonnew file mode 100644index 00000000..cd2220e7--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/grouping-node-unknown-result/summary.json@@ -0,0 +1,16 @@+{+  "title": "Test - prism",+  "startTime": 1000000000,+  "finishTime": 1000000001,+  "environmentDescription": "prism · Debug · macOS",+  "topInsights": [],+  "result": "Passed",+  "totalTestCount": 1,+  "passedTests": 1,+  "failedTests": 0,+  "skippedTests": 0,+  "expectedFailures": 0,+  "statistics": [],+  "devicesAndConfigurations": [],+  "testFailures": []+}diff --git a/Tools/Tests/Fixtures/check-test-results/grouping-node-unknown-result/tests.json b/Tools/Tests/Fixtures/check-test-results/grouping-node-unknown-result/tests.jsonnew file mode 100644index 00000000..57900c17--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/grouping-node-unknown-result/tests.json@@ -0,0 +1,34 @@+{+  "testPlanConfigurations": [],+  "devices": [],+  "testNodes": [+    {+      "nodeType": "Test Plan",+      "name": "prism",+      "result": "unknown",+      "children": [+        {+          "nodeType": "Unit test bundle",+          "name": "prismTests",+          "result": "Passed",+          "children": [+            {+              "nodeType": "Test Suite",+              "name": "SampleTests",+              "result": "Passed",+              "children": [+                {+                  "nodeType": "Test Case",+                  "name": "testOne()",+                  "result": "Passed",+                  "duration": "0.01s",+                  "children": []+                }+              ]+            }+          ]+        }+      ]+    }+  ]+}diff --git a/Tools/Tests/Fixtures/check-test-results/malformed-children-object-nested/summary.json b/Tools/Tests/Fixtures/check-test-results/malformed-children-object-nested/summary.jsonnew file mode 100644index 00000000..cd2220e7--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/malformed-children-object-nested/summary.json@@ -0,0 +1,16 @@+{+  "title": "Test - prism",+  "startTime": 1000000000,+  "finishTime": 1000000001,+  "environmentDescription": "prism · Debug · macOS",+  "topInsights": [],+  "result": "Passed",+  "totalTestCount": 1,+  "passedTests": 1,+  "failedTests": 0,+  "skippedTests": 0,+  "expectedFailures": 0,+  "statistics": [],+  "devicesAndConfigurations": [],+  "testFailures": []+}diff --git a/Tools/Tests/Fixtures/check-test-results/malformed-children-object-nested/tests.json b/Tools/Tests/Fixtures/check-test-results/malformed-children-object-nested/tests.jsonnew file mode 100644index 00000000..d3394826--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/malformed-children-object-nested/tests.json@@ -0,0 +1,46 @@+{+  "testPlanConfigurations": [],+  "devices": [],+  "testNodes": [+    {+      "nodeType": "Test Plan",+      "name": "prism",+      "result": "Passed",+      "children": [+        {+          "nodeType": "Unit test bundle",+          "name": "prismTests",+          "result": "Passed",+          "children": [+            {+              "nodeType": "Test Suite",+              "name": "SampleTests",+              "result": "Passed",+              "children": [+                {+                  "nodeType": "Test Case",+                  "name": "testOne()",+                  "result": "Passed",+                  "duration": "0.01s",+                  "children": [+                    {+                      "nodeType": "Repetition",+                      "name": "Repetition 1",+                      "result": "Passed",+                      "duration": "0.1s",+                      "children": {+                        "nodeType": "Test Case Run",+                        "name": "Run 1",+                        "result": "Failed"+                      }+                    }+                  ]+                }+              ]+            }+          ]+        }+      ]+    }+  ]+}diff --git a/Tools/Tests/Fixtures/check-test-results/malformed-children-object/summary.json b/Tools/Tests/Fixtures/check-test-results/malformed-children-object/summary.jsonnew file mode 100644index 00000000..cd2220e7--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/malformed-children-object/summary.json@@ -0,0 +1,16 @@+{+  "title": "Test - prism",+  "startTime": 1000000000,+  "finishTime": 1000000001,+  "environmentDescription": "prism · Debug · macOS",+  "topInsights": [],+  "result": "Passed",+  "totalTestCount": 1,+  "passedTests": 1,+  "failedTests": 0,+  "skippedTests": 0,+  "expectedFailures": 0,+  "statistics": [],+  "devicesAndConfigurations": [],+  "testFailures": []+}diff --git a/Tools/Tests/Fixtures/check-test-results/malformed-children-object/tests.json b/Tools/Tests/Fixtures/check-test-results/malformed-children-object/tests.jsonnew file mode 100644index 00000000..b0cb3319--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/malformed-children-object/tests.json@@ -0,0 +1,39 @@+{+  "testPlanConfigurations": [],+  "devices": [],+  "testNodes": [+    {+      "nodeType": "Test Plan",+      "name": "prism",+      "result": "Passed",+      "children": [+        {+          "nodeType": "Unit test bundle",+          "name": "prismTests",+          "result": "Passed",+          "children": [+            {+              "nodeType": "Test Suite",+              "name": "SampleTests",+              "result": "Passed",+              "children": [+                {+                  "nodeType": "Test Case",+                  "name": "testOne()",+                  "result": "Passed",+                  "duration": "0.01s",+                  "children": {+                    "nodeType": "Repetition",+                    "name": "Repetition 1",+                    "result": "Failed",+                    "duration": "0.1s"+                  }+                }+              ]+            }+          ]+        }+      ]+    }+  ]+}diff --git a/Tools/Tests/Fixtures/check-test-results/orphan-attempt-node/summary.json b/Tools/Tests/Fixtures/check-test-results/orphan-attempt-node/summary.jsonnew file mode 100644index 00000000..cd2220e7--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/orphan-attempt-node/summary.json@@ -0,0 +1,16 @@+{+  "title": "Test - prism",+  "startTime": 1000000000,+  "finishTime": 1000000001,+  "environmentDescription": "prism · Debug · macOS",+  "topInsights": [],+  "result": "Passed",+  "totalTestCount": 1,+  "passedTests": 1,+  "failedTests": 0,+  "skippedTests": 0,+  "expectedFailures": 0,+  "statistics": [],+  "devicesAndConfigurations": [],+  "testFailures": []+}diff --git a/Tools/Tests/Fixtures/check-test-results/orphan-attempt-node/tests.json b/Tools/Tests/Fixtures/check-test-results/orphan-attempt-node/tests.jsonnew file mode 100644index 00000000..7b60f84e--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/orphan-attempt-node/tests.json@@ -0,0 +1,41 @@+{+  "testPlanConfigurations": [],+  "devices": [],+  "testNodes": [+    {+      "nodeType": "Test Plan",+      "name": "prism",+      "result": "Passed",+      "children": [+        {+          "nodeType": "Unit test bundle",+          "name": "prismTests",+          "result": "Passed",+          "children": [+            {+              "nodeType": "Test Suite",+              "name": "SampleTests",+              "result": "Passed",+              "children": [+                {+                  "nodeType": "Test Case",+                  "name": "testOne()",+                  "result": "Passed",+                  "duration": "0.01s",+                  "children": []+                },+                {+                  "nodeType": "Repetition",+                  "name": "Repetition 1",+                  "result": "unknown",+                  "duration": "0.9s",+                  "children": []+                }+              ]+            }+          ]+        }+      ]+    }+  ]+}diff --git a/Tools/Tests/Fixtures/check-test-results/test-case-no-result/summary.json b/Tools/Tests/Fixtures/check-test-results/test-case-no-result/summary.jsonnew file mode 100644index 00000000..cd2220e7--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/test-case-no-result/summary.json@@ -0,0 +1,16 @@+{+  "title": "Test - prism",+  "startTime": 1000000000,+  "finishTime": 1000000001,+  "environmentDescription": "prism · Debug · macOS",+  "topInsights": [],+  "result": "Passed",+  "totalTestCount": 1,+  "passedTests": 1,+  "failedTests": 0,+  "skippedTests": 0,+  "expectedFailures": 0,+  "statistics": [],+  "devicesAndConfigurations": [],+  "testFailures": []+}diff --git a/Tools/Tests/Fixtures/check-test-results/test-case-no-result/tests.json b/Tools/Tests/Fixtures/check-test-results/test-case-no-result/tests.jsonnew file mode 100644index 00000000..aa923cda--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/test-case-no-result/tests.json@@ -0,0 +1,33 @@+{+  "testPlanConfigurations": [],+  "devices": [],+  "testNodes": [+    {+      "nodeType": "Test Plan",+      "name": "prism",+      "result": "Passed",+      "children": [+        {+          "nodeType": "Unit test bundle",+          "name": "prismTests",+          "result": "Passed",+          "children": [+            {+              "nodeType": "Test Suite",+              "name": "SampleTests",+              "result": "Passed",+              "children": [+                {+                  "nodeType": "Test Case",+                  "name": "testOne()",+                  "duration": "0.01s",+                  "children": []+                }+              ]+            }+          ]+        }+      ]+    }+  ]+}diff --git a/Tools/Tests/Fixtures/check-test-results/unknown-attempt-in-passed-case/summary.json b/Tools/Tests/Fixtures/check-test-results/unknown-attempt-in-passed-case/summary.jsonnew file mode 100644index 00000000..eb69d5b2--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/unknown-attempt-in-passed-case/summary.json@@ -0,0 +1,16 @@+{+  "title": "Test - prism",+  "startTime": 1000000000,+  "finishTime": 1000000001,+  "environmentDescription": "prism · Debug · iOS Simulator",+  "topInsights": [],+  "result": "Passed",+  "totalTestCount": 1,+  "passedTests": 1,+  "failedTests": 0,+  "skippedTests": 0,+  "expectedFailures": 0,+  "statistics": [],+  "devicesAndConfigurations": [],+  "testFailures": []+}diff --git a/Tools/Tests/Fixtures/check-test-results/unknown-attempt-in-passed-case/tests.json b/Tools/Tests/Fixtures/check-test-results/unknown-attempt-in-passed-case/tests.jsonnew file mode 100644index 00000000..1f4b8580--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/unknown-attempt-in-passed-case/tests.json@@ -0,0 +1,65 @@+{+  "testPlanConfigurations": [+    {+      "configurationId": "1",+      "configurationName": "en (base)"+    }+  ],+  "devices": [+    {+      "deviceId": "D1",+      "deviceName": "iPhone 17 Pro",+      "architecture": "arm64",+      "modelName": "iPhone 17 Pro",+      "osVersion": "26.0"+    }+  ],+  "testNodes": [+    {+      "nodeType": "Test Plan",+      "name": "prism",+      "result": "Passed",+      "children": [+        {+          "nodeType": "Unit test bundle",+          "name": "prismTests",+          "result": "Passed",+          "children": [+            {+              "nodeType": "Test Suite",+              "name": "WebContentTerminationWiringTests",+              "result": "Passed",+              "children": [+                {+                  "name": "controllerObservesItsOwnPageNavigationStream()",+                  "nodeIdentifier": "WebContentTerminationWiringTests/controllerObservesItsOwnPageNavigationStream()",+                  "nodeIdentifierURL": "test://com.apple.xcode/prism/prismTests/WebContentTerminationWiringTests/controllerObservesItsOwnPageNavigationStream()",+                  "nodeType": "Test Case",+                  "result": "Passed",+                  "duration": "0.1s",+                  "durationInSeconds": 0.104,+                  "children": [+                    {+                      "nodeType": "Repetition",+                      "name": "Repetition 1",+                      "result": "unknown",+                      "duration": "0.9s",+                      "children": []+                    },+                    {+                      "nodeType": "Repetition",+                      "name": "Repetition 2",+                      "result": "Passed",+                      "duration": "0.3s",+                      "children": []+                    }+                  ]+                }+              ]+            }+          ]+        }+      ]+    }+  ]+}diff --git a/Tools/Tests/Fixtures/check-test-results/unknown-test-case-result/summary.json b/Tools/Tests/Fixtures/check-test-results/unknown-test-case-result/summary.jsonnew file mode 100644index 00000000..cd2220e7--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/unknown-test-case-result/summary.json@@ -0,0 +1,16 @@+{+  "title": "Test - prism",+  "startTime": 1000000000,+  "finishTime": 1000000001,+  "environmentDescription": "prism · Debug · macOS",+  "topInsights": [],+  "result": "Passed",+  "totalTestCount": 1,+  "passedTests": 1,+  "failedTests": 0,+  "skippedTests": 0,+  "expectedFailures": 0,+  "statistics": [],+  "devicesAndConfigurations": [],+  "testFailures": []+}diff --git a/Tools/Tests/Fixtures/check-test-results/unknown-test-case-result/tests.json b/Tools/Tests/Fixtures/check-test-results/unknown-test-case-result/tests.jsonnew file mode 100644index 00000000..bf501f76--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/unknown-test-case-result/tests.json@@ -0,0 +1,34 @@+{+  "testPlanConfigurations": [],+  "devices": [],+  "testNodes": [+    {+      "nodeType": "Test Plan",+      "name": "prism",+      "result": "Passed",+      "children": [+        {+          "nodeType": "Unit test bundle",+          "name": "prismTests",+          "result": "Passed",+          "children": [+            {+              "nodeType": "Test Suite",+              "name": "SampleTests",+              "result": "Passed",+              "children": [+                {+                  "nodeType": "Test Case",+                  "name": "testOne()",+                  "result": "unknown",+                  "duration": "0.01s",+                  "children": []+                }+              ]+            }+          ]+        }+      ]+    }+  ]+}diff --git a/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-summary/summary.exit b/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-summary/summary.exitnew file mode 100644index 00000000..7f8f011e--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-summary/summary.exit@@ -0,0 +1 @@+7diff --git a/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-summary/summary.json b/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-summary/summary.jsonnew file mode 100644index 00000000..5a0be932--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-summary/summary.json@@ -0,0 +1,16 @@+{+  "title": "Test - prism",+  "startTime": 1000000000,+  "finishTime": 1000000010,+  "environmentDescription": "prism · Debug · macOS",+  "topInsights": [],+  "result": "Passed",+  "totalTestCount": 6,+  "passedTests": 5,+  "failedTests": 0,+  "skippedTests": 0,+  "expectedFailures": 1,+  "statistics": [],+  "devicesAndConfigurations": [],+  "testFailures": []+}diff --git a/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-summary/tests.json b/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-summary/tests.jsonnew file mode 100644index 00000000..27ef0166--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-summary/tests.json@@ -0,0 +1,48 @@+{+  "testPlanConfigurations": [],+  "devices": [],+  "testNodes": [+    {+      "nodeType": "Test Plan",+      "name": "prism",+      "result": "Passed",+      "children": [+        {+          "nodeType": "Unit test bundle",+          "name": "prismTests",+          "result": "Passed",+          "children": [+            {+              "nodeType": "Test Suite",+              "name": "SampleTests",+              "result": "Passed",+              "children": [+                {+                  "nodeType": "Test Case",+                  "name": "testOne()",+                  "result": "Passed",+                  "duration": "0.01s",+                  "children": []+                },+                {+                  "nodeType": "Test Case",+                  "name": "testTwo()",+                  "result": "Passed",+                  "duration": "0.02s",+                  "children": []+                },+                {+                  "nodeType": "Test Case",+                  "name": "testExpectedFailure()",+                  "result": "Expected Failure",+                  "duration": "0.01s",+                  "children": []+                }+              ]+            }+          ]+        }+      ]+    }+  ]+}diff --git a/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-tests/summary.json b/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-tests/summary.jsonnew file mode 100644index 00000000..5a0be932--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-tests/summary.json@@ -0,0 +1,16 @@+{+  "title": "Test - prism",+  "startTime": 1000000000,+  "finishTime": 1000000010,+  "environmentDescription": "prism · Debug · macOS",+  "topInsights": [],+  "result": "Passed",+  "totalTestCount": 6,+  "passedTests": 5,+  "failedTests": 0,+  "skippedTests": 0,+  "expectedFailures": 1,+  "statistics": [],+  "devicesAndConfigurations": [],+  "testFailures": []+}diff --git a/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-tests/tests.exit b/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-tests/tests.exitnew file mode 100644index 00000000..f11c82a4--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-tests/tests.exit@@ -0,0 +1 @@+9\ No newline at end of filediff --git a/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-tests/tests.json b/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-tests/tests.jsonnew file mode 100644index 00000000..27ef0166--- /dev/null+++ b/Tools/Tests/Fixtures/check-test-results/xcresulttool-nonzero-tests/tests.json@@ -0,0 +1,48 @@+{+  "testPlanConfigurations": [],+  "devices": [],+  "testNodes": [+    {+      "nodeType": "Test Plan",+      "name": "prism",+      "result": "Passed",+      "children": [+        {+          "nodeType": "Unit test bundle",+          "name": "prismTests",+          "result": "Passed",+          "children": [+            {+              "nodeType": "Test Suite",+              "name": "SampleTests",+              "result": "Passed",+              "children": [+                {+                  "nodeType": "Test Case",+                  "name": "testOne()",+                  "result": "Passed",+                  "duration": "0.01s",+                  "children": []+                },+                {+                  "nodeType": "Test Case",+                  "name": "testTwo()",+                  "result": "Passed",+                  "duration": "0.02s",+                  "children": []+                },+                {+                  "nodeType": "Test Case",+                  "name": "testExpectedFailure()",+                  "result": "Expected Failure",+                  "duration": "0.01s",+                  "children": []+                }+              ]+            }+          ]+        }+      ]+    }+  ]+}

diff --git a/CHANGELOG.md b/CHANGELOG.mdindex bb517458..3f9ec845 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -22,6 +22,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0  ### Fixed +- `Tools/check-test-results.sh` no longer double-reports a malformed `children` field found below a Test Case, under the wrong label (T-2243). A round-3 review fix closed this only where the malformed field sits directly on the Test Case itself; one level deeper — a `Repetition`'s own `children`, say — the same field was still reported twice: once correctly, and once mislabelled as the Test Case's own. `collect_attempts()`'s own subtree scan no longer reports the problem itself, since `walk()`'s separate, correctly-labelled traversal already does so for every node.+- `Tools/check-test-results.sh` now fails closed on six more ways an xcresult bundle can look clean while being unreadable (T-2243): a nonzero exit from either `xcresulttool` call is no longer masked by its non-empty stdout; a Test Case or attempt result of `unknown` is no longer accepted as a valid outcome; a `children` field that is present but not an array is no longer silently read as "no children"; result validation now covers grouping nodes (e.g. Test Plan) as well as Test Case and attempt nodes; an attempt node is now validated wherever it sits in the tree rather than only when it happens to fall inside some Test Case's own subtree, so one recorded outside that scope no longer sails through unchecked; and a Test Case whose result is absent or null — previously compared against nothing but `"Failed"` — is now flagged instead of silently counted as fine. - An image that is tiny as a file but enormous as a picture can no longer exhaust memory or terminate Prism, whether it comes from the web, from a file beside the document, or written directly into the markdown (T-2132, T-2149, T-2151, T-1867). A picture is stored compressed, and a plain-coloured one compresses at about a thousand to one — so a 400 KB download can be a 20,000 by 20,000 image that needs about 400 MB the moment anything tries to display it, and four times that if it is in colour. Prism's limits were all written on the wrong side of that: a 50 MB cap on the download said nothing about the picture inside it, and the 2 MB cap on a local SVG was applied only after the whole file had already been read, so a very large one could freeze the app on its way to being refused. The limits that did exist covered only images fetched from the web; the same image referenced from a file next to your document, or embedded inline in the markdown, went straight to the renderer unchecked. Prism now reads the picture's dimensions from its header — a few bytes, before anything is decoded — and decides from that. An ordinary image is displayed as before. A very large one referenced from the web or from a file is scaled down to fit. One beyond any reasonable size is refused outright and shows the usual "Image failed to load" placeholder, rather than being handed to a decoder that would have to build the whole thing first. How large a picture is now also accounts for how much detail each dot of it carries: most pictures store one byte per colour, but some store two or four, and Prism previously assumed the smaller size for all of them and so under-counted the deep ones by half or three quarters. One consequence you may see: a very large deep-colour photograph that used to display at full size is now scaled down, because its true size was always above the limit and is now measured as such. Files are now read up to their limit instead of read whole and then measured — including the copy Prism keeps of a document you have not saved yet, which is restored when the app reopens. How much decoding happens at once is limited by how much memory those pictures actually need rather than by how many of them there are, so a page full of large images no longer overruns while appearing to stay within its bounds. Two things behave differently, both deliberately. An image whose file does not say how big it is, or what kind of dots it stores, now shows the "Image failed to load" placeholder instead of being displayed — there is no way to know what it would cost until it has already cost it. And an image written directly into the markdown is treated more strictly than the same image kept in a file beside the document: it is either small enough to display as it is or refused, never scaled down. That difference is about memory rather than effort. Scaling a picture that is written into the markdown means rebuilding it and writing the smaller version back into the page, where it then stays for as long as the document is open — which costs more memory, for longer, than not showing it. A picture in a file has somewhere else to keep its smaller version, so it can be scaled instead of refused. Animated images are unaffected in either case: they play as before, however many frames they have. - A verification scan that starts during the app's initial entitlement bootstrap can no longer publish a stale result while a newer scan is still in flight (T-2152). While `entitlementState` was still `.loading`, any scan's result was accepted regardless of whether a more recent scan — for example one started right after `AppStore.sync()` — was still reading the world; the older scan finishing first could briefly flip the paywall to locked (or unlocked) ahead of the newer, more current answer. An older result that arrives while a newer scan is still outstanding is now held back rather than published. If the newer scan goes on to answer, its fresher result is published and the held-back one is simply dropped; if instead it is cancelled without ever answering, the held-back result is released, so a cancelled scan cannot leave the paywall stranded on `.loading`. The trade is that the brief loading state now ends when the last overlapping scan answers rather than the first, so it can last marginally longer; every control it gates is disabled meanwhile, so nothing silently does nothing. - Saving a pasted document to a file no longer disturbs whatever document you opened next (T-2213). A save finishes in two parts: the file is written straight away, but the document only becomes that file once its notes have been moved across, and on a slow iCloud connection that second part can still be running after you have closed the document or opened another one. When it finished late, it acted on the document then on screen instead of the one it had saved: the pasted text of that other document was deleted from the place Prism keeps unsaved documents — so it could no longer be recovered after a relaunch — its entry in Recent Files was labelled with the wrong document's title, and an action you had queued behind its own Save prompt could run without you confirming it. A save that failed to move its notes also raised an alert naming a file you were no longer looking at. Each of these now belongs to the document that was actually saved, and the document on screen is left alone. Its Recent Files entry is labelled with its own title rather than the other document's. Where that other document had itself started saving in the meantime, the late save no longer takes over the shortcut that document had prepared for its own file, which can leave the saved file without a Recent Files entry of its own. The file is saved either way, and can be opened from the Files app.diff --git a/docs/agent-notes/development-tooling.md b/docs/agent-notes/development-tooling.mdindex 94fd86ac..819e50e8 100644--- a/docs/agent-notes/development-tooling.md+++ b/docs/agent-notes/development-tooling.md@@ -95,7 +95,7 @@ Four things about the guard itself, all learned by it failing on this repo: - **`SVGWebViewTests` no longer needs skipping** (fixed on the T-1541 bugfix branch; `specs/bugfixes/svgwebviewtests-offmain-crash/report.md`). Root cause of the full-run host crash: the suite was the only one constructing `WKWebView` in *synchronous* test bodies. A sync `@MainActor` function has no hop-on-entry — its isolation depends on the caller hopping, and swift-testing's `nonisolated(nonsending)` thunks (Swift 5 mode + Approachable Concurrency, no default isolation in the test target) run on the runner's cooperative pool under full-suite load. WebKit's init `RELEASE_ASSERT(main thread)` then killed the host. The `@MainActor` added in 81c1636 could only move the trap. The rule for this target: **touch WebKit only from an `async` test in a `@MainActor` suite** (an ACTOR-ISOLATED async function hops on entry as ABI — `async` on its own hops nowhere); the suite additionally wraps construction in `MainActor.run`, which is runtime-enforced. - **The app and test targets have different actor-isolation defaults.** `SWIFT_DEFAULT_ACTOR_ISOLATION = MainActor` is set on the app target only; `prismTests` and `prismUITests` have no default. Production code is therefore implicitly main-actor while the same type called from a test is not, which silently turns "safe because everything is on the main actor" into a real race in tests (see `RecentFileEntry.relativeDateFormatter`). Do not assume a production type's isolation holds inside a test. - **`check-test-results.sh`'s arithmetic check now folds in `expectedFailures`, and it gates the build.** An expected failure is counted in neither `passedTests` nor `failedTests`, so the original `passed+failed+skipped != total` check tripped on every run containing an `XCTExpectFailure` (currently 2, T-1985 and T-1986) and printed a `WARN` everyone learned to ignore. Fixed in both directions (T-1993): `expectedFailures` is part of the sum, so those runs now reconcile exactly, and a mismatch is a hard FAIL rather than a warning — counts that do not reconstruct the whole mean nothing below them can be trusted. Do not expect a benign warning here any more; if this fires, the bundle is wrong.-- **`check-test-results.sh` fails closed on anything it cannot read, including the per-test tree.** Beyond the counts it reads `xcrun xcresulttool get test-results tests` and refuses to report OK when the tree contains an unrecognised `nodeType` or result value, contains no `Test Case` node while the summary counts tests, records a `Test Case` as `Failed` while `failedTests` is zero, or shows any test whose final result was reached only after a failed attempt (T-2224). Two things worth knowing before debugging a surprise FAIL: a future Xcode that adds a `TestNodeType` will trip the unrecognised-node check until the enum copy inside the script is updated (deliberate — a guard that goes quiet exactly when the format moves is worse), and the retry scan walks a Test Case's WHOLE subtree rather than its direct children, because on any run without `-only-test-configuration` — i.e. `make test` and `make test-ui` — a Test Case's direct children are `Test Plan Configuration` nodes and nothing else structural (measured: 5,627 of 5,627 cases across every multi-configuration bundle on this machine, four configurations each). Where a per-attempt node would land inside that subtree is inference, not measurement — no bundle on this machine has ever contained a `Repetition` or `Test Case Run` node, which is exactly why the scan accepts either type at any depth. Regression suite: `Tools/Tests/test-check-test-results.sh`, run by `make verify-make-guards`.+- **`check-test-results.sh` fails closed on anything it cannot read, including the per-test tree.** Beyond the counts it reads `xcrun xcresulttool get test-results tests` and refuses to report OK when: the tree contains an unrecognised `nodeType` or result value; contains no `Test Case` node while the summary counts tests; records a `Test Case` as `Failed` while `failedTests` is zero; shows any test whose final result was reached only after a failed attempt (T-2224); either `xcresulttool` call it depends on (the summary read or the per-test tree read) exits non-zero, even when its stdout still parses cleanly; a Test Case or attempt result is `"unknown"` — a genuine `TestResult` enum member, just never a RESOLVED one; a `children` field is present but not a JSON array; a grouping node (Test Plan, Test Suite, ...) itself carries an invalid or unresolved result; an attempt node (`Repetition` / `Test Case Run`) sits somewhere OUTSIDE any Test Case's own subtree, a shape no real bundle here has shown but nothing in the schema forbids; or a Test Case's `result` is absent or null (T-2243, closed in two rounds — the last two only found on a follow-up review of the first four). Two things worth knowing before debugging a surprise FAIL: a future Xcode that adds a `TestNodeType` will trip the unrecognised-node check until the enum copy inside the script is updated (deliberate — a guard that goes quiet exactly when the format moves is worse), and the retry scan walks a Test Case's WHOLE subtree rather than its direct children, because on any run without `-only-test-configuration` — i.e. `make test` and `make test-ui` — a Test Case's direct children are `Test Plan Configuration` nodes and nothing else structural (measured: 5,627 of 5,627 cases across every multi-configuration bundle on this machine, four configurations each). Where a per-attempt node would land inside that subtree is inference, not measurement — no bundle on this machine has ever contained a `Repetition` or `Test Case Run` node, which is exactly why the scan accepts either type at any depth. Regression suite: `Tools/Tests/test-check-test-results.sh`, run by `make verify-make-guards`. - **One `MockURLProtocol` used to be shared by five suites** across `URLDocumentLoaderTests`, `ImageLoaderTests`, and `SVGSourceLoaderTests`. `.serialized` only orders tests *within* a suite, so the suites overwrote each other's static handler and served each other's payloads. Handlers are now registered per scope (`MockURLScope`), carried as a request header set from the session configuration. If you add a networked suite, give it its own scope rather than a global handler.  ## "The test runner hung before establishing connection" on macOS (T-2146)