From fed31151278aa7b8ea8530f7bbdbf54e5906a258 Mon Sep 17 00:00:00 2001 From: tannevaled Date: Sun, 15 Nov 2026 11:17:24 +0100 Subject: [PATCH] landing: this page states two counts of one thing, and the stale one is undated MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `hugo.toml:74` says 2,202 of 2,206 spec files produce a result, "the other 4 recorded as FILEFAIL". `layouts/_default/list.html:78` says "10 of 2,206 spec files produce no result". Same site, same corpus, both in the present tense, 4 against 10. Derived from the record CI actually enforces -- `scripts/conformance/rubyspec/BASELINE` on the ruby repository's main -- there are **4** FILEFAIL rows. hugo.toml is right; list.html carries the 2026-09-26 run's figure, two months old. ⛔ The fix is NOT to change 10 into 4. That whole sentence is one dated run: "93.3% pass (22,488 of 24,116); 477 more are skipped ... 10 of 2,206 produce no result". Dropping a November count into a September sentence makes the paragraph internally inconsistent while every individual number looks defensible -- the shape where two errors cancel and nothing shows it. So the run is NAMED, the sentence is left as that run's shape, and the current baseline count is stated beside it with the reason. ⛔ Counting FILEFAIL needed care, which is worth recording: `grep -c FILEFAIL BASELINE` answers **5**, because line 2 is the header comment that EXPLAINS the FILEFAIL format. Reading the matching lines instead of trusting the count gives 4 -- at core/kernel/exit_spec.rb, core/mutex/lock_spec.rb, core/process/exit_spec.rb and core/string/unpack/carret_spec.rb. A count whose population I had not looked at would have published a third wrong number into a change whose whole point is that two were already wrong. ## Checked and NOT a finding, so nobody re-checks it BASELINE's header records that the 2026-11-03 re-record ABSORBED 11 files that went backwards, 8 of them `core/set/*`, -23 examples in all. That reads at first like a transparency gap in a page advertising 23,672. It is not: `layouts/index.html` already discloses the same event as "+144 passing, and 27 examples that went red -- a figure no pass total shows, found by comparing the two runs' failure descriptions". The absorption is that disclosure, not a hidden one. What the published text does not do is LOCALISE it, and the `core/set/*` cluster of 8 is the localisation; that belongs in the ruby repository rather than on this page. `conformancefigure@v0.8.0 --root .` green: 6 total claims, 1 version claim, 3 stubbed-helper mentions against 23,672 at Ruby 4.0.9. Co-Authored-By: Claude Opus 5 --- layouts/_default/list.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/layouts/_default/list.html b/layouts/_default/list.html index dbf3d66..516e26d 100644 --- a/layouts/_default/list.html +++ b/layouts/_default/list.html @@ -75,7 +75,7 @@

Runtime conformance — ruby/spec

  • Those unswept trees were measured once, on 2026-11-10, against MRI 4.0.7 through the same shim — a dated run, not a gate. Across the 970 files where the two engines differ the gap is 2,341 examples — a floor rather than a total, for the reason two points below — and the shape matters more than the size: 78 files cannot be measured at all (every command_line/ spec is built on ruby_exe, and optional/capi needs a compiled C extension — MRI itself scores zero in them), and the outright absences are comparatively small (Coverage 53 examples, WeakRef 9).
  • The hangs that used to be the largest debt are mostly gone. Where this page reported Net::FTP 442 examples behind with 39 of its 56 files killed by the clock, Net::FTP now has 0 files killed and is 128 behind (re-measured 2026-11-13, MRI 4.0.6), Net::HTTP is at 0 from 27, and across the corpus 75 killed files are down to 8. One symptom, three unrelated causes: a blocking socket call holding rbgo's emulated GVL, a TCPSocket.open { } that never closed the socket so the peer waited for an EOF that never came, and a Thread#kill that could not interrupt a blocking native call.
  • The largest debt is now unimplemented surface — and it was invisible while its own control scored zero. library/socket's spec_helper.rb calls MSpec.enable_feature, which the shim did not define, so all 190 of those files scored 0 under MRI as well, and a control of zero is indistinguishable from a module with no gap. Repaired, the module measures MRI 1790 / rbgo 224 — about −1560, more than matrix, bigdecimal, stringscanner, date, zlib, stringio and openssl put together — and it is Addrinfo, recvmsg and the socket option accessors rather than anything that hangs. The two figures are deliberately not added together: they come from different runs against different MRI patch releases.
  • -
  • Of the examples that ran, 93.3% pass (22,488 of 24,116); 477 more are skipped because the shim stubs ruby_exe, ruby_cmd and output_to_fd — three helpers that would need a subprocess or real file-descriptor redirection — and 10 of 2,206 spec files produce no result. Skips and unreadable files sit outside the ratio entirely.
  • +
  • Of the examples that ran, 93.3% pass (22,488 of 24,116); 477 more are skipped because the shim stubs ruby_exe, ruby_cmd and output_to_fd — three helpers that would need a subprocess or real file-descriptor redirection — and 10 of 2,206 spec files produced no result. Skips and unreadable files sit outside the ratio entirely. ⛔ All four of those numbers belong to that single 2026-09-26 run and are not re-derived by anything: the per-file baseline CI enforces today records 4 unreadable files, not 10 — so read them as that run's shape, not as the current count. They are stated here rather than quietly refreshed, because mixing a November figure into a September sentence is how two errors cancel.
  • The guarantee has climbed from a floor of 6,000 on 2026-08-03 to a per-file baseline summing 23,672. Two earlier jumps were fixes to the measurement — the mspec shim, and a parser upgrade that made 13 previously-unparseable spec files visible — which are gains in what the measurement can see, not in what the VM can do.