Skip to content

landing: this page states two counts of one thing, and the stale one is undated - #43

Merged
tannevaled merged 1 commit into
mainfrom
dated-run-attribution
Nov 15, 2026
Merged

tannevaled merged 1 commit into
mainfrom
dated-run-attribution

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

landing: this page states two counts of one thing, and the stale one is undated

hugo.toml:74 says 2,202 of 2,206 spec files produce a result, "the other 4
recorded as FILEFAIL". layouts/_default/list.html:78 says "10 of 2,206 spec
files produce no result". Same site, same corpus, both in the present tense,
4 against 10.

Derived from the record CI actually enforces -- scripts/conformance/rubyspec/BASELINE
on the ruby repository's main -- there are 4 FILEFAIL rows. hugo.toml is
right; list.html carries the 2026-09-26 run's figure, two months old.

⛔ The fix is NOT to change 10 into 4. That whole sentence is one dated run:
"93.3% pass (22,488 of 24,116); 477 more are skipped ... 10 of 2,206 produce no
result". Dropping a November count into a September sentence makes the paragraph
internally inconsistent while every individual number looks defensible -- the
shape where two errors cancel and nothing shows it. So the run is NAMED, the
sentence is left as that run's shape, and the current baseline count is stated
beside it with the reason.

⛔ Counting FILEFAIL needed care, which is worth recording: grep -c FILEFAIL BASELINE answers 5, because line 2 is the header comment that EXPLAINS the
FILEFAIL format. Reading the matching lines instead of trusting the count gives
4 -- at core/kernel/exit_spec.rb, core/mutex/lock_spec.rb,
core/process/exit_spec.rb and core/string/unpack/carret_spec.rb. A count whose
population I had not looked at would have published a third wrong number into a
change whose whole point is that two were already wrong.

Checked and NOT a finding, so nobody re-checks it

BASELINE's header records that the 2026-11-03 re-record ABSORBED 11 files that
went backwards, 8 of them core/set/*, -23 examples in all. That reads at first
like a transparency gap in a page advertising 23,672.

It is not: layouts/index.html already discloses the same event as "+144
passing, and 27 examples that went red -- a figure no pass total shows, found by
comparing the two runs' failure descriptions". The absorption is that disclosure,
not a hidden one. What the published text does not do is LOCALISE it, and the
core/set/* cluster of 8 is the localisation; that belongs in the ruby
repository rather than on this page.

conformancefigure@v0.8.0 --root . green: 6 total claims, 1 version claim, 3
stubbed-helper mentions against 23,672 at Ruby 4.0.9.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com

…is undated

`hugo.toml:74` says 2,202 of 2,206 spec files produce a result, "the other 4
recorded as FILEFAIL". `layouts/_default/list.html:78` says "10 of 2,206 spec
files produce no result". Same site, same corpus, both in the present tense,
4 against 10.

Derived from the record CI actually enforces -- `scripts/conformance/rubyspec/BASELINE`
on the ruby repository's main -- there are **4** FILEFAIL rows. hugo.toml is
right; list.html carries the 2026-09-26 run's figure, two months old.

⛔ The fix is NOT to change 10 into 4. That whole sentence is one dated run:
"93.3% pass (22,488 of 24,116); 477 more are skipped ... 10 of 2,206 produce no
result". Dropping a November count into a September sentence makes the paragraph
internally inconsistent while every individual number looks defensible -- the
shape where two errors cancel and nothing shows it. So the run is NAMED, the
sentence is left as that run's shape, and the current baseline count is stated
beside it with the reason.

⛔ Counting FILEFAIL needed care, which is worth recording: `grep -c FILEFAIL
BASELINE` answers **5**, because line 2 is the header comment that EXPLAINS the
FILEFAIL format. Reading the matching lines instead of trusting the count gives
4 -- at core/kernel/exit_spec.rb, core/mutex/lock_spec.rb,
core/process/exit_spec.rb and core/string/unpack/carret_spec.rb. A count whose
population I had not looked at would have published a third wrong number into a
change whose whole point is that two were already wrong.

## Checked and NOT a finding, so nobody re-checks it

BASELINE's header records that the 2026-11-03 re-record ABSORBED 11 files that
went backwards, 8 of them `core/set/*`, -23 examples in all. That reads at first
like a transparency gap in a page advertising 23,672.

It is not: `layouts/index.html` already discloses the same event as "+144
passing, and 27 examples that went red -- a figure no pass total shows, found by
comparing the two runs' failure descriptions". The absorption is that disclosure,
not a hidden one. What the published text does not do is LOCALISE it, and the
`core/set/*` cluster of 8 is the localisation; that belongs in the ruby
repository rather than on this page.

`conformancefigure@v0.8.0 --root .` green: 6 total claims, 1 version claim, 3
stubbed-helper mentions against 23,672 at Ruby 4.0.9.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 76a0182 into main Nov 15, 2026
3 checks passed
@tannevaled
tannevaled deleted the dated-run-attribution branch November 15, 2026 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant