Skip to content

2026.11.15.1: CI time cost round two: main's test build reused, tests at profile ci, a long case divided by the cases it declared, the matrix scan side by side - #937

Draft
Sunrisepeak wants to merge 5 commits into
mainfrom
ci/acceleration-w1-w4
Draft

Sunrisepeak wants to merge 5 commits into
mainfrom
ci/acceleration-w1-w4

Conversation

@Sunrisepeak

@Sunrisepeak Sunrisepeak commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Design: .agents/docs/2026-11-14-ci-and-release-time-cost-design.md §6 (round two; the owner confirmed W1–W4, the division of a case by the cases it declares, and the macOS paths on 2026-11-15).

Measured (PR run 43602794812, 97.6 min wall, 265 runner-minutes queued)

Interval (min) Cause
0 → 9.1 the cancelled previous run's target-matrix / coverage (if: always()) queued 8.5 min for a runner and held the concurrency group
9.1 → 20.7 the first jobs queued 11.4 min: three full CI runs in the organization at once
20.7 → 69.3 the archive stage: 9.3 queued, 39.4 built (the tests and the library at -O2: this package's default profile is release)
69.3 → 97.4 the shards: 28.0 / 27.9 / 12.4 / 9.8 min; e2e/openkal/trees 1373 s and sets 1375 s ran their cases in one process each, the longest case 564 s

The macOS unit step took 31 min and the macOS matrix scan 29.4 min (59 cells one after another).

Changes

  • W1, main's test build reused. main's archive stage keeps its build directory, with the blob of each tracked file (.github/tools/source_mtimes.py record), as the artifact mcpp-unit-build-<host>; a pull request's archive stage restores it and gives each unchanged file a time older than every output, so only what changed is built again. An artifact and not the cache: the repository's cache holds 9.98 of its 10 GB, and a build directory there would evict the hosts' homes.
  • W2, [profile.ci] (-O0, no debug information, strip) for the tests on Linux (archive), macOS and Windows (mcpp-unit input profile): the library and its unit tests took 8988 s of compiler time at the default release profile and 4183 s with it.
  • W3, a long case divided by the cases it declared (SPEC-017 O10 and P4.3, mtest 0.9.0, 0.9.0: a selection excludes and selects where declared (SPEC-017 P4.3), so a runner divides a long case by the cases it declares mtest#24). When a program's selection states except and optional, a listed case longer than its share with two or more measured declared cases is divided by the innermost of them: each part but the last selects its cases with ?<id> (a case since removed is not missing), the last selects the case without them (!<id>), so an unmeasured case runs once. A divided case's estimate is at least the sum of its declared cases. The store's case durations are kept by program and id (layers and sets exist in two programs each).
  • W4: coverage runs if: ${{ !cancelled() }}.
  • Matrix scan side by side: scan.sh runs each cell in its own directory, MATRIX_JOBS at a time (the processors), and prints in the table's order.
  • mtest 0.9.0 for this package's tests and for the scaffold of mcpp new (kMtestRelease), so the division applies to them.
  • Fixed, found by the division's parallelism: an install for a project takes the home's state turn (xlings::state_dir). It saves the home's environment (subos/default/.xlings.json) as well as the project's, and with the turn in the project's directory it collided on Windows with another project's install (e2e images/1033, run 43632995462). Test XlingsState/AProjectsInstallTakesTheHomesTurn.
  • Fixed, found by the matrix scan's cells side by side: a host tool built by two builds of one home at once is published by both. The publish goes through mcpp.build.stage::stage_file (a temporary of each writer's own, an equal destination counted as published); with one temporary name for every writer, one removed the other's before its rename (publishing host tool 'openkal-tools:okld' failed: No such file or directory, macos-arm64, run 43636471152). Staging temporaries carry a random part as well as the clock. Test BuildStage/WritersOfOneDestinationDoNotCollide.
  • CI: the Windows legacy code-page check runs its two cases with the profile the job built the tests with (MCPP_TEST_PROFILE=ci); under the default profile it built every test again and passed its 600 s timeout.
  • Test: ConcurrentChildren/AReaderReturnsWhenItsOwnChildExits starts both programs once before its rounds; a first start on a fresh Windows runner outlasted the slow child (round 0 of run 43626158365, -427 ms, where a shared pipe returns both together).
  • Docs: SPEC-017 2.3 (P4.3, O10), docs/08 (en, zh); CHANGELOG; version 2026.11.15.1.

Local: TestSchedule/ACaseIsDividedByTheCasesItDeclared and the scheduling tests; test_source_mtimes.py; a demo of four declared cases 8.2 s → 3.0 s, a renamed case run once in the last part and its old name not missing; W1 restore with no change 0.4 s, one interface module changed 88 s (cold 248 s).

… built with profile ci; main's test build reused; the matrix scan side by side; a cancelled run releases its group
…so a long case is divided by the cases it declares
… first start on a fresh Windows runner outlasted the slow child
…me's environment too, and collided on Windows with another project's install
… legacy code-page check runs the tests as the job built them (profile ci)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant