Skip to content

test: wait for the Autobahn server to start - #599

Open
mafredri wants to merge 4 commits into
masterfrom
mafredri/autobahn-startup-wait
Open

mafredri wants to merge 4 commits into
masterfrom
mafredri/autobahn-startup-wait

Conversation

@mafredri

@mafredri mafredri commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

TestAutobahn gave the wstest container 5s after docker run to accept a WebSocket handshake. On GitHub's ubuntu-26.04 runners that is often not enough, and the daily test jobs fail with autobahn_test.go:72: context deadline exceeded (#598). Since the daily jobs moved to 26.04 on 2026-10-30, 13 of 27 failed this way.

The image is already pulled when the wait starts. The time goes to the first container start after docker pull: on 26.04 (Docker 29.4.2 with the containerd image store), that start took about 2.8s on AMD EPYC 7763 VMs and 2.9s to 7.3s on EPYC 9V45, EPYC 9V74 and Xeon 8573C VMs. wstest starts listening about 1.1s later. Under the full ci/test.sh load, the server took 2.4s to 13.5s to accept a handshake. On 24.04 (Docker 28.0.4), the same start took under 0.5s on every VM type. The daily failures follow the VM type: 0 of 11 jobs on EPYC 7763 failed, and 13 of 15 on the other types.

wstestServer now returns once the server accepts a handshake. It dials every 100ms instead of in a busy loop, fails as soon as docker run exits, and gives up after 2 minutes.

Killing the docker run client does not stop the container, so each run left a wstest container running. closeFn now removes the container by name.

Refs #598

Verification
  • Startup timings on runners: 43594056234 (24.04 and 26.04 under the test load), 43594595034 (first start without load), 43594994307 (any image or network mode), 43595595196 (time since pull, not since boot), 43598451098 (runner image and VM type).
  • AUTOBAHN=1 make test on ubuntu-26.04 with earlier versions of this branch: 43599366425 (5 minute limit, same container removal), 8 of 8 passed (5 on the slower VM types), with no container left afterwards. 43595298461 (1 minute limit), 6 of 6 passed. Master's test in the same run passed 5 of 6 and failed once with the 5s error.
  • The daily workflow on this branch's head: 43606350993.
  • Locally, AUTOBAHN=fast passes and leaves no container. With an invalid wstest mode, the test fails after 2.2s with wstest exited: exit status 1.
  • make lint passed.
Decision log
  • 2 minutes: the wait covers only the container start and wstest's startup, not a download. The longest measured time to accept a handshake was 13.5s. A server that fails to start makes docker run exit, which ends the wait right away.
  • What Docker 29 does during the delay is not known. It happens before containerd starts the container's shim, for any image and network mode, and not when 60s pass between the pull and the run. The test does not work around it with a throwaway container or a sleep after the pull, because that would build one Docker version's behavior into the test.
  • Not pinning the daily jobs to ubuntu-24.04: that hides the slow start until 24.04 is retired.
  • The busy loop did not cause the delay: dialing every 50ms gave the same 2.6s to 8.7s start times. It made 15k to 64k dials per startup.

🤖 This PR was created with the help of Coder Agents.

TestAutobahn gave the wstest container 5s to accept a handshake after
docker run. Since the daily jobs moved to ubuntu-26.04 (Docker 29.4.2),
the first container start on a fresh runner takes 2.8s to 12s, then
wstest needs about 1s more to listen. On 24.04 (Docker 28.0.4) the
first start took under 0.5s. The test failed in 13 of 27 daily test jobs
since 2026-10-30.

waitWS now allows a minute, sleeps 100ms between dials instead of
dialing in a busy loop, and returns the last dial error.
Killing the docker run client leaves the container running, so each
TestAutobahn run left a wstest container behind. closeFn now removes
the container by name.

wstestServer now returns once the server accepts handshakes, and stops
waiting when docker run exits, so a server that fails to start fails
the test right away. That lets the wait allow 5 minutes for a slow
start.
The image is pulled before docker run, so the wait covers only the
container start and wstest's startup, which took at most 13.5s on
the runners measured. 2 minutes is enough for that.

docker run exiting now cancels the wait's context with the exit error
as its cause, so waitWS needs no separate channel.
@mafredri
mafredri marked this pull request as ready for review November 14, 2026 16:45

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant