Skip to content

fix(api): apply a timeout to every API request - #960

Open
xianshijing-lk wants to merge 1 commit into
mainfrom
sxian/CLTCU-24/python-sdks-livekit-api-default-request-timeout
Open

xianshijing-lk wants to merge 1 commit into
mainfrom
sxian/CLTCU-24/python-sdks-livekit-api-default-request-timeout

Conversation

@xianshijing-lk

Copy link
Copy Markdown
Contributor

Fixes CLTCU-24. Found while investigating INC-1765; it was not a factor in that incident.

Problem

LiveKitAPI creates its session with a 10-second timeout. That timeout never applied to API calls.

TwirpClient.request forwarded its timeout argument to session.post even when it was None:

async with self._session.post(
    url, headers=headers, data=serialized_data, timeout=timeout
) as resp:

aiohttp reads an explicit timeout=None as no timeout at all. Only a missing argument selects the session timeout — from ClientSession._request:

if timeout is sentinel:
    real_timeout: ClientTimeout = self._timeout      # session default
else:
    if not isinstance(timeout, ClientTimeout):
        real_timeout = ClientTimeout(total=timeout)  # None -> total=None -> unbounded

So any call that did not pass its own timeout ran unbounded. A host that accepts the connection and never answers — a load balancer with no backend responding — blocked the call indefinitely, and region failover never started, because failover only begins after an attempt fails.

The same function already read self._session.timeout.total to size the failover budget, so request() assumed the session timeout governed the request while the post three lines below discarded it.

Calls that pass their own timeout (the SIP dial paths, connector) were unaffected. The line has been present since api-v1.0.0 (d976d67, #403).

Reproduction

Against a local server that accepts the request and never answers, with livekit-api 2.0.0 / aiohttp 3.14.5:

session default timeout: 10 s
SDK still waiting at 25.0s - harness gave up

Fix

Resolve the timeout once — per-call override → client default → session — and pass the result explicitly, so the failover budget and the actual request deadline derive from the same expression:

request_timeout = timeout
if request_timeout is None:
    request_timeout = self._timeout
if request_timeout is None:
    request_timeout = self._session.timeout

Second change: timeout= now works with a caller-supplied session

timeout was read only inside the branch where the SDK builds the session, so LiveKitAPI(session=..., timeout=...) silently ignored it while the docstring documented it as the request timeout.

The value now reaches TwirpClient via Service, so a caller can give API calls a deadline of their own without changing the timeout of other traffic sharing the session:

api = LiveKitAPI(session=shared_http_session, timeout=aiohttp.ClientTimeout(total=10))

This matters for callers that always pass a session — livekit-agents builds one with no timeout=, so its API calls currently inherit aiohttp's generic 300s default rather than the SDK's 10s.

Deliberately conservative: a supplied session with no explicit timeout keeps its own timeout, exactly as before. No existing behavior changes.

Testing

New tests/api/test_request_timeout.py, 8 cases covering the resolution chain, the custom-session path, back-compat, and an end-to-end check against a server that never answers. All 8 fail on main.

  • tests/api: 48 passed, 28 skipped (was 40 passed)
  • mypy livekit-api: clean, 15 files
  • ruff format --check: clean; no new ruff check findings beyond the file's existing conventions

Verified behavior against a hanging host:

case before after
SDK-built session, no timeout= unbounded TimeoutError at ~10s
custom session (20s), no timeout= unbounded TimeoutError at ~20s
custom session (20s), timeout=5s unbounded TimeoutError at ~5s, session still 20s

Note for reviewers: livekit-api supports Python 3.9 and four of the six service modules lack from __future__ import annotations, so the new parameters use Optional[...] rather than PEP 604. Verified on Python 3.9.6 + aiohttp 3.13.5 that all six service constructors evaluate and that timeout propagates.

Related

  • CS-2064 / CSCU-113 put a deadline on RoomService requests server-side; this is the client side of the same symptom. A server deadline does not help when the request never reaches a backend.
  • The Rust SDK is not affected (livekit-api 0.9.1 times out at 10s per attempt in the same test).
  • Anything below MIN_FAILOVER_TIMEOUT (5s) makes failover_attempts() return 1, disabling region failover — worth knowing before tuning this value down.

🤖 Generated with Claude Code

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

TwirpClient.request forwarded its `timeout` argument to session.post even
when it was None. aiohttp reads an explicit `timeout=None` as "no timeout
at all" — only a *missing* argument selects the session timeout — so every
call that did not pass its own timeout ran unbounded and the session's
10s default never applied.

A host that accepts the connection and never answers therefore blocked the
call forever, and region failover never started, because failover only
begins once an attempt fails. The same function already read
`self._session.timeout.total` to size the failover budget, so it assumed
the session timeout governed the request while the post call discarded it.

Resolve the timeout once, per call override -> client default -> session,
and pass the result explicitly, so both uses agree.

Also make `LiveKitAPI(timeout=...)` mean something when the caller supplies
a session. It was read only on the path where the SDK built the session, so
`LiveKitAPI(session=..., timeout=...)` silently ignored the timeout even
though it is documented as the request timeout. The value now reaches
TwirpClient through Service, letting a caller give API calls a deadline
without changing the timeout of other traffic on a shared session. A
supplied session with no explicit timeout keeps its own, as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@xianshijing-lk
xianshijing-lk force-pushed the sxian/CLTCU-24/python-sdks-livekit-api-default-request-timeout branch from 327d058 to bc4e386 Compare November 20, 2026 20:05

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant