Skip to content

chore(spanner): bound streaming result buffer and disable unsafe retries - #19165

Open
olavloite wants to merge 1 commit into
mainfrom
spanner-bound-stream-buffer
Open

olavloite wants to merge 1 commit into
mainfrom
spanner-bound-stream-buffer

Conversation

@olavloite

Copy link
Copy Markdown
Contributor

Previously, streaming reads and queries buffered incoming chunks without a limit while waiting for a resume token from Spanner. If a stream sent many chunks without a resume token, memory usage could grow unboundedly and row delivery to the caller would stall.

With this change:

  • Streaming results now buffer at most 512 chunks while waiting for a resume token.
  • Once that limit is exceeded, buffered chunks are flushed immediately to the caller and retries are disabled so already-yielded rows cannot be duplicated.
  • Subsequent chunks without a resume token continue streaming directly without buffering, and any transient stream failure is propagated immediately instead of being retried.
  • Retries are re-enabled only if a newer resume token arrives later in the stream.
  • Both the synchronous and asynchronous clients are updated consistently, with unit and mock-server tests covering the threshold, failure, and recovery paths.

Previously, streaming reads and queries buffered incoming chunks without
a limit while waiting for a resume token from Spanner. If a stream sent
many chunks without a resume token, memory usage could grow unboundedly
and row delivery to the caller would stall.

With this change:
- Streaming results now buffer at most 512 chunks while waiting for a
  resume token.
- Once that limit is exceeded, buffered chunks are flushed immediately
  to the caller and retries are disabled so already-yielded rows cannot
  be duplicated.
- Subsequent chunks without a resume token continue streaming directly
  without buffering, and any transient stream failure is propagated
  immediately instead of being retried.
- Retries are re-enabled only if a newer resume token arrives later in
  the stream.
- Both the synchronous and asynchronous clients are updated consistently,
  with unit and mock-server tests covering the threshold, failure, and
  recovery paths.
@olavloite
olavloite requested a review from a team as a code owner November 21, 2026 09:23

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a buffer limit (_MAX_BUFFERED_CHUNKS = 512) for PartialResultSet chunks in both synchronous and asynchronous snapshot implementations to prevent unbounded buffering when waiting for a resume_token. If the limit is exceeded, the chunks are flushed downstream and retries are disabled until a newer resume_token is received. Comprehensive unit and mock server tests are added to validate this behavior. The review feedback suggests adding an explanatory comment in the synchronous implementation to match the asynchronous one for consistency, and removing redundant parentheses in a generator expression to comply with PEP 8.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant