From fe62e20957c33318cb279d9372e0d44a30e5fb64 Mon Sep 17 00:00:00 2001 From: Koichi ITO Date: Fri, 20 Nov 2026 12:05:53 +0900 Subject: [PATCH] Keep a session with a request in flight from idle expiry ## Motivation and Context `StreamableHTTPTransport` counted a session's idle time from the start of its latest request, so a request that ran longer than `session_idle_timeout` lost its session while the client was still waiting for the response. Removing the session closes its streams, which dropped the response over SSE, and since #692 it also cancels the requests in flight, so in JSON response mode the POST was answered 202 with no body instead of the result. A session is now idle only while none of its requests is running, and its idle time counts from the later of the last activity that touched it and the end of its last request. `ServerSession` records when a request leaves the in-flight registry, and the transport reads that together with whether any request is left. With that, a session that expires has no request to cancel: the cancellation #692 added applies when the client ends the session with `DELETE`, when its GET stream breaks without an event store, and when the transport is closed. The tests of #692 that expired a session with a request in flight now assert that it stays, and the docs say so. ## How Has This Been Tested? `test/mcp/server/transports/streamable_http_transport_test.rb`; the test that lets the reaper, a request, a notification, a broadcast and an `initialize` check a session whose request outlived the timeout fails on the previous library, where the request is cancelled. ## Breaking Changes None. A session whose request outlives `session_idle_timeout` is kept until the request ends. --- docs/_server/cancellation.md | 5 +- docs/_server/transports.md | 14 +- .../transports/streamable_http_transport.rb | 12 +- lib/mcp/server_session.rb | 12 +- .../streamable_http_transport_test.rb | 165 +++++++++++++++--- 5 files changed, 172 insertions(+), 36 deletions(-) diff --git a/docs/_server/cancellation.md b/docs/_server/cancellation.md index 5c378655..c6919577 100644 --- a/docs/_server/cancellation.md +++ b/docs/_server/cancellation.md @@ -108,8 +108,9 @@ the opt-in detection only wraps the context when the block signature asks for it When `StreamableHTTPTransport` removes a stateful session while requests are still running in it, those requests are cancelled as if the client had sent `notifications/cancelled` for each of them. This happens when the client terminates -the session with `DELETE`, when the session passes `session_idle_timeout`, when its GET stream breaks without an event store -to resume from, and when the transport is closed. `server_context.cancelled?` turns true, `on_cancel` hooks registered on +the session with `DELETE`, when its GET stream breaks without an event store to resume from, and when the transport is closed. +A session that passes `session_idle_timeout` has no request running, since a running request keeps its session from expiring. +`server_context.cancelled?` turns true, `on_cancel` hooks registered on `server_context.cancellation` run with the reason `"Session terminated"`, and the response is suppressed, since no client is left to receive it. diff --git a/docs/_server/transports.md b/docs/_server/transports.md index 5ffd659e..9967d471 100644 --- a/docs/_server/transports.md +++ b/docs/_server/transports.md @@ -315,9 +315,17 @@ with at most 1024 JSON values in its `params`; a larger one is rejected with HTT The byte bound can be raised or removed with `nil`, in which case `max_request_bytes` still caps the body; the value bound always applies, since a session keeps the parsed data and many small values parse into far more objects than their size suggests. Together they bound both the serialized input and the number of parsed values each of the `max_sessions` sessions can retain. -A session's idle timer is reset by activity that touches it (a GET, or a regular-request POST), and expired sessions are collected by -a background reaper roughly once a minute, so cleanup lags inactivity by up to that interval. At the cap, the transport first reclaims -any already-expired slots and then, if still full, rejects a new `initialize` with HTTP 503 (it does not evict an existing session). +A session's idle timer is reset by activity that touches it (a GET, or a regular-request POST) and does not run while one of +its requests is still being handled: it starts again when that request ends, so a tool call longer than the timeout keeps +its session and its response. A notification, which has no response to wait for, does not hold the timer, +and neither does an open GET stream: opening it resets the timer once, so a client that only listens keeps its session +by sending a request, such as a `ping`, within the timeout. +The timeout does not bound a request itself: a handler that never returns holds its session, and its slot under `max_sessions`, +for as long as it runs, so a limit on how long a handler may run belongs in the handler, +as a deadline it checks the way it checks `server_context.cancelled?`. +Expired sessions are collected by a background reaper roughly once a minute, so cleanup lags inactivity by up to that interval. +At the cap, the transport first reclaims any already-expired slots (a session with a request still being handled is +never one of them) and then, if still full, rejects a new `initialize` with HTTP 503 (it does not evict an existing session). ```ruby # Tune the limits diff --git a/lib/mcp/server/transports/streamable_http_transport.rb b/lib/mcp/server/transports/streamable_http_transport.rb index 69f20a21..860bdf8b 100644 --- a/lib/mcp/server/transports/streamable_http_transport.rb +++ b/lib/mcp/server/transports/streamable_http_transport.rb @@ -142,7 +142,7 @@ class InvalidJsonError < StandardError; end # object instead of an SSE stream. # @param session_idle_timeout [Numeric, nil] seconds before an idle session is reaped; defaults # to `DEFAULT_SESSION_IDLE_TIMEOUT` (1800) in stateful mode, and an explicit `nil` disables - # expiry. Not supported in stateless mode. + # expiry. A session is not idle while one of its requests is running. Not supported in stateless mode. # @param max_sessions [Integer, nil] cap on the concurrent session count in stateful mode; a new # `initialize` past the cap is rejected with HTTP 503, and `nil` disables the cap. # @param allowed_origins [Array, :any, nil] extra `Origin` values accepted in addition to @@ -2960,10 +2960,18 @@ def send_keepalive_ping(session_id, stream) raise # Re-raise to exit the keepalive loop end + # A session is idle only while none of its requests is running, and its idle time counts from the later of + # the last activity that touched it and the end of its last request. `last_active_at` alone is the start of + # the latest request, so a request longer than the timeout would lose its session, and with it its response, + # while the client is still waiting for it. def session_expired?(session) return false unless @session_idle_timeout - Process.clock_gettime(Process::CLOCK_MONOTONIC) - session[:last_active_at] > @session_idle_timeout + busy, last_request_finished_at = session[:server_session]&.in_flight_activity + return false if busy + + idle_since = [session[:last_active_at], last_request_finished_at].compact.max + Process.clock_gettime(Process::CLOCK_MONOTONIC) - idle_since > @session_idle_timeout end # Stream ids carry the session id, so a `Last-Event-ID` is matched to the session presenting it with diff --git a/lib/mcp/server_session.rb b/lib/mcp/server_session.rb index a07429e8..18dfc981 100644 --- a/lib/mcp/server_session.rb +++ b/lib/mcp/server_session.rb @@ -29,6 +29,7 @@ def initialize(server:, transport:, session_id: nil, era: nil) @protocol_version = nil @in_flight = {} @in_flight_mutex = Mutex.new + @last_request_finished_at = nil @initialized = false end @@ -85,10 +86,19 @@ def unregister_in_flight(request_id, cancellation: nil) @in_flight_mutex.synchronize do next if cancellation && !@in_flight[request_id].equal?(cancellation) - @in_flight.delete(request_id) + @last_request_finished_at = Process.clock_gettime(Process::CLOCK_MONOTONIC) if @in_flight.delete(request_id) end end + # What a transport needs to tell whether the session is idle, read in one step so the two cannot disagree: + # whether a request is still in flight, and the monotonic clock reading at which the last one finished + # (`nil` when none has). + # + # @api private + def in_flight_activity + @in_flight_mutex.synchronize { [!@in_flight.empty?, @last_request_finished_at] } + end + # Whether `request_id` is currently in flight, so a transport can refuse a colliding request # before registering any state of its own for it. def in_flight?(request_id) diff --git a/test/mcp/server/transports/streamable_http_transport_test.rb b/test/mcp/server/transports/streamable_http_transport_test.rb index d6df2192..724ff387 100644 --- a/test/mcp/server/transports/streamable_http_transport_test.rb +++ b/test/mcp/server/transports/streamable_http_transport_test.rb @@ -4085,47 +4085,73 @@ def string end test "every other path that removes a session cancels the requests in flight in it" do - expire = ->(transport, session_id) { transport.instance_variable_get(:@sessions)[session_id][:last_active_at] -= 61 } - session_post = ->(session_id, body) do - create_rack_request("POST", "/", { "CONTENT_TYPE" => "application/json", "HTTP_MCP_SESSION_ID" => session_id }, body.to_json) - end attach_get_stream = ->(transport, session_id) do TestStream.new.tap { |stream| transport.instance_variable_get(:@sessions)[session_id][:get_sse_stream] = stream } end removals = { "closing the transport" => ->(transport, _session_id) { transport.close }, - "the reaper finding it expired" => ->(transport, session_id) do - expire.call(transport, session_id) - transport.send(:reap_expired_sessions) + "a failed write to its GET stream" => ->(transport, session_id) do + transport.send(:drop_broken_stream, session_id, attach_get_stream.call(transport, session_id), nil) end, - "a request finding it expired" => ->(transport, session_id) do - expire.call(transport, session_id) + } + + removals.each do |removal, remove_session| + server = Server.new(name: "test", tools: [], prompts: [], resources: []) + transport = StreamableHTTPTransport.new(server, enable_json_response: true) + server.transport = transport + cancellations = [] + started = Queue.new + release = Queue.new + define_held_tool(server, cancellations: cancellations, started: started, release: release) + session_id, = start_session(transport) + + call = Thread.new { transport.handle_request(held_tool_call(session_id, "call-1")) } + started.pop + + remove_session.call(transport, session_id) + + assert_equal([["call-1", "Session terminated"]], cancellations, "session removed by #{removal}") + ensure + release.push(true) + call.join + transport.close + end + end + + test "a session with a request in flight does not expire, whichever path checks it" do + session_post = ->(session_id, body) do + create_rack_request("POST", "/", { "CONTENT_TYPE" => "application/json", "HTTP_MCP_SESSION_ID" => session_id }, body.to_json) + end + + checks = { + "the reaper" => ->(transport, _session_id) { transport.send(:reap_expired_sessions) }, + "a request" => ->(transport, session_id) do transport.handle_request(session_post.call(session_id, { jsonrpc: "2.0", id: "ping-1", method: "ping" })) end, - "a notification finding it expired" => ->(transport, session_id) do - expire.call(transport, session_id) + "a notification" => ->(transport, session_id) do transport.handle_request(session_post.call(session_id, { jsonrpc: "2.0", method: "notifications/initialized" })) end, - "a notification sent to it after it expired" => ->(transport, session_id) do - expire.call(transport, session_id) + "a notification sent to it" => ->(transport, session_id) do transport.send_notification("notifications/message", { level: "info", data: "late" }, session_id: session_id) end, - "a broadcast finding it expired" => ->(transport, session_id) do - attach_get_stream.call(transport, session_id) - expire.call(transport, session_id) + "a broadcast" => ->(transport, session_id) do + transport.instance_variable_get(:@sessions)[session_id][:get_sse_stream] = TestStream.new transport.send_notification(Methods::NOTIFICATIONS_TOOLS_LIST_CHANGED) end, - "an initialize reclaiming its expired slot" => ->(transport, session_id) do - expire.call(transport, session_id) - start_session(transport) - end, - "a failed write to its GET stream" => ->(transport, session_id) do - transport.send(:drop_broken_stream, session_id, attach_get_stream.call(transport, session_id), nil) + "an initialize looking for a slot to reclaim" => ->(transport, _session_id) do + transport.handle_request( + create_rack_request( + "POST", + "/", + { "CONTENT_TYPE" => "application/json" }, + { jsonrpc: "2.0", method: "initialize", id: "init", params: initialize_params }.to_json, + ), + ) end, } - removals.each do |removal, remove_session| + checks.each do |check, run_check| server = Server.new(name: "test", tools: [], prompts: [], resources: []) transport = StreamableHTTPTransport.new(server, enable_json_response: true, session_idle_timeout: 60, max_sessions: 1) server.transport = transport @@ -4138,16 +4164,99 @@ def string call = Thread.new { transport.handle_request(held_tool_call(session_id, "call-1")) } started.pop - remove_session.call(transport, session_id) + # The request started longer ago than the timeout allows, and is still running. + transport.instance_variable_get(:@sessions)[session_id][:last_active_at] -= 61 + checked = run_check.call(transport, session_id) - assert_equal([["call-1", "Session terminated"]], cancellations, "session removed by #{removal}") + assert_empty(cancellations, "session checked by #{check}") + # The only slot is taken by a session that is not idle, so nothing is reclaimed for the newcomer. + assert_equal(503, checked[0]) if check == "an initialize looking for a slot to reclaim" + + release << true + response = call.value + assert_equal(200, response[0], "session checked by #{check}") + assert_equal("done", JSON.parse(response[2].first).dig("result", "content", 0, "text")) ensure - release&.push(true) - call&.join - transport&.close + release.push(true) + call.join + transport.close end end + test "a session's idle time counts from the end of its last request" do + server = Server.new(name: "test", tools: [], prompts: [], resources: []) + transport = StreamableHTTPTransport.new(server, enable_json_response: true, session_idle_timeout: 60) + server.transport = transport + started = Queue.new + release = Queue.new + define_held_tool(server, cancellations: [], started: started, release: release) + session_id, server_session = start_session(transport) + sessions = transport.instance_variable_get(:@sessions) + + call = Thread.new { transport.handle_request(held_tool_call(session_id, "call-1")) } + started.pop + sessions[session_id][:last_active_at] -= 61 + release << true + assert_equal(200, call.value[0]) + + transport.send(:reap_expired_sessions) + assert(sessions.key?(session_id), "the request has only just finished") + + finished_at = server_session.instance_variable_get(:@last_request_finished_at) + server_session.instance_variable_set(:@last_request_finished_at, finished_at - 61) + sessions[session_id][:last_active_at] = Process.clock_gettime(Process::CLOCK_MONOTONIC) + transport.send(:reap_expired_sessions) + assert(sessions.key?(session_id), "touched after the request finished") + + sessions[session_id][:last_active_at] -= 61 + transport.send(:reap_expired_sessions) + refute(sessions.key?(session_id), "idle for longer than the timeout since both") + ensure + release.push(true) + call.join + transport.close + end + + test "a session stays while one of two overlapping requests is still in flight" do + server = Server.new(name: "test", tools: [], prompts: [], resources: []) + transport = StreamableHTTPTransport.new(server, enable_json_response: true, session_idle_timeout: 60) + server.transport = transport + started = Queue.new + first_release = Queue.new + second_release = Queue.new + releases = { "call-1" => first_release, "call-2" => second_release } + server.define_tool(name: "held_tool") do |server_context:| + started << true + releases.fetch(server_context.cancellation.request_id).pop + + Tool::Response.new([{ type: "text", text: "done" }]) + end + session_id, server_session = start_session(transport) + sessions = transport.instance_variable_get(:@sessions) + + first = Thread.new { transport.handle_request(held_tool_call(session_id, "call-1")) } + second = Thread.new { transport.handle_request(held_tool_call(session_id, "call-2")) } + 2.times { started.pop } + first_release << true + assert_equal(200, first.value[0]) + + # Both the last touch and the end of the first request are older than the timeout; the second request still runs. + sessions[session_id][:last_active_at] -= 61 + finished_at = server_session.instance_variable_get(:@last_request_finished_at) + server_session.instance_variable_set(:@last_request_finished_at, finished_at - 61) + transport.send(:reap_expired_sessions) + assert(sessions.key?(session_id)) + + second_release << true + assert_equal(200, second.value[0]) + ensure + first_release.push(true) + second_release.push(true) + first.join + second.join + transport.close + end + test "cancellation hooks of a terminated session's requests run outside the transport lock" do server = Server.new(name: "test", tools: [], prompts: [], resources: []) transport = StreamableHTTPTransport.new(server, enable_json_response: true)