Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions docs/_server/cancellation.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,9 @@ the opt-in detection only wraps the context when the block signature asks for it

When `StreamableHTTPTransport` removes a stateful session while requests are still running in it, those requests
are cancelled as if the client had sent `notifications/cancelled` for each of them. This happens when the client terminates
the session with `DELETE`, when the session passes `session_idle_timeout`, when its GET stream breaks without an event store
to resume from, and when the transport is closed. `server_context.cancelled?` turns true, `on_cancel` hooks registered on
the session with `DELETE`, when its GET stream breaks without an event store to resume from, and when the transport is closed.
A session that passes `session_idle_timeout` has no request running, since a running request keeps its session from expiring.
`server_context.cancelled?` turns true, `on_cancel` hooks registered on
`server_context.cancellation` run with the reason `"Session terminated"`, and the response is suppressed, since no client
is left to receive it.

Expand Down
14 changes: 11 additions & 3 deletions docs/_server/transports.md
Original file line number Diff line number Diff line change
Expand Up @@ -315,9 +315,17 @@ with at most 1024 JSON values in its `params`; a larger one is rejected with HTT
The byte bound can be raised or removed with `nil`, in which case `max_request_bytes` still caps the body; the value bound always applies,
since a session keeps the parsed data and many small values parse into far more objects than their size suggests.
Together they bound both the serialized input and the number of parsed values each of the `max_sessions` sessions can retain.
A session's idle timer is reset by activity that touches it (a GET, or a regular-request POST), and expired sessions are collected by
a background reaper roughly once a minute, so cleanup lags inactivity by up to that interval. At the cap, the transport first reclaims
any already-expired slots and then, if still full, rejects a new `initialize` with HTTP 503 (it does not evict an existing session).
A session's idle timer is reset by activity that touches it (a GET, or a regular-request POST) and does not run while one of
its requests is still being handled: it starts again when that request ends, so a tool call longer than the timeout keeps
its session and its response. A notification, which has no response to wait for, does not hold the timer,
and neither does an open GET stream: opening it resets the timer once, so a client that only listens keeps its session
by sending a request, such as a `ping`, within the timeout.
The timeout does not bound a request itself: a handler that never returns holds its session, and its slot under `max_sessions`,
for as long as it runs, so a limit on how long a handler may run belongs in the handler,
as a deadline it checks the way it checks `server_context.cancelled?`.
Expired sessions are collected by a background reaper roughly once a minute, so cleanup lags inactivity by up to that interval.
At the cap, the transport first reclaims any already-expired slots (a session with a request still being handled is
never one of them) and then, if still full, rejects a new `initialize` with HTTP 503 (it does not evict an existing session).

```ruby
# Tune the limits
Expand Down
12 changes: 10 additions & 2 deletions lib/mcp/server/transports/streamable_http_transport.rb
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ class InvalidJsonError < StandardError; end
# object instead of an SSE stream.
# @param session_idle_timeout [Numeric, nil] seconds before an idle session is reaped; defaults
# to `DEFAULT_SESSION_IDLE_TIMEOUT` (1800) in stateful mode, and an explicit `nil` disables
# expiry. Not supported in stateless mode.
# expiry. A session is not idle while one of its requests is running. Not supported in stateless mode.
# @param max_sessions [Integer, nil] cap on the concurrent session count in stateful mode; a new
# `initialize` past the cap is rejected with HTTP 503, and `nil` disables the cap.
# @param allowed_origins [Array<String>, :any, nil] extra `Origin` values accepted in addition to
Expand Down Expand Up @@ -2960,10 +2960,18 @@ def send_keepalive_ping(session_id, stream)
raise # Re-raise to exit the keepalive loop
end

# A session is idle only while none of its requests is running, and its idle time counts from the later of
# the last activity that touched it and the end of its last request. `last_active_at` alone is the start of
# the latest request, so a request longer than the timeout would lose its session, and with it its response,
# while the client is still waiting for it.
def session_expired?(session)
return false unless @session_idle_timeout

Process.clock_gettime(Process::CLOCK_MONOTONIC) - session[:last_active_at] > @session_idle_timeout
busy, last_request_finished_at = session[:server_session]&.in_flight_activity
return false if busy

idle_since = [session[:last_active_at], last_request_finished_at].compact.max
Process.clock_gettime(Process::CLOCK_MONOTONIC) - idle_since > @session_idle_timeout
end

# Stream ids carry the session id, so a `Last-Event-ID` is matched to the session presenting it with
Expand Down
12 changes: 11 additions & 1 deletion lib/mcp/server_session.rb
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ def initialize(server:, transport:, session_id: nil, era: nil)
@protocol_version = nil
@in_flight = {}
@in_flight_mutex = Mutex.new
@last_request_finished_at = nil
@initialized = false
end

Expand Down Expand Up @@ -85,10 +86,19 @@ def unregister_in_flight(request_id, cancellation: nil)
@in_flight_mutex.synchronize do
next if cancellation && !@in_flight[request_id].equal?(cancellation)

@in_flight.delete(request_id)
@last_request_finished_at = Process.clock_gettime(Process::CLOCK_MONOTONIC) if @in_flight.delete(request_id)
end
end

# What a transport needs to tell whether the session is idle, read in one step so the two cannot disagree:
# whether a request is still in flight, and the monotonic clock reading at which the last one finished
# (`nil` when none has).
#
# @api private
def in_flight_activity
@in_flight_mutex.synchronize { [!@in_flight.empty?, @last_request_finished_at] }
end

# Whether `request_id` is currently in flight, so a transport can refuse a colliding request
# before registering any state of its own for it.
def in_flight?(request_id)
Expand Down
165 changes: 137 additions & 28 deletions test/mcp/server/transports/streamable_http_transport_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4085,47 +4085,73 @@ def string
end

test "every other path that removes a session cancels the requests in flight in it" do
expire = ->(transport, session_id) { transport.instance_variable_get(:@sessions)[session_id][:last_active_at] -= 61 }
session_post = ->(session_id, body) do
create_rack_request("POST", "/", { "CONTENT_TYPE" => "application/json", "HTTP_MCP_SESSION_ID" => session_id }, body.to_json)
end
attach_get_stream = ->(transport, session_id) do
TestStream.new.tap { |stream| transport.instance_variable_get(:@sessions)[session_id][:get_sse_stream] = stream }
end

removals = {
"closing the transport" => ->(transport, _session_id) { transport.close },
"the reaper finding it expired" => ->(transport, session_id) do
expire.call(transport, session_id)
transport.send(:reap_expired_sessions)
"a failed write to its GET stream" => ->(transport, session_id) do
transport.send(:drop_broken_stream, session_id, attach_get_stream.call(transport, session_id), nil)
end,
"a request finding it expired" => ->(transport, session_id) do
expire.call(transport, session_id)
}

removals.each do |removal, remove_session|
server = Server.new(name: "test", tools: [], prompts: [], resources: [])
transport = StreamableHTTPTransport.new(server, enable_json_response: true)
server.transport = transport
cancellations = []
started = Queue.new
release = Queue.new
define_held_tool(server, cancellations: cancellations, started: started, release: release)
session_id, = start_session(transport)

call = Thread.new { transport.handle_request(held_tool_call(session_id, "call-1")) }
started.pop

remove_session.call(transport, session_id)

assert_equal([["call-1", "Session terminated"]], cancellations, "session removed by #{removal}")
ensure
release.push(true)
call.join
transport.close
end
end

test "a session with a request in flight does not expire, whichever path checks it" do
session_post = ->(session_id, body) do
create_rack_request("POST", "/", { "CONTENT_TYPE" => "application/json", "HTTP_MCP_SESSION_ID" => session_id }, body.to_json)
end

checks = {
"the reaper" => ->(transport, _session_id) { transport.send(:reap_expired_sessions) },
"a request" => ->(transport, session_id) do
transport.handle_request(session_post.call(session_id, { jsonrpc: "2.0", id: "ping-1", method: "ping" }))
end,
"a notification finding it expired" => ->(transport, session_id) do
expire.call(transport, session_id)
"a notification" => ->(transport, session_id) do
transport.handle_request(session_post.call(session_id, { jsonrpc: "2.0", method: "notifications/initialized" }))
end,
"a notification sent to it after it expired" => ->(transport, session_id) do
expire.call(transport, session_id)
"a notification sent to it" => ->(transport, session_id) do
transport.send_notification("notifications/message", { level: "info", data: "late" }, session_id: session_id)
end,
"a broadcast finding it expired" => ->(transport, session_id) do
attach_get_stream.call(transport, session_id)
expire.call(transport, session_id)
"a broadcast" => ->(transport, session_id) do
transport.instance_variable_get(:@sessions)[session_id][:get_sse_stream] = TestStream.new
transport.send_notification(Methods::NOTIFICATIONS_TOOLS_LIST_CHANGED)
end,
"an initialize reclaiming its expired slot" => ->(transport, session_id) do
expire.call(transport, session_id)
start_session(transport)
end,
"a failed write to its GET stream" => ->(transport, session_id) do
transport.send(:drop_broken_stream, session_id, attach_get_stream.call(transport, session_id), nil)
"an initialize looking for a slot to reclaim" => ->(transport, _session_id) do
transport.handle_request(
create_rack_request(
"POST",
"/",
{ "CONTENT_TYPE" => "application/json" },
{ jsonrpc: "2.0", method: "initialize", id: "init", params: initialize_params }.to_json,
),
)
end,
}

removals.each do |removal, remove_session|
checks.each do |check, run_check|
server = Server.new(name: "test", tools: [], prompts: [], resources: [])
transport = StreamableHTTPTransport.new(server, enable_json_response: true, session_idle_timeout: 60, max_sessions: 1)
server.transport = transport
Expand All @@ -4138,16 +4164,99 @@ def string
call = Thread.new { transport.handle_request(held_tool_call(session_id, "call-1")) }
started.pop

remove_session.call(transport, session_id)
# The request started longer ago than the timeout allows, and is still running.
transport.instance_variable_get(:@sessions)[session_id][:last_active_at] -= 61
checked = run_check.call(transport, session_id)

assert_equal([["call-1", "Session terminated"]], cancellations, "session removed by #{removal}")
assert_empty(cancellations, "session checked by #{check}")
# The only slot is taken by a session that is not idle, so nothing is reclaimed for the newcomer.
assert_equal(503, checked[0]) if check == "an initialize looking for a slot to reclaim"

release << true
response = call.value
assert_equal(200, response[0], "session checked by #{check}")
assert_equal("done", JSON.parse(response[2].first).dig("result", "content", 0, "text"))
ensure
release&.push(true)
call&.join
transport&.close
release.push(true)
call.join
transport.close
end
end

test "a session's idle time counts from the end of its last request" do
server = Server.new(name: "test", tools: [], prompts: [], resources: [])
transport = StreamableHTTPTransport.new(server, enable_json_response: true, session_idle_timeout: 60)
server.transport = transport
started = Queue.new
release = Queue.new
define_held_tool(server, cancellations: [], started: started, release: release)
session_id, server_session = start_session(transport)
sessions = transport.instance_variable_get(:@sessions)

call = Thread.new { transport.handle_request(held_tool_call(session_id, "call-1")) }
started.pop
sessions[session_id][:last_active_at] -= 61
release << true
assert_equal(200, call.value[0])

transport.send(:reap_expired_sessions)
assert(sessions.key?(session_id), "the request has only just finished")

finished_at = server_session.instance_variable_get(:@last_request_finished_at)
server_session.instance_variable_set(:@last_request_finished_at, finished_at - 61)
sessions[session_id][:last_active_at] = Process.clock_gettime(Process::CLOCK_MONOTONIC)
transport.send(:reap_expired_sessions)
assert(sessions.key?(session_id), "touched after the request finished")

sessions[session_id][:last_active_at] -= 61
transport.send(:reap_expired_sessions)
refute(sessions.key?(session_id), "idle for longer than the timeout since both")
ensure
release.push(true)
call.join
transport.close
end

test "a session stays while one of two overlapping requests is still in flight" do
server = Server.new(name: "test", tools: [], prompts: [], resources: [])
transport = StreamableHTTPTransport.new(server, enable_json_response: true, session_idle_timeout: 60)
server.transport = transport
started = Queue.new
first_release = Queue.new
second_release = Queue.new
releases = { "call-1" => first_release, "call-2" => second_release }
server.define_tool(name: "held_tool") do |server_context:|
started << true
releases.fetch(server_context.cancellation.request_id).pop

Tool::Response.new([{ type: "text", text: "done" }])
end
session_id, server_session = start_session(transport)
sessions = transport.instance_variable_get(:@sessions)

first = Thread.new { transport.handle_request(held_tool_call(session_id, "call-1")) }
second = Thread.new { transport.handle_request(held_tool_call(session_id, "call-2")) }
2.times { started.pop }
first_release << true
assert_equal(200, first.value[0])

# Both the last touch and the end of the first request are older than the timeout; the second request still runs.
sessions[session_id][:last_active_at] -= 61
finished_at = server_session.instance_variable_get(:@last_request_finished_at)
server_session.instance_variable_set(:@last_request_finished_at, finished_at - 61)
transport.send(:reap_expired_sessions)
assert(sessions.key?(session_id))

second_release << true
assert_equal(200, second.value[0])
ensure
first_release.push(true)
second_release.push(true)
first.join
second.join
transport.close
end

test "cancellation hooks of a terminated session's requests run outside the transport lock" do
server = Server.new(name: "test", tools: [], prompts: [], resources: [])
transport = StreamableHTTPTransport.new(server, enable_json_response: true)
Expand Down