Skip to content

Ksn/update - #1

Merged
AmyrAhmady merged 1246 commits into
masterfrom
ksn/update
Nov 16, 2026
Merged

AmyrAhmady merged 1246 commits into
masterfrom
ksn/update

Conversation

@AmyrAhmady

Copy link
Copy Markdown
Member

No description provided.

yhirose and others added 30 commits September 11, 2026 16:54
199d7ee made read() return the new ReadResult::Timeout for every read
timeout and leave the connection open. The compile-time server default
(CPPHTTPLIB_WEBSOCKET_SERVER_READ_TIMEOUT_SECOND, 300s) is always in
effect, so a handler written as `while (ws.read(msg))`, the form the
README's Quick Start uses, no longer ended when a peer went quiet:
Timeout is non-zero, so the loop ran its body again with the previous
message still in `msg`, and the worker the backstop is meant to reclaim
was never released. Nothing caught it because every test of the new
result set a timeout explicitly and checked the result by value, and the
heartbeat tests keep the connection alive with pings.

The two timeouts mean different things. One the caller sets through
set_read_timeout() is a request for control back, and is reported as
Timeout on a still-open connection. The compile-time default is a
backstop against a peer that has gone quiet, and elapsing it is now a
failure again: read() returns Fail and closes the connection, as it did
before 199d7ee. WebSocket tracks whether set_read_timeout() was called,
and WebSocketClient carries the same flag over to the WebSocket it
creates on connect().

Tests use the heartbeat binary, which compiles both defaults down to 3s:
a `while (ws.read(msg))` server handler runs its body once and exits
when the client falls silent, and a client that never set a timeout gets
Fail with the connection closed. The README and cookbook now say which
timeout produces Timeout.

Claude-Session: https://claude.ai/code/session_01EF5uZ1X2kaHhqJ8VgfjVaQ
`sed -i ''` is BSD-only: GNU sed takes the '' as the script and the expression as a file name, so `just release --run` failed on Linux before touching anything.
The WebSocket upgrade path matched the route and switched protocols
without setting req.matched_route or calling pre_request_handler, so a
check placed there (e.g. authentication) never ran for WebSocket
routes. Set matched_route and run the handler before the upgrade; if it
handles the request, reply with a regular HTTP response instead of 101.

Also write rejected upgrade responses (from pre_routing_handler too)
with write_response_with_content, so they carry Content-Length. Without
it, a client reading the body waited until the keep-alive timeout.
The server wrote 100 Continue as soon as it saw the expectation, before
pre_routing_handler, pre_request_handler, or routing ran. A request
those handlers rejected, or one that matched no route, still invited
the client to send a body the server would never read.

Defer the interim response until the body is about to be read. If the
request is answered without reading the body, 100 Continue is never
sent and the connection is closed, since whether and when the client
sends the body is unknown.

Also treat a 417 returned by expect_100_continue_handler as the final
response. It used to be written as a bare status line, after which the
request was processed and a second response was written.
tls::shutdown() on OpenSSL called SSL_shutdown() a second time to wait
for the peer's close_notify. An idle keep-alive client never sends one,
so closing its connection held the worker thread until the read timeout,
and Server::stop() waited for it. Send close_notify and return, as the
Mbed TLS and wolfSSL backends already do.
…e#2585)

* reject control characters in chunk extensions in read_payload

* Bound every chunk-size line scan by the line terminator

read_payload() ended its scans of one line buffer two different ways: the
hex-size parse and the space skip that follows stopped on the NUL that
stream_line_reader::append() writes, while the new chunk-ext check walked
to an explicit end pointer. Compute that end pointer first and bound all
of them by it, so no scan depends on the buffer's NUL and the terminator
can never be read as line content.

The bare-LF branch is reachable only under
CPPHTTPLIB_ALLOW_LF_AS_LINE_TERMINATOR, where getline() ends the line on
an LF that is the terminator rather than extension text. Say so: the
comment below it explains why a bare LF inside the line is rejected, and
without that note the two read as contradictory. Its guard no longer
depends on the scan cursor either, since all it ever needed was a check
that there is a byte to look at.

* Reuse the chunked-body helper in the chunk-ext acceptance test

AcceptsChunkExtension repeated expect_chunked_body_rejected()'s body
verbatim apart from the expected status, so parameterise the helper on
the status and keep the rejection wrapper for the existing callers. The
decoded body is already checked by the /chunked handler, so asserting
the status is all the new test needs.

Also record why the control-character literal stays split: a hex escape
consumes every hex digit that follows it, so "\x01b" would be the single
byte \x1b rather than \x01 followed by 'b', and joining the halves would
quietly change what the test sends.

---------

Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
write_request_line checked the request target for CR/LF but concatenated
the method verbatim. A method carrying CR/LF could smuggle a whole
request ahead of the real one, and the client would take the smuggled
request's response as its own. A method with a space or an empty method
put a malformed request line on the wire.

Require the method to be a token (RFC 9110 Section 9.1) before anything
is written. All three callers (the buffered client path, open_stream and
the WebSocket handshake) go through this function and fail with
Error::Write, as they already do for a rejected target.

Claude-Session: https://claude.ai/code/session_01NTDesJQTQPEuu4o4XCu69g
process_request reads the response even after write_request fails, so
that an early response (e.g. 413/414) sent while the body is still being
uploaded is not lost. A request line or header rejected while the
request is being built in memory never reaches the socket, though, so
no response will come and the client blocked until the read timeout (or
until the server closed the idle connection).

write_request now reports such a local rejection, and process_request
returns immediately in that case. Socket write failures still read the
response as before.
open_stream wrote the request line and each header straight to the
socket, so a header rejected by check_and_write_headers left the request
line and the headers before it on the wire. Build them in a BufferStream
first and flush once, as write_request and the WebSocket handshake do.
yhirose#2578)

* Added a feature test to auto enable/disable CPPHTTPLIB_USE_NON_BLOCKING_GETADDRINFO

* turned status: WARNING 'GetAddrInfoExCancel is unavailable; disabling non-blocking getaddrinfo.' into a warning

* added ws2_32 for the GetAddrInfoExCancel. this catches previous false negatives

---------

Co-authored-by: Tobias Wallner <tobias.wallner@qtlabs.at>
…opy (yhirose#2589)

A file served from a mount point or through set_file_content() left in two
writes, one for the status line and headers and one for the body, because the
body came from a content provider. A small file is now read into the header
buffer so the whole response leaves in a single write. Only file-backed
providers are coalesced this way: a user-supplied provider may produce its data
over time, and holding the headers back until it finishes would stall the
client.

A large set_content() body was copied into the header buffer before being
sent. A body of CPPHTTPLIB_SEND_BUFSIZ or more is now written directly after
the headers, which saves the copy at the cost of one extra write.
parse_quality accepted values such as q=0.5junk because it checked only the conversion error and ignored the returned end pointer. Require the numeric parser to consume the complete q parameter so malformed Accept values are rejected and invalid Accept-Encoding weights are ignored. Add regression cases for both headers.
req.path is percent-decoded, so a request like GET /%0D%0A... put a
literal CR/LF into the NGINX-style log lines and let a client forge
extra entries. Log the raw req.target (matching NGINX's $request) and
escape '"', '\', control and non-ASCII bytes as \xHH the way NGINX
does. Also note in the README logging section that req.path may
contain control characters and should be escaped before logging.
RFC 9112 §3.2 does not allow control characters in the request-target,
and §2.2 requires a bare CR to be treated as invalid. parse_request_line
accepted them, so e.g. "GET /a\rb HTTP/1.1" was routed normally. Reject
any byte that is not VCHAR or obs-text with 400 Bad Request. obs-text is
still allowed since some clients send raw UTF-8 in the target.
Share the server's request-target check as fields::is_request_target()
and use it in write_request_line too. The client previously used
is_field_value(), which let an embedded SP or HTAB through.

encode_path() only escaped CR/LF among the control characters, so with
path encoding enabled a path like "/a\tb" would now be rejected instead
of sent. Percent-encode every control character (0x00-0x1F, 0x7F).
parse_sse_line checked only the error code of from_chars, which accepts
a leading '-' and stops at the first non-digit, so retry: -1 made the
client reconnect without waiting and retry: 10s set 10 ms. The SSE spec
ignores a retry value that is not all ASCII digits.
SSEClient::wait_for_reconnect() sleeps in 100ms steps until the
reconnect interval has elapsed. With an interval of 0 (for example
"retry: 0" from the server, or set_reconnect_interval(0)) it never
slept at all, so a server that sends "retry: 0" and closes the stream
made the client reconnect in a tight loop. set_max_reconnect_attempts()
does not stop this either, because each successful connection resets
the attempt counter.

Always wait at least one step (100ms). Intervals of 1-99ms already
waited 100ms because of the step size, so only 0 and negative values
change behavior.
A request whose Content-Length was present but not a valid decimal length
(e.g. "42, 42", "+42", "0x2e" or empty) was treated as having no body
unless a handler read it: no 400 was returned, the body was not drained,
and the bytes after the header block were parsed as the next request on
the keep-alive connection. Reject such a request with 400 and close the
connection before routing, as RFC 9112 Section 6.3 requires.

The server also kept reading after a response that announced
Connection: close. A rejected request line or header block left the rest
of the message to be parsed as a new request, and an error response to a
bodyless request did the same with whatever followed. Close the
connection whenever the final response carries Connection: close
(RFC 9112 Section 9.6), and mark the two request-head rejection paths
closed explicitly as the other rejection paths already do.
…hirose#2602)

CryptoAPI got only the leaf, so it fetched an issuer from the leaf's AIA
URL instead of using the intermediates the server sent. For
accounts.spotify.com that issuer chains to Certainly Root R1, which
Windows does not trust, while the server's own chain ends at Starfield
Root G2.

Add tls::get_peer_certs(), which returns the certificates the peer sent
in the same way get_ca_certs() returns the CA certificates, for every
backend. The CryptoAPI check puts them into a memory store that it
passes to CertGetCertificateChain(), skipping any certificate that
cannot be added. wolfSSL keeps the received chain only when built with
SESSION_CERTS; without it, CryptoAPI still gets the leaf alone.

Refs yhirose#2596
yhirose#2604)

Every backend loads the Windows ROOT and CA stores, but Windows adds a
root to them only when CryptoAPI needs it to build a chain. On a machine
that had not needed a root yet, the backend rejected the chain before
the CryptoAPI check ran, so Windows never fetched the root (for example
OpenSSL error 20 for accounts.spotify.com under Starfield Root G2).

With Windows verification enabled, the backend's chain verdict is no
longer used. Mbed TLS and wolfSSL skip chain verification during the
handshake, and the post-handshake verify result is ignored. The
CryptoAPI check becomes mandatory: a leaf that cannot be encoded now
fails the connection instead of skipping the check, and the chain must
allow server authentication, which the backends used to check.

A server certificate verifier works on the backend's chain verification,
so when one is set the backend still decides and CryptoAPI only adds its
own check, as before.

ClientCertMissing now disables hostname verification: cert.pem does not
match HOST, and on Windows the hostname check fails before the chain
check.

Refs yhirose#2596
parse_port checked only the error code of from_chars, which stops at
the first non-digit, so http://host:80abc was accepted as port 80, and
a redirect Location with such a port was followed. RFC 3986 defines
port as *DIGIT. Require the whole string to be consumed, as yhirose#2590 does
for quality values.
harbinresearcher and others added 28 commits October 25, 2026 20:05
open_stream_() created a decoder whatever set_decompress() said, so a response stream decoded the body when decompression was disabled, and failed with UnsupportedContentEncoding in a build without that decoder. The buffered path already checks the option; the streaming path now does too. The new test covers open_stream() and stream::Get().

Fixes yhirose#2639.
…#2641)

The macOS system CA loader took every certificate that has Trust Settings, in the system, admin or user domain, as a trusted root without reading those settings. A root set to "Never Trust" in Keychain Access, or trusted only for S/MIME or code signing, still verified TLS servers.

The loader now evaluates the settings for the SSL policy. A per-user setting overrides the admin domain, which overrides the system domain, and only TrustRoot and TrustAsRoot are loaded. The OpenSSL path shares the helper that the Mbed TLS and wolfSSL paths already use.
…e#2642)

write_websocket_frame and the request-body write in open_stream_ called
Stream::write once and only failed on a negative return, so a short
write (one record per mbedtls_ssl_write call, or SO_SNDTIMEO expiring
after part of the data went out) dropped the rest of the frame or body.
Route both through write_data, as the other body writers do.

Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
…ose#2644)

last_event_id() returned a reference to a string that run_event_loop() assigns on its own thread under start_async(), so a caller reading it from another thread raced the writes. The member is now guarded by a mutex and the getter returns a copy.

The return type changes from const std::string & to std::string. Code that binds the result to a const reference or copies it still compiles.
On NTFS a ':' in a path names a stream of the file, so /dir/test.html::$DATA opened the default stream of dir/test.html and passed a set_pre_routing_handler() guard on /dir/test.html, and /setup.exe:Zone.Identifier served the stream Windows attaches to downloaded files. is_valid_path() now refuses a ':' on Windows.

Elsewhere a ':' is an ordinary file name character, so it is still served: a file such as 2024-01-01T10:00:00.log would otherwise answer 404. The test pins both sides.
resolve_relative_location removed "." and ".." segments only from a relative-path Location. A Location that starts with "/" or an absolute URI was requested as is, so a redirect to /dir/../target went out as /dir/../target and a server that does not normalize the path answered 404.

RFC 3986 section 5.2.2 applies remove_dot_segments to the target path of every reference that has a path, and Go and urllib3 do the same. The loop now lives in detail::remove_dot_segments, and ClientImpl::redirect applies it to an absolute path as well. Relative locations resolve as before.
…e#2651)

On Windows a ':' ends a drive prefix or starts an NTFS stream name, so a filename such as "C:.." or "a.txt:stream" survived sanitize_filename and could replace the upload directory or write to an alternate data stream. Cut at ':' on Windows as well.
…hirose#2649)

Mbed TLS and wolfSSL read a TLS EOF without close_notify as a clean end, so a response delimited by the connection close could be cut short and still come back complete. Both reads now end cleanly only on a close_notify, as on OpenSSL.
…p() (yhirose#2647)

Through a CONNECT proxy it is the proxy that connects to the origin, and the CONNECT target was always the original hostname, so there was no way to send the tunnel to another address. set_tunnel_hostname_addr_map() sets that target on SSLClient, Client and WebSocketClient. The CONNECT request's Host follows it, including the Digest retry, while the Host header, the SNI and the certificate check inside the tunnel keep the original hostname. The map is also copied to a redirect client.

set_hostname_addr_map() is deprecated in favor of set_direct_hostname_addr_map(), which behaves the same and names the connection it applies to. The two maps are independent and correspond to curl's --resolve and --connect-to; neither applies to the proxy's own address.
…ose#2652)

ClientImpl and SSLClient let callers set origin credentials with
set_basic_auth()/set_bearer_token_auth(), but WebSocketClient could only
take a fixed Authorization header through its constructor. Add both
setters: each connect() adds the Authorization header to the upgrade
request from the current credentials (Basic over Bearer), unless the
constructor headers already carry one, and never to the proxy's CONNECT
request.
…e#2653)

stop() shuts the socket down to end a request in flight, but on Windows
that does not wake a read waiting in WSAPoll (yhirose#1396). The request then
waited until the server answered, the read timeout passed, or the OS
dropped the connection.

stop() now also sets a flag, and send_() gives the request's stream an
interrupt check that reads it, as SSEClient::stop() does since 801bf2e.
The wait for data then ends within 100 ms on every platform. A request
started after stop() clears the flag. A redirect or a digest retry sent
inside the request keeps it.

select_read_interruptible() now rounds its last wait up to a whole
millisecond. poll() takes milliseconds, and the wait ended up to 1 ms
before the deadline, which MaxTimeoutTest caught once every client
request waited through this function.
…irose#2654)

* keep a relative Location on its host in resolve_relative_location

* Shorten the comment on on_this_host in resolve_relative_location

---------

Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* Cap a send() or wolfSSL_read() length at INT_MAX

On 64-bit Windows SocketStream::write() handed size to send(), whose
length is an int, so a single write of 4 GiB or more went out as its low
32 bits: a 4 GiB + 1 byte write sent one byte, and write_data() then
retried the remaining 4 GiB as zero-byte sends forever. The INT_MAX cap
that read() has was limited to 32-bit Windows in df20c27 (yhirose#1294), when
the body was still written with one unchecked write(); write_data() now
sends the rest, so the cap applies to every Windows build again.

The wolfSSL tls::read() had the same cast, so a read into a buffer over
INT_MAX failed. Cap it as the OpenSSL backend has done since f80864c.

* Drop the 4 GiB write test and shorten comments

The Windows-only test allocated 4 GiB and took about 5 s, which is too
heavy for CI.

---------

Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
RFC 9110 14.2 says an origin server MUST ignore a Range header field
with a range unit it does not understand. parse_range_header() accepts
only bytes=, so a request such as "Range: items=1-2" got a 416 instead
of the full response. The unit is now checked first, and only a bytes
range is parsed and answered with a 416 when it is invalid.
A request sent with send(Request) and params got those params appended
to the Location of a redirect that had no query, on the same origin or
another one. The Location alone names the next target, as it already
did for Get/Post(path, params).
)

A client using set_address_family(AF_UNIX) took the host of an absolute Location as a socket path, so a redirect to http://localhost/next failed and one to http://other.sock/next went to ./other.sock. Every hop now stays on the client's socket, as with curl's --unix-socket.

The same host is still decided from the Location, as curl does: only localhost (the Host this client sends) counts as this client, so a redirect to any other host goes through the cross-host client and drops Authorization and Cookie. That client reaches the original socket through addr_map_, and a host the caller mapped with set_direct_hostname_addr_map keeps its mapping. An https Location runs TLS over the socket with the Location's host.

The mapped-target lookup shared with make_connect_request_target is factored into detail::addr_map_target.
ThreadPool::enqueue constructed the dynamic thread without handling
std::system_error. When thread creation failed (e.g. EAGAIN under a
pids limit), the exception escaped the accept loop and destroying the
pool's joinable threads called std::terminate. Treat the failure as
"no spare capacity" and leave the job for an existing worker.
Co-authored-by: astunm <astunm@users.noreply.github.com>
Move the line splitting and event assembly out of
SSEClient::run_event_loop() into a parser that is fed chunks and reports
events, ids and retry values through handlers. The event loop is left
with I/O and reconnection, and the parser can be driven without a
server, which the following test and fuzzer changes rely on.

No behavior change.
Move the SSE parsing tests from a fixture that started a server and an
SSEClient for every case to unit tests of detail::SSEParser, and turn
NonDigitRetryFieldIgnored, which slept for 500ms, into one of them. Add
tests that feed each stream split at every position and byte by byte,
and that cover the max event length and an event left incomplete at the
end of the stream.
Add websocket_fuzzer covering frame reading, UTF-8 and close payload validation, and the upgrade response parser. Add websocket_fuzzer.options to raise max_len above the 64 KiB frame length boundary.
Add cases for relative location resolution, dot segment removal, URI encoding and decoding, and filename sanitization.
Feed the event stream to the parser in chunks and abort if the reported events, ids and retries differ from parsing it in one piece. Also run with a length limit to cover the early-stop path.
Fuzz Server::handle_file_request through a mount point backed by a temporary directory that holds symlinks escaping the mount root, one of them into a sibling directory sharing its name prefix. Abort if a token stored outside the root appears in a response. The seeds are based on those of yhirose#2661.

Move FuzzedStream into fuzzed_stream.h so that both server fuzzers share it, and add FuzzableServer::MarkAsRunning(), without which a server that is not listening writes no file.
@AmyrAhmady
AmyrAhmady merged commit a47663d into master Nov 16, 2026
38 of 45 checks passed
@AmyrAhmady
AmyrAhmady deleted the ksn/update branch November 16, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.