Repository navigation
Ksn/update - #1
Merged
Merged
Conversation
199d7ee made read() return the new ReadResult::Timeout for every read timeout and leave the connection open. The compile-time server default (CPPHTTPLIB_WEBSOCKET_SERVER_READ_TIMEOUT_SECOND, 300s) is always in effect, so a handler written as `while (ws.read(msg))`, the form the README's Quick Start uses, no longer ended when a peer went quiet: Timeout is non-zero, so the loop ran its body again with the previous message still in `msg`, and the worker the backstop is meant to reclaim was never released. Nothing caught it because every test of the new result set a timeout explicitly and checked the result by value, and the heartbeat tests keep the connection alive with pings. The two timeouts mean different things. One the caller sets through set_read_timeout() is a request for control back, and is reported as Timeout on a still-open connection. The compile-time default is a backstop against a peer that has gone quiet, and elapsing it is now a failure again: read() returns Fail and closes the connection, as it did before 199d7ee. WebSocket tracks whether set_read_timeout() was called, and WebSocketClient carries the same flag over to the WebSocket it creates on connect(). Tests use the heartbeat binary, which compiles both defaults down to 3s: a `while (ws.read(msg))` server handler runs its body once and exits when the client falls silent, and a client that never set a timeout gets Fail with the connection closed. The README and cookbook now say which timeout produces Timeout. Claude-Session: https://claude.ai/code/session_01EF5uZ1X2kaHhqJ8VgfjVaQ
`sed -i ''` is BSD-only: GNU sed takes the '' as the script and the expression as a file name, so `just release --run` failed on Linux before touching anything.
The WebSocket upgrade path matched the route and switched protocols without setting req.matched_route or calling pre_request_handler, so a check placed there (e.g. authentication) never ran for WebSocket routes. Set matched_route and run the handler before the upgrade; if it handles the request, reply with a regular HTTP response instead of 101. Also write rejected upgrade responses (from pre_routing_handler too) with write_response_with_content, so they carry Content-Length. Without it, a client reading the body waited until the keep-alive timeout.
The server wrote 100 Continue as soon as it saw the expectation, before pre_routing_handler, pre_request_handler, or routing ran. A request those handlers rejected, or one that matched no route, still invited the client to send a body the server would never read. Defer the interim response until the body is about to be read. If the request is answered without reading the body, 100 Continue is never sent and the connection is closed, since whether and when the client sends the body is unknown. Also treat a 417 returned by expect_100_continue_handler as the final response. It used to be written as a bare status line, after which the request was processed and a second response was written.
tls::shutdown() on OpenSSL called SSL_shutdown() a second time to wait for the peer's close_notify. An idle keep-alive client never sends one, so closing its connection held the worker thread until the read timeout, and Server::stop() waited for it. Send close_notify and return, as the Mbed TLS and wolfSSL backends already do.
…e#2585) * reject control characters in chunk extensions in read_payload * Bound every chunk-size line scan by the line terminator read_payload() ended its scans of one line buffer two different ways: the hex-size parse and the space skip that follows stopped on the NUL that stream_line_reader::append() writes, while the new chunk-ext check walked to an explicit end pointer. Compute that end pointer first and bound all of them by it, so no scan depends on the buffer's NUL and the terminator can never be read as line content. The bare-LF branch is reachable only under CPPHTTPLIB_ALLOW_LF_AS_LINE_TERMINATOR, where getline() ends the line on an LF that is the terminator rather than extension text. Say so: the comment below it explains why a bare LF inside the line is rejected, and without that note the two read as contradictory. Its guard no longer depends on the scan cursor either, since all it ever needed was a check that there is a byte to look at. * Reuse the chunked-body helper in the chunk-ext acceptance test AcceptsChunkExtension repeated expect_chunked_body_rejected()'s body verbatim apart from the expected status, so parameterise the helper on the status and keep the rejection wrapper for the existing callers. The decoded body is already checked by the /chunked handler, so asserting the status is all the new test needs. Also record why the control-character literal stays split: a hex escape consumes every hex digit that follows it, so "\x01b" would be the single byte \x1b rather than \x01 followed by 'b', and joining the halves would quietly change what the test sends. --------- Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
write_request_line checked the request target for CR/LF but concatenated the method verbatim. A method carrying CR/LF could smuggle a whole request ahead of the real one, and the client would take the smuggled request's response as its own. A method with a space or an empty method put a malformed request line on the wire. Require the method to be a token (RFC 9110 Section 9.1) before anything is written. All three callers (the buffered client path, open_stream and the WebSocket handshake) go through this function and fail with Error::Write, as they already do for a rejected target. Claude-Session: https://claude.ai/code/session_01NTDesJQTQPEuu4o4XCu69g
process_request reads the response even after write_request fails, so that an early response (e.g. 413/414) sent while the body is still being uploaded is not lost. A request line or header rejected while the request is being built in memory never reaches the socket, though, so no response will come and the client blocked until the read timeout (or until the server closed the idle connection). write_request now reports such a local rejection, and process_request returns immediately in that case. Socket write failures still read the response as before.
open_stream wrote the request line and each header straight to the socket, so a header rejected by check_and_write_headers left the request line and the headers before it on the wire. Build them in a BufferStream first and flush once, as write_request and the WebSocket handshake do.
yhirose#2578) * Added a feature test to auto enable/disable CPPHTTPLIB_USE_NON_BLOCKING_GETADDRINFO * turned status: WARNING 'GetAddrInfoExCancel is unavailable; disabling non-blocking getaddrinfo.' into a warning * added ws2_32 for the GetAddrInfoExCancel. this catches previous false negatives --------- Co-authored-by: Tobias Wallner <tobias.wallner@qtlabs.at>
…opy (yhirose#2589) A file served from a mount point or through set_file_content() left in two writes, one for the status line and headers and one for the body, because the body came from a content provider. A small file is now read into the header buffer so the whole response leaves in a single write. Only file-backed providers are coalesced this way: a user-supplied provider may produce its data over time, and holding the headers back until it finishes would stall the client. A large set_content() body was copied into the header buffer before being sent. A body of CPPHTTPLIB_SEND_BUFSIZ or more is now written directly after the headers, which saves the copy at the cost of one extra write.
parse_quality accepted values such as q=0.5junk because it checked only the conversion error and ignored the returned end pointer. Require the numeric parser to consume the complete q parameter so malformed Accept values are rejected and invalid Accept-Encoding weights are ignored. Add regression cases for both headers.
req.path is percent-decoded, so a request like GET /%0D%0A... put a literal CR/LF into the NGINX-style log lines and let a client forge extra entries. Log the raw req.target (matching NGINX's $request) and escape '"', '\', control and non-ASCII bytes as \xHH the way NGINX does. Also note in the README logging section that req.path may contain control characters and should be escaped before logging.
RFC 9112 §3.2 does not allow control characters in the request-target, and §2.2 requires a bare CR to be treated as invalid. parse_request_line accepted them, so e.g. "GET /a\rb HTTP/1.1" was routed normally. Reject any byte that is not VCHAR or obs-text with 400 Bad Request. obs-text is still allowed since some clients send raw UTF-8 in the target.
Share the server's request-target check as fields::is_request_target() and use it in write_request_line too. The client previously used is_field_value(), which let an embedded SP or HTAB through. encode_path() only escaped CR/LF among the control characters, so with path encoding enabled a path like "/a\tb" would now be rejected instead of sent. Percent-encode every control character (0x00-0x1F, 0x7F).
parse_sse_line checked only the error code of from_chars, which accepts a leading '-' and stops at the first non-digit, so retry: -1 made the client reconnect without waiting and retry: 10s set 10 ms. The SSE spec ignores a retry value that is not all ASCII digits.
SSEClient::wait_for_reconnect() sleeps in 100ms steps until the reconnect interval has elapsed. With an interval of 0 (for example "retry: 0" from the server, or set_reconnect_interval(0)) it never slept at all, so a server that sends "retry: 0" and closes the stream made the client reconnect in a tight loop. set_max_reconnect_attempts() does not stop this either, because each successful connection resets the attempt counter. Always wait at least one step (100ms). Intervals of 1-99ms already waited 100ms because of the step size, so only 0 and negative values change behavior.
A request whose Content-Length was present but not a valid decimal length (e.g. "42, 42", "+42", "0x2e" or empty) was treated as having no body unless a handler read it: no 400 was returned, the body was not drained, and the bytes after the header block were parsed as the next request on the keep-alive connection. Reject such a request with 400 and close the connection before routing, as RFC 9112 Section 6.3 requires. The server also kept reading after a response that announced Connection: close. A rejected request line or header block left the rest of the message to be parsed as a new request, and an error response to a bodyless request did the same with whatever followed. Close the connection whenever the final response carries Connection: close (RFC 9112 Section 9.6), and mark the two request-head rejection paths closed explicitly as the other rejection paths already do.
…hirose#2602) CryptoAPI got only the leaf, so it fetched an issuer from the leaf's AIA URL instead of using the intermediates the server sent. For accounts.spotify.com that issuer chains to Certainly Root R1, which Windows does not trust, while the server's own chain ends at Starfield Root G2. Add tls::get_peer_certs(), which returns the certificates the peer sent in the same way get_ca_certs() returns the CA certificates, for every backend. The CryptoAPI check puts them into a memory store that it passes to CertGetCertificateChain(), skipping any certificate that cannot be added. wolfSSL keeps the received chain only when built with SESSION_CERTS; without it, CryptoAPI still gets the leaf alone. Refs yhirose#2596
yhirose#2604) Every backend loads the Windows ROOT and CA stores, but Windows adds a root to them only when CryptoAPI needs it to build a chain. On a machine that had not needed a root yet, the backend rejected the chain before the CryptoAPI check ran, so Windows never fetched the root (for example OpenSSL error 20 for accounts.spotify.com under Starfield Root G2). With Windows verification enabled, the backend's chain verdict is no longer used. Mbed TLS and wolfSSL skip chain verification during the handshake, and the post-handshake verify result is ignored. The CryptoAPI check becomes mandatory: a leaf that cannot be encoded now fails the connection instead of skipping the check, and the chain must allow server authentication, which the backends used to check. A server certificate verifier works on the backend's chain verification, so when one is set the backend still decides and CryptoAPI only adds its own check, as before. ClientCertMissing now disables hostname verification: cert.pem does not match HOST, and on Windows the hostname check fails before the chain check. Refs yhirose#2596
parse_port checked only the error code of from_chars, which stops at the first non-digit, so http://host:80abc was accepted as port 80, and a redirect Location with such a port was followed. RFC 3986 defines port as *DIGIT. Require the whole string to be consumed, as yhirose#2590 does for quality values.
open_stream_() created a decoder whatever set_decompress() said, so a response stream decoded the body when decompression was disabled, and failed with UnsupportedContentEncoding in a build without that decoder. The buffered path already checks the option; the streaming path now does too. The new test covers open_stream() and stream::Get(). Fixes yhirose#2639.
…#2641) The macOS system CA loader took every certificate that has Trust Settings, in the system, admin or user domain, as a trusted root without reading those settings. A root set to "Never Trust" in Keychain Access, or trusted only for S/MIME or code signing, still verified TLS servers. The loader now evaluates the settings for the SSL policy. A per-user setting overrides the admin domain, which overrides the system domain, and only TrustRoot and TrustAsRoot are loaded. The OpenSSL path shares the helper that the Mbed TLS and wolfSSL paths already use.
…e#2642) write_websocket_frame and the request-body write in open_stream_ called Stream::write once and only failed on a negative return, so a short write (one record per mbedtls_ssl_write call, or SO_SNDTIMEO expiring after part of the data went out) dropped the rest of the frame or body. Route both through write_data, as the other body writers do. Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
…ose#2644) last_event_id() returned a reference to a string that run_event_loop() assigns on its own thread under start_async(), so a caller reading it from another thread raced the writes. The member is now guarded by a mutex and the getter returns a copy. The return type changes from const std::string & to std::string. Code that binds the result to a const reference or copies it still compiles.
On NTFS a ':' in a path names a stream of the file, so /dir/test.html::$DATA opened the default stream of dir/test.html and passed a set_pre_routing_handler() guard on /dir/test.html, and /setup.exe:Zone.Identifier served the stream Windows attaches to downloaded files. is_valid_path() now refuses a ':' on Windows. Elsewhere a ':' is an ordinary file name character, so it is still served: a file such as 2024-01-01T10:00:00.log would otherwise answer 404. The test pins both sides.
resolve_relative_location removed "." and ".." segments only from a relative-path Location. A Location that starts with "/" or an absolute URI was requested as is, so a redirect to /dir/../target went out as /dir/../target and a server that does not normalize the path answered 404. RFC 3986 section 5.2.2 applies remove_dot_segments to the target path of every reference that has a path, and Go and urllib3 do the same. The loop now lives in detail::remove_dot_segments, and ClientImpl::redirect applies it to an absolute path as well. Relative locations resolve as before.
…e#2651) On Windows a ':' ends a drive prefix or starts an NTFS stream name, so a filename such as "C:.." or "a.txt:stream" survived sanitize_filename and could replace the upload directory or write to an alternate data stream. Cut at ':' on Windows as well.
…hirose#2649) Mbed TLS and wolfSSL read a TLS EOF without close_notify as a clean end, so a response delimited by the connection close could be cut short and still come back complete. Both reads now end cleanly only on a close_notify, as on OpenSSL.
…p() (yhirose#2647) Through a CONNECT proxy it is the proxy that connects to the origin, and the CONNECT target was always the original hostname, so there was no way to send the tunnel to another address. set_tunnel_hostname_addr_map() sets that target on SSLClient, Client and WebSocketClient. The CONNECT request's Host follows it, including the Digest retry, while the Host header, the SNI and the certificate check inside the tunnel keep the original hostname. The map is also copied to a redirect client. set_hostname_addr_map() is deprecated in favor of set_direct_hostname_addr_map(), which behaves the same and names the connection it applies to. The two maps are independent and correspond to curl's --resolve and --connect-to; neither applies to the proxy's own address.
…ose#2652) ClientImpl and SSLClient let callers set origin credentials with set_basic_auth()/set_bearer_token_auth(), but WebSocketClient could only take a fixed Authorization header through its constructor. Add both setters: each connect() adds the Authorization header to the upgrade request from the current credentials (Basic over Bearer), unless the constructor headers already carry one, and never to the proxy's CONNECT request.
…e#2653) stop() shuts the socket down to end a request in flight, but on Windows that does not wake a read waiting in WSAPoll (yhirose#1396). The request then waited until the server answered, the read timeout passed, or the OS dropped the connection. stop() now also sets a flag, and send_() gives the request's stream an interrupt check that reads it, as SSEClient::stop() does since 801bf2e. The wait for data then ends within 100 ms on every platform. A request started after stop() clears the flag. A redirect or a digest retry sent inside the request keeps it. select_read_interruptible() now rounds its last wait up to a whole millisecond. poll() takes milliseconds, and the wait ended up to 1 ms before the deadline, which MaxTimeoutTest caught once every client request waited through this function.
…irose#2654) * keep a relative Location on its host in resolve_relative_location * Shorten the comment on on_this_host in resolve_relative_location --------- Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
* Cap a send() or wolfSSL_read() length at INT_MAX On 64-bit Windows SocketStream::write() handed size to send(), whose length is an int, so a single write of 4 GiB or more went out as its low 32 bits: a 4 GiB + 1 byte write sent one byte, and write_data() then retried the remaining 4 GiB as zero-byte sends forever. The INT_MAX cap that read() has was limited to 32-bit Windows in df20c27 (yhirose#1294), when the body was still written with one unchecked write(); write_data() now sends the rest, so the cap applies to every Windows build again. The wolfSSL tls::read() had the same cast, so a read into a buffer over INT_MAX failed. Cap it as the OpenSSL backend has done since f80864c. * Drop the 4 GiB write test and shorten comments The Windows-only test allocated 4 GiB and took about 5 s, which is too heavy for CI. --------- Co-authored-by: yhirose <yuji.hirose.bug@gmail.com>
RFC 9110 14.2 says an origin server MUST ignore a Range header field with a range unit it does not understand. parse_range_header() accepts only bytes=, so a request such as "Range: items=1-2" got a 416 instead of the full response. The unit is now checked first, and only a bytes range is parsed and answered with a 416 when it is invalid.
A request sent with send(Request) and params got those params appended to the Location of a redirect that had no query, on the same origin or another one. The Location alone names the next target, as it already did for Get/Post(path, params).
) A client using set_address_family(AF_UNIX) took the host of an absolute Location as a socket path, so a redirect to http://localhost/next failed and one to http://other.sock/next went to ./other.sock. Every hop now stays on the client's socket, as with curl's --unix-socket. The same host is still decided from the Location, as curl does: only localhost (the Host this client sends) counts as this client, so a redirect to any other host goes through the cross-host client and drops Authorization and Cookie. That client reaches the original socket through addr_map_, and a host the caller mapped with set_direct_hostname_addr_map keeps its mapping. An https Location runs TLS over the socket with the Location's host. The mapped-target lookup shared with make_connect_request_target is factored into detail::addr_map_target.
ThreadPool::enqueue constructed the dynamic thread without handling std::system_error. When thread creation failed (e.g. EAGAIN under a pids limit), the exception escaped the accept loop and destroying the pool's joinable threads called std::terminate. Treat the failure as "no spare capacity" and leave the job for an existing worker.
Co-authored-by: astunm <astunm@users.noreply.github.com>
Move the line splitting and event assembly out of SSEClient::run_event_loop() into a parser that is fed chunks and reports events, ids and retry values through handlers. The event loop is left with I/O and reconnection, and the parser can be driven without a server, which the following test and fuzzer changes rely on. No behavior change.
Move the SSE parsing tests from a fixture that started a server and an SSEClient for every case to unit tests of detail::SSEParser, and turn NonDigitRetryFieldIgnored, which slept for 500ms, into one of them. Add tests that feed each stream split at every position and byte by byte, and that cover the max event length and an event left incomplete at the end of the stream.
Add websocket_fuzzer covering frame reading, UTF-8 and close payload validation, and the upgrade response parser. Add websocket_fuzzer.options to raise max_len above the 64 KiB frame length boundary.
Add cases for relative location resolution, dot segment removal, URI encoding and decoding, and filename sanitization.
Feed the event stream to the parser in chunks and abort if the reported events, ids and retries differ from parsing it in one piece. Also run with a length limit to cover the early-stop path.
Fuzz Server::handle_file_request through a mount point backed by a temporary directory that holds symlinks escaping the mount root, one of them into a sibling directory sharing its name prefix. Abort if a token stored outside the root appears in a response. The seeds are based on those of yhirose#2661. Move FuzzedStream into fuzzed_stream.h so that both server fuzzers share it, and add FuzzableServer::MarkAsRunning(), without which a server that is not listening writes no file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.