README (41944B)
1 GNU libmicrohttpd -- in-process fuzzing harnesses 2 ================================================= 3 4 This directory contains four in-process fuzzing harnesses for MHD. All 5 of them are *dual mode*: 6 7 * they export the libFuzzer entry point 8 9 int LLVMFuzzerTestOneInput (const uint8_t *data, size_t size); 10 11 so the very same source can be linked with clang/libFuzzer, AFL++ or 12 OSS-Fuzz, and 13 14 * they ship a **built-in standalone driver** (`fuzz_common.h`) with a 15 deterministic, seeded generator + mutator loop, so they are useful 16 with nothing but gcc and `-fsanitize=address,undefined`. 17 18 The standalone driver is compiled unless `FUZZ_NO_MAIN` is defined. 19 20 21 ------------------------------------------------------------------- 22 1. The harnesses 23 ------------------------------------------------------------------- 24 25 fuzz_request.c the flagship. Feeds arbitrary bytes into a real 26 `struct MHD_Daemon` through a `socketpair()`, 27 using MHD_USE_NO_LISTEN_SOCKET + 28 MHD_add_connection() and external polling 29 (MHD_run()). Everything runs in one thread, so 30 the harness is deterministic and fast (~20k 31 requests/s under ASAN+UBSAN). 32 33 fuzz_str.c direct fuzzing of the string primitives in 34 src/microhttpd/mhd_str.c. Every output buffer is 35 malloc()ed at *exactly* the documented size so 36 that ASAN's redzone catches a one-byte overrun. 37 38 fuzz_auth_header.c direct fuzzing of the "Authorization:" header 39 parsers, MHD_get_rq_dauth_params_() and 40 MHD_get_rq_bauth_params_() (gen_auth.c), through 41 a minimal fabricated `struct MHD_Connection`. 42 ~250k execs/s. 43 44 fuzz_postprocessor.c fuzzing of MHD_post_process() with random 45 Content-Type (urlencoded / multipart with random 46 boundaries), random post-processor buffer sizes 47 and random chunking of the POST data. 48 49 Shared code lives in `fuzz_common.h` (header-only, so every harness 50 stays a single translation unit). 51 52 53 ------------------------------------------------------------------- 54 2. Design of fuzz_request 55 ------------------------------------------------------------------- 56 57 2.1 Why a socketpair 58 -------------------- 59 60 MHD_add_connection() accepts any already-connected socket, so a 61 `socketpair(AF_UNIX, SOCK_STREAM)` is enough: no listen socket, no port, 62 no TCP stack, no second thread. The daemon is started with 63 MHD_USE_NO_LISTEN_SOCKET and *without* MHD_USE_INTERNAL_POLLING_THREAD, 64 and the harness pumps it with MHD_run() between sends. A fake 65 127.0.0.1 `struct sockaddr_in` is passed so that per-IP accounting and 66 MHD_get_connection_info() see something sane. 67 68 2.2 Input format 69 ---------------- 70 71 byte 0 connection memory limit selector 72 (index into {default,128,192,256,320,384,512,768,1024, 73 1400,1500,2048,4096,32768}) 74 byte 1 handler behaviour bitmask 75 0x01 call MHD_digest_auth_check3() / 76 MHD_queue_auth_required_response3() 77 0x02 call MHD_basic_auth_get_username_password3() 78 0x04 run the request body through MHD_post_process() 79 0x08 iterate MHD_get_connection_values() over headers, 80 GET arguments, cookies and footers 81 0x10 unused (see 2.2.1) 82 0x20 reply with a larger, copied response body 83 (only meaningful for response kind 0) 84 0x40 reply 403 instead of 200 85 0x80 unused 86 byte 2 low nibble: MHD_OPTION_CLIENT_DISCIPLINE_LVL selector 87 (index into {-3,-2,-1,0,1,2}); 88 high nibble: reserved for MHD_OPTION_SERVER_INSANITY 89 (MHD 1.0.7 only defines MHD_DSC_SANE, so the value is 0) 90 byte 3 digest configuration: bits 0-1 select the algorithm of the 91 401 challenge {SHA-256, MD5, SHA-512-256, SHA-256}, 92 bit 2 selects the QOP, bits 4-5 select MHD_OPTION_NONCE_NC_SIZE 93 byte 4-9 the API-selection block, see 2.2.1 94 byte 10. a sequence of *send segments*. Each segment starts with a 95 little-endian 16 bit header: 96 97 (op << 14) | length length <= 0x3FFF 98 99 op 0 send `length` bytes on the current connection 100 op 1 the payload is the *expected decoded request body* 101 (ground truth for the body oracle, see 2.4); it is 102 not sent 103 op 2 send, then pump the daemon for extra rounds 104 op 3 close the current connection, open a fresh one on 105 the same daemon, then send 106 107 An explicit length encoding (rather than a magic delimiter) is used so 108 that the fuzzer can move a split point without having to invent an 109 escaping scheme. Splitting matters: MHD's parser is incremental and 110 several bugs only appear for particular split points. 111 112 An input shorter than ten bytes is rejected: the configuration block is 113 mandatory, and an input that short has no segment stream either. 114 115 2.2.1 The API-selection block 116 ----------------------------- 117 118 Bytes 4-9 pick which parts of the public API the iteration touches. 119 They are always present. Bytes 0-3 alone decide how MHD *parses* the 120 request; these six decide which of the response constructors, which 121 authentication entry point, which event loop, and which introspection 122 calls run against the parsed result. 123 124 The all-zero setting is the plainest one -- a static two byte buffer 125 response, MHD_run() as the event loop, no suspend, no upgrade, no extra 126 introspection -- so zeroing bytes 4-9 of any input reduces it to the 127 request-parsing-only behaviour that the harness had before these bytes 128 existed. 129 130 These bytes were gated behind bit 0x10 of byte 1 while they were being 131 brought up, so that the corpus predating them kept its byte-exact 132 meaning. The gate is gone; bit 0x10 is left unused rather than 133 reassigned, so that a corpus file written while it was a gate cannot 134 silently change meaning. The reproducers in known-findings/ that predate 135 the change (K1-K6, seven files) were migrated by inserting six zero bytes 136 at offset 4 -- the identity transformation, since the all-zero block is 137 the old behaviour -- and all seven still drive MHD along their recorded 138 paths, verified by comparing the --verbose daemon, handler, body and 139 challenge counts before and after. 140 141 byte 4 response construction 142 bits 0-3 which constructor to use: 143 0 buffer_static 1 buffer_copy 2 empty 144 3 buffer/PERSISTENT 4 buffer/MUST_FREE 5 buffer/MUST_COPY 145 6 buffer_with_free_callback 7 data (copy) 146 8 data (free) 9 callback, known length 147 10 callback, MHD_SIZE_UNKNOWN (chunked reply) 148 11 fd 12 fd_at_offset 13 fd64 149 14 pipe 15 iovec 150 bits 4-5 number of response headers to add (0-3) 151 bit 6 also add a response footer (a chunked trailer) 152 bit 7 exercise MHD_get_response_header(), 153 MHD_get_response_headers(), 154 MHD_del_response_header() and 155 MHD_set_response_options() 156 byte 5 authentication entry point 157 bits 0-3 0 check3 (as before) 1 check 158 2 check2 3 check_digest 159 4 check_digest2 5 check_digest3 160 6 get_username 7 get_username3 161 8 get_request_info3 9 as 0 162 bits 4-5 challenge variant: 0 queue_auth_required_response3, 163 1 queue_auth_fail_response, 164 2 queue_auth_fail_response2, 165 3 the basic-auth pair 166 bit 6 use the v1 MHD_basic_auth_get_username_password() 167 bit 7 also call the connection-less digest helpers 168 byte 6 event loop and introspection 169 bits 0-1 0 MHD_run(), 1 MHD_get_fdset()+run_from_select(), 170 2 and 3 the *2 variants 171 bit 2 query all four MHD_get_timeout*() forms 172 bit 3 MHD_lookup_connection_value(), 173 MHD_lookup_connection_value_n(), 174 MHD_get_connection_URI_path_n() 175 bit 4 MHD_set_connection_value() 176 bit 5 MHD_get_connection_info() / MHD_get_daemon_info() 177 bit 6 MHD_set_connection_option() 178 bit 7 MHD_quiesce_daemon() before stopping 179 byte 7 suspend and upgrade 180 bits 0-1 0 none, 1 suspend+resume in the handler, 181 2 suspend and resume from the pump loop, 182 3 as 1 183 bit 2 allow HTTP "Upgrade" (only acted on when the 184 request really asks for one, see below) 185 bits 3-4 which MHD_upgrade_action() to issue first 186 byte 8 seed picking the response header names and values, and the 187 MHD_RF_* flags for MHD_set_response_options() 188 byte 9 seed for the content-reader callback: body length, block 189 size, and whether it fails part way through 190 191 Two constraints on the harness are worth stating, because both are 192 application-contract requirements rather than things worth fuzzing, and 193 violating either makes MHD abort on input that is perfectly legal: 194 195 * the connection is only suspended when the handler is about to return 196 MHD_YES. Returning MHD_NO asks MHD to terminate the connection, and 197 terminating one that the same callback just suspended trips 198 mhd_assert (! connection->suspended) in MHD_connection_close_(); 199 200 * a 101 response is only queued for a request that is actually an 201 upgrade request (HTTP/1.1, an "Upgrade" header, and a "Connection" 202 header naming the upgrade token and not "close"). All three come 203 off the wire, so the path stays attacker-driven. 204 205 This is exactly the check a real application can perform, and no 206 more. It deliberately does not try to predict whether MHD will 207 accept the upgrade: a request that passes it used to abort MHD, which 208 is finding K7 in section 6, and MHD_queue_response() now answers 209 MHD_NO for that case instead. Do not tighten the condition to keep 210 the suite green -- an application cannot do better than this, so 211 neither should the harness. 212 213 The message-framing headers (Content-Length, Transfer-Encoding) are 214 deliberately absent from the response-header table: MHD generates them 215 itself from the response object, so an application that also sets them 216 by hand is lying to the library about its own body. 217 MHD_RF_INSANITY_HEADER_CONTENT_LENGTH is the sanctioned way to explore 218 that corner and is reachable through bit 7 of byte 4. 219 220 2.3 The %%NONCE%% placeholder 221 ----------------------------- 222 223 Interesting parts of digestauth.c are only reached *after* the client 224 presents a nonce that MHD itself generated. A stateless fuzzer can 225 never guess one. Therefore the harness rewrites the literal ASCII token 226 227 %%NONCE%% 228 229 inside a segment, at send time, into the most recent `nonce="..."` value 230 seen in a response from the daemon. A generated (or hand-written) input 231 can thus be: 232 233 request 1: GET /a -> handler calls MHD_digest_auth_check3(), 234 gets MHD_DAUTH_WRONG_HEADER and replies 235 401 + WWW-Authenticate: Digest ... nonce="X" 236 op 3: new connection 237 request 2: GET /a with Authorization: Digest ... nonce="%%NONCE%%" 238 239 which walks all the way into the 'response' comparison. Without this 240 the over-long `response=` stack overflow (see 5.4) is unreachable. 241 242 2.4 Oracles 243 ----------- 244 245 Memory errors are caught by ASAN/UBSAN and aborts by the signal 246 handlers. In addition fuzz_request installs two behavioural oracles: 247 248 a) MHD_set_panic_func() -- any MHD_PANIC() reached from network input 249 is a finding (a remote abort), not a legitimate "API violation". 250 251 b) A request-body oracle. Framing bugs (chunked transfer coding, 252 Content-Length) do not corrupt memory, they corrupt *data*, which 253 is exactly what HTTP request smuggling exploits. The input can 254 therefore declare the expected decoded body in an `op 1` segment. 255 Every byte that MHD hands to the application must be the next 256 expected byte, and when MHD completes the request the delivered 257 body must be complete. MHD is free to reject the request at any 258 point -- only what it *does* deliver is checked. 259 260 The declaration is honoured only for un-mutated inputs (the driver 261 exposes this as `fuzz_pristine`), because a random mutation would 262 of course invalidate the ground truth. 263 264 2.5 The generator 265 ----------------- 266 267 Purely random bytes essentially never form a valid HTTP request, so the 268 standalone driver uses a small HTTP grammar (`fuzz_generate()`), and 269 then optionally applies byte-level mutations on top. Shapes: 270 271 0 SHAPE_PLAIN random method/target/version + headers 272 1 SHAPE_NOHDR_QARG *no header lines at all* plus a trailing 273 query argument without '=' (this is the 274 exact shape needed for the read-buffer 275 shift-back bug; the generator also forces a 276 small connection memory pool for it) 277 2 SHAPE_CL_BODY Content-Length body + body oracle 278 3 SHAPE_CHUNKED chunked body with chunk extensions 279 (";ext", ";ext=val", ";ext=\"quoted\"", 280 ";a=1;b=2;c") + trailers + body oracle 281 4 SHAPE_DIGEST_SIMPLE Authorization: Digest with a randomised 282 parameter set, including unknown 283 `algorithm=` tokens and `response=` values 284 of every length up to 128 hex digits 285 5 SHAPE_DIGEST_REPLAY the two-request nonce handshake of 2.3 286 6 SHAPE_BASIC Authorization: Basic with random base64 287 7 SHAPE_POST_FORM urlencoded / multipart POST bodies 288 8 SHAPE_WEIRD folded headers, bare CR, bare LF, 289 whitespace before the colon, percent 290 encoding, absolute-form targets, ... 291 292 `MHD_FUZZ_SHAPE=<n>` restricts the generator to a single shape, which is 293 very handy for triage and for regression-testing a specific past bug. 294 295 296 ------------------------------------------------------------------- 297 3. Running the harnesses 298 ------------------------------------------------------------------- 299 300 Build (gcc only, no clang required): 301 302 SRC=/path/to/libmicrohttpd # configured build tree 303 gcc -g -O1 -Wall -Wextra \ 304 -fsanitize=address,undefined -fno-sanitize-recover=all \ 305 -I$SRC -I$SRC/src/include -I$SRC/src/microhttpd -I$SRC/src/fuzz \ 306 -o fuzz_request $SRC/src/fuzz/fuzz_request.c \ 307 $SRC/src/microhttpd/.libs/libmicrohttpd.a -lpthread 308 309 (the same command line for fuzz_str, fuzz_auth_header and 310 fuzz_postprocessor; the static archive is required because fuzz_str and 311 fuzz_auth_header use functions that are hidden in the shared object). 312 313 Options of the built-in driver (identical for all harnesses): 314 315 --iterations=N number of generate/mutate iterations [3000] 316 --seed=N PRNG seed; (harness, seed) fully determines a run 317 --corpus-dir=DIR replay every regular file in DIR and exit 318 --file=PATH replay a single input and exit (crash repro) 319 --crash-dir=DIR where reproducers are written [crashes] 320 --timeout=SEC per-iteration watchdog, 0 disables [20] 321 --write-corpus=DIR dump the built-in seed corpus to DIR 322 --skip-seeds do not replay the built-in corpus first 323 --verbose enable MHD's error log + print statistics 324 --help 325 326 Environment variables (all optional): 327 328 MHD_FUZZ_ITERATIONS, MHD_FUZZ_SEED, MHD_FUZZ_TIMEOUT, 329 MHD_FUZZ_CRASH_DIR, MHD_FUZZ_VERBOSE, MHD_FUZZ_SKIP_SEEDS 330 331 MHD_FUZZ_SHAPE=<n> (fuzz_request) restrict the generator 332 to one grammar shape 333 MHD_FUZZ_MIN_DISCIPLINE=<n> (fuzz_request) lower bound for 334 MHD_OPTION_CLIENT_DISCIPLINE_LVL, 335 default -3 (the full range) 336 MHD_FUZZ_MIN_MEM_LIMIT=<n> (fuzz_request) lower bound for 337 MHD_OPTION_CONNECTION_MEMORY_LIMIT, 338 default 0 (the full range) 339 MHD_FUZZ_MODEL_DIGEST_SINK=1 (fuzz_str) enable the modelled 340 digest 'response' call site, see 5.4 341 342 Typical use: 343 344 # quick smoke test (a couple of seconds) 345 ./fuzz_request 346 347 # a real session 348 ./fuzz_request --iterations=5000000 --seed=$RANDOM 349 350 # regression: replay the whole checked-in corpus 351 ./fuzz_request --corpus-dir=corpus 352 ./fuzz_str --corpus-dir=corpus # ignores foreign files gracefully 353 354 # reproduce a crash 355 ./fuzz_request --file=crashes/crash-fuzz_request-seed3-iter55.bin 356 357 358 ------------------------------------------------------------------- 359 4. Reproducing a failure 360 ------------------------------------------------------------------- 361 362 Whenever the process dies -- ASAN error, UBSAN error, `mhd_assert()`, 363 MHD_PANIC(), a body-oracle finding, or the watchdog -- the input of the 364 running iteration is written to 365 366 $MHD_FUZZ_CRASH_DIR/crash-<harness>-seed<S>-iter<N>.bin 367 368 and a line is printed telling you the harness, the seed and the 369 iteration. The dump is produced from 370 371 * `__sanitizer_set_death_callback()` (weakly linked; present whenever 372 the binary is built with ASAN), and 373 * SIGABRT/SIGSEGV/SIGBUS/SIGILL/SIGFPE/SIGALRM handlers, 374 375 using only async-signal-safe calls. Replay with `--file=...`; the run 376 is fully deterministic, so `--seed=S --iterations=N+1` reproduces the 377 whole session as well. 378 379 380 ------------------------------------------------------------------- 381 5. What these harnesses find (regression coverage) 382 ------------------------------------------------------------------- 383 384 The four vulnerabilities fixed in MHD 1.0.7+1 are all rediscovered from 385 scratch. Each has a dedicated seed in `corpus/`, and the generator 386 finds each of them on its own within a few thousand iterations. 387 388 5.1 digestauth.c: unknown `algorithm=` token -> MHD_PANIC() 389 An `algorithm=` token MHD does not know parses to 390 MHD_DIGEST_AUTH_ALGO3_INVALID, which is 0, so the allow-mask test 391 `c_algo == (c_algo & malgo3)` passes for *any* mask; the code then 392 calls digest_init_one_time() with an invalid algorithm and panics. 393 Found by: SHAPE_DIGEST_SIMPLE / SHAPE_DIGEST_REPLAY, the panic hook, 394 corpus seed `digest-unknown-algorithm`. 395 396 5.2 connection.c get_req_headers(): read-buffer shift-back underflow 397 Needs, all at once: a small MHD_OPTION_CONNECTION_MEMORY_LIMIT 398 (MHD_BUF_INC_SIZE (1500) > read_buffer_size), *no header lines*, and 399 a trailing query argument without '=' (whose `value` is NULL). 400 Found by: SHAPE_NOHDR_QARG, corpus seeds 401 `small-pool-trailing-query-arg[-2]`. 402 403 5.3 connection.c process_request_body(): chunk-extension CRLF 404 `chunk_size_line_len = i` instead of `i + 2` leaves the CRLF of the 405 chunk-size line in the stream, so the following chunk data is 406 shifted -- a body desync, i.e. a request-smuggling primitive. This 407 corrupts no memory, so it is caught by the body oracle (2.4). 408 Found by: SHAPE_CHUNKED, corpus seeds `chunked-with-extensions`, 409 `chunked-split`. 410 411 5.4 digestauth.c: over-long `response=` -> stack buffer overflow 412 `response` was accepted up to `digest_size * 4` characters (128 for 413 SHA-256) and then decoded with MHD_hex_to_bin() into 414 `uint8_t hash1_bin[MAX_DIGEST]` (32 bytes) -- up to 64 bytes 415 written, 32 bytes of stack smashed. Reaching it requires a *valid* 416 nonce, hence the %%NONCE%% mechanism of 2.3. 417 Found by: SHAPE_DIGEST_REPLAY, corpus seed 418 `digest-overlong-response`. 419 420 fuzz_str additionally reproduces the underlying primitive: 421 MHD_hex_to_bin() has no output-size parameter and writes len/2 422 bytes, so any caller with a fixed-size buffer must bound the input 423 length itself. `MHD_FUZZ_MODEL_DIGEST_SINK=1` enables a target that 424 replays exactly the pre-fix call site (32 byte heap buffer, input 425 length bounded only by 4 * 32) and ASAN reports the overflow 426 immediately. The target models a *caller*, not the library, so it 427 is off by default. 428 429 430 ------------------------------------------------------------------- 431 6. Findings against MHD 1.0.7 - all fixed, kept as regressions 432 ------------------------------------------------------------------- 433 434 Running these harnesses against v1.0.7 built with `--enable-asserts` 435 reported the following *additional* issues on top of the four 436 vulnerabilities of section 5. All of them were `mhd_assert()`s reachable 437 from network input, i.e. a remote abort in builds that keep assertions 438 enabled, and all of them are fixed on master. They are documented here 439 because the reproducers are kept as a regression corpus: a failure of one 440 of them means the corresponding fix has been undone. Byte-exact reproducers are in `corpus/known-findings/`; replay 441 one with 442 443 ./fuzz_request --file=corpus/known-findings/K1-digest-empty-realm.bin 444 445 K1 digestauth.c:2467 is_param_equal(): 446 mhd_assert (0 != param->value.len) -> fixed in 300a2ab0 447 Trigger (default daemon configuration!), one request: 448 GET /a HTTP/1.1 449 Host: x 450 Authorization: Digest username="user", realm="", nonce="0000", 451 uri="/a", response="00" 452 digest_auth_check_all_inner() rejects a *missing* realm/username but 453 not an *empty* one, so a zero-length parameter reaches 454 is_param_equal(), whose documented precondition is a non-empty 455 value. Requires only that the application calls 456 MHD_digest_auth_check3(). Real defect (missing validation). 457 Repro: corpus/known-findings/K1-digest-empty-realm.bin 458 459 K2 connection.c:3582 handle_recv_no_space(): 460 mhd_assert ((MHD_PROC_RECV_BODY_CHUNKED != stage) || 461 ! c->rq.some_payload_processed) -> fixed in 68c83f22 462 Trigger: small MHD_OPTION_CONNECTION_MEMORY_LIMIT (<= ~400 bytes) 463 plus a chunked body whose *second* chunk-size line carries a chunk 464 extension that does not fit into the remaining read buffer. The 465 flag reflects the last application callback only and survives later 466 reads, so the assertion is over-strong; the code below it already 467 handles the situation. Stale assertion. 468 Repro: corpus/known-findings/K2-chunkext-no-space.bin 469 470 K3 connection.c:2881 transmit_error_response_len(): 471 mhd_assert (! connection->stop_with_error) -> fixed in e04eb218 472 Trigger: small connection memory pool plus an over-long, unterminated 473 chunk extension: 474 POST /a HTTP/1.1 / Transfer-Encoding: chunked 475 d;ext="qqqqqqqq... (longer than the read buffer) 476 handle_req_chunk_size_line_no_space() is missing a `return` after it 477 has already queued the "chunk extension too big" response. Real 478 defect: in a release build the second call forces the connection to 479 MHD_CONNECTION_CLOSED and the 413 response is never sent. 480 Repro: corpus/known-findings/K3-chunkext-stop-with-error.bin 481 482 K4 connection.c:6099 get_req_header(): 483 mhd_assert ((0 == c->rq.hdrs.hdr.value_start) || 484 (0 != c->rq.hdrs.hdr.name_len)) -> fixed in 0b750975 485 Trigger a) MHD_OPTION_CLIENT_DISCIPLINE_LVL <= -1, first header line 486 starting with whitespace: 487 GET /a HTTP/1.1\r\n Host: x\r\n\r\n 488 Trigger b) MHD_OPTION_CLIENT_DISCIPLINE_LVL <= -2, empty header 489 name: 490 GET /a HTTP/1.1\r\n: value\r\nHost: x\r\n\r\n 491 Both shapes are explicitly allowed by those discipline levels. 492 Stale assertion. 493 Repro: corpus/known-findings/K4a-wsp-first-header.bin, 494 corpus/known-findings/K4b-empty-header-name.bin 495 496 K5 connection.c:6394 get_req_header(): 497 mhd_assert ('\r' != chr) -> fixed in 6fcdfd43 498 Trigger: MHD_OPTION_CLIENT_DISCIPLINE_LVL = -3, which sets 499 `bare_cr_keep = true`; the branch that keeps a bare CR falls through 500 into the "not a whitespace, not the end of the line" arm whose 501 assertion predates that mode. 502 GET /a HTTP/1.1\r\nHost: x\r\nX: y\r\r\n\r\n 503 Stale assertion. 504 Repro: corpus/known-findings/K5-bare-cr-keep.bin 505 506 K6 digestauth.c:860 check_nonce_nc(): 507 mhd_assert (0 == nn->nonce[noncelen]) 508 The nonce-nc slot array is indexed by a hash of the nonce, but the 509 slot content is compared assuming the *stored* nonce has the same 510 length as the presented one. A client can therefore make MHD read 511 the terminator of a nonce at the wrong offset by presenting a nonce 512 whose length belongs to a different digest algorithm. 513 Note that this one is *timing dependent*: the nonce carries a 514 millisecond timestamp and whether it counts as stale depends on the 515 wall clock, so the same input reproduces only in a fraction of the 516 replays (about 1 in 30 for the corpus file below). Replay it in a 517 loop. 518 Trigger (MHD_OPTION_NONCE_NC_SIZE = 1 makes every nonce land in slot 519 0, which turns the collision into a certainty; larger arrays only 520 need more attempts): 521 request 1: GET /a -> 401 with a SHA-256 nonce 522 (76 chars) stored in the slot 523 request 2: Authorization: Digest username="user", 524 realm="TestRealm", nonce="<44 zeros>", uri="/a", 525 response="e" 526 (no algorithm parameter -> MD5 -> nonce length 44, 527 all-zero timestamp -> not stale) 528 The generator needs ~150k iterations to hit it on its own. 529 Repro: corpus/known-findings/K6-nonce-length-collision.bin 530 531 K7 connection.c:2596 build_header_response(): 532 mhd_assert ((NULL == r->upgrade_handler) || 533 (MHD_CONN_MUST_UPGRADE == c->keepalive)) 534 Fixed by commit acef58a0. Was driven entirely from the wire, with 535 default daemon options. 536 537 While parsing the request headers, connection.c sets 538 c->keepalive = MHD_CONN_MUST_CLOSE 539 in three places for requests whose message framing it distrusts: 540 541 4988 two "Content-Length" headers with different values 542 (client discipline -3 only) 543 5012 "Transfer-Encoding" on an HTTP/1.0 request 544 (client discipline <= 0) 545 5051 "Content-Length" *and* "Transfer-Encoding: chunked" in the 546 same request -- no discipline gate at all, so this one 547 fires with the default configuration 548 549 In every case the request is then handed to the access handler as a 550 normal request. If the handler answers 101 with a response from 551 MHD_create_response_for_upgrade(), MHD_queue_response() accepts it: 552 it checks MHD_ALLOW_UPGRADE, the status code, the "Connection" 553 header and the HTTP version, but not whether the connection can 554 still be kept open. keepalive_possible() then returns 555 MHD_CONN_MUST_CLOSE from its very first test, which is placed 556 *before* the upgrade branch, and the assertion above fires while the 557 reply header is being built. 558 559 The application cannot defend itself. The harness checks the 560 request the way an application would -- HTTP/1.1, an "Upgrade" 561 header, and a "Connection" header naming the upgrade token and not 562 "close" -- and the reproducer satisfies all three. MHD's decision 563 is not exposed through any public accessor. 564 565 It needs --enable-asserts and an application that answers upgrade 566 requests, so it is a robustness defect rather than a memory-safety 567 one; but it is reachable from a single well-formed request against a 568 default-configured server, which is a good deal worse than the 569 "application misuse" it first looked like. 570 571 The fix rejects the response in MHD_queue_response() 572 (connection.c:8338) next to the other upgrade preconditions, so the 573 application gets MHD_NO -- which it already has to handle -- instead 574 of an abort. 575 Repro: corpus/known-findings/K7-upgrade-after-must-close.bin 576 GET / HTTP/1.1 577 Host: x 578 Connection: Upgrade 579 Upgrade: fuzz-protocol 580 Content-Length: 0 581 Transfer-Encoding: chunked 582 (then "0\r\n\r\n") 583 584 K8 digestauth.c MHD_digest_auth_check_digest2(): 585 MHD_PANIC ("Wrong 'malgo3' value, only one base hashing 586 algorithm ... must be specified, API violation") 587 Fixed by commit 07c051dd. An API-contract defect rather than a 588 parsing bug: unlike K7 the offending argument came from the 589 application, not from the network. 590 591 MHD_DIGEST_ALG_AUTO is a documented member of 592 enum MHD_DigestAuthAlgorithm, and the `algo` parameter of 593 MHD_digest_auth_check_digest2() is documented only as "digest 594 algorithms allowed for verification". But the function maps AUTO to 595 MHD_DIGEST_AUTH_MULT_ALGO3_ANY_NON_SESSION and forwards it to 596 MHD_digest_auth_check_digest3(), which requires *exactly one* base 597 hashing algorithm because the caller supplies a pre-computed digest 598 of one specific length -- and aborts through MHD_PANIC() when more 599 than one is named. 600 601 So MHD_digest_auth_check_digest2 (..., MHD_DIGEST_ALG_AUTO) 602 deterministically kills the process. The same is not true of 603 MHD_digest_auth_check2(), which takes a password rather than a 604 digest and whose AUTO handling is fine. 605 606 Unlike K7 this is entirely under the application's control -- no 607 network input is involved -- so it is a usability trap rather than a 608 robustness bug: AUTO simply cannot be used with the 609 pre-computed-digest entry points, and no mapping can rescue it, 610 because the digest length does not identify the algorithm (SHA-256 611 and SHA-512/256 digests are both 32 bytes). 612 613 The fix documents the restriction in microhttpd.h and returns MHD_NO 614 instead of aborting. AUTO is left unusable with the 615 pre-computed-digest entry points, deliberately: there is nothing to 616 map it to. 617 Repro: none committed. The argument is the application's, so the 618 harness never passes AUTO there (see run_digest_check() in 619 fuzz_request.c); to see it, change the MHD_DIGEST_ALG_* argument of 620 the variant-4 call to MHD_DIGEST_ALG_AUTO and replay 621 corpus/fuzz_request-37.bin. 622 623 K9 postprocessor.c:1119 post_process_multipart(): 624 LeakSanitizer: direct leak, strdup() from MHD_post_process() 625 Found by fuzz_postprocessor, not fuzz_request -- the first finding 626 that did not come from the daemon harness. Remotely triggerable, 627 default configuration, no assertions needed. 628 629 On entering PP_PerformCheckMultipart the code did 630 631 pp->nested_boundary = strstr (pp->content_type, "boundary="); 632 ... 633 pp->nested_boundary = strdup (&pp->nested_boundary[9]); 634 635 The first assignment stores an *interior pointer into 636 pp->content_type* over whatever pp->nested_boundary held. If the 637 post processor already owned a boundary -- which it does for every 638 nested "multipart/mixed" part after the first, unless the state 639 machine happened to pass through PP_PerformCleanup in between -- that 640 allocation is lost. MHD_destroy_post_processor() frees only the last 641 one. 642 643 So a body with N nested multipart/mixed parts, each carrying its own 644 "boundary=", leaks N-1 blocks, and the client picks how long each one 645 is. That makes it a memory-exhaustion vector against any application 646 that calls MHD_post_process() on multipart input, not the one byte 647 the reproducer happens to leak (its boundary is the empty string). 648 649 Fixed by copying into a local first and releasing any previously 650 owned boundary before taking ownership of the new one; the error 651 path no longer overwrites the old pointer either. 652 Repro: corpus/known-findings/K9-fuzz_postprocessor-nested-boundary-leak.bin 653 654 K10 memorypool.c MHD_pool_deallocate(): end block of an exactly-full 655 pool was never returned; the front/end dispatch tested pool->pos, 656 where the two ranges meet. Fixed. 657 658 K11 memorypool.c MHD_pool_reallocate(): returned non-NULL for a 659 wrapping @a new_size, handing the caller a block it believed was 660 nearly SIZE_MAX bytes long. Fixed. 661 662 K12 daemon.c MHD_start_daemon_va(): leaked the GnuTLS DH parameters 663 built from MHD_OPTION_HTTPS_MEM_DHPARAMS on every startup-failure 664 exit. The application cannot free them. Fixed. 665 666 K13 daemon.c MHD_epoll() / MHD_quiesce_daemon(): both threads remove 667 the listening socket from the epoll set, and the loser aborts on 668 ENOENT. Open; two alternative fixes are proposed as 669 ../../patches/K13.diff (tolerate ENOENT everywhere) and 670 ../../patches/K13b.diff (take a lock, keep the strict check). 671 672 K14 daemon.c:1358 call_handlers(): 673 mhd_assert (! force_close || MHD_CONNECTION_CLOSED == con->state) 674 Open, fix proposed in ../../patches/K14.diff. 675 676 MHD_connection_handle_read() closes the connection when its 677 @a socket_error argument is set -- but only once it gets far enough 678 to try. It returns early, leaving the state untouched, when the 679 connection is suspended, when a TLS handshake is in progress, and 680 when the read buffer has no free space at all. The caller asserts 681 the post-condition unconditionally. 682 683 Instrumenting the assertion site with the reproducer gives 684 685 state=12 (HEADERS_PROCESSED) suspended=0 686 rb_size=2021 rb_off=2021 evinfo=READ 687 688 i.e. the read buffer is exactly full. A client that fills the 689 connection memory pool without the application consuming the body 690 gets there; no unusual API use is involved. Observed through 691 MHD_run_from_select2(), i.e. from the external event loop. 692 693 The reproducer does NOT fire on a plain in-tree gcc build: ASan's 694 pool redzones change the buffer arithmetic enough to matter, so 695 confirm it against the OSS-Fuzz build configuration. 696 Repro: corpus/known-findings/K14-fuzz_eventloop-force-close-not-closed.bin 697 698 Status 699 ------ 700 701 K1-K12 are fixed on master: 702 703 K1 300a2ab0 K4a 0b750975 K7 acef58a0 K10 (memorypool) 704 K2 68c83f22 K4b 0b750975 K8 07c051dd K11 (memorypool) 705 K3 e04eb218 K5 6fcdfd43 K9 (postproc) K12 (daemon/TLS) 706 K6 f438804c 707 708 K13 and K14 are open, with proposed fixes in ../../patches/. Until K14 709 lands, `make check-corpus` fails on an `--enable-asserts` build, because 710 its reproducer is committed and still reproduces -- which is the 711 documented behaviour for an open finding, but worth knowing before a CI 712 run. 713 714 The reproducers in `corpus/known-findings/` otherwise all replay clean 715 on a build configured with `--enable-asserts`, and 716 `make check-corpus` asserts exactly that -- it replays that directory 717 along with the generated corpus. K8 has no reproducer, because its 718 trigger is an argument the application chooses rather than anything that 719 comes off the wire. 720 721 A reproducer belongs to the harness that found it, and says so in its 722 name: `K<n>-<harness>-<what>.bin`. The ones without a harness in the 723 name predate the convention and are all fuzz_request inputs. 724 `contrib/oss-fuzz/make_seed_corpus.sh` routes each one into the right 725 target's seed corpus on that basis -- a fuzz_postprocessor reproducer in 726 fuzz_request's corpus would just be an uninteresting input. 727 728 When the next finding is opened, its reproducer goes into 729 `corpus/known-findings/` and will make `make check-corpus` fail until the 730 fix lands. That is the intended behaviour: it is the same arrangement as 731 the `XFAIL_TESTS` entries in `src/microhttpd/Makefile.am`, except that 732 here the expectation is not encoded, so the commit that adds a live 733 reproducer should say so. `contrib/oss-fuzz/make_seed_corpus.sh` has to 734 skip such a reproducer as well, or every ClusterFuzz run starts by 735 rediscovering it; see the comment there. 736 737 The `make check` defaults in Makefile.am 738 739 MHD_FUZZ_ITERATIONS = 50000 740 MHD_FUZZ_MIN_DISCIPLINE = -3 (full range) 741 MHD_FUZZ_MIN_MEM_LIMIT = 0 (full range) 742 743 exercise the whole matrix and take about three seconds in total under 744 ASAN+UBSAN. A real session: 745 746 MHD_FUZZ_MIN_DISCIPLINE=-3 MHD_FUZZ_MIN_MEM_LIMIT=0 \ 747 ./fuzz_request --iterations=1000000 --seed=1 748 749 ------------------------------------------------------------------- 750 7. Building with clang/libFuzzer or AFL++ 751 ------------------------------------------------------------------- 752 753 Both need `-DFUZZ_NO_MAIN` so that the driver's `main()` is left out. 754 755 7.1 libFuzzer 756 ------------- 757 758 # build the library itself with the same instrumentation 759 ./configure --enable-static --disable-shared --enable-asserts \ 760 CC=clang \ 761 CFLAGS="-g -O1 -fsanitize=fuzzer-no-link,address,undefined \ 762 -fno-sanitize-recover=all -fprofile-instr-generate \ 763 -fcoverage-mapping" 764 make -C src/microhttpd 765 766 clang -g -O1 -DFUZZ_NO_MAIN \ 767 -fsanitize=fuzzer,address,undefined -fno-sanitize-recover=all \ 768 -I. -Isrc/include -Isrc/microhttpd -Isrc/fuzz \ 769 -o fuzz_request src/fuzz/fuzz_request.c \ 770 src/microhttpd/.libs/libmicrohttpd.a -lpthread 771 772 mkdir -p CORPUS && ./fuzz_request --help >/dev/null 2>&1 || true 773 ./fuzz_request CORPUS src/fuzz/corpus \ 774 -max_len=8192 -rss_limit_mb=4096 -timeout=20 775 776 # minimise a crash found by libFuzzer 777 ./fuzz_request -minimize_crash=1 -runs=100000 crash-<hash> 778 779 7.2 AFL++ 780 --------- 781 782 export CC=afl-clang-lto AFL_USE_ASAN=1 AFL_USE_UBSAN=1 783 ./configure --enable-static --disable-shared --enable-asserts 784 make -C src/microhttpd 785 786 afl-clang-lto -g -O1 -DFUZZ_NO_MAIN \ 787 -I. -Isrc/include -Isrc/microhttpd -Isrc/fuzz \ 788 -o fuzz_request src/fuzz/fuzz_request.c \ 789 $(afl-config --libdir 2>/dev/null || echo /usr/local/lib/afl)/afl-compiler-rt.o \ 790 /usr/local/lib/afl/libAFLDriver.a \ 791 src/microhttpd/.libs/libmicrohttpd.a -lpthread 792 793 afl-fuzz -i src/fuzz/corpus -o findings -- ./fuzz_request @@ 794 795 (`libAFLDriver.a` provides a `main()` that reads the file named on the 796 command line and calls LLVMFuzzerTestOneInput(); that is why 797 `-DFUZZ_NO_MAIN` is required. With `AFL_LLVM_PERSISTENT` / 798 `__AFL_LOOP` the same binary can be used in persistent mode.) 799 800 7.3 OSS-Fuzz 801 ------------ 802 803 Ready to go: see `../../contrib/oss-fuzz/` and its README. That 804 directory holds the OSS-Fuzz `build.sh`, `project.yaml` and `Dockerfile`, 805 per-harness dictionaries (`dicts/fuzz_*.dict`), per-harness `.options` 806 files (`max_len`, `dict`) and `make_seed_corpus.sh`, which packages 807 `corpus/` — including the `corpus/known-findings/` reproducers of the 808 findings that are already *fixed*, so that they become permanent 809 regressions — into the `<fuzzer>_seed_corpus.zip` files OSS-Fuzz 810 expects. 811 812 `build.sh` configures out of tree with 813 814 --enable-static --disable-shared --with-pic --enable-fuzzing 815 --enable-asserts --disable-https --disable-curl --disable-doc 816 --disable-examples --disable-tools --enable-build-type=neutral 817 818 honouring OSS-Fuzz's `$CFLAGS` (no `--enable-sanitizers`: OSS-Fuzz 819 supplies the instrumentation), and then compiles each harness exactly as 820 in 7.1 but against `$LIB_FUZZING_ENGINE`. HTTPS is off on purpose — the 821 harnesses never speak TLS, and leaving GnuTLS out is what makes the 822 MemorySanitizer build possible. 823 824 Two things to know before reading a report from there: 825 826 * the request-body oracle of 2.4 is **inactive** under libFuzzer. It 827 is gated on `fuzz_pristine`, which nothing sets when `FUZZ_NO_MAIN` 828 is defined — correctly so, since libFuzzer's mutations invalidate the 829 declared ground truth. Pure framing/desync defects (5.3) therefore 830 remain the job of the built-in driver, i.e. of `make check`; 831 * `primary_contact` in `project.yaml` is a placeholder. OSS-Fuzz needs 832 an address the maintainer controls; it has to be filled in before the 833 project can be submitted. 834 835 The token list in `fuzz_common.h` (`fuzz_interesting_str`) is the 836 generator's equivalent of those dictionaries; the two are intentionally 837 similar but are not generated from each other. 838 839 OSS-Fuzz is deliberately not part of `contrib/ci/jobs/`; the bounded 840 in-tree equivalents for CI are `make -C src/fuzz check` and 841 `make -C src/fuzz check-corpus`. 842 843 844 ------------------------------------------------------------------- 845 8. Adding a harness 846 ------------------------------------------------------------------- 847 848 Create `fuzz_<name>.c` with 849 850 #define FUZZ_HARNESS_NAME "fuzz_<name>" 851 #include "fuzz_common.h" 852 853 and implement 854 855 int LLVMFuzzerTestOneInput (const uint8_t *, size_t); 856 static size_t fuzz_generate (struct fuzz_rng *, uint8_t *, size_t); 857 static size_t fuzz_seed_count (void); 858 static const uint8_t *fuzz_seed_get (size_t, size_t *); 859 860 then add it to `check_PROGRAMS` in Makefile.am. Report non-crashing 861 findings with `fuzz_report_finding("...")`, which dumps the reproducer 862 and aborts. 863 864 Two rules learnt the hard way: 865 866 * do not violate documented *preconditions* of the function under 867 test (e.g. MHD_str_remove_token_caseless_() asserts that the token 868 contains no space, tab, comma or NUL) -- otherwise the harness only 869 finds its own bugs; 870 871 * never evaluate a macro argument twice when it contains a PRNG call.