CAMPAIGN.md (26786B)
1 # Running a libextractor fuzzing campaign 2 3 This is the measured answer to two questions: **how should a fixed 4 number of core-hours be divided across the targets**, and **how long is 5 it still worth running**. Both answers come from a controlled 12-core 6 hour on the 2026-07-29 tree, not from intuition, and both are 7 uncomfortable: a flat allocation wastes most of its budget, and the 8 productive length of a campaign is far shorter than it looks. 9 10 `../../contrib/oss-fuzz/run_campaign.sh` implements the conclusion. 11 12 13 ## 1. The measurement 14 15 Every target got an **identical** budget, so the curves are comparable 16 and the allocation could be derived from the data instead of assumed: 17 18 36 jobs x 1200 s / 12 cores = 3 full waves = 3715 s wall, 97% busy 19 20 32 of the jobs were the targets; 4 were replicates of `fuzz_unzip`, 21 `fuzz_msoffice`, `fuzz_rtf` and `fuzz_datasource` with a different PRNG 22 seed and their own corpus, to find out how much of any difference is 23 just noise. 24 25 The tree carried the twelve fixes from `issues.txt`. Result: 26 27 | | | 28 |---|---| 29 | wall clock | 3715 s (62 min), 12 cores, 12.0 core-hours | 30 | crashes / leaks / OOM / timeouts | **0** — all 36 jobs reported `0/0/0` | 31 | artifacts | **0** | 32 | corpus | 1727 seed inputs grew to 6045 | 33 | replay of the whole result under ASan+UBSan | **0 sanitizer reports** | 34 35 The replay pass is not optional and not redundant. `-fork=1` reads only 36 the child's exit status, so a *recovering* UBSan report leaves no 37 artifact and nothing in the log — and seven of the twelve fixed defects 38 were exactly that. 39 40 41 ## 2. Feature counts are noise; edge coverage is not 42 43 The four replicate pairs differ only in PRNG seed: 44 45 | target | run 1 `ft` | run 2 `ft` | Δ | run 1 `cov` | run 2 `cov` | Δ | 46 |---|---|---|---|---|---|---| 47 | fuzz_rtf | 6608 | 7201 | **+9.0 %** | 1047 | 1072 | +2.4 % | 48 | fuzz_msoffice | 5332 | 5739 | **+7.6 %** | 1083 | 1100 | +1.6 % | 49 | fuzz_unzip | 4560 | 4430 | −2.9 % | 805 | 804 | −0.1 % | 50 | fuzz_datasource | 3254 | 3244 | −0.3 % | 650 | 650 | 0.0 % | 51 52 libFuzzer's `ft` counter includes value-profile entries and is unbounded, 53 so it keeps rising after the fuzzer has stopped reaching new code. Any 54 ranking built on it at this scale is fitting noise. Everything below 55 therefore uses **edge coverage** and **llvm-cov region coverage**, both 56 of which reproduce to within about 2 %. 57 58 59 ## 3. How long a target is worth running 60 61 Seconds each target needed to reach a given fraction of the edge 62 coverage it ended the 1200 s slice with: 63 64 | | median over 32 targets | 65 |---|---| 66 | 90 % of final | **12 s** | 67 | 95 % of final | **34 s** | 68 | 99 % of final | **93 s** | 69 70 Only **4 of 32** targets were still gaining at t = 1200 s 71 (`fuzz_unzip`, `fuzz_rtf`, `fuzz_qt`, `fuzz_msoffice`); twelve had 72 converged within nine seconds. 73 74 Aggregated over all 32 targets, the discovery rate collapses by a factor 75 of **340** across a single twenty-minute slice: 76 77 | window | new edges | rate | 78 |---|---|---| 79 | 0 – 75 s | 2026 | 3039 edges/core-hour | 80 | 75 – 150 s | 117 | 175 edges/core-hour | 81 | 150 – 300 s | 80 | 60 edges/core-hour | 82 | 300 – 600 s | 61 | 23 edges/core-hour | 83 | 600 – 1200 s | 48 | **9 edges/core-hour** | 84 85 87 % of everything the slice found arrived in its first 75 seconds. The 86 second half of the run — six of the twelve core-hours — produced 2 % of 87 the gain. 88 89 Cumulative gain fits `G(t) = 1655 · t^0.050`, which extrapolates to: 90 91 | per target | total edges | vs the 20-min slice | cost | 92 |---|---|---|---| 93 | 20 min | 2332 | — | 10.7 core-hours | 94 | 1 h | 2488 | +6.7 % | 32 core-hours | 95 | 2 h | 2576 | +10.5 % | 64 core-hours | 96 | 8 h | 2760 | +18.3 % | 256 core-hours | 97 | 24 h | 2915 | +25.0 % | 768 core-hours | 98 | 168 h | 3211 | +37.7 % | 5376 core-hours | 99 100 Seventy-two times the budget buys a quarter more coverage. The marginal 101 cost per edge roughly doubles with every doubling of runtime: 0.14 102 core-hours per edge going from 20 min to 1 h per target, 1.4 at 8 h, 103 3.9 at 24 h. 104 105 106 ## 4. Where the remaining code actually is 107 108 The campaign corpora were replayed through a coverage build. Including 109 `fuzz_ole2`, which was given its own matching 1200 s slice afterwards, 110 the union across all targets is **6488 of 7328 library regions, 88.5 %**. 111 The 840 that are left are not evenly spread, and not all of them are 112 reachable: 113 114 | source file | regions | covered | left | best target | 115 |---|---|---|---|---| 116 | `common/unzip.c` | 674 | 555 | **119** | fuzz_unzip | 117 | `plugins/pack.c` | 128 | 39 | 89 → *unreachable* | fuzz_elf | 118 | `plugins/ole2_extractor.c` | 799 | 721 | 78 | fuzz_ole2 | 119 | `main/extractor_datasource.c` | 545 | 471 | 74 | fuzz_datasource | 120 | `plugins/msoffice_extractor.c` | 548 | 475 | 73 | fuzz_msoffice | 121 | `plugins/ogg_extractor.c` | 235 | 162 | 73 | fuzz_ogg | 122 | `plugins/qt_extractor.c` | 390 | 334 | 56 | fuzz_qt | 123 | `plugins/rtf_extractor.c` | 746 | 690 | 56 | fuzz_rtf | 124 | `plugins/png_extractor.c` | 315 | 267 | 48 | fuzz_png | 125 | `plugins/msoffice_biff.h` | 181 | 140 | 41 | fuzz_ole2 | 126 | `plugins/flac_extractor.c` | 130 | 89 | 41 | fuzz_flac | 127 | `plugins/mime_extractor.c` | 52 | 24 | 28 | fuzz_mime | 128 | everything else | | | ≤ 10 each | | 129 130 Four findings matter more than the table: 131 132 **`fuzz_ole2` was the highest-yield target in the tree and had never been 133 run.** Its 1200 s slice took `ole2_extractor.c` from 47.6 % to 90.2 % 134 — 341 new regions, 1023 regions/core-hour, against 780 for the next best 135 (`fuzz_elf`) and 150 for `fuzz_unzip`. It also covers 136 `plugins/msoffice_biff.h` better than `fuzz_msoffice` does (41 uncovered 137 against 64). It was excluded from `contrib/oss-fuzz/build.sh` because 138 the plugin needs libgsf's compiler flags rather than just a `-l`, and 139 nobody revisited the exclusion; issue 13 was waiting in it, reachable 140 from a file already in `src/plugins/testdata/`. Build it with 141 `LE_FUZZ_GSF=1`. Note that most of what it *reports* belongs to libgsf 142 and glib rather than to libextractor — see `issues.txt` — so it earns a 143 place in local campaigns but not in the OSS-Fuzz default set. 144 145 **`plugins/pack.c`'s 89 uncovered regions are dead code.** It is a 146 general-purpose (un)packer with sixteen format codes. Its only two 147 callers pass fixed format strings — `"4bW16bH"`, `"WWW"`, 148 `"hhwwwwwhhhhhh"`, `"wwwwwwwwww"` and their upper-case variants — so no 149 input can reach the rest. `fuzz_applefile` looks like it has 102 150 regions of headroom and actually has none; do not fund it. 151 152 **`fuzz_zip`, `fuzz_odf` and `fuzz_msoffice` do not meaningfully cover 153 `unzip.c`.** Their corpora were merged with `fuzz_unzip`'s (961 inputs 154 total) and replayed: union coverage of `unzip.c` is 83.4 % against 155 `fuzz_unzip`'s 82.3 % alone. Three targets, 743 inputs, seven extra 156 regions. `unzip.c` should be funded through `fuzz_unzip` and nowhere 157 else; `zip_extractor.c` itself is already at 100 %. 158 159 **What is left in `unzip.c` is the interesting part.** The uncovered 160 regions concentrate in `parse_current_file_coherency_header` (29) and 161 `unzip_open_using_ffd` (18) — the central-directory / local-header 162 consistency checks, which is precisely where a ZIP reader gets attacked. 163 Reaching them needs a better ZIP generator, not a longer run. 164 165 166 ## 5. The recommendation 167 168 **Balance.** Weight each target by the library regions still uncovered in 169 the files it covers better than any other target, excluding the 170 unreachable ones, and damp the third-party wrappers (gif, jpeg, tiff, 171 flac, ogg, archive, mime) to 35 % — their remaining regions are error 172 returns from giflib/libjpeg/libtiff/FLAC/libvorbis/libarchive/libmagic, 173 which those projects fuzz themselves. Give every target a 300 s floor so 174 that a regression anywhere still gets caught. That table is built into 175 `run_campaign.sh`. 176 177 **Length.** Three tiers, and the top one is smaller than it looks: 178 179 | profile | wall clock on 12 cores | core-hours | what it is for | 180 |---|---|---|---| 181 | `ci` | ~11 min (300 s floor, every target) | 2.8 | every push; recovers ~93 % of the edges a 20-min-per-target run reaches | 182 | `nightly` | 2 h | 24 | floor plus the headroom-weighted share | 183 | `deep` | 8 h | 96 | weekly; past here each additional 1 % of edge coverage costs >30 core-hours | 184 185 No single job is ever given more than the wall clock, however heavy its 186 weight: a target handed nine hours inside an eight-hour campaign would 187 still be running when everything else had finished. `run_campaign.sh` 188 water-fills instead — share by weight, clamp at the wall clock, 189 redistribute the spill, repeat — so `-t 8` really does take eight hours. 190 191 Do not run flat campaigns longer than `deep`. Beyond roughly 100 192 core-hours per run the fitted curve says the money is better spent on: 193 194 1. **A structure-aware ZIP generator**, for the `unzip.c` coherency 195 checks above — 119 regions, and the ones that matter. 196 2. **New targets.** `fuzz_ole2` had 419 uncovered regions and one live 197 defect purely because nobody had built it; that is a far better 198 return than another 100 core-hours on the 32 targets that were 199 already running. `fuzz_extract` is still absent from the libFuzzer 200 set for a good reason (it `dlopen()`s uninstrumented modules), but it 201 is worth running under the built-in driver. 202 3. **The replay pass after every fix.** Issue 12 existed only because 203 issue 02 fired first on almost every PNG input and masked it; it was 204 found by re-running after patching, not by running longer. 205 206 The general shape: on this codebase a campaign is a **corpus-building 207 and regression exercise**, not a bug-discovery lottery. Of the twelve 208 defects fixed, four were found within 73 seconds and seven more were 209 reachable from the seed corpus with *no fuzzing at all* — replaying 210 `src/plugins/testdata/` under UBSan finds them. Exactly one needed 211 fuzzing time, and only after another defect was unmasked. 212 213 214 ## 6. Reproducing the measurement 215 216 ```sh 217 # 1. build, out of tree, with gsf so ole2 is included 218 rsync -a --exclude=.git /path/to/libextractor/ /tmp/le/libextractor/ 219 ( cd /tmp/le/libextractor && make distclean ) 220 SRC=/tmp/le WORK=/tmp/w OUT=/tmp/out SANITIZER=address \ 221 LE_FUZZ_EXTRA_PLUGINS=1 LE_FUZZ_GSF=1 \ 222 /tmp/le/libextractor/contrib/oss-fuzz/build.sh 223 224 # 2. flat slice, for comparable curves 225 contrib/oss-fuzz/run_campaign.sh -b /tmp/out -o /tmp/camp -c 12 -F 1200 -t 0 226 227 # 3. coverage build, replay each corpus, diff against the seed corpus 228 SRC=/tmp/le WORK=/tmp/wc OUT=/tmp/cov SANITIZER=coverage \ 229 LE_FUZZ_EXTRA_PLUGINS=1 LE_FUZZ_GSF=1 \ 230 /tmp/le/libextractor/contrib/oss-fuzz/build.sh 231 LLVM_PROFILE_FILE=p.profraw /tmp/cov/fuzz_unzip -runs=0 /tmp/camp/corpus/fuzz_unzip 232 llvm-profdata merge -sparse p.profraw -o p.profdata 233 llvm-cov report /tmp/cov/fuzz_unzip -instr-profile=p.profdata 234 ``` 235 236 The per-job stats lines libFuzzer prints in fork mode 237 (`#N: cov: .. ft: .. corp: .. time: Ns job: M`) are a ready-made time 238 series; that is where sections 2 and 3 come from. 239 240 241 ## 7. The staged forensic plugins (measured 2026-08-07) 242 243 Sixteen plugins — `apk diskimage ebook geotiff gpx heif id3 iso9660 kml 244 lnk mbox pecoff plist sqlite tar webp`, about 24 000 lines of 245 hand-written parser — were added to the tree with in-tree harnesses in 246 `src/fuzz/Makefile.am`, but they were **not** in 247 `contrib/oss-fuzz/build.sh` and not in the weight table, so no campaign 248 had ever run one. They are in both now. 249 250 ### The run 251 252 48 jobs x 1200 s / 16 cores = 3 waves = 3718 s wall, 16 core-hours 253 254 The 1200 s slice is deliberately the same one section 1 used, so the 255 headroom numbers below are on the same basis as the older ones. Of the 256 48 jobs, 32 were the sixteen new targets run **twice** with different 257 PRNG seeds and independent corpora, and 16 were the highest-weight 258 legacy targets as a regression check. 259 260 | | | 261 |---|---| 262 | crashes in the sixteen new targets | **0** | 263 | crashes elsewhere | 8 artifacts, all `fuzz_ole2`, all one defect (below) | 264 | recovering UBSan reports | **1 defect**, `geotiff_extractor.c` | 265 | corpus | 170 MB across 48 corpora | 266 | union region coverage | 15 607 / 17 380 = **89.8 %** | 267 268 ### Two findings 269 270 **`geotiff_extractor.c:1669-1670`, float-cast-overflow.** `emit_image_tags` 271 guards `width` and `height` with `isfinite()` and `> 0.0` but with no 272 upper bound, then casts both to `unsigned long long`. A TIFF tag 273 carrying 7.0e22 makes that cast undefined. The two cases immediately 274 below it — `bits` and `samples` — already carry the `4096.0 >` bound 275 that this one is missing, so the fix is to match them. Reproduced in 276 both replicates, and it is exactly the shape section 1 warns about: a 277 *recovering* report, invisible to `-fork=1`, found only by the replay 278 pass. 279 280 **The eight `fuzz_ole2` artifacts are one defect, and it is libgsf's.** 281 All eight have an identical stack: `iconv` reads past the end of the 282 buffer `le_input_read` handed it, called from 283 `gsf_doc_meta_data_read_from_msole` via `g_convert_with_iconv`. This is 284 the class section 4 already documents for this target. `issue13` no 285 longer reproduces, so nothing regressed. 286 287 ### These plugins do not converge like the old ones 288 289 This is the result that matters for scheduling, and it contradicts 290 section 3 for the new set: 291 292 | time to reach | legacy median (32 targets) | staged median (16 targets) | 293 |---|---|---| 294 | 90 % of final edge coverage | 12 s | 21 s | 295 | 95 % | 34 s | 115 s | 296 | 99 % | 93 s | **690 s** | 297 | still gaining at 1200 s | 4 of 32 | **11 of 16** | 298 299 The 300 s floor was derived from targets that are done in 93 s. It cuts 300 these off mid-climb. **Run with `-F 900` while they are in the set.** 301 302 ### Replicate noise is much larger here, so do not over-read the ranking 303 304 Section 2 established that edge coverage reproduces to about 2 % on the 305 legacy targets. It does not here. Same target, same slice, different 306 seed: 307 308 | target | Δ edge coverage | target | Δ edge coverage | 309 |---|---|---|---| 310 | fuzz_sqlite | 0.0 % | fuzz_kml | −3.6 % | 311 | fuzz_id3 | 0.0 % | fuzz_diskimage | −3.7 % | 312 | fuzz_lnk | −0.1 % | fuzz_apk | −5.7 % | 313 | fuzz_heif | −0.1 % | fuzz_plist | −6.7 % | 314 | fuzz_iso9660 | +0.2 % | fuzz_gpx | −7.4 % | 315 | fuzz_webp | +0.3 % | fuzz_mbox | −9.1 % | 316 | fuzz_geotiff | +0.7 % | **fuzz_ebook** | **−16.6 %** | 317 | fuzz_tar | −0.8 % | fuzz_pecoff | +4.4 % | 318 319 The noisy ones are the deep, slow parsers, and they are noisy for the 320 same reason they are slow: `fuzz_mbox` managed 487 k executions in its 321 slice where `fuzz_sqlite` managed 60.7 M, a factor of 125, so a single 322 seed samples far less of the input space. The weights below therefore 323 come from **llvm-cov region coverage of the merged a+b corpus**, not 324 from one run's edge count, and a difference of less than about 20 % in 325 this table should not be treated as a real ordering. 326 327 ### Where the remaining code is 328 329 | source file | regions | covered | left | best target | 330 |---|---|---|---|---| 331 | `diskimage_extractor.c` | 1030 | 753 | **277** | fuzz_diskimage | 332 | `apk_extractor.c` | 927 | 824 | 103 | fuzz_apk | 333 | `pecoff_extractor.c` | 943 | 842 | 101 | fuzz_pecoff | 334 | `tar_extractor.c` | 356 | 257 | 99 | fuzz_tar | 335 | `ebook_extractor.c` | 775 | 678 | 97 | fuzz_ebook | 336 | `forensics.c` | 263 | 201 | 62 | fuzz_lnk | 337 | `kml_extractor.c` | 807 | 751 | 56 | fuzz_kml | 338 | `mbox_extractor.c` | 1382 | 1327 | 55 | fuzz_mbox | 339 | `gpx_extractor.c` | 881 | 846 | 35 | fuzz_gpx | 340 | `geotiff_extractor.c` | 901 | 868 | 33 | fuzz_geotiff | 341 | `id3_extractor.c` | 886 | 855 | 31 | fuzz_id3 | 342 | `heif_extractor.c` | 713 | 695 | 18 | fuzz_heif | 343 | `plist_extractor.c` | 594 | 580 | 14 | fuzz_plist | 344 | `iso9660_extractor.c` | 281 | 274 | 7 | fuzz_iso9660 | 345 | `webp_extractor.c` | 225 | 220 | 5 | fuzz_webp | 346 | `sqlite_extractor.c` | 549 | 548 | 1 | fuzz_sqlite | 347 348 Three things follow. 349 350 **`fuzz_diskimage` is the new `fuzz_ole2`.** 277 uncovered regions, more 351 than any other target in the tree, and 73 % coverage where every other 352 staged plugin is near 90 %. The cause is the corpus, not the runtime: 353 the plugin dispatches on four container formats — QCOW2, VHD, VHDX, 354 VMDK — and `src/plugins/testdata/` holds exactly one small sample of 355 each. Fund it first, and give it better seeds. 356 357 **`forensics.c` is shared by all sixteen and 24 % of it is uncovered.** 358 It is attributed to `fuzz_lnk` only because that target happens to cover 359 it best; no single target owns it. 62 regions in code every one of 360 these plugins calls is worth more attention than its position in the 361 table suggests. 362 363 **`fuzz_sqlite` and `fuzz_webp` are done.** One and five regions left 364 after 60.7 M and 39.7 M executions. Give them the floor and nothing 365 more. 366 367 ### What to spend the next campaign on 368 369 1. **Seeds for `fuzz_diskimage`**, and to a lesser extent `fuzz_tar` 370 (72 % covered). Both are corpus-starved rather than time-starved; 371 another hour of the same inputs will not find the other 277 regions. 372 2. **`-F 900`**, per the convergence table above. 373 3. **A structure-aware generator** is still the honest answer for 374 `unzip.c` (61 regions left, concentrated in the central-directory 375 coherency checks). Note that `fuzz_apk` and `fuzz_ebook` link 376 `unzip.c` but cover it *worse* than `fuzz_unzip` does — 527 and 521 377 regions against 607 — so section 4's conclusion stands unchanged: 378 fund `unzip.c` through `fuzz_unzip` and nowhere else. 379 380 381 ## 8. The tuned 4-hour run (2026-08-07) 382 383 The campaign of section 7 was re-run with everything it taught applied: 384 the measured weight table, `-F 900` for the slow-converging staged 385 plugins, and enriched seeds for the two targets that turned out to be 386 corpus-starved. 387 388 49 targets / 16 cores / 4 h = 64 core-hours, water-filled 389 390 ### Seeds beat runtime, decisively 391 392 Section 7 attributed `fuzz_diskimage`'s 277 uncovered regions to a 393 corpus holding exactly one sample of each of its four container formats, 394 and `fuzz_tar`'s 99 to a single ustar archive. Generating the missing 395 variants — QCOW2 v2/v3 at three cluster sizes, lazy-refcount, zstd and 396 backed images, all five VMDK subformats qemu-img will write, VHDX at 397 four block/log combinations, fixed and dynamic VHD; and GNU/ustar/pax/ 398 v7/oldgnu/posix tars plus sparse members and pax extended headers — 399 moved coverage *before any fuzzing at all*: 400 401 | file | seeds only, before | seeds only, after | 1200 s campaign of §7 | 402 |---|---|---|---| 403 | `diskimage_extractor.c` | 55.5 % | **66.2 %** | 73.1 % | 404 | `tar_extractor.c` | 54.2 % | **80.1 %** | 72.2 % | 405 406 The tar figure is the one to remember: eight generated files reach more 407 of the parser than a twenty-minute fuzzing slice did. When a target's 408 headroom is concentrated in format variants the corpus does not contain, 409 generating them is worth more than any amount of additional runtime. 410 411 ### Findings 412 413 **`qt_extractor.c`, heap-buffer-overflow — fixed.** An 80-byte input. 414 `getAtomSize` and `getAtomHeaderSize` both document that 415 `checkAtomValid` must have run first, because a size field of 1 selects 416 the 64-bit `struct LongAtom` and is read sixteen bytes wide rather than 417 eight. `ilstHandler` validates each entry and passes the real end down, 418 but `iTunesTagHandler` ignored its `size` parameter and tested only the 419 *declared* size (`asize < hdr + 8`) before handing `pos + hdr` to 420 `processDataAtom`, which dereferences the child atom immediately. A 421 child declaring the 64-bit escape therefore read eight bytes past what 422 had been guaranteed. The fix restores the documented precondition with 423 the same `checkAtomValid` call the sibling handlers already use. 424 Reproducer: `corpus/known-findings/fuzz_qt-issue15.bin`. 425 426 **`geotiff_extractor.c`, float-cast-overflow — fixed.** See section 7. 427 Reproducer: `corpus/known-findings/fuzz_geotiff-issue14.bin`. 428 429 **The ole2 artifacts are a libgsf defect — patch sent to `patches/`.** 430 Diagnosed rather than merely classified this time: `msole_prop_read` 431 checks a dictionary entry's length against `end - data + 8` where it 432 must be `end - data - 8`, and omits `section->char_size`, which is 2 for 433 the Unicode property sets. A crafted document reads up to `2*len + 16` 434 bytes past the property-set section. It is invisible with libgsf's own 435 stdio and memory inputs, whose padded buffers absorb it, and becomes a 436 hard ASan report only because `le_input_read` in `ole2_extractor.c` 437 hands libgsf a tightly sized buffer. Rebuilding libgsf 1.14.53 with the 438 fix takes all twenty artifacts to zero and leaves `test_ole2` and 439 `test_msoffice` passing. 440 441 Note that `fuzz_qt` ran this campaign against the *unpatched* plugin — 442 the defect was found an hour in, and rebuilding mid-run would have 443 invalidated the schedule. Its artifacts are all the one defect. 444 445 446 ## 9. Independent runs beat long runs 447 448 The four-hour campaign of section 8 gave its six heaviest targets a 449 single uninterrupted four-hour slice each, because that is what 450 water-filling a weighted budget does. Replaying the corpora afterwards 451 showed that this was the wrong thing to do, and by a wide margin. 452 453 Region coverage of each plugin's own source file, same binary, same 454 seed corpus, only the shape of the schedule differing: 455 456 | target | 2 x 1200 s, corpora unioned | 1 x 14 000 s | best single 1200 s run | 457 |---|---|---|---| 458 | `fuzz_ebook` | **678** / 775 | 595 | 669 | 459 | `fuzz_apk` | **824** / 927 | 792 | 822 | 460 461 Forty core-minutes beat four core-hours, and for both targets even *one* 462 twenty-minute run beat the four-hour one. 463 464 The mechanism is visible in the logs. `fuzz_ebook`'s four-hour job 465 reached 1193 edges at t = 2570 s and never found another one — it then 466 spent 82 % of its slice and 195 million further executions standing 467 still, while a twenty-minute run from a different PRNG seed reached 468 1395. libFuzzer commits early to a corpus and mutates within it; when 469 that corpus is a poor basin there is no mechanism to leave it, and more 470 time cannot help. A second run from a different seed is not more of the 471 same work — it is a second sample of a distribution whose spread 472 (section 7: up to 16.6 % on these plugins) dwarfs what extra runtime 473 buys. 474 475 This is not universal. Measuring how long each target went without a 476 new edge before its slice ended: 477 478 | still gaining at the end | plateaued early | 479 |---|---| 480 | `fuzz_rtf` 0 %, `fuzz_apk` 1 %, `fuzz_gpx` 2 %, `fuzz_pecoff` 3 %, `fuzz_msoffice` 4 %, `fuzz_diskimage` 5 %, `fuzz_mbox` 5 % | `fuzz_ogg` 87 %, `fuzz_ebook` 82 %, `fuzz_unzip` 61 %, `fuzz_datasource` 48 %, `fuzz_kml` 48 % | 481 482 Note that `fuzz_apk` is in the *left* column and still lost to two short 483 runs: a target can keep finding new edges at a slow rate and yet be in a 484 worse basin than a fresh run would have found. Time-to-plateau alone 485 does not identify which targets to shard. 486 487 ### What was changed 488 489 `run_campaign.sh` grew `-S SECONDS` (default 1800). Any target allotted 490 more than one shard's worth is now run as several independent jobs of 491 that length, each with its own copy of the seed corpus and its own 492 `-seed`, and the shard corpora are unioned into `corpus-merged/` at the 493 end. The core-hour budget and the weight table are untouched; only the 494 shape changes. On 16 cores for 4 h that is 148 jobs instead of 49. 495 496 The weight table itself was re-derived from the union of every corpus 497 both campaigns produced, which is the least seed-dependent basis 498 available: union coverage 17559 / 18889 regions, 93.0 %. 499 500 ### The standing advice, revised 501 502 1. **Shard.** Leave `-S` alone. It is worth more than any weight in the 503 table. 504 2. **Seeds before time.** Section 8's tar result and section 9's ebook 505 result point the same way: eight generated files beat a twenty-minute 506 slice, and a twenty-minute slice beat four hours. When a target 507 looks starved, ask whether the corpus is missing a format variant 508 before asking for more core-hours. 509 3. **Then weights**, for apportioning what is left. 510 511 512 ## 10. Sharding is a cold-start effect 513 514 Section 9 concluded that several independent short runs beat one long 515 run, and `-S` was added on that basis. A two-hour, 16-core, 85-job run 516 tested whether it still holds once the corpus is no longer small. It 517 does not, and the qualification matters more than the original result. 518 519 The run differed from section 8's in two ways: every target started 520 from the accumulated corpus of the previous two campaigns rather than 521 from the shipped seeds, and the oversized allocations were sharded. The 522 corpus was distilled first (`-merge=1`, 42 707 inputs down to 16 748, 523 338 MB to 116 MB, coverage-neutral: `diskimage_extractor.c` 831 to 830 524 regions, `gpx_extractor.c` 855 to 855). 525 526 Final edge coverage of the shards of one target, against the 527 cold-start seed-to-seed spread that motivated sharding in the first 528 place: 529 530 | target | shards (warm) | spread | cold-start Δ (§7) | 531 |---|---|---|---| 532 | fuzz_ebook | 1433 1435 1433 1434 | **0.14 %** | −16.6 % | 533 | fuzz_mbox | 1833 1833 | 0.00 % | −9.1 % | 534 | fuzz_gpx | 1264 1264 | 0.00 % | −7.4 % | 535 | fuzz_apk | 1467 1462 1460 1464 | 0.48 % | −5.7 % | 536 | fuzz_kml | 1078 1081 | 0.28 % | −3.6 % | 537 | fuzz_diskimage | 824 825 | 0.12 % | −3.7 % | 538 | fuzz_pecoff | 1284–1288 | 0.31 % | +4.4 % | 539 | fuzz_lnk, heif, id3, geotiff | identical | 0.00 % | ~0 % | 540 541 From a large corpus the PRNG seed stops mattering: every shard lands in 542 the same place to within half a percent, where cold-start replicates 543 differed by up to a sixth. **It is the corpus, not the seed, that 544 decides where a run ends up.** Sharding buys diversity only while the 545 corpus is small enough for trajectory to be in play. 546 547 That does not make `-S` wrong — it still prevents the section 9 548 pathology, where a single long job plateaus and burns hours on nothing 549 (`fuzz_ebook` idle for 82 % of a four-hour slice), and it costs nothing. 550 But it should be understood as insurance against a bad trajectory, not 551 as a way to buy coverage from a mature corpus. 552 553 The warm start itself was the larger effect. `fuzz_ebook` finished at 554 1433 edges from every shard, against 1395 for the best cold-start 555 twenty-minute run and 1163 for the worst — the corpus removed the 556 variance that section 7 spent its whole budget measuring. 557 558 ### Diminishing returns, quantified 559 560 | campaign | cost | union region coverage | 561 |---|---|---| 562 | §7 flat measurement | 16 core-hours | 89.8 % | 563 | §8 tuned, 4 h | 64 core-hours | 93.0 % | 564 | §10 warm + sharded, 2 h | 32 core-hours | **93.1 %** | 565 566 Thirty-two core-hours bought 36 regions. The library is at the point 567 this document predicted in section 5: campaigns are now a regression 568 exercise, and new coverage has to come from seeds, new targets, or 569 structure-aware generators rather than from runtime. 570 571 ### Findings 572 573 None new. All 22 artifacts were `fuzz_ole2`, all carrying the 574 `g_convert_with_iconv` signature of the libgsf defect in `patches/`, and 575 all 22 stop reporting against a patched libgsf. The replay pass over 576 all 85 corpora produced no recovering UndefinedBehaviorSanitizer reports 577 and no other sanitizer output.