BUILD-INTEGRATION.md (7647B)
1 # Build integration for `src/fuzz/` 2 3 > **Status: applied.** The three changes below are already present in 4 > this tree — `configure.ac` carries `--enable-fuzzing` and the 5 > `ENABLE_FUZZING` conditional, `AC_CONFIG_FILES` lists 6 > `src/fuzz/Makefile`, and `src/Makefile.am` adds `fuzz` to `SUBDIRS` 7 > under that conditional. This file is kept as the record of what 8 > changed and as the recipe for porting `src/fuzz/` to another branch. 9 > 10 > For running these same harnesses on OSS-Fuzz — a different build path 11 > that does **not** go through `src/fuzz/Makefile.am` — see 12 > `../../contrib/oss-fuzz/` and §5 at the end of this file. 13 14 `src/fuzz/Makefile.am` is complete. Three files outside `src/fuzz/` 15 had to be touched, plus the removal of the script this directory 16 replaces. 17 18 The harnesses are guarded by the automake conditional `ENABLE_FUZZING` 19 (`--enable-fuzzing`, default **no**), because they are only meaningful in 20 a build with sanitizers and because they need static archives of the 21 library: they call functions that are hidden in the shared objects. 22 23 --- 24 25 ## 1. `configure.ac` 26 27 ### 1.1 The option and the conditional 28 29 Added where the old `AC_CHECK_PROG([HAVE_ZZUF], ...)` pair used to be: 30 31 ```m4 32 AC_MSG_CHECKING([whether to build the fuzzing harnesses]) 33 AC_ARG_ENABLE([fuzzing], 34 [AS_HELP_STRING([--enable-fuzzing], 35 [build the in-process fuzzing harnesses in src/fuzz and run them as ] 36 [part of "make check"; requires a static build of the library and is ] 37 [only really useful together with a sanitizer build [no]])], 38 [], [enable_fuzzing="no"]) 39 AS_IF([test "x$enable_fuzzing" = "xyes"], 40 [AS_IF([test "x$enable_static" = "xno"], 41 [AC_MSG_RESULT([no]) 42 AC_MSG_ERROR([--enable-fuzzing requires --enable-static])], 43 [AC_MSG_RESULT([yes])])], 44 [enable_fuzzing="no" 45 AC_MSG_RESULT([no])]) 46 AM_CONDITIONAL([ENABLE_FUZZING], [test "x$enable_fuzzing" = "xyes"]) 47 ``` 48 49 ### 1.2 Register the new `Makefile` 50 51 `src/fuzz/Makefile` added to the final `AC_CONFIG_FILES([...])` list. 52 53 --- 54 55 ## 2. `src/Makefile.am` 56 57 ```make 58 if ENABLE_FUZZING 59 FUZZ_SUBDIRS = fuzz 60 endif 61 62 SUBDIRS = include $(INTLEMU_SUBDIRS) main common plugins $(FUZZ_SUBDIRS) . 63 ``` 64 65 Automake derives `DIST_SUBDIRS` from all branches of `SUBDIRS` 66 automatically, so `make dist` keeps shipping `src/fuzz/` regardless of 67 the conditional. 68 69 --- 70 71 ## 3. `src/plugins/Makefile.am` — remove the old zzuf test 72 73 `fuzz_default.sh` is gone. Three hunks were removed with it: its 74 `EXTRA_DIST` entry, the 75 76 ```make 77 if HAVE_ZZUF 78 fuzz_tests=fuzz_default.sh 79 endif 80 ``` 81 82 block, and `$(fuzz_tests)` from `TESTS`. The `HAVE_ZZUF` conditional 83 itself is left in `configure.ac` because `contrib/coverage.sh` still 84 drives zzuf. 85 86 --- 87 88 ## 4. Build and run 89 90 ```sh 91 ./bootstrap 92 ./configure --enable-fuzzing --enable-static \ 93 CC=clang \ 94 CFLAGS="-g -O1 -fno-omit-frame-pointer \ 95 -fsanitize=address,undefined \ 96 -fsanitize-address-use-after-scope" \ 97 LDFLAGS="-fsanitize=address,undefined" 98 make 99 make -C src/fuzz check 100 ``` 101 102 A longer session, and the corpus replay a CI job should run after a fix: 103 104 ```sh 105 make -C src/fuzz check LE_FUZZ_ITERATIONS=5000000 LE_FUZZ_SEED=$RANDOM 106 make -C src/fuzz check-corpus 107 ``` 108 109 --- 110 111 ## 5. Notes and caveats 112 113 * **`--enable-static` is mandatory.** `fuzz_datasource`, `fuzz_unzip`, 114 `fuzz_ipc` and `fuzz_convert` call `EXTRACTOR_datasource_read_()`, 115 `EXTRACTOR_common_unzip_open()`, `EXTRACTOR_IPC_process_reply_()` and 116 friends. `src/main/Makefile.am` builds the library with 117 `$(HIDDEN_VISIBILITY_CFLAGS)` and an `-export-symbols-regex`, so those 118 symbols are not in `libextractor.so`. The `-static` in the per-target 119 `_LDFLAGS` makes libtool pick `.libs/libextractor.a`. 120 121 * **`fuzz_extract` is the exception** — it deliberately does *not* link 122 `-static`, because it needs `lt_dlopen()` to find the plugin modules 123 at run time. It also needs `LIBEXTRACTOR_PREFIX` to point at them; 124 `AM_TESTS_ENVIRONMENT` sets it to `src/plugins/.libs`. 125 126 * **Per-plugin targets compile plugin sources a second time.** Each 127 `fuzz_<plugin>` lists the plugin's `.c` files in 128 `nodist_<target>_SOURCES` with its own `_CPPFLAGS`, so automake gives 129 the objects a per-target prefix (`fuzz_riff-riff_extractor.o`) and 130 they never collide with the ones `src/plugins/` builds. Automake 131 warns about `subdir-objects` for these; the warning is about a future 132 automake changing where those objects land, not about the current 133 build being wrong. Enabling `subdir-objects` globally is the eventual 134 fix and is a tree-wide change, so it is deliberately not made here. 135 136 * **The plugin list in `Makefile.am` must stay in sync with three other 137 places** when a target is added: `fuzz_plugin_name.h` (the format 138 description), `contrib/oss-fuzz/build.sh` (`PLUGIN_FUZZERS`) and 139 `contrib/oss-fuzz/make_seed_corpus.sh` (`testdata_glob`). §6 of the 140 README lists the steps. 141 142 * `make check` here is a *smoke test*, not a campaign: 20000 iterations 143 per harness of the built-in generator. It exists so that a harness 144 that stops compiling, stops linking or starts crashing on its own seed 145 corpus is caught by an ordinary `make check`. 146 147 --- 148 149 ## 6. The OSS-Fuzz build path (does not use this `Makefile.am`) 150 151 `contrib/oss-fuzz/build.sh` builds the *same harness sources* for 152 libFuzzer/AFL++/honggfuzz without going through `src/fuzz/Makefile.am` 153 at all. It configures the library out of tree, then compiles each 154 harness by hand with `-DFUZZ_NO_MAIN` and links it against 155 `$LIB_FUZZING_ENGINE`. 156 157 Two consequences for anyone editing `src/fuzz/`: 158 159 * **`LLVMFuzzerTestOneInput()` must stay unconditional.** Only `main()` 160 may be inside `#ifndef FUZZ_NO_MAIN`; a harness whose fuzz target is 161 itself conditional silently produces an empty OSS-Fuzz binary. 162 163 * **Anything the fuzz target needs must not live inside the 164 `#ifndef FUZZ_NO_MAIN` block of `fuzz_common.h`.** The PRNG, the crash 165 bookkeeping, `fuzz_report_finding()`, `fuzz_env_ulong()` and 166 `fuzz_ignore_sigpipe()` are outside it; the generator loop, the 167 mutator, the corpus walker and `main()` are inside. 168 169 This split is easy to get wrong and the failure is silent, so it is 170 worth spelling out the shape of it. `signal (SIGPIPE, SIG_IGN)` 171 belongs to the *target*: under the built-in driver everything would 172 look perfect, while under `-DFUZZ_NO_MAIN` the call would vanish and 173 the process would die on the first `SIGPIPE` with no stack trace, no 174 artifact and no crash report — which reads exactly like a clean run 175 that found nothing. It is therefore `fuzz_ignore_sigpipe()`, called 176 from both the driver and every `LLVMFuzzerTestOneInput()`. 177 178 The lesson generalises: a bug in this split cannot be caught by 179 `make -C src/fuzz check`, because that path always defines `main()`. 180 After touching `fuzz_common.h`, build the OSS-Fuzz way as well and 181 confirm the targets still run: 182 183 ```sh 184 rsync -a --exclude=.git . /tmp/le-fuzz-src/ # a tree with no 185 # in-tree config.status 186 WORK=/tmp/le-fuzz-work OUT=/tmp/le-fuzz-out \ 187 LE_SRC=/tmp/le-fuzz-src /tmp/le-fuzz-src/contrib/oss-fuzz/build.sh 188 /tmp/le-fuzz-out/fuzz_unzip -runs=10000 189 echo "exit=$?" # anything but 0 here is a bug in the harness, 190 # not a finding 191 ``` 192 193 `contrib/oss-fuzz/` also relies on two things this directory provides: 194 `make -C src/fuzz refresh-corpus` (which regenerates `corpus/`) and the 195 `corpus/known-findings/` reproducers, which it packages into the 196 per-target `_seed_corpus.zip` so that they become permanent regressions.