# Fuzzing

Fuzzing is a technique to find potential bugs by providing randomly generated
invalid inputs. To detect potential bugs such as programming errors we use
fuzzing in combination with ASan (Address Sanitizer), MSan (Memory Sanitizer),
UBSan (Undefined Behavior Sanitizer) and asserts in the code. An invalid input
will likely produce a decoding error (some API function returning error), which
is absolutely not a problem, but what it should not do is access memory out of
bounds, use uninitialized memory or hit a false assert condition.

## Automated Fuzzing with oss-fuzz

libjxl fuzzing is integrated into [oss-fuzz](https://github.com/google/oss-fuzz)
as the project `libjxl`. oss-fuzz regularly runs the fuzzers on the `main`
branch and reports bugs into their bug tracker which remains private until the
bugs are fixed in main.

## Fuzzer targets

There are several fuzzer executable targets defined in the `tools/` directory
to fuzz different parts of the code. The main one is `djxl_fuzzer`, which uses
the public C decoder API to attempt to decode an image. The fuzzer input is not
directly the .jxl file, the last few bytes of the fuzzer input are used to
decide *how* will the API be used (if preview is requested, the pixel format
requested, if the .jxl input data is provided altogether, etc) and the rest of
the fuzzer input is provided as the .jxl file to the decoder. Some bugs might
reproduce only if the .jxl input is decoded in certain way.

The remaining fuzzer targets execute a specific portion the codec that might be
easier to fuzz independently from the whole codec.

## Reproducing fuzzer bugs

A fuzzer target, like `djxl_fuzzer` accepts as a parameter one or more files
that will be used as inputs. This runs the fuzzer program in test-only mode
where no new inputs are generated and only the provided files are tested. This
is the easiest way to reproduce a bug found by the fuzzer using the generated
test case from the bug report.

oss-fuzz uses a specific compiler version and flags, and it is built using
Docker. Different compiler versions will have different support for detecting
certain actions as errors, so we want to reproduce the build from oss-fuzz as
close as possible. To reproduce the build as generated by oss-fuzz there are a
few helper commands in `ci.sh` as explained below.

### Generate the gcr.io/oss-fuzz/libjxl image

First you need the ossfuzz libjxl builder image. This is the base oss-fuzz
builder image with a few dependencies installed. To generate it you need to
check out the oss-fuzz project and build it:

```bash
git clone https://github.com/google/oss-fuzz.git ~/oss-fuzz
cd ~/oss-fuzz
sudo infra/helper.py build_image libjxl
```

This will create the `gcr.io/oss-fuzz/libjxl` docker image. You can check if it
was created verifying that it is listed in the output of the `sudo docker image
ls` command.

### Build the fuzzer targets with oss-fuzz

To build the fuzzer targets from the current libjxl source checkout, use the
`./ci.sh ossfuzz_msan` command for MSan, `./ci.sh ossfuzz_asan` command for ASan
or `./ci.sh ossfuzz_ubsan` command for UBSan. All the `JXL_ASSERT` and
`JXL_DASSERT` calls are enabled in all the three modes. These ci.sh helpers will
reproduce the oss-fuzz docker call to build libjxl mounting the current source
directory into the Docker container. Ideally you will run this command in a
different build directory separated from your regular builds.

For example, for MSan builds run:

```bash
BUILD_DIR=build-fuzzmsan ./ci.sh ossfuzz_msan
```

After this, the fuzzer program will be generated in the build directory like
for other build modes: `build-fuzzmsan/tools/djxl_fuzzer`.

### Iterating changes with oss-fuzz builds

After modifying the source code to fix the fuzzer-found bug, or to include more
debug information, you can rebuild only a specific fuzzer target to save on
rebuilding time and immediately run the test case again. For example, for
rebuilding and testing only `djxl_fuzzer` in MSan mode we can run:

```bash
BUILD_DIR=build-fuzzmsan ./ci.sh ossfuzz_msan djxl_fuzzer && build-fuzzmsan/tools/djxl_fuzzer path/to/testcase.bin
```

When MSan and ASan fuzzers fail they will print a stack trace at the point where
the error occurred, and some related information. To make these these stack
traces useful we need to convert the addresses to function names and source file
names and lines, which is done with the "symbolizer". For UBSan to print a stack
trace we need to set the `UBSAN_OPTIONS` environment variables when running the
fuzzer.

Set the following environment variables when testing the fuzzer binaries. Here
`clang` should match the compiler version used by the container, you can pass a
different compiler version in the following example by first installing the
clang package for that version outside the container and using `clang-NN`
(for example `clang-11`) instead of `clang` in the following commands:

```bash
symbolizer=$($(realpath $(which clang)) -print-prog-name=llvm-symbolizer)
export MSAN_SYMBOLIZER_PATH="${symbolizer}"
export UBSAN_SYMBOLIZER_PATH="${symbolizer}"
export ASAN_SYMBOLIZER_PATH="${symbolizer}"
export ASAN_OPTIONS=detect_leaks=1
export UBSAN_OPTIONS=print_stacktrace=1
```

Note: The symbolizer binary must be a program called `llvm-symbolizer`, any
other file name will fail. There are normally symlinks already installed with
the right name which the `-print-prog-name` would print.

## Running the fuzzers locally

Running the fuzzer targets in fuzzing mode can be achieved by running them with
no parameters, or better with a parameter with the path to a *directory*
containing a seed of files to use as a starting point. Note that passing a
directory is considered a corpus to use for fuzzing while passing a file is
considered an input to evaluate. Multi-process fuzzing is also supported. For
details about all the fuzzing options run:

```bash
build-fuzzmsan/tools/djxl_fuzzer -help=1
```

## Writing fuzzer-friendly code

Fuzzing on itself can't find programming bugs unless an input makes the program
perform an invalid operation (read/write out of bounds, perform an undefined
behavior operation, etc). You can help the fuzzer find invalid situations by
adding asserts:

 * `JXL_ASSERT()` is enabled in Release mode by default. It can be disabled
   with `-DJXL_ENABLE_ASSERT=0` but the intention is that it will run for all
   the users in released code. If performance of the check is not an issue (like
   checks done once per image, once per channel, once per group, etc) a
   JXL_ASSERT is appropriate. A failed assert is preferable to an out of bounds
   write.

 * `JXL_DASSERT()` is only enabled in Debug builds, which includes all the ASan,
   MSan and UBSan builds. Performance of these checks is not an issue if kept
   within reasonable limits (automated msan/asan test should finish withing 1
   hour for example). Fuzzing is more effective when the given input runs
   faster, so keep that in mind when adding a complex DASSERT that runs multiple
   times per output pixel.

 * For MSan builds it is also possible to specify that certain values must be
   initialized. This is automatic for values that are used to make decisions
   (like when used in an `if` statement or in the ternary operator condition)
   but those checks can be made explicit for image data using the
   `JXL_CHECK_IMAGE_INITIALIZED(image, rect)` macro. This helps document and
   check (only in MSan builds) that a given portion of the image is expected to
   be initialized, allowing to catch errors earlier in the process.

## Dealing with use-of-uninitialized memory

In MSan builds it is considered an error to *use* uninitialized memory. Using
the memory normally requires something like a decision / branch based on the
uninitialized value, just running `memcpy()` or simple arithmetic over
uninitialized memory is not a problem. Notably, computing `DemoteTo()`,
`NearestInt()` or similar expressions that create a branch based on the value of
the uninitialized memory will trigger an MSan error.

In libjxl we often run vectorized operations over a series of values, rounding
up to the next multiple of a vector size, thus operating over uninitialized
values past the end of the requested region. These values are part of the image
padding but are not initialized. This behavior would not create an MSan error
unless the processing includes operations like `NearestInt()`. For such cases
the preferred solution is to use `msan::UnpoisonMemory` over the portion of
memory of the last SIMD vector before processing, and then running
`msan::PoisonMemory` over the corresponding value in the output side. A note
including why this is safe to do must be added, for example if the processing
doesn't involve any cross-lane computation.

Initializing padding memory in MSan builds is discouraged because it may hide
bugs in functions that weren't supposed to read from the padding. Initializing
padding memory in all builds, including Release builds, would mitigate the
MSan potential security issue but it would hide the logic bug for a longer time
and potentially incur in a performance hit.