Testing Guide

First time setup

make ze-setup    # install all dev tools: build deps, linters, appliance tools (one-time)
make ze-smoke    # verify everything works: lint + unit + build (~2 min)

If ze-smoke passes, your environment is ready.

The escalation ladder

Use the narrowest test that covers your change. Escalate only when needed.

make ze-verify is the pre-commit gate. Everything below that is a development tool.

Component groups for step 3

Pick the group matching your change.

Test types

Ze has four levels of testing, each covering a different concern.

Unit tests (*_test.go)

Standard Go tests. They validate logic in isolation: does the parser produce the right output, does the state machine transition correctly, does the encoder round-trip.

go test -race ./internal/component/bgp/message/...    # one package
go test -race -run TestParseOrigin ./internal/...      # one test
make ze-unit-test                                      # all packages (~5 min)

Functional tests (.ci files)

These spin up real Ze processes and test behavior end-to-end: does the config parse, does the BGP session establish, does the plugin produce the right output.

Each test/ subdirectory has its own runner and format:

Run a single test by one-based id or exact name, list the available ids, or resume from the last printed id after an interrupted run:

bin/ze-test bgp plugin 42          # test id 42
bin/ze-test bgp encode --list      # list N/TOTAL, id, and name
bin/ze-test bgp plugin --start 42  # run id 42 and every later test
bin/ze-test editor 7               # editor test id 7
bin/ze-test editor -p nav          # editor tests matching "nav"
bin/ze-test exabgp --start 20      # resume ExaBGP compatibility

Run a full suite:

make ze-encode-test     # all encode tests
make ze-plugin-test     # all plugin tests
make ze-functional-test # all release-gate suites
make ze-exabgp-test     # ExaBGP compatibility through ze-test

Mutation tests (gomu)

Coverage tells you which lines ran. Mutation testing tells you whether the tests would notice if those lines did something different. gomu rewrites the AST (arithmetic, conditional, logical, bitwise, branch, return value, and error handling operators), runs the test suite against each mutation, and reports which mutations survived.

gomu uses overlay-based execution, so it never modifies source files on disk. It is vendored in tools.go and runs via go run; no separate install is needed.

make ze-mutation-changed                              # changed files only (fast)
make ze-mutation-pkg PKG=./internal/core/textbuf/     # one package
make ze-mutation-test                                 # all non-excluded packages (slow)
make ze-mutation-report                               # full run with HTML report

Mutation testing is advisory. It never gates ze-verify or CI. A surviving mutant is a signal that a test could be stronger, not a blocking failure.

Files with custom build tags and cmd/ze/ are excluded via .gomuignore because gomu has no --tags support. Reports land in tmp/ (gitignored). Mutation score history is tracked in test/mutation/history.ndjson.

Tuning: GOMU_WORKERS (default: half CPU cores), GOMU_TIMEOUT (default: 120s), GOMU_THRESHOLD (default: 0%).

Interop and integration tests

These require external infrastructure (Docker, root/CAP_NET_ADMIN, QEMU, or internet). They are not part of the normal development cycle.

make ze-interop-test              # FRR/BIRD in Docker
make ze-integration-test          # netns tests (needs root)
make ze-qemu-integration-test     # same tests in QEMU (macOS-friendly)
make ze-live-rpki-test            # real RPKI data (needs internet)

See make help-test for the full list.

How ze-verify works

ze-verify is the pre-commit gate. It uses a two-pass strategy to stay fast:

1. Lint (27 linters via golangci-lint)
2. Vet evidence (cross-compile evidence scripts for Linux)
3. Cached full pass (go test without -race): Go caches by source hash, so when nothing changed this is instant. Catches logic regressions everywhere.
4. Race pass on changed groups (go test -race only on packages with modified .go files): catches data races in what you touched.
5. Functional tests (the gating functional suites; see mk/test-functional.mk)
6. ExaBGP compatibility

Common case (one group changed): ~2 min total instead of 6+.

Output is captured to tmp/ze-verify.log. On failure:

grep -E "^--- FAIL|^FAIL|TEST FAILURE" tmp/ze-verify.log

Interpreting output

Unit test failures

Standard Go test output. Look for --- FAIL: TestName lines:

--- FAIL: TestParseOrigin (0.00s)
    origin_test.go:42: expected 0, got 1
FAIL    github.com/ze-software/ze/internal/core/bgp/attribute   0.003s

Functional test failures

ze-test prints a summary at the end of each suite. Look for the test name and the expectation that failed:

FAIL  encode/addpath.ci
  expect=bgp:conn=1:seq=1:hex=...
  got: FFFF...0038...

Fuzz failures

Go places the failing input in testdata/fuzz/<TestName>/ under the package. The file contains the input that triggered the crash. Fix the code, then the fuzz corpus entry becomes a regression test automatically.

Cheat sheet