Interoperability Testing

Ze validates protocol correctness against production BGP daemons in two complementary ways: live session interop tests (Docker containers running real daemons) and byte-level wire format validation against ExaBGP (Ze's predecessor, a BGP implementation in Python).

For BGP terminology used in this document, see docs/features.md.

Tested Daemons

Prerequisites

The interop test network uses 172.30.0.0/24. MD5 authentication scenarios require NET_ADMIN capability (granted automatically by the orchestrator).

Live Interop Tests (test/interop/)

Each scenario runs Ze and one or more peer daemons in Docker containers on a shared network (172.30.0.0/24), establishes real BGP sessions, and asserts correct behavior via each daemon's native CLI.

How It Works

A Python orchestrator (run.py) iterates over scenario directories in test/interop/scenarios/. For each scenario, interop.py manages the container lifecycle:

1. Create a Docker network (172.30.0.0/24)
2. Start Ze (always) and peer daemons (conditionally, based on which config files exist in the scenario directory)
3. Wait for all containers to become healthy
4. Import and run the scenario's check.py
5. Tear down containers and network

Daemons start conditionally: FRR if frr.conf exists, BIRD if bird.conf exists, GoBGP if gobgp.toml exists. This means each scenario only runs the daemons it needs.

Container Addresses

Container names include the runner PID as suffix, so concurrent runs do not conflict.

Scenario Structure

Each scenario is a directory under test/interop/scenarios/:

scenarios/01-ebgp-ipv4-frr/
  ze.conf        # Ze configuration (required)
  frr.conf       # FRR configuration (starts FRR container)
  check.py       # Python assertions (required)

The check.py file defines a check() function that uses daemon helper classes (FRR, BIRD, GoBGP, Ze) from interop.py to query sessions, routes, and attributes via each daemon's native CLI.

Daemon Helpers

interop.py provides helper classes for querying each daemon:

Methods follow a naming convention:

All classes (FRR, BIRD, GoBGP, Ze) are defined in interop.py. Each wraps the daemon's native CLI (vtysh, birdc, gobgp, ze) via docker exec. Start with an existing scenario (e.g., 01-ebgp-ipv4-frr/check.py) as a template.

All session waiters poll with a configurable timeout (default 90s, override via SESSION_TIMEOUT env var).

Scenario Inventory

The suite has grown to 97 scenario directories in test/interop/scenarios/. The table below lists the core BGP scenarios (01-37); beyond these, the suite also covers route reflection, policy import/export, RPKI origin validation, BMP monitoring, PATHS-LIMIT, max-prefix cease, GTSM, AS112, and full IS-IS (auth, convergence, dual-stack, LAN DIS, P2P, redistribution) and OSPFv2/OSPFv3 (auth, BFD, TE, LFA/TI-LFA, graceful restart, segment routing, opaque LSAs, stub/NSSA, virtual links, and more) interop families.

Running

make ze-interop-test                                  # all scenarios
make ze-interop-test INTEROP_SCENARIO=01-ebgp-ipv4-frr  # single scenario
VERBOSE=1 make ze-interop-test                         # debug output
NO_BUILD=1 make ze-interop-test                        # skip image rebuilds
FRR_IMAGE=quay.io/frrouting/frr:10.3 make ze-interop-test  # override FRR version

Interop tests require Docker and are not part of make ze-verify (which runs without Docker). They are available as a separate target for protocol validation.

The first run builds Docker images (takes a few minutes). Subsequent runs with NO_BUILD=1 skip rebuilds. The full suite takes roughly 5-10 minutes depending on session establishment times.

Debugging Failures

On failure, the orchestrator automatically dumps the last 20 lines of container logs. For more detail:

• VERBOSE=1 enables debug output (polling status, container commands, raw CLI output)
• SESSION_TIMEOUT=120 increases the session establishment timeout (default 90s)
• Single-scenario runs isolate the problem: make ze-interop-test INTEROP_SCENARIO=13-graceful-restart-frr

Writing a New Scenario

1. Create test/interop/scenarios/NN-description/
2. Write ze.conf (required) and peer configs (frr.conf, bird.conf, gobgp.toml) as needed
3. Write check.py with a check() function that imports helpers from interop
4. Run make ze-interop-test INTEROP_SCENARIO=NN-description

Example check.py:

import sys, os
# Make the interop module (two directories up) importable.
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", ".."))
from interop import FRR, ZE_IP

def check():
    frr = FRR()
    frr.wait_session(ZE_IP)
    frr.check_route("10.0.0.0/24")

For Ze's configuration syntax, see docs/architecture/config/syntax.md. Copy an existing scenario's ze.conf as a starting point.

ExaBGP Wire Compatibility (test/exabgp-compat/)

A separate test suite validates that Ze's wire encoding produces identical bytes to ExaBGP (main branch, JSON API 6.0.0). Rather than establishing live BGP sessions, it compares encoded output byte-for-byte using a Python harness.

What It Tests

The harness runs Ze with ExaBGP-derived configurations and compares the wire bytes Ze produces against known-good ExaBGP output. 42 test cases are defined as .ci files in test/exabgp-compat/encoding/. These .ci files use a format specific to the ExaBGP compat harness (option=file:, option=serial, 1:cmd:, 1:raw:, 1:json: lines), not the standard .ci format used by Ze's functional tests. option=serial marks process-driven fixtures that must not overlap other ExaBGP harness instances; the runner executes those after the parallel batch.

Coverage includes:

Running

make ze-exabgp-test   # runs via uv with psutil dependency

ExaBGP compatibility is part of make ze-verify (the pre-commit gate).

Test Hierarchy

Interop tests are intentionally separate from the pre-commit gate because they require Docker and take longer to run. ExaBGP wire compatibility tests run as part of the standard verification suite.

Current Scope

Interop scenarios cover core BGP: session establishment, route exchange, withdrawal, capabilities (4-byte ASN, ADD-PATH, GR, route refresh, PATHS-LIMIT), communities, MD5 auth, route server behavior, route reflection, policy import/export, RPKI origin validation, BMP monitoring, BFD failover, ECMP, SRv6 VPNv6, remove-private-as export policy, GTSM, AS112, and non-unicast address families (EVPN, VPN, FlowSpec). The suite also includes full IS-IS and OSPFv2/OSPFv3 interop families (adjacency, flooding, SPF, dual-stack, authentication, TE, LFA/TI-LFA, graceful restart, and segment routing). ExaBGP compat covers wire encoding for all supported address families.

Not yet covered by interop tests:

• Long-Lived Graceful Restart with live peers

Known Vendor Limitations

Related Documents

• .ci test format -- Ze's standard functional test file format
• Functional test system -- complete guide to the functional test system
• BGP implementation comparison -- feature matrix comparing Ze with FRR, BIRD, GoBGP, ExaBGP, and others
• ExaBGP comparison report -- detailed implementation differences between Ze and ExaBGP