BGP performance testing with Ze

AMS-IX used ExaBGP as a lightweight BGP route source. bgperf formalized the same idea with tester, target, and monitor containers. Ze can run that shape with ze-perf: Ze supplies the sender and receiver, the target is whatever BGP speaker you want to test, and the output is a repeatable report instead of a stopwatch and screenshots.

This page shows Ze in the tester role. The DUT can be FRR, BIRD, GoBGP, OpenBGPD, a route-server appliance, or Ze itself.

What a good test answers

A performance test has to answer these questions, and publish the evidence for each.

A good run proves a complete path: generated routes enter the DUT, pass through the DUT policy and RIB path, leave the DUT, and arrive at a real BGP receiver. A route count alone is wiring. A repeated run with zero loss and published inputs is evidence.

Topology

The test is three nodes: a sender and a receiver that Ze runs, with the device under test between them.

ze-perf sender  --->  DUT under test  --->  ze-perf receiver
AS 65001             AS 65000             AS 65002

ze-perf matches the prefixes the sender injected against the ones the receiver saw, so it can report convergence, throughput, latency percentiles, withdrawal time, and lost routes. In bgperf terms, the sender is the tester, the DUT is the target, and the receiver is the monitor.

Fastest working test

Use the existing Docker runner from the main Ze checkout when you want the whole setup built for you.

python3 test/perf/run.py --build --test frr
ze-perf report test/perf/results/*.json

Run a smaller wiring test first when you are adding a new DUT target.

ze-perf run \
  --dut-addr 172.31.0.2 \
  --dut-asn 65000 \
  --dut-name frr \
  --routes 10 \
  --repeat 1 \
  --warmup-runs 0

If 10 routes do not converge, fix the BGP sessions or DUT policy before running a scale test.

A 100k route run

This is the Ze equivalent of using ExaBGP as the injector in the original route-server tests.

ze-perf run \
  --dut-addr 172.31.0.2 \
  --dut-asn 65000 \
  --dut-name frr \
  --sender-addr 172.31.0.10 \
  --sender-asn 65001 \
  --receiver-addr 172.31.0.11 \
  --receiver-asn 65002 \
  --routes 100000 \
  --repeat 10 \
  --warmup-runs 2 \
  --iter-delay 3s \
  --output results/frr-100k.json

The sender and receiver addresses must be valid local addresses in the lab where ze-perf runs. The Docker runner handles that wiring. In a manual lab, bind those addresses on the host or run ze-perf in the right network namespace.

Then render the report.

ze-perf report results/frr-100k.json

Use --family ipv6/unicast for IPv6. Use --force-mp to encode IPv4 NLRI through MP_REACH_NLRI and test the DUT multiprotocol path. Use --batch-size 1 when you want one prefix per UPDATE instead of packed UPDATEs.

FRR as the DUT

This FRR config accepts the Ze sender and exports the received routes to the Ze receiver.

frr defaults traditional
hostname dut-frr
log stdout informational
!
route-map ACCEPT permit 10
!
router bgp 65000
 bgp router-id 172.31.0.2
 no bgp ebgp-requires-policy
 neighbor 172.31.0.10 remote-as 65001
 neighbor 172.31.0.11 remote-as 65002
 !
 address-family ipv4 unicast
  neighbor 172.31.0.10 activate
  neighbor 172.31.0.10 route-map ACCEPT in
  neighbor 172.31.0.11 activate
  neighbor 172.31.0.11 route-map ACCEPT out
 exit-address-family
!

Useful checks while debugging:

show bgp ipv4 unicast summary
show bgp ipv4 unicast 10.0.0.0/24

PfxRcd on the sender neighbor should reach the generated route count. The receiver side should receive the same count.

BIRD as the DUT

This BIRD 2 config imports routes from the Ze sender and exports them to the Ze receiver.

log stderr all;
router id 172.31.0.2;

protocol device {
}

protocol bgp ze_sender {
    local 172.31.0.2 as 65000;
    neighbor 172.31.0.10 as 65001;
    ipv4 {
        import all;
        export none;
    };
}

protocol bgp ze_receiver {
    local 172.31.0.2 as 65000;
    neighbor 172.31.0.11 as 65002;
    ipv4 {
        import none;
        export all;
    };
}

Useful checks:

birdc show protocols
birdc show route count

What Ze reports

ze-perf reports the fields that are easy to lose when the test is driven by an external injector.

Use JSON for anything you intend to publish.

ze-perf run --dut-addr 172.31.0.2 --dut-asn 65000 --routes 100000 --output result.json

What to publish with the number

Publish the method with the result.

How close Ze is to the full route-server test

Ze already covers the core of the test: sender, DUT, receiver, route count, convergence, loss, and repeatable output.

The current ze-perf shape is one sender session and one receiver session. That is enough to test route propagation and DUT table pressure. It is not the same as hundreds of independent BGP client sessions in one run.

If the target is the full AMS-IX or bgperf style route-server scale test, the useful Ze feature would be a multi-peer mode:

That would let Ze reproduce the full route-server topology while keeping Ze's timing and JSON reporting.

Prior art

AMS-IX published route-server performance slides at RIPE64 with 500 IPv4 peers, 128,000 prefixes, BIRD table-mode comparisons, an MTU test with 1012 IPv6 peers, and a note to check out ExaBGP as a lightweight tool to open BGP sessions and inject prefixes: AMS-IX Route-Server Performance Test.

Job Snijders' bgp-performance repository is a bgperf2-based performance suite. Its docs describe a target, monitor, and tester topology, and its MRT mode can use ExaBGP as an MRT injector: job/bgp-performance, how bgperf works, and MRT injection.

Lab evidence

Ze's automated performance evidence already wires sender, DUT, and receiver the same way.

Run the Ze performance suite from the main checkout:

python3 test/perf/run.py --build --test frr bird gobgp
ze-perf report test/perf/results/*.json

The full benchmarking guide is at docs/guide/benchmarking.