Guide

Traffic Usage

Ze accounts per-(port, protocol) and, optionally, per-IP byte totals on operator-selected interfaces using eBPF programs attached via TCX. The counters are exported as Prometheus metrics and viewed with show traffic usage. The trafficusage plugin loads only when a traffic { usage { } } section is present in the config.

Traffic usage is monitoring only. The eBPF programs read packet headers and update counters; they never drop, redirect, or modify traffic, and they leave each packet to the programs and tc filters that follow them on the interface, so a traffic-control policer, an sFlow sampler or a mirror on the same interface still runs. Accounting is IPv4 only. The plugin is Linux only and is a no-op on other platforms.

How It Works

The plugin attaches one ingress and one egress eBPF program to each configured interface using TCX (the kernel's traffic-control express attach point, kernel 6.6 and later). Each packet's IPv4 header is inspected and a byte count is added to per-key maps:

  • Per-port (always on): ingress traffic is keyed by destination port and protocol, egress traffic by source port and protocol. Packets with no L4 port (ICMP and other non-port protocols) aggregate under port 0.
  • Per-IP (opt-in via track-ip): ingress traffic is additionally keyed by source IPv4 address, egress traffic by destination IPv4 address. This is off by default to bound metric cardinality.

A user-space poller reads the maps on a fixed interval (interval), publishes the absolute byte totals as Prometheus gauges, and removes metric series that have not been seen within stale-timeout so that /metrics cardinality stays bounded. Each map is an LRU of capacity max-entries; when a map fills, the kernel evicts the least-recently-used keys, so the live keys are the current top talkers.

Pure-Go eBPF

The eBPF programs are assembled in pure Go as asm.Instructions using the github.com/cilium/ebpf library and loaded from memory. There is no C source, no committed .o object file, and no clang/LLVM toolchain in the build. This deliberately differs from the L2TP XDP-policing plugin, which commits bpf2go generated .o files. The cost of the pure-Go approach is hand-written assembly, which is validated by BPF_PROG_TEST_RUN tests that feed crafted packets through the loaded programs and assert on the resulting map state.

Configuration

All traffic usage configuration lives under a single traffic { usage { } } section.

traffic {
    usage {
        enabled true
        interfaces {
            interface eth0 {
            }
            interface wan {           // a ze interface name (resolved to its OS device)
                track-ip false        // override the global track-ip just for wan
                max-entries 65536     // and give it a larger top-talker map
            }
        }
        interval 1000
        track-ip true                 // global default, inherited by eth0
        stale-timeout 300000
        max-entries 10240
    }
}

Interfaces are listed under interfaces { }, keyed by name, mirroring the OSPF and IS-IS interface blocks. Each name is a ze interface name (the logical name configured under interface { }); it is resolved to its OS/kernel device (honoring the os-name / mac-match selectors) before the eBPF programs are attached.

track-ip, stale-timeout, and max-entries are global defaults that each interface inherits unless it sets its own value, so you can (for example) track top-talker IPs with a large map on the WAN while leaving the LAN to port/protocol counters only. interval is global (a single poll loop). Changing an interface's track-ip or max-entries rebuilds and re-attaches its eBPF program (its counters reset); changing interval takes effect on the next poll.

Field Type Default Range Description
enabled boolean false - Enable accounting.
interfaces / interface <name> keyed list - - ze interface names to account on, resolved to their OS devices.
interfaces / interface <name> / enabled boolean true - Account traffic on this interface.
interfaces / interface <name> / track-ip boolean global - Per-interface override of track-ip.
interfaces / interface <name> / stale-timeout uint32 (ms) global 0 disables Per-interface override of stale-timeout.
interfaces / interface <name> / max-entries uint32 global 1..4294967295 Per-interface override of max-entries.
interval uint32 (ms) 1000 100..3600000 Map poll interval in milliseconds (global).
stale-timeout uint32 (ms) 300000 0 disables Global default: remove a metric series unseen for this long.
track-ip boolean false - Global default: also account per source (ingress) / destination (egress) IPv4. Off by default to bound metric cardinality.
max-entries uint32 10240 1..4294967295 Global default: per-map LRU capacity; the kernel evicts the least-recently-used keys (top-talker retention) when a map fills.

Operational Command

show traffic usage [name <interface>] reports the accounted byte totals. With no argument it lists every monitored interface; with name <interface> it returns one interface. Output is structured per interface: ingress-ports, egress-ports, and (only when track-ip is enabled) ingress-ips and egress-ips, plus map-entries reporting per-map fill levels.

ze cli -c 'show traffic usage'
ze cli -c 'show traffic usage name eth0'

The command answers structured data, which ze cli -c renders in the format environment cli format default names. The registered value is text. Append | json, | table, | yaml or | ndjson to select another one, and the full set of pipe operators applies.

The terminal lab enables accounting on traffic0 through a live SSH commit. Real ICMP and HTTP requests establish a baseline for source 10.77.0.2 and TCP destination port 8080. A heavier burst must increase those specific byte totals, not merely create more map entries. The validator also requires positive exported metrics filtered to that source, TCP/8080 and ICMP/0. The lab sizes both finite send counts for the recording container's one-hour budget: ICMP runs at 10 requests per second with a matching wall-time bound, and HTTP at no more than 20 requests per second with a two-second limit per request. The visible stop control terminates only the owned sender PIDs, leaving the HTTP server and Ze running; lab cleanup also owns those senders. After the stop, the validator requires a stable two-second window within ten seconds. The totals remain cumulative: stopping traffic does not reset them to zero.

Ze terminal lab

Attribute a live traffic burst

Enable eBPF accounting on the running router, attribute an unexpected ICMP and HTTP burst to its source and port, stop the sender, and observe recovery.

Reproducible
Ze 26.10.283 minutes 9 secondsLinux namespace labTerminalZe recorder Plain-text transcript
Read the demonstration transcript
Traffic usage live attribution walkthrough

Lab: traffic0 at 10.77.0.1, real sender 10.77.0.2 and HTTP server port 8080.
$ ze-demo card topic-traffic overview
$ cat /src/tmp/terminal-demos/state/showcase-lab/ze.pid
$ ze cli -c 'show traffic usage | no-more'

Load: bounded eBPF accounting and traffic0 attachment in one SSH candidate.
$ sshpass -e ssh ze-demo
ze# load file absolute merge /src/demos/terminal/topics/traffic/1-accounting.conf
ze# load file absolute merge /src/demos/terminal/topics/traffic/2-interface.conf
ze# show | compare
ze# commit now

Proof: real ping and curl populate source, ICMP and TCP/8080 byte totals.
Long shell commands below quote their first painted-row prefix, not invented output.
$ ze-demo card topic-traffic proof
$ ip netns exec traffic-peer ping -c 3 -W 2 10.77.0.1
$ ip netns exec traffic-peer curl --fail --silent --show-error --max-time 5
$ ze cli -c 'show traffic usage name traffic0 | no-more'

Failure: bounded heavier senders increase those specific totals in successive snapshots; filtered metrics identify the source.
$ ze-demo card topic-traffic failure
$ ze-demo run showcase-lab control traffic burst
$ ze cli -c 'show traffic usage name traffic0 | no-more'
$ ze cli -c 'show traffic usage name traffic0 | no-more'
$ ze cli -c 'show metrics name ze_traffic_usage_ingress_bytes_total label interface traffic0 label src_ip 10.77.0.2 | no-more'
$ ze cli -c 'show metrics name ze_traffic_usage_ingress_port_bytes_total label interface traffic0 label dst_port 8080 label protocol tcp
$ ze cli -c 'show metrics name ze_traffic_usage_ingress_port_bytes_total label interface traffic0 label dst_port 0 label protocol icmp

Fix: stop the senders and require cumulative totals to settle for two seconds within ten seconds, without resetting counters or replacing Ze.
$ ze-demo card topic-traffic fix
$ ze-demo run showcase-lab control traffic stop
$ ze cli -c 'show traffic usage name traffic0 | no-more'
$ ze cli -c 'show traffic usage name traffic0 | no-more'
$ cat /src/tmp/terminal-demos/state/showcase-lab/ze.pid
$ ze-demo card topic-traffic recap

Prometheus Metrics

The plugin registers the following metrics. All are GaugeVec carrying absolute cumulative byte totals.

Metric Type Labels When Description
ze_traffic_usage_ingress_port_bytes_total GaugeVec interface, dst_port, protocol always Ingress bytes per destination port and protocol.
ze_traffic_usage_egress_port_bytes_total GaugeVec interface, src_port, protocol always Egress bytes per source port and protocol.
ze_traffic_usage_ingress_bytes_total GaugeVec interface, src_ip track-ip only Ingress bytes per source IPv4.
ze_traffic_usage_egress_bytes_total GaugeVec interface, dst_ip track-ip only Egress bytes per destination IPv4.
ze_traffic_usage_map_entries GaugeVec interface, map always Live entry count per BPF map (LRU fill level).

The protocol label is one of tcp, udp, icmp, gre, esp, ah, or icmpv6; unknown protocols render as their decimal protocol number. ICMP and other packets with no L4 port aggregate under dst_port/src_port 0.

Doctor Check

The doctor-traffic-usage-ebpf check warns when traffic-usage is enabled in the config but eBPF/TCX is unavailable on the running kernel, so the operator learns that accounting is configured but inert rather than silently producing no metrics.

The check probes TCX only when traffic-usage is enabled. It runs in the process that answers doctor: the running daemon when one answers ze doctor or show doctor, and your own ze doctor process only when no daemon answers. When the probe cannot measure readiness (that process lacks CAP_BPF or CAP_NET_ADMIN), it does not guess: it warns under doctor-traffic-usage-ebpf-unknown with the reason, and its fix is sudo systemctl stop ze && sudo ze install systemd --force --start. When the daemon answered, the daemon is what lacks the capability: under systemd a ze unit written by an older ze does not grant CAP_BPF, and that one line stops ze, which the install needs, rewrites the unit and starts ze again; without systemd, start ze with CAP_BPF and CAP_NET_ADMIN. When no daemon answered, the answer also carries doctor-runtime-not-checked, and sudo ze doctor measures the host as root. doctor-traffic-usage-ebpf is kept for a kernel that lacks TCX, and names the Linux 6.6 and CONFIG_NET_XGRESS requirement.

Limitations

  • IPv4 only. Per-port and per-IP accounting cover IPv4 traffic; IPv6 is not counted.
  • Monitoring only. The eBPF programs only read headers and update counters. They never drop, redirect, or modify traffic, and they never end the interface's tc hook: the filters after them still run.
  • Per-IP requires track-ip. The ze_traffic_usage_ingress_bytes_total and ze_traffic_usage_egress_bytes_total series are only produced when track-ip is enabled. Per-port accounting is always on.
  • Linux >= 6.6, no-op elsewhere. TCX attach requires Linux 6.6 or later, plus CAP_BPF and CAP_NET_ADMIN. On other platforms the plugin is a no-op. The doctor-traffic-usage-ebpf check reports when the kernel cannot support the feature.
  • Bounded cardinality. Stale metric series (unseen within stale-timeout) are deleted, and each map is an LRU capped at max-entries; under high key churn only the current top talkers are retained.
  • IPsec bare ESP can be counted as UDP to port 4500. On a link where Ze's IPsec ESP form fast path runs, the program counts some ingress ESP under udp; Bare ESP counted as UDP to port 4500 says when, and how to read the figures.

Bare ESP counted as UDP to port 4500

A Child SA on UDP port 4500 receives ESP in both forms, and Ze's ESP form fast path turns the peer's bare ESP into ESP in UDP at tc ingress, before XFRM decrypts it (Both ESP wire forms). The fast path attaches its program at the head of the interface's TCX chain. The traffic-usage program attaches behind it, so it counts the datagram after the rewrite: in ze_traffic_usage_ingress_port_bytes_total, bare ESP for that Child SA is counted under protocol="udp" and dst_port="4500", not under esp, and each datagram counts 8 bytes more, the UDP header the rewrite added.

Only ingress is affected, and only while the fast path rewrites: ESP for an SPI no Child SA on UDP port 4500 uses, and all bare ESP while the fast path is off (show doctor says when it is), stays under esp. The per-IP series are keyed by the peer's address either way. Ingress UDP to port 4500 from an IPsec peer then holds both its IKE messages and its ESP. The Child SA's own counters, bytes-in and packets-in in ze show vpn ipsec sa, count its ESP in either form.