Feature

Interface Management

Ze manages Linux network interfaces via pure netlink (no iproute2 shell-outs). JunOS-style two-layer model: physical interfaces with named logical units.

Interface operation journals survive the inner operation commit. A later owner failure or outer candidate-publication failure replays their original inverses; whole-reload acceptance releases them through the acknowledged config-complete callback. Config restoration also restores the active interface configuration and reconciles its DHCP, PPPoE and Router Advertisement workers.

Capability Table

Category Feature Status Priority
Interface Types Ethernet (physical) have
Dummy (virtual) have
Veth pairs have
Bridge (basic) have
VLAN 802.1Q have
VLAN 802.1p QoS maps (ingress/egress PCP-priority) have
Class-of-service named profiles (802.1p, interface-level inheritance) have
Loopback have
Bonding / LACP missing high
VXLAN (a tunnel encapsulation, netlink and VPP) have
GRE / GRETAP / IPIP / SIT tunnels have
IP6GRE / IP6GRETAP / IP6TNL / IPIP6 tunnels have
ERSPAN missing lower
WireGuard (declarative peers, $9$-encoded keys) have
MACsec missing lower
MACVLAN missing lower
Geneve missing lower
PPPoE client (pppoe-client; the session device carries the configured name, so routes, firewall and NAT name it directly) have
L2TPv3 missing lower
WiFi missing lower
XFRM (route-based IPsec, if_id) have
VTI (legacy IPsec tunnel) missing lower
QinQ (802.1ad) missing lower
Logical Model Two-layer physical + unit have
Unit 0 implicit have
VLAN units create subinterfaces have
Lifecycle Create (dummy, veth, bridge, VLAN, tunnel) have
Delete have
Auto-up on create have
Admin state control (explicit up/down) have
Interface rename missing lower
Address Management Add/remove IPv4/IPv6 CIDR have
DAD-aware monitoring have
MAC address set/get have
Gratuitous ARP on add missing medium
Neighbor table (ARP/NDP) missing lower
DHCP DHCPv4 (RFC 2131, full DORA) have
DHCPv6 (SARR, IA_NA, IA_PD) have
Concurrent v4+v6 have
Direct netlink install have
DHCPv4 address lifetime equals the lease, refreshed by every renewal; an infinite lease (0xffffffff) installs as infinite and is not renewed have
DHCPv4 lease kept for as long as a server answers: RENEW unicast to the leasing server at every T1, REBIND broadcast to any server at T2, both retransmitted on the RFC 2131 Section 4.4.5 schedule (half the time left, at least 60 s); the address goes only when the lease expires. Deadlines count from the instant the REQUEST was sent have
DHCPv4 T1 and T2 from the server (options 58 and 59) when T1 < T2 < lease, otherwise the defaults (half and seven eighths of the lease) have
DHCPv4 DHCPNAK to a RENEW or REBIND removes the address and route and restarts DORA at once. The RENEW is sent from the leased address but its answer is read from the device, so a DHCPNAK the server broadcasts is seen as well as a DHCPACK it unicasts have
DHCPv4 ACK without a lease time (option 51), or with 0, is refused: nothing is installed and DORA retries after 10 s; during a renewal it counts as no answer. The Warn names the server and says option 51 is missing have
DHCPv4 DORA, RENEW and REBIND open their sockets on the kernel device the interface name resolves to (os-name, mac/match) have
DHCPv4 RENEW and REBIND carry the same client-id (option 61) and hostname (option 12) as the DORA REQUEST have
DHCPv4 lease registered as leased, so a commit keeps it; iface-dhcp refuses to run as an external plugin, because the registry is a same-process call have
Bus events (acquired/renewed/expired) have
Config-driven (dhcp { enabled true }) have
Default route from DHCP Router option; a failed install is retried while the lease is held (5 s, doubling to 60 s), and a renewal naming another router, or none, removes the old route have
Route priority per unit (route-priority) have
Link-down route deprioritization (metric + 1024) have
IPv6 default route from RA with configurable metric have
Routes carry proto 253 (ze-iface), so a teardown removes only its own have
Learned default routes rank at metric 254, clear of a static route's metric have
DNS from DHCP and from PPPoE IPCP (RFC 1877) to, tracked per lease and ordered DHCPv4, DHCPv6, PPP: a renewal or renegotiation without DNS, or a lease or session end, removes that lease's servers only. /tmp/resolv.conf have
Hostname in DHCPv4 (option 12) have
Client-ID in DHCPv4 (option 61) have
DHCPv4 flags bits 1 to 15 sent as zero, even when the server's OFFER or ACK sets them (RFC 2131 section 2) have
NTP servers from DHCP (option 42) have
DHCPv6 proper Renew (not re-solicit) have
DHCP relay missing lower
DHCPv4 server (RFC 2131) on each listen-interface, with pools, static mappings and PXE options have
DHCPv4 server: several subnets on one interface (shared-network), a static mapping in any subnet of the segment before any pool, new addresses from the subnet of the interface's primary address first; a listener answers only for the shared networks its interface is addressed in have
DHCPv4 server identifier (option 54) is the interface's own address in the client's subnet, the interface's primary address for a relay-only subnet, or the server-identifier leaf while the interface holds it; a REQUEST naming any of the server's addresses on the segment is accepted (RFC 2131 section 4.1) have
Monitoring Netlink multicast (link + addr + neigh) have
Virtual iface state detection have
11 bus topics, JSON payloads have
Interface statistics/counters have
Persistent counter tracking missing medium
Per-Interface Tuning IPv4 forwarding have
ARP filter / ARP accept have
IPv6 autoconf (SLAAC), netlink backend only have
IPv6 accept-ra (0/1/2), netlink backend only have
IPv6 forwarding have
Proxy ARP have
ARP announce / ARP ignore have
RPF / source validation (IPv4 rp_filter sysctl, plus VPP uRPF on backend vpp; IPv6 nftables fib lookup in ip6 ze_iface_rpf, netlink backend only) have See "Reverse-path check"
TCP MSS clamping (v4+v6) missing medium
Directed broadcast missing lower
IPv6 Extended EUI-64 address generation missing medium
DAD configuration (messages, accept) missing medium
Custom interface identifiers missing lower
Router Advertisement RA sender per unit (RFC 4861) have
Prefix Information options for SLAAC have
RDNSS resolver option (RFC 8106) have
Router Solicitation answers with rate limit have
Route Information option (RFC 4191), for the delegated prefix of a unit that numbers from one have One option per delegated prefix, with the delegation's remaining valid lifetime; see "Send loop"
A LAN /64 from a DHCPv6 delegated prefix, with lifetimes bounded by the delegation (RFC 7084 L-2, RFC 9915 Section 6.3) have ipv6 delegated-prefix <pppoe-client> { subnet-id; interface-identifier }; the source must be a pppoe-client whose dhcpv6 asks for a prefix ("PPPoE client IPv6") . docs/guide/pppoe.md
CE Router Lifetime: 0 without a WAN default route or a prefix, and an immediate zero-lifetime RA when either is lost (RFC 7084 G-4, G-5, L-4) have Applies only to a unit with delegated-prefix; see "Send loop"
Traffic Mirroring Ingress/egress via tc mirred have
Idempotent setup/cleanup have
Traffic redirect (vs mirror) missing lower
Traffic Control QoS / shaping missing lower
Policing missing lower
Queuing disciplines missing lower
Migration Make-before-break 5-phase have
BGP readiness signaling have
Per-phase rollback have
BGP Integration Reactor subscribes to addr events have
Listener start/stop on addr change have
bgp/listener/ready publish have
Bridge Features Create and bring up have
STP have
VLAN filtering missing medium
Add/remove member ports have
Multicast snooping missing lower
Port isolation missing lower
Ageing/forward delay/hello/max age missing lower
Bonding Mode selection missing high
Hash policy missing high
LACP rate missing high
MII monitoring missing high
Min active links missing high
Member management missing high
VRF YANG leaf exists ∿ high
Route table isolation missing high
Per-VRF address assignment missing high
VRF-aware DHCP missing medium
Gateway Redundancy VRRP / keepalived have
Virtual MAC have
State monitoring/failover missing medium
Physical Layer Speed / duplex / autoneg missing medium
Hardware offload (GRO/GSO/TSO/LRO) missing medium
Ring buffer sizing missing lower
RPS / RFS missing lower
ethtool integration missing medium
Security 802.1X / EAPoL missing lower
Storm control missing lower
Configuration YANG model (all types + units) have
Input validation (name, VLAN, MTU) have A pppoe-client key is the kernel device name, so it is 1 to 15 octets (IFNAMSIZ), starts with a letter, holds only letters, digits, _, . and -, and may not repeat the key of any other configured interface of any kind
Platform Pluggable backend interface have
Linux netlink backend (default) have
YANG backend leaf (config-driven selection) have backend vpp starts the VPP component with the interface plugin. A build without ze_vpp refuses it when the config is validated, committed or booted, naming the leaf and the tag, rather than at apply
VPP backend (ifacevpp, via GoVPP) have ResetCounters via sw_interface_clear_stats; ListKernelRoutes via ip_route_v2_dump; RouteLookup via IPRouteLookupV2 (VPP FIB is authoritative); ListNeighbors via ip_neighbor_dump; VXLAN/GRE/IPIP/LCP/stats-socket/mirror/STP pending third-party vendoring
macOS / Darwin missing lower
FreeBSD / OpenBSD missing lower
systemd-networkd missing lower
Quality Context-wrapped errors have
Panic recovery have
14 test files (unit + integration) have

Creating a device

Each apply creates the devices the config names: dummy, veth, bridge, tunnel, wireguard, xfrm, and the VLAN of each unit with a vlan-id. Whether a device exists is read before its create, never after, and it is read from the link alone (GetLink), as are the MTU and admin state the undo puts back: a failed address read cannot make a present device read as absent. Only the backend's "no such device" answer is absence: a link read that fails (a socket that does not open, ENOBUFS, a dump interrupted on every retry) fails the commit, naming the device and ze show interface <name>, rather than reading as absent and creating a present device again. A device that existed is kept, and the undo of a failed apply leaves it in place. A create that fails for a device that did not exist fails the commit, even when the backend left a partial device behind, and the undo deletes only the devices the apply created.

Unnumbered IPv4 units

A unit can borrow the IPv4 address of another interface instead of holding a subnet of its own (Cisco ip unnumbered, Junos unnumbered-address):

interface {
	loopback { unit 0 { ipv4 { address [ 192.0.2.1/32 ]; } } }
	ethernet eth0 { unit 0 { ipv4 { unnumbered loopback; } } }
}

The value names the donor, any interface in the config, or loopback for the loopback block. Ze installs the donor's address on the unit as a /32, the shape FRR and the Linux kernel use for an unnumbered point-to-point link. When the donor holds several IPv4 addresses, unnumbered-source <a.b.c.d> picks one; without it ze takes the donor's first address, in unit name order and then in address order (Junos preferred-source-address).

The commit is refused when the unit also sets address or enables dhcp, when the donor is not in the config, is disabled, is itself unnumbered or holds no IPv4 address, when unnumbered-source names an address the donor does not hold, and when unnumbered-source is set without unnumbered.

The /32 is a copy of the donor's address, so Ze keeps it only while the donor holds that address on a device that is up. When the donor goes down or loses the address, Ze removes the /32 from the unnumbered unit; when the donor recovers, Ze installs it again. Every interface link or address event, and every config apply, re-checks this. A protocol on the unit sees the address go: OSPF takes the link out of service until the address returns.

Reverse-path check

Each apply builds the complete set of reverse-path checks the config asks for, one per unit and address family, and hands it to the backend whole: ReplaceIPv4RPF and ReplaceIPv6RPF. A check whose leaf, unit or interface left the config is absent from the set, so the backend removes it, and the apply journal restores the previous set when a later step fails.

nftables refuses even the removal of ip6 ze_iface_rpf to a process without CAP_NET_ADMIN. When the config asks for no IPv6 check, the netlink backend therefore accepts that refusal with a warning, so an unprivileged ze still applies its interfaces. A config that asks for an IPv6 check still fails without the capability.

The table name is reserved for the reverse-path check: a firewall table named iface_rpf (kernel name ze_iface_rpf) is refused at verify, naming the owner iface-rpf, because the firewall backend and the reverse-path check would each delete and recreate the shared table.

Family netlink backend vpp backend
IPv4 net.ipv4.conf.<dev>.rp_filter, written by the sysctl phase; ReplaceIPv4RPF installs nothing more urpf_update on the receive path of the VPP interface (strict or loose), read back with urpf_interface_dump so a check no longer configured is turned off. The sysctl still reaches the Linux Control Plane tap, which judges only what VPP punts to the host
IPv6 nftables table ip6 ze_iface_rpf refused at commit (ze:backend "netlink") and by the backend

VPP's uRPF (urpf_plugin.so, always enabled in the startup.conf ze writes) passes a packet to a multicast destination or to 255.255.255.255, so DHCP from 0.0.0.0 passes. Loose accepts a source whose route has a real path, a default route included, and drops one that only VPP's drop default covers. IPv6 is not offered on VPP because VPP's fe80::/10 entry carries an empty uRPF list: measured on VPP 26.06 with the packet generator, strict and loose both drop a unicast packet from a link-local source, which would break neighbor discovery, DHCPv6 and link-local BGP. An external VPP without the plugin accepts a config with no IPv4 rpf-check and refuses one that sets it, naming the plugin.

Architecture

Config (YANG: ze-iface-conf.yang, "backend" leaf selects backend)
  |
  v
iface component (register.go) -- OnConfigure() loads backend, starts monitor
  |
  v
Backend interface (backend.go) -- 45 methods: lifecycle, address, route, bridge, mirror, monitor
  |
  v
+------------------+--------------------+
|                  |                    |
netlink backend    DHCP plugin          (future: networkd, FreeBSD)
(ifacenetlink/)    (ifacedhcp/)
|                  |
netlink calls      lease negotiation
|                  |
v                  v
Bus topics -- interface/{created,deleted,up,down,addr/*,dhcp/*}
  |
  v
Subscribers -- BGP reactor (starts/stops listeners on address changes)

Interface Discovery

During ze init, Ze discovers OS network interfaces and generates initial configuration entries. The DiscoverInterfaces function enumerates interfaces via ListInterfaces (netlink on Linux, stdlib on other platforms) and classifies each by Ze type: ethernet, bridge, veth, dummy, or loopback. On Linux, the netlink device type maps to ethernet (except lo, which maps to loopback). Results are sorted by type then name.

A managed client (ze init --managed) and an appliance seed (ze init --seed) skip discovery and write no initial config: the managed client takes its first config from its hub, and the appliance discovers on the device at first boot.

The generated config uses descriptive names as YANG list keys (the OS interface name at discovery time). The MAC address serves as the physical binding between configuration and hardware. For ethernet, veth, and bridge interfaces, the MAC address (mac { address }) is optional and, when set, must be unique within each list. Omit it to keep the hardware-assigned MAC, or set it to override the address and pin the named config entry to a specific physical device.

Discovery writes one selector per entry. An ethernet that reports a factory address (IFLA_PERM_ADDRESS) is bound by mac { match } against that address, so the entry follows the NIC across a kernel rename. Every other discovered kind records an os-name hidden leaf holding the original OS interface name, which remains available for debugging and internal binding after the user renames the config entry. A discovered ethernet gets no mac { address } override: that leaf imposes an address on whatever device the entry resolves to, which is a wrong write the moment the name reaches a different port.

Operator-facing interface operations resolve the logical name to its kernel device through the shared resolver before touching the kernel, so a configured name that differs from the OS device (via the os-name or mac/match selector) is honored uniformly: the iface CLI ops (set MTU, add/remove address, admin up/down, bridge, mirror, ...), the DHCP client socket binding, and the routing/protocol consumers all act on the bound kernel device. The dispatch layer performs this translation for the by-name backend ops and leaves GetInterface and ListInterfaces raw because the resolver is built on them. ./le arch iface-resolution check keeps new consumers from resolving the kernel directly instead of through the resolver.

The config apply path resolves separately, and on purpose. It takes ONE interface listing per apply and binds every ethernet entry from it, so each logical name resolves once and Phase 2, Phase 2c and the address reconcile all key their work by the same kernel device. Going through the dispatch wrappers instead would translate at each of about twenty call sites and still miss the sites that take a device string rather than make a backend call: the per-interface sysctl keys and the ethtool offload ioctls.

An entry whose selector names no present device is unbound: every phase skips it, and none falls back to the logical name. That fallback is what made an aliased entry configure whatever else carried its name. An entry whose mac/match names more than one device refuses the apply, and a device only answers a mac/match when the address it is matched on is its own: a VLAN inherits its parent's, and a bridge or bond wears a member's, so neither is a candidate. ResolveDevice draws the same line for the by-name dispatch ops: a name with no selector passes through unchanged, and a name WITH a selector that fails to resolve returns an error rather than the name.

The two settings that name a SECOND interface resolve through the same map: a bridge member is enslaved by the kernel device its entry selects, and a mirror sends its copy to the capture port's kernel device.

Created kinds need no resolution, because Ze makes the device under its configured name: dummy, veth, bridge, tunnel, WireGuard, XFRM, and the pppoe-client session device. The PPPoE client creates its PPP unit under the list key on every session and every reconnect, so pppoe-client pppoe0 is the kernel device pppoe0 and never a pppN. A static route next { interface pppoe0 }, a firewall output-interface "pppoe0" match and a masquerade rule therefore reach the session. The device exists only while a session is up. A static route through it waits for the device and follows it across a reconnect (docs/guide/static-routes.md, "Interface-only next-hops").

A MAC address validator (ze:validate "mac-address") provides format checking (colon-separated hex octets) and live OS autocomplete. The CompleteFn calls DiscoverInterfaces on each tab press, returning MAC addresses from currently active OS interfaces. The os-name selector carries the sibling validator ze:validate "os-device-name", which screens the FORM of a kernel device name and completes to the device names present on the box. It does not check that the device exists: the YANG promises a binding that defers until its device appears, and a config is validated on machines that will never run it.

Key Design Decisions

  • Descriptive names as keys, MAC as binding. Interface names in config are user-chosen descriptive labels. The MAC address ties the config entry to physical hardware. This separates the logical identity (name) from the physical identity (MAC).
  • Pluggable backends. The iface component defines a Backend interface (34 methods). OS-specific operations live in backend plugins (ifacenetlink for Linux). The YANG backend leaf selects the backend (default: netlink). DHCP is a separate plugin (ifacedhcp) that uses the backend for address operations.
  • Pure netlink, no shell-outs. The netlink backend uses github.com/vishvananda/netlink.
  • Event-driven. Monitor publishes to bus; consumers subscribe and react. No polling.
  • Per-family addresses. Addresses live inside ipv4 { address [...]; } and ipv6 { address [...]; } containers within each unit, following the Junos/Nokia model.
  • DAD-aware. IPv6 addresses with IFA_F_TENTATIVE flag are held until DAD completes.
  • The config owns its addresses, not every address on the unit. Every commit, VPP connect and owned-address change reconciles each enabled unit: it adds the addresses the config and the owned-address registry name, and removes the other addresses on the unit, such as one an operator added by hand or one a dropped config line left behind. Three kinds stay in place. A DHCPv4 lease stays because its client registers it as leased while it holds it, finite or infinite: an infinite lease (RFC 2131 lease time 0xffffffff) is installed with the kernel's infinite lifetime, which makes it permanent like a static address, so only the registration tells the two apart. The reconcile never installs a leased address and never changes its lifetime, so a lease its client removed is not put back. A DHCPv6 lease stays the same way: its client registers every leased address, finite or infinite, and installs it with exactly the lifetimes the lease carries. An address with a finite valid lifetime stays: a SLAAC address. Its owner renews it and the kernel expires it. An IPv6 link-local address the replaced config did not name stays, because the kernel generates one on every IPv6 link (RFC 4291 Section 2.1). A link-local the operator configured and then dropped is removed.

The DHCPv6 client follows RFC 9915. It acquires a lease with Solicit, Advertise, Request and Reply, or with a Rapid Commit Reply. It extends the lease with Renew to the leasing server at the earliest T1 of its IAs and with Rebind to any server at the earliest T2, for as long as a server answers. When every valid lifetime has run out it removes the addresses, publishes one dhcp-expired per lease (a delegated prefix included) and solicits again. Each exchange retransmits on the RFC 9915 Section 15 schedule, with Solicit backing off to SOL_MAX_RT (3600 s, or the value a server sends between 60 and 86400) and never stopping, and at most 20 messages in any 20 s per interface. One Warn reports an outage, and retransmissions log at Debug. A valid lifetime of 0 removes the lease; an address with a preferred lifetime above its valid lifetime, or an IA with T1 above T2, is discarded. With pd set, every message carries an IA_PD, with a prefix-length hint when pd { length } is set. Every message carries one stable DUID: the duid leaf when set, otherwise a DUID-LL built from the device MAC. A DUID-LL is a known deviation from RFC 9915 Section 11.4, which advises against it when the client cannot tell the interface is permanently attached; set duid where that matters. A duid is colon-separated hex octets, the 2-octet type code then 1 to 128 octets of identifier (RFC 9915 Section 11.1), and a commit refuses any other form. Ze changed from a fresh time-stamped DUID per Solicit to this stable one, so a server hands out one new binding once after the upgrade. On stop the client removes its addresses, then sends a Release and waits at most 3 s for the answer. A delegated prefix a LAN unit took a /64 from is left out of that Release, and when nothing else is held no Release is sent: RFC 9915 Section 6.3 forbids releasing a delegated prefix while any of its space is outstanding.

The kernel runs duplicate address detection on every leased address the client installs. The client reads the outcome while the address is tentative: when the kernel marks the address dadfailed, or deletes it (what it does to an address of finite lifetime whose DAD failed) while the link is up, the client removes it from the lease, logs a Warn naming the address, and sends a Decline for it to the server that leased it. The other bindings are kept, and when nothing is left the client solicits again. Whether the link is up is read from the kernel, not only from link events, because the kernel deletes the addresses of a device set down before the client can receive the down event. The kernel backend reads the link before the addresses, so an address found gone with the link up is declined only when a second read, whose link state is read after those addresses, still reports the link up. A device the kernel reports down deleted the address with the link, so nothing is declined, and the next up counts as a move. When the backend reports a link state that is neither up nor down, the client declines nothing, logs one Warn for each such state naming it and the interface, and reads again at the next DAD poll. A read that fails declines nothing either. The VPP backend reports interface addresses but not their DAD outcome, because VPP's address dump carries no DAD state. On it the client cannot read a DAD outcome, so it does not take an unflagged address as unique: it declines nothing and logs one Warn saying so. When the bound device goes down and comes back up, the client may have moved to a new link: with a delegated prefix held it sends a Rebind for every lease, on the Confirm timers (RFC 9915 Section 18.2.12), at most once every 30 s, and it installs every leased address again with what is left of its lifetimes, because the kernel deletes them when the interface is set down, then runs DAD on them again. A lease of addresses alone sends a Confirm instead, after a random delay of up to 1 s and under the same 30 s limit, carrying every leased address with its lifetimes set to 0. A Reply with Success, stated or implied by a missing Status Code, keeps the lease, and so does no answer at all. When every Reply that arrives in a listen window says NotOnLink, the client removes every leased address, publishes one dhcp-expired for each, and solicits again.

  • A VLAN unit carries its parent's MTU. A unit has no mtu leaf, so every commit sets each VLAN sub-interface of an ethernet entry with an mtu to that MTU, and a VLAN already at it gets no call. Neither backend does it alone: Linux gives a VLAN its parent's MTU when it is created and keeps it when the parent's later rises (it caps it when the parent's falls), and VPP rewrites a sub-interface's MTU only when its parent's link MTU changes, not when the parent takes a protocol MTU alone. An entry with no mtu leaf leaves its VLANs alone; each backend still creates a VLAN at its parent's MTU. A commit that fails after changing a parent's MTU puts the parent's old MTU back first, then the MTU each existing VLAN held before the commit: Linux caps a VLAN when its parent falls and refuses a VLAN above its parent, so a VLAN restored first, or read after the parent changed, would stay capped.
  • An omitted mtu leaves the device's MTU alone. The schema declares no MTU default, because each kind has its own: a device ze creates keeps the kernel's value, 1500 for a dummy, veth or bridge, 1420 for wireguard, and the underlay MTU less the encapsulation for a tunnel (1476 for gre).
  • The first configured address is the primary. A unit's address leaf-list is installed in the order it is written, IPv4 before IPv6, so the first address is added first and the kernel lists it first. That address is the interface's primary: the DHCPv4 server takes the segment's home subnet from it, and source-address selection prefers it. Every install path keeps the order: the commit's reconcile, the reconcile after a VPP connect or a registry change, the additive pass a not-ready backend takes, and the transaction's add-address operations. Addresses a plugin registers follow the configured ones. A reconcile adds only the addresses missing from the device, so reordering the leaf-list of a live interface does not move an address that is already installed.
  • Make-before-break. Migration adds new address, waits for BGP readiness, then removes old. Prevents session loss during address moves.
  • Same-subnet renumbers keep working. Because the new address is added first, a renumber inside one subnet (10.0.0.1/24 -> 10.0.0.2/24) briefly leaves the new address as a Linux IPv4 secondary of the old one, and Linux deletes every secondary of a subnet along with its primary. Before such a removal the netlink backend enables net.ipv4.conf.<device>.promote_secondaries so the kernel promotes a secondary instead of flushing the subnet, logging enabled promote_secondaries when it changes the knob. If the knob cannot be set the removal fails, naming the addresses that would have been destroyed, rather than silently emptying the interface. The knob is left enabled: restoring it would re-arm the same hazard on the next removal. IPv6 has no primary/secondary distinction and is untouched, as is the VPP backend, which deletes exactly the requested address.
  • Point-to-point address removal. RemoveAddress selects by local CIDR and retains the installed peer metadata for deletion. RemoveAddressP2P requires the exact installed local/peer pair before deletion. A successful dump proving that pair absent completes removal without a delete call. A failed dump or lookup remains an error. IPv4 deletion also accepts the kernel's EADDRNOTAVAIL when the pair disappeared after the dump. Requests and installed addresses use the same normalization, including equal IPv4 endpoints. The primary/secondary guard uses the selected pair and its peer subnet to protect other addresses before deletion.
  • Virtual interface state. Dummy/bridge/veth report OperUnknown not OperUp; monitor checks IFF_UP flag as fallback.
  • Tunnel encapsulation as YANG choice/case. The tunnel list at the iface level is one container with an encapsulation choice that branches into one case per Linux netlink kind (gre, gretap, ip6gre, ip6gretap, ipip, sit, ip6tnl, ipip6). Per-case leaf sets are constrained by the schema: key only appears in GRE-family cases, hoplimit/tclass/encaplimit only in v6-underlay cases. Local and remote endpoints use the same local { ip ... } remote { ip ... } shape as the BGP peer connection block, with local { interface ... } as an alternative when the source should be taken from another interface. The remote is mandatory and must be a specific address, on every tunnel kind: config verify refuses 0.0.0.0 and :: (and a remote that is not an address), so ze config validate and a commit refuse it on either backend, naming the set interface tunnel <name> encapsulation <kind> remote ip <address> command that fixes it. The decapsulator matches the outer source address against the remote, and an unspecified remote accepts packets from any source (RFC 4213 Section 3.6 requires the decapsulator to verify the tunnel source). The netlink and VPP backends run the same check again before they program a tunnel.
  • Idempotent cleanup. Delete and mirror removal succeed even if already gone.
  • The mirror shares its qdisc, so it owns only its filters. Mirroring attaches a tc mirred filter at priority 1 to the qdisc at handle ffff: on the source interface. That qdisc is a shared attachment point: flow-export sampling attaches its own filter at priority 100 to the same object, and both hooks, ingress and egress, hang off it. So mirror setup accepts a qdisc another subsystem created (EEXIST is not an error), and mirror teardown deletes its own filters and leaves the qdisc standing, empty. A teardown cannot know who created a shared qdisc, and the set of filters on it can change between the moment it is read and the moment it is acted on. An empty qdisc carries no filter and classifies no packet, and the next mirror or sampling setup adopts it. Only the rollback of a failed setup deletes a qdisc, because it created that qdisc moments earlier and knows so. Ze always creates clsact, never the older ingress qdisc, because clsact carries both hooks and can therefore serve a mirror with a different destination per direction. tc qdisc show on an interface that once mirrored reports a clsact qdisc with no filter on it, which is expected.
  • A mirror the config drops is removed. applyConfig compares the mirrors the new config asks for against the mirrors the previous config installed, and removes every one that was dropped or changed before it installs the new set. A changed destination is a remove followed by an install, because tc filters are additive: installing the new destination would otherwise leave the old one duplicating traffic.
  • The reconcile reads the dataplane, so a restart retires a stranded mirror. A second pass runs inside the config reconcile. It compares the mirrors the dataplane carries against the mirrors the configuration asks for, so it needs no previous config. A mirror the operator deleted while ze was down is therefore removed on the next boot. So is a mirror an earlier apply failed to remove. A backend that cannot read its live mirrors removes nothing, because "the read failed" is not "no mirror is installed".
  • The pass acts only on the interfaces the configuration names. A priority-1 matchall mirred filter is a shape, not a mark of ownership, and another tool can install the same one. So ze removes a mirror only on an interface named by the current config or the previous one. One case is out of reach: an interface whose whole stanza was deleted while ze was down keeps its mirror. Nothing then tells that mirror apart from a filter ze never installed. To retire that mirror, delete the interface from the configuration while ze runs, or clear the mirror leaves first.

Tunnel Configuration

interface {
    tunnel gre0 {
        encapsulation {
            gre {
                local  { ip 192.0.2.1; }
                remote { ip 198.51.100.1; }
                key 42
            }
        }
        unit default {
            ipv4 {
                address [ 10.0.0.1/30 ]
            }
        }
    }

    tunnel sixin4 {
        encapsulation {
            sit {
                local  { ip 192.0.2.1; }
                remote { ip 198.51.100.1; }
            }
        }
        unit default {
            ipv6 {
                address [ 2001:db8::1/64 ]
            }
        }
    }

    tunnel v6ov6 {
        encapsulation {
            ip6tnl {
                local  { ip 2001:db8::1; }
                remote { ip 2001:db8::2; }
                hoplimit 64
                encaplimit 4
            }
        }
    }
}

The nine supported encapsulation kinds map to Linux netlink kinds:

Kind Linux netlink Underlay Layer Notes
gre gre IPv4 L3 RFC 2784, RFC 2890 key
gretap gretap IPv4 L2 (bridgeable) Ethernet over GRE
ip6gre ip6gre IPv6 L3 hoplimit/tclass per RFC 2473
ip6gretap ip6gretap IPv6 L2 (bridgeable)
ipip ipip IPv4 L3 RFC 2003, no key
sit sit IPv4 L3 6in4 per RFC 4213
ip6tnl ip6tnl IPv6 L3 RFC 2473 with Proto=IPV6
ipip6 ip6tnl IPv6 L3 RFC 2473 with Proto=IPIP
vxlan vxlan IPv4 (UDP) L2 (bridgeable) RFC 7348. Mandatory vni 1..16777215, port default 4789

ipip6 shares the kernel ip6tnl netdev with a different inner protocol byte (4 vs 41). Both surface as distinct YANG cases so the schema and config are unambiguous.

L2 tunnel kinds (gretap, ip6gretap) support an optional mac container (with an address leaf) inside the case container. L3 kinds do not carry a MAC address (the kernel does not assign one).

Outer-header TTL

A tunnel that names no ttl carries an outer TTL of 64, and a tunnel that names no hoplimit carries an outer hop limit of 64. The value 0 keeps its meaning: it tells the kernel to copy the inner packet's TTL onto the outer header, and an operator who wants that mode writes ttl 0.

64 is the default because inherit blackholes a multi-hop underlay. A locally originated packet can carry a small inner TTL, and every underlay router decrements the outer header, so an inherited value expires the encapsulated packet before it reaches the far endpoint. The tunnel then works between directly connected endpoints and drops everything else.

Each default is declared once, in the YANG leaf, and the parse materializes it into the tunnel spec before the backend reads the leaves. No Go constant repeats it, so the schema, the CLI completion, the config diff and the device cannot disagree.

The defaults reach netlink-backed tunnels. The VPP tunnel calls carry no outer-TTL field, so the ttl leaf is declared ze:backend "netlink" and a commit that names one on the vpp backend is refused by path:

/interface/tunnel/t0/encapsulation/gre/ttl: feature not supported by backend "vpp" (supported: netlink)

A vpp-backed tunnel that names no ttl still commits, because the gate reads what the operator wrote and the schema default is materialized after it. sit and the other netlink-only kinds are refused at the kind rather than at this leaf.

ERSPAN, GRE keepalives, VRF underlay/overlay leaves, and ignore-df on gretap are out of scope for v1.

All nine kinds are creatable on the gokrazy appliance image. Nine kinds need six kernel symbols plus the GRE demux gate, and the appliance kernel builds all seven in: NET_IPGRE_DEMUX, NET_IPGRE, IPV6_GRE, NET_IPIP, IPV6_TUNNEL, IPV6_SIT, VXLAN. runtime.require pins the same seven and the required-symbol check accepts =y alone, so a Kconfig default m answer fails the image build rather than shipping a kind nobody can create. Earlier images carried two of the nine, so a config Ze accepted produced a device that never appeared. Measured cost of the seven: vmlinuz grew 98304 bytes, 0.60%.

Tunnel Reload Behaviour

On config reload (SIGHUP or transaction commit), applyConfig compares each tunnel's spec against the previously applied config, which indexTunnelSpecs indexes by name. Tunnels whose spec is unchanged are left alone; MTU, MAC, and addresses still reconcile through later phases, so non-spec changes still take effect. Tunnels whose spec changed (encapsulation kind, local, remote, key, ttl, hoplimit, and the rest of the per-case leaves) are deleted and recreated, because Linux does not support in-place modification of most tunnel kinds. The recreate briefly drops traffic on the changed tunnel only; unrelated tunnels are not disturbed.

A third case applies when the previously applied config is not available. The comparison above reads the config this process applied before, and a netdev outlives the process that made it. A tunnel therefore has no previous spec at every plugin start, at every daemon start, and in the second apply of a reload that starts the interface plugin, so the apply tries to create a link that is already there. What holds the name decides the result:

What holds the name Result
Nothing The create runs. A failure here is the kernel's own answer, and it fails the apply
A tunnel of the configured kind Ze keeps it and logs a WARN. The link is not deleted and not rebuilt, so traffic crossing it is not interrupted. Its encapsulation parameters are not compared: the read-back carries the link type and no encapsulation field
Any other device: a dummy, a bridge, a physical NIC, or a tunnel of a different kind The apply fails. Ze does not delete a device it did not create, and it does not push this tunnel's MTU, admin state and addresses onto a device that is not the tunnel

The last row is also what an operator meets after editing encapsulation while the daemon is down. There is no previous spec to compare against, so the edit does not reach the delete-and-recreate path. Ze refuses the config instead of running the old encapsulation under the new addresses. Delete the tunnel, or give the new encapsulation a new interface name.

Tunnel Validation Scope

ze config validate, API pre-save validation, and CLI commit validation run YANG schema checks plus registered side-effect-free in-process plugin verifiers. They do not call live external plugin OnConfigVerify callbacks because those callbacks are runtime transaction participants. Live external plugin verification runs when the daemon loads, reloads, or commits config; failed API commits roll the saved file back to the previous content before returning the reload error.

Interface validation that has an in-process verifier, such as tunnel case consistency and backend feature gates, is visible in static validation. Any third-party external plugin that only implements a live OnConfigVerify callback is verified at daemon transaction time, not by ze config validate.

WireGuard Configuration

WireGuard interfaces are a top-level wireguard list under interface, alongside ethernet, tunnel, and the other iface kinds. Each entry carries interface-level parameters plus a nested peer list; unit-level addresses ride the same interface-unit grouping used everywhere else.

interface {
    wireguard wg0 {
        listen-port 51820
        fwmark 0
        private-key "$9$ABCabc..."        # $9$-encoded; see below
        peer site2 {
            public-key "YYYY..."           # base64, plaintext
            preshared-key "$9$DEF..."      # optional, also $9$-encoded
            endpoint {
                ip 198.51.100.2
                port 51820
            }
            allowed-ips [ 10.0.0.2/32 192.168.10.0/24 ]
            persistent-keepalive 25
        }
        unit default {
            ipv4 {
                address [ 10.0.0.1/24 ]
            }
        }
    }
}

Key material and $9$ encoding

private-key and peer preshared-key are marked ze:sensitive in YANG. The config parser auto-decodes $9$-prefixed values on load (internal/component/config/parser.go); ze config show / ze config dump always re-encodes them on output (internal/component/config/cli/cmd_dump.go), so the plaintext base64 form never reaches the config file on disk. Public keys are public and stored plaintext.

$9$ is JunOS-compatible obfuscation, not encryption. Anyone with read access to the config file (or the database/ store) can trivially recover the plaintext key via secret.Decode. The protection is on the filesystem layer: chmod 600 /etc/ze/ze.conf (the store's tree is 0700 and its files 0600, see storage-backends.md). This is the same posture ze uses for BGP MD5 passwords, SSH secrets, MCP tokens, and API tokens.

Reconciliation

On reload, applyConfig compares the new spec to the previously applied spec via wireguardSpecEqual. Unchanged entries are a no-op; the kernel is not touched and peer handshake state is preserved. Changed entries trigger a single ConfigureWireguardDevice call with wgtypes.Config{ReplacePeers: true} -- the kernel matches unchanged peers by public-key and preserves their handshake state, so "apply entire spec on every change" is functionally equivalent to a per-peer diff at a tiny fraction of the code. New wireguard entries get a CreateWireguardDevice (netlink) before the Configure call. Wireguard list entries removed from config are deleted by Phase 4 reconciliation, same as tunnels.

Peer names in config (peer site2 { ... }) are operator-chosen labels. The kernel tracks peers only by public key, so the label can change freely without affecting the handshake. ze init emits discovered peers with synthetic names (peer0, peer1, ...) which operators typically rename via ze config edit.

Port conflict detection

listen-port participates in the same conflict-detection mechanism as TCP services (web, ssh, mcp, etc.) with one Phase-5 extension: ListenerEndpoint gained a Protocol field so wireguard's UDP ports never clash with a TCP service on the same port. Two wireguards with the same listen-port are rejected at reload time.

Dependencies

WireGuard peer and key configuration uses golang.zx2c4.com/wireguard/wgctrl, the reference Go client maintained by the WireGuard authors (Donenfeld, Layher). It is vendored under vendor/golang.zx2c4.com/wireguard/wgctrl along with its transitive dependencies github.com/mdlayher/genetlink and github.com/mdlayher/netlink. WireGuard has no RFC; reference material is the original whitepaper (https://www.wireguard.com/papers/wireguard.pdf), the Linux kernel genetlink spec (https://www.kernel.org/doc/html/latest/userspace-api/netlink/specs/wireguard.html), and wg(8).

IPv6 Router Advertisements

Ze sends IPv6 Router Advertisements on a LAN unit, which is the job radvd does on other systems. Hosts on the link build addresses by stateless address autoconfiguration (SLAAC), learn a default router, and learn DNS resolvers. The iface-ra plugin owns the socket, the timers, and the answers to Router Solicitations (RFC 4861).

The router-advertisement container sits inside the per-unit ipv6 container. It carries ze:os "linux" and ze:backend "netlink". The schema walker prunes it on another platform, and a tree with backend vpp rejects it at config verify with feature not supported by backend "vpp".

Send and accept are separate

An advertising interface tells hosts to send it their off-link traffic. Set ipv6 forwarding true on that unit, or the kernel drops that traffic and every host on the link loses connectivity. Linux forwards IPv6 only while net.ipv6.conf.all.forwarding is 1 (or the receiving device's force_forwarding is), so while any unit forwards Ze holds that switch at 1 and writes back each unit's own forwarding leaf, which the kernel overwrites when the switch changes. Leave accept-ra at 0 unless the same interface also learns from another router.

Config verify cannot catch this, because forwarding can arrive from a sysctl profile after verify runs. Ze reports the state instead. The doctor-iface-ra-forwarding check emits one warning for each advertising interface whose net.ipv6.conf.<device>.forwarding is 0, and one for each whose kernel forwards no IPv6 (all.forwarding 0 and the device's force_forwarding not 1).

The autoconf and accept-ra leaves of the per-unit ipv6 container each carry ze:backend "netlink". The kernel builds a SLAAC address only when Router Advertisements reach the device it owns. On the netlink backend that device is the NIC. On the vpp backend VPP owns the NIC, the kernel sees the Linux Control Plane tap, and no advertisement reaches it. Ze writes net.ipv6.conf.<device>.autoconf and net.ipv6.conf.<device>.accept_ra on the tap, the kernel waits, and no address appears.

A tree that sets either leaf with backend vpp is rejected at config verify with feature not supported by backend "vpp" (supported: netlink), and the message names the unit. This matches the refusal the dhcp, dhcpv6 and router-advertisement containers already carry.

Container leaves

Leaf Type Range Unit Default Meaning
enabled boolean false Send advertisements on this unit. RFC 4861 Section 6.2.1 requires the default false, so a node never becomes a router by accident
maximum-interval uint16 4..1800 seconds 600 Longest wait between unsolicited advertisements (MaxRtrAdvInterval)
minimum-interval uint16 3..1350 seconds 200 Shortest wait between unsolicited advertisements (MinRtrAdvInterval)
router-lifetime uint16 0..9000 seconds 1800 How long hosts keep Ze in their default router list (AdvDefaultLifetime)
hop-limit uint8 0..255 64 Value hosts put in the Hop Limit field of their outgoing packets. 0 states no value
managed boolean false The M flag: hosts get their addresses from DHCPv6
other-config boolean false The O flag: hosts get other configuration, such as DNS, from DHCPv6
reachable-time uint32 0..3600000 milliseconds 0 How long a host treats a neighbor as reachable after a confirmation. 0 states no value
retransmit-timer uint32 milliseconds 0 Time between retransmitted Neighbor Solicitations on this link. 0 states no value

Ze picks each unsolicited interval at random between minimum-interval and maximum-interval, which keeps two routers on one link from synchronizing (RFC 4861 Section 6.2.4). The first three advertisements after a start wait 16 seconds at most, so a new router is found quickly.

Prefix list

Each prefix entry becomes one Prefix Information option (RFC 4861 Section 4.6.2). The list key is the prefix in CIDR notation. SLAAC needs a 64-bit prefix (RFC 4862 Section 5.5.3).

Leaf Type Unit Default Meaning
on-link boolean true The L flag: hosts treat addresses in this prefix as on-link
autonomous boolean true The A flag: hosts build addresses from this prefix by SLAAC
valid-lifetime uint32 seconds 2592000 How long the prefix stays valid. 30 days. 4294967295 never expires
preferred-lifetime uint32 seconds 604800 How long addresses from the prefix stay preferred. 7 days

Resolvers (RDNSS)

The rdnss container points a link at a resolver without a DHCPv6 server (RFC 8106 Section 5.1). server is a leaf-list of up to 8 IPv6 addresses, and all of them share one lifetime.

lifetime has no default, and the two zero-like cases stay apart because of that.

You write Ze advertises Hosts do
✕ 3 x maximum-interval Use the resolvers, and refresh them on each advertisement
lifetime 0 0 Stop using these resolvers
lifetime 4294967295 4294967295 Use the resolvers, and never expire them

rdnss dns-source chooses where the advertised resolvers come from: the dns-source list of docs/features/dns-resolver.md. configured is server, received is every IPv6 resolver Ze learned (DHCPv6, an accepted Router Advertisement, the host's resolv.conf), and self is the unit's own global or unique-local address. radius is refused. With no dns-source the list is source [ configured received ], mode first-usable: a unit with server set advertises exactly that, and a unit with none advertises the learned resolvers instead of no RDNSS option.

Case What the RDNSS option carries
a learned resolver a lifetime no longer than what remains of the lease that supplied it, recomputed at every send
a resolver the unit stops advertising (lease withdrawn, server committed) one advertisement, sent at once, with a second RDNSS option of lifetime 0 for it; past 8 withdrawals the rest go in the next advertisements, at the RA rate limit
withdrawals queued while nothing can be sent (the link is down) at most 24, what the 3 advertisements of the initial burst carry; each older one dropped is logged at WARN, and hosts keep it until its last advertised lifetime ends
a loopback or zone-less link-local address never advertised; each drop is logged at WARN with the path
a link-local address learned on another link (on a LAN unit). fe80::1%wan0 never advertised: the option carries no zone, so a host would query it on its own link (RFC 8106 Section 5.1); each drop is logged at WARN. One learned on the unit's own device is advertised
more than 8 selected the first 8 in list order; the rest are logged

Validation

The YANG ranges bound each leaf on its own. Config verify applies the cross-leaf rules of RFC 4861 that no single range can express, and every one of them rejects the commit.

Rule Rejected input Message
minimum-interval is at most 0.75 x maximum-interval minimum-interval 500 with maximum-interval 600 above three quarters of maximum-interval
router-lifetime is 0, or at least maximum-interval router-lifetime 300 with maximum-interval 600 below maximum-interval, and not 0
preferred-lifetime is at most valid-lifetime valid-lifetime 3600 alone preferred-lifetime 604800 is above valid-lifetime 3600
The prefix carries no host bits prefix 2001:db8:1::1/64 host bits set below the prefix length, write 2001:db8:1::/64
The prefix is not link-local prefix fe80::/64 the link-local prefix is not advertised

Two of these rows are traps worth reading twice.

router-lifetime 0 is valid input, and it means that Ze advertises its prefixes and its resolvers while it is not a default router (RFC 4861 Section 4.2). The "at least maximum-interval" rule applies to every other value.

preferred-lifetime defaults to 604800, so lowering only valid-lifetime below that number is rejected. Set both leaves together.

Ze rejects a prefix with host bits rather than masking them, because a masked prefix advertises something the operator did not write.

Example

A LAN unit that advertises one prefix, two resolvers, and itself as the default router:

interface {
    backend netlink;
    ethernet eth0 {
        unit 0 {
            ipv6 {
                address [ 2001:db8:1::1/64 ];
                forwarding true;
                router-advertisement {
                    enabled true;
                    maximum-interval 900;
                    minimum-interval 300;
                    router-lifetime 1800;
                    hop-limit 64;
                    prefix 2001:db8:1::/64 {
                        on-link true;
                        autonomous true;
                        valid-lifetime 86400;
                        preferred-lifetime 43200;
                    }
                    rdnss {
                        server [ 2001:db8:1::53 2001:db8:1::54 ];
                        lifetime 3600;
                    }
                }
            }
        }
    }
}

Send loop

One goroutine owns the socket and every timer of one sender. It joins ff02::2 to receive Router Solicitations, and it sends to ff02::1. Every advertisement leaves with Hop Limit 255, which RFC 4861 Section 6.1.2 makes a receiver check.

A solicitation draws an answer after a random wait of 500 milliseconds at most, and a burst of solicitations draws one answer, timed from the first of them. Consecutive multicast advertisements stay 3 seconds apart, so a flood of solicitations cannot become a flood of advertisements (RFC 4861 Section 6.2.6). A sender that stops sends ONE advertisement with a Router Lifetime of 0, so each host drops Ze from its default router list at once instead of waiting the lifetime out (RFC 4861 Section 6.2.5). The RFC permits up to three. Ze sends one because three leave in a single scheduler tick, so a receiver cannot tell them apart and a link that drops one drops all three; radvd, FRR and BIRD each send one as well.

Nothing leaves a link that is down. The next link-up event restarts the initial burst.

A config change or a lease renew updates a running sender in place. It is never restarted, because a restart would send the zero Router Lifetime above and withdraw Ze from every host for nothing.

A unit with ipv6 delegated-prefix <wan> is in the CE role. Its advertisement carries the unit's /64 with the L and A flags, and a Route Information option (RFC 4191) for the whole delegated prefix. Both carry the lifetimes that remain of the delegation at the moment of each send, so they never outlive it (RFC 9915 Section 6.3). Its Router Lifetime is the configured value only while the main table holds an IPv6 default route through the WAN device and the unit advertises a prefix; otherwise it is 0 (RFC 7084 G-4 and L-4). When either is lost after a non-zero Router Lifetime went out, an advertisement with Router Lifetime 0 leaves at once rather than at the next periodic one (G-5), still carrying the prefix and route while the delegation holds (L-3). The change back, when both hold again, leaves at once the same way, and "at once" still keeps the 3 second spacing above. A /64 whose delegation ended is advertised once with zero lifetimes, then dropped. A unit without delegated-prefix keeps its configured Router Lifetime whatever the WAN does. A WAN whose device Ze cannot resolve holds no default route, and Ze logs it at WARN, naming the LAN and the WAN, when it starts failing, and at INFO when it resolves again.

Counters

Metric Type Labels Counts
ze_iface_ra_sent_total CounterVec interface Every advertisement put on the wire, unsolicited and solicited together
ze_iface_ra_solicited_total CounterVec interface The advertisements that answered a Router Solicitation

A solicited advertisement increments both counters, so ze_iface_ra_sent_total stays the total.

Plugin-Owned Devices (Macvlan)

Beyond the plugin-owned address registry (a plugin declares addresses ze should keep on an interface), iface offers a plugin-owned device registry for bridge-mode macvlan devices. A same-process plugin asks iface to maintain a macvlan carrying a caller-chosen unicast MAC on a parent interface; iface creates it, keeps it reconciled, and deletes it on release or after a crash. The one consumer today is VRRP (one macvlan per group carrying the RFC virtual MAC), but the mechanism is generic and carries zero VRRP knowledge.

An owned macvlan derives no IPv6 link-local of its own: iface sets its IPv6 address generation mode to none before bringing it up, because the MAC it carries is the caller's choice (for VRRP, a MAC every router of the group shares) and Linux would otherwise build the link-local's identifier from it. The owner installs whatever IPv6 address the device needs.

The API is a small value struct plus three calls (mirroring the address registry):

  • MacvlanSpec{Name, Parent, MAC} -- Parent is the parent's OS device name; mode is always bridge (no field).
  • ComposeOwnedDeviceName(prefix, parentIfindex, id) -- builds a deterministic, collision-free name <prefix>-<parentIfindex>-<id> that fits the 15-char IFNAMSIZ budget, rejecting (never truncating) an over-budget candidate.
  • RegisterOwnedMacvlan(owner, spec), UnregisterOwnedMacvlan(owner, name), UnregisterOwnedMacvlans(owner).

Ownership marker. The device's kernel link alias (IFLA_IFALIAS) carries ze:owned:<owner>. The kernel ignores the alias in the RTM_NEWLINK that creates a device (Linux limitation LL-RTNL-NEWLINK-NO-ALIAS, docs/architecture/linux-limitations.md), so CreateMacvlanDevice creates the device with its MAC, then sets the alias with a second message, and deletes the device if that fails, before it sets the IPv6 address generation mode to none and brings the device up. A crash between the two messages can leave an unmarked device under a registered name; the reconcile pass adopts and re-marks it. The reconcile pass reads ownership back from the kernel rather than remembering history, so orphan cleanup works across restarts and crashes -- the one structural difference from the address registry (which must track its own stale interfaces because an address carries no owner).

Reconcile ordering. The device pass runs inside the same reconcile as addresses, BEFORE the address loops: registered devices are created (or re-asserted on MAC/parent/MTU drift by delete + recreate) first, so a VIP registered on the device name via the address registry lands on an existing device in the same pass. Then the orphan scan deletes any macvlan carrying the ze:owned: alias that has no registration; deletion requires BOTH kind macvlan AND the alias, so an operator's own macvlan is never touched. Macvlans are not in the Phase 4 prune set (zeManageable) -- they are alias-guarded, not YANG-managed. A registered device whose parent is absent is retried when the parent appears (monitor interface/created / interface/up re-triggers the pass). Non-netlink backends (VPP, non-Linux) reject CreateMacvlanDevice fail-closed (exact-or-reject).

A ze doctor check (doctor-iface-macvlan) probes kernel macvlan capability (create + delete a throwaway device pair), and the gauge ze_iface_owned_devices{owner} tracks the registry.

Bus Topics

Topic Trigger Payload
interface/created First RTM_NEWLINK for an index name, unit, type, index, mtu
interface/up OperUp or OperUnknown+IFF_UP name, unit, index
interface/down Other oper states or RTM_DELLINK name, unit, index; deletion also carries type and mtu
interface/addr-added Usable address added, initial netlink dump, or retained address replay name, unit, index, address, prefix-length, family, origin
interface/addr-removed Address removed, or its interface deleted name, unit, index, address, prefix-length, family, origin
interface/address-replay-request Consumer requests the active monitor's current usable addresses none
interface/dhcp-acquired DHCPv4 ACK name, unit, address, prefix-length, router, dns, lease-time
interface/dhcp-renewed Renewal success name, unit, address, prefix-length, router, dns, lease-time
interface/dhcp-expired Lease timeout name, unit, address, prefix-length, router, dns, lease-time

The netlink monitor starts after the boot-time apply has assigned the configured addresses, and a netlink subscription delivers only changes. It therefore asks the kernel for every existing address on the subscribed socket when it starts, and announces each as interface/addr-added before any later change. Without that dump no consumer heard of an address that predated the monitor: connected published no prefix for it, so the system RIB could not prove a static gateway inside it, and a multipath group that also names a device-only next-hop dropped that gateway. A consumer can meet the same address announced twice (the dump overlapping a live change, or the kernel announcing an IPv6 address again when its lifetimes change), so a consumer that counts addresses keys them by identity: connected keeps the set of addresses that cover each prefix, not a count of events.

Links and addresses reach the monitor on two netlink subscriptions, and nothing orders one against the other: an interface created and addressed straight away (ip link add, then ip addr add) can deliver its address before the monitor has read the link. The monitor then asks the kernel for that link by index and announces the address under its name, so the address is not lost; the link's own update still announces interface/created and its state afterwards. An address whose link the kernel no longer holds is not announced, since its link deletion follows. Any other failure of that lookup leaves the address unannounced until it changes, so the monitor logs it at WARN and raises a doctor-iface-address-unattributed error naming the address, the index and the kernel's error, with the commands that add the address again.

A late subscriber emits interface/address-replay-request after subscribing to the address events. The active monitor publishes its retained addresses as ordinary addr-added events. Replay and live changes share one publication order, so a removed address cannot return from an older snapshot. Requests coalesce while a replay is pending. A request starts no backend and has no answer when the interface provider is inactive.

The VPP monitor seeds its retained addresses from the interface and address API dumps. Monitor startup and complete address changes share a lock, so an address changed during startup cannot overwrite the newer state with an older dump. Stopping either monitor removes its replay subscription and joins its worker before returning.

Neighbor table changes are not a bus topic. These topics come from the interface monitor, which starts only when an interface block configures a backend, and BGP needs neighbor changes in every config. The interface component registers the neighbor table in internal/core/network as a reader and a watcher (network.RegisterNeighborTable). The BGP reactor starts the watch when it starts, so a next hop naming another router on a shared subnet carries that router's link-local (RFC 2545 Section 3) in the advertisements made after the router's entry changes, with or without an interface block.

The watch loads the default backend when none is loaded, as the reader does. The netlink backend watches with its own netlink subscription to neighbor changes, which starts with a dump of the table, so an entry that existed before the watch started is known and its deletion signals. If the kernel flags that dump as interrupted, a deletion of an entry the watch never saw signals too. It signals only when an IPv6 entry (global or link-local) appears, goes away, changes its link-layer address, or crosses between resolved (reachable, stale, delay, probe, permanent) and unresolved. A flip inside one class signals nothing. Closing the backend, at a reload or at boot when the interface component loads its backend a second time, ends that backend's watch, and the interface component moves the watch to the backend that replaced it and signals once, so BGP reads the new table and logs nothing. A watch that ends any other way (its netlink subscription failed, or the backend was closed with none in its place) is reported to BGP, which logs that the table is not watched and starts the watch again after a delay that doubles from one second to one minute. The VPP backend refuses the watch, because it subscribes to no VPP neighbor event. While no watch runs, BGP logs a warning that names the reason, and a next hop sent without its router's link-local says that the table it read is not watched.

BGP does not read interface/addr-added or interface/addr-removed either, for the same reason: it needs address changes with or without an interface block. The netlink backend package registers a watcher of the kernel address table in internal/core/network (network.RegisterAddressWatcher) at init, whichever backend is active, with its own netlink subscription to address changes. The BGP reactor opens the watch when it starts, re-reads the table on each signal, starts a listener for a passive peer's local address that appeared, on the port that peer listens on, stops it when the address goes away, and re-settles RFC 2545 Section 3's common-subnet condition. On a platform with no netlink backend the watch cannot start, and BGP logs a warning that names the cause. Other consumers (OSPF, the unnumbered donor, the connected-route plugin) still read the bus topics.

Each RTM_NEWLINK also emits the current up or down state, including the first update that emits created. An interface can exist before the monitor starts, so its first observed update can already be a link failure.

Compound Commands (Auto-Ensure Parent)

When a command creates a sub-resource (VLAN unit, address) on an interface that may not exist yet, the compound form auto-creates the parent:

Command Behavior
create interface dummy name <name> unit <vid> Creates dummy <name> if missing, then creates VLAN <name>.<vid>
create interface dummy name <name> address <prefix> Creates dummy <name> if missing, then adds address
create interface bridge name <name> unit <vid> Creates bridge <name> if missing, then creates VLAN
create interface bridge name <name> address <prefix> Creates bridge <name> if missing, then adds address
create interface <name> unit <vid> Direct form: parent must already exist
create interface <name> address <prefix> Direct form: interface must already exist

The type keyword (dummy, bridge) is required when the parent does not exist, because the system needs to know what kind of interface to create. If the parent already exists with a different type, the command fails (e.g., create interface dummy name br0 unit 100 rejects if br0 is a bridge).

Rollback: if the sub-resource creation fails after the parent was auto-created, the parent is deleted. Pre-existing parents are never deleted on failure.

The mechanism is driven by the ze:ensure-exists YANG extension on the type containers (dummy, bridge). The dispatch system builds an ensure chain at registration time and wraps the leaf handler automatically.

Telling those two rollback cases apart requires the creation handler to report whether it created the resource or found it already present. That report is mandatory: if a handler on an ensure-exists path returns no created flag, the command is refused with ensure-exists contract violation naming the handler, and the sub-resource step never runs. Ze fails closed here rather than guessing, because each guess corrupts a different case: assuming "created" would delete an interface the operator already owned, and assuming "not created" would leave a half-built interface behind.

veth has no compound form. Creating a veth creates a pair, so it needs a peer name, and the ensure chain invokes parent handlers with no arguments (it only knows the parent's name selector). There is nowhere in create interface veth name <name> unit <vid> to put the peer, so veth is deliberately absent from the table above; create the pair first, then use the direct form on either end.

Backend Implementations

The Backend interface declares 34 methods. Three implementations ship in tree; coverage varies per platform and dataplane. The table below lists each method against the backend and whether it is wired or returns an error. err means the method is implemented as a stub that rejects every call with a descriptive error. real means the method drives the underlying mechanism. Cells with a footnote carry a caveat.

Category Method netlink (Linux) VPP stub (non-Linux)
Lifecycle CreateDummy real real (CreateLoopback) err
CreateVeth real err (VPP uses memif/TAP) err
CreateBridge real real (BridgeDomainAddDelV2) err
CreateVLAN real real (CreateVlanSubif) err
CreateTunnel real [1] err (pending GoVPP tunnel API) err
CreateWireguardDevice real (rtnetlink) err (requires VPP wg plugin) err
ConfigureWireguardDevice real (wgctrl) err (requires VPP wg plugin) err
GetWireguardDevice real (wgctrl) err (requires VPP wg plugin) err
CreateXFRM real (rtnetlink) err (XFRM is Linux netlink only) err
GetXFRMInfo real (rtnetlink+xfrm) err (XFRM is Linux netlink only) err
DeleteInterface real real (DeleteLoopback/DeleteSubif) err
Address AddAddress real real (SwInterfaceAddDelAddress) err
RemoveAddress real real (SwInterfaceAddDelAddress) err
ReplaceAddressWithLifetime real ∿ err
AddAddressP2P real err (PPP NCP not supported yet) err
RemoveAddressP2P exact installed pair err (PPP NCP not supported yet) err
Route AddRoute real err (use fib-vpp plugin) err
RemoveRoute real err (use fib-vpp plugin) err
ListRoutes real err (use fib-vpp plugin) err
RouteLookup real (netlink RouteGet) real (IPRouteLookupV2 LPM) err
ListKernelRoutes real err [3] err
Link state SetAdminUp real real (SwInterfaceSetFlags) err
SetAdminDown real real (SwInterfaceSetFlags) err
Properties SetMTU real real (HwInterfaceSetMtu, then SwInterfaceSetMtu) err
SetMACAddress real real (SwInterfaceSetMacAddress) err
GetMACAddress real real (via SwInterfaceDump) err
GetStats real err (pending GoVPP stats API) err
ResetCounters real [4] err (pending sw_interface_clear_stats) err
Query ListInterfaces real real (SwInterfaceDump, IPAddressDump per family; no DAD state) real (net.Interfaces)
GetInterface real real (SwInterfaceDump, IPAddressDump per family; no DAD state) real (net.InterfaceByName)
GetLink real (no address dump; ENODEV is not-found) real (SwInterfaceDump, no address dump) real (net.Interfaces)
ListNeighbors real err (pending ip_neighbor_dump) err
Bridge BridgeAddPort real real (SwInterfaceSetL2Bridge) err
BridgeDelPort real real (SwInterfaceSetL2Bridge) err
BridgeSetSTP real (rtnetlink IFLA_BR_STP_STATE; works with a read-only /sys) err (VPP STP varies by version) err
Mirror SetupMirror real (tc mirred) real (device SPAN) err
RemoveMirror real real (device SPAN) err
ListMirrors real (tc filter dump) real (SwInterfaceSpanDump) err
Monitor StartMonitor real (netlink multicast) real (WantInterfaceEvents) err
StopMonitor real real ✕
Close real real ✕

[1] CreateTunnel rejects an unknown kind with unsupported tunnel kind <k>. Valid kinds are gre, gretap, ip6gre, ip6gretap, ipip, sit, ip6tnl, ipip6.

[2] VPP has no kernel-style address lifetimes. The VPP backend ignores the validLft/preferredLft arguments and installs the address without an expiry, matching the exact-or-reject rule only when the operator does not actually need DHCP lease-aware behaviour on VPP. DHCP runs against the netlink backend today.

[3] On VPP the kernel FIB is not authoritative; the VPP backend rejects ListKernelRoutes rather than return misleading data. A VPP FIB dump via ip_route_v2_dump is the correct replacement and is not yet wired.

[4] Linux netlink has no generic counter-reset syscall. The netlink backend returns iface.ErrCountersNotResettable and the dispatch layer falls back to a per-interface baseline-delta model: the current values become a baseline and GetStats / ListInterfaces / GetInterface subtract the baseline before returning, so the operator sees "since last clear" values.

Raw snapshots also carry counter-generation, an opaque process-local continuity token. The Linux backend reads counters and link lifecycle events on one socket subscribed to RTNLGRP_LINK, so a delete and recreate at the same ifIndex changes the token even if the replacement's counters have already overtaken the previous values. Renames and ordinary down/up transitions preserve it. Decreases in exported raw 64-bit counters change it; truncation to a 32-bit wire counter does not.

Socket errors discard the incomplete snapshot and invalidate notification history. The next successful read assigns fresh tokens rather than treating possibly replaced interfaces as continuous. Backends without this metadata report zero. Display-counter clearing only changes the baseline, not raw continuity.

The backend owns this subscribed socket. Close stops its monitor, waits for any bounded counter read, closes the socket and rejects later reads. The iface component calls Close when replacing or shutting down its backend; direct backend users must do the same.

Linux requires statistics to persist “across routine operations like bringing the interface down and up” (networking statistics documentation). Its generic statistics API provides no driver-reset epoch. A driver reset whose new counters exceed the previous sample before the next read cannot be inferred from these values; the token does not claim to detect it.

Pure netlink via vishvananda/netlink, with WireGuard peer/key operations via golang.zx2c4.com/wireguard/wgctrl. Every method is implemented; the only caveats are CreateTunnel rejecting unknown kinds and ResetCounters using baseline-delta (both noted above). No iproute2 shell-outs. Non-Linux builds of this package compile into the stub backend described below.

VPP (opt-in, via GoVPP)

Selected by the backend vpp leaf. 17 methods drive the VPP binary API through GoVPP; 16 return errNotSupported with a string naming the missing GoVPP call or the plugin gap. The channel to VPP is acquired lazily on first method call; before the first successful acquire, every method returns iface.ErrBackendNotReady.

Nothing makes the interface plugin wait for the GoVPP handshake, so its first apply can run before VPP answers. An apply whose step meets that answer is deferred whole rather than failed: it creates nothing, reports no error, logs iface config: apply deferred, the vpp backend has not connected yet at info level, and the plugin performs it in full (loopbacks and their LCP pairs, MTU, admin state, addresses) when vppevents.EventConnected fires, then logs iface config: deferred apply performed at info level. A commit that arrives first replaces the deferred apply. Any other error still fails the apply. A deferred apply that then fails is logged as a warning that names the error and the fix, and stays deferred, so the next commit is measured against what the dataplane was last told. Nothing retries it on a timer: only a VPP restart (vppevents.EventReconnected) or the next commit does. After a VPP restart, vppevents.EventReconnected reloads the backend to drop the dead channel and its name map, then performs a deferred apply or reconciles the addresses.

An MTU on a NIC goes to its link first (hw_interface_set_mtu), which hands the frame size to the driver, so a NIC that takes no larger frame refuses it and the commit fails naming the interface, the asked MTU, the link MTU VPP reports and the set interface ethernet <name> mtu <link MTU> command that fits it. VPP then writes the new MTU on every sub-interface of that NIC. The protocol MTUs (sw_interface_set_mtu) follow, with the per-protocol values left at 0 so each tracks the L3 MTU. That message alone is never enough: VPP stores it without reading the link, so it would accept mtu 9000 over a NIC capped at 1500. An interface whose frame size VPP cannot change (a loopback, a tunnel) takes the protocol MTU alone, up to the link MTU it reports, and a sub-interface is bounded by its parent's link MTU.

Some DPDK drivers change the frame size of a stopped port only: i40e answers EBUSY while the port runs, and VPP relays it as Busy. A raise that meets that answer on an interface that is admin up is retried with the port stopped: Ze sets the interface down, raises the link, and sets it up again, which interrupts the link for that moment. An MTU the link already carries never stops the port, since the protocol MTU serves it, and an interface left down because VPP would not set it up again fails the commit saying so.

A VLAN sub-interface Ze creates in VPP takes its parent's MTU: VPP creates it with an MTU of 0, which it reads as 9000, so without this a parent with no mtu leaf at 1500 would carry a 9000 VLAN. A sub-interface whose MTU VPP refuses is deleted and the create fails. So is one whose ingress-qos-map or egress-qos-map VPP refuses to program: left in place, the existing VLAN would let the commit succeed with its QoS maps not applied.

Ze sets no MTU on a linux-cp tap itself. With lcp sync true, the default, VPP's linux-cp copies every MTU change of a paired interface to its tap, on the VPP side and on the Linux side, and gives a sub-interface with no MTU its parent's (lcp_itf_mtu_change and lcp_itf_pair_sync_state in src/plugins/linux-cp/lcp_interface_sync.c, VPP 25.10). With lcp sync false the tap keeps the MTU it was created with, which is the drift that leaf describes.

Use the VPP backend when VPP owns the dataplane. Use netlink elsewhere. Routes, stats, counter reset, neighbour table, mirrors, and STP are the main gaps against feature parity.

Stub (non-Linux)

On darwin, macOS, BSD, and Windows the netlink backend package compiles to a stub whose constructor succeeds. ListInterfaces, GetInterface and GetLink read the host's interfaces through the Go standard library, so ze init discovers them and writes an initial config: the loopback is typed by its flag, and any other interface with a hardware address is classified as ethernet. Every other method returns "interface management not supported on <GOOS>". StopMonitor and Close are no-ops. The stub exists so the rest of the daemon can load and the binary remains testable on developer machines; real interface management requires Linux.

The stub never installs itself as the default silently. A macOS daemon can list its interfaces, but any config-driven reconciliation that creates, addresses or changes one sees the explicit error and rejects under exact-or-reject.