Interface Management
Ze manages Linux network interfaces via pure netlink (no iproute2 shell-outs). JunOS-style two-layer model: physical interfaces with named logical units.
Interface operation journals survive the inner operation commit. A later owner
failure or outer candidate-publication failure replays their original inverses;
whole-reload acceptance releases them through the acknowledged config-complete
callback. Config restoration also restores the active interface configuration
and reconciles its DHCP, PPPoE and Router Advertisement workers.
Capability Table
| Category | Feature | Status | Priority |
|---|---|---|---|
| Interface Types | Ethernet (physical) | have | |
| Dummy (virtual) | have | ||
| Veth pairs | have | ||
| Bridge (basic) | have | ||
| VLAN 802.1Q | have | ||
| VLAN 802.1p QoS maps (ingress/egress PCP-priority) | have | ||
| Class-of-service named profiles (802.1p, interface-level inheritance) | have | ||
| Loopback | have | ||
| Bonding / LACP | missing | high | |
VXLAN (a tunnel encapsulation, netlink and VPP) |
have | ||
| GRE / GRETAP / IPIP / SIT tunnels | have | ||
| IP6GRE / IP6GRETAP / IP6TNL / IPIP6 tunnels | have | ||
| ERSPAN | missing | lower | |
WireGuard (declarative peers, $9$-encoded keys) |
have | ||
| MACsec | missing | lower | |
| MACVLAN | missing | lower | |
| Geneve | missing | lower | |
PPPoE client (pppoe-client; the session device carries the configured name, so routes, firewall and NAT name it directly) |
have | ||
| L2TPv3 | missing | lower | |
| WiFi | missing | lower | |
| XFRM (route-based IPsec, if_id) | have | ||
| VTI (legacy IPsec tunnel) | missing | lower | |
| QinQ (802.1ad) | missing | lower | |
| Logical Model | Two-layer physical + unit | have | |
| Unit 0 implicit | have | ||
| VLAN units create subinterfaces | have | ||
| Lifecycle | Create (dummy, veth, bridge, VLAN, tunnel) | have | |
| Delete | have | ||
| Auto-up on create | have | ||
| Admin state control (explicit up/down) | have | ||
| Interface rename | missing | lower | |
| Address Management | Add/remove IPv4/IPv6 CIDR | have | |
| DAD-aware monitoring | have | ||
| MAC address set/get | have | ||
| Gratuitous ARP on add | missing | medium | |
| Neighbor table (ARP/NDP) | missing | lower | |
| DHCP | DHCPv4 (RFC 2131, full DORA) | have | |
| DHCPv6 (SARR, IA_NA, IA_PD) | have | ||
| Concurrent v4+v6 | have | ||
| Direct netlink install | have | ||
| DHCPv4 address lifetime equals the lease, refreshed by every renewal; an infinite lease (0xffffffff) installs as infinite and is not renewed | have | ||
| DHCPv4 lease kept for as long as a server answers: RENEW unicast to the leasing server at every T1, REBIND broadcast to any server at T2, both retransmitted on the RFC 2131 Section 4.4.5 schedule (half the time left, at least 60 s); the address goes only when the lease expires. Deadlines count from the instant the REQUEST was sent | have | ||
| DHCPv4 T1 and T2 from the server (options 58 and 59) when T1 < T2 < lease, otherwise the defaults (half and seven eighths of the lease) | have | ||
| DHCPv4 DHCPNAK to a RENEW or REBIND removes the address and route and restarts DORA at once. The RENEW is sent from the leased address but its answer is read from the device, so a DHCPNAK the server broadcasts is seen as well as a DHCPACK it unicasts | have | ||
| DHCPv4 ACK without a lease time (option 51), or with 0, is refused: nothing is installed and DORA retries after 10 s; during a renewal it counts as no answer. The Warn names the server and says option 51 is missing | have | ||
DHCPv4 DORA, RENEW and REBIND open their sockets on the kernel device the interface name resolves to (os-name, mac/match) |
have | ||
| DHCPv4 RENEW and REBIND carry the same client-id (option 61) and hostname (option 12) as the DORA REQUEST | have | ||
DHCPv4 lease registered as leased, so a commit keeps it; iface-dhcp refuses to run as an external plugin, because the registry is a same-process call |
have | ||
| Bus events (acquired/renewed/expired) | have | ||
Config-driven (dhcp { enabled true }) |
have | ||
| Default route from DHCP Router option; a failed install is retried while the lease is held (5 s, doubling to 60 s), and a renewal naming another router, or none, removes the old route | have | ||
Route priority per unit (route-priority) |
have | ||
| Link-down route deprioritization (metric + 1024) | have | ||
| IPv6 default route from RA with configurable metric | have | ||
Routes carry proto 253 (ze-iface), so a teardown removes only its own |
have | ||
| Learned default routes rank at metric 254, clear of a static route's metric | have | ||
DNS from DHCP and from PPPoE IPCP (RFC 1877) to, tracked per lease and ordered DHCPv4, DHCPv6, PPP: a renewal or renegotiation without DNS, or a lease or session end, removes that lease's servers only. /tmp/resolv.conf |
have | ||
| Hostname in DHCPv4 (option 12) | have | ||
| Client-ID in DHCPv4 (option 61) | have | ||
| DHCPv4 flags bits 1 to 15 sent as zero, even when the server's OFFER or ACK sets them (RFC 2131 section 2) | have | ||
| NTP servers from DHCP (option 42) | have | ||
| DHCPv6 proper Renew (not re-solicit) | have | ||
| DHCP relay | missing | lower | |
DHCPv4 server (RFC 2131) on each listen-interface, with pools, static mappings and PXE options |
have | ||
DHCPv4 server: several subnets on one interface (shared-network), a static mapping in any subnet of the segment before any pool, new addresses from the subnet of the interface's primary address first; a listener answers only for the shared networks its interface is addressed in |
have | ||
DHCPv4 server identifier (option 54) is the interface's own address in the client's subnet, the interface's primary address for a relay-only subnet, or the server-identifier leaf while the interface holds it; a REQUEST naming any of the server's addresses on the segment is accepted (RFC 2131 section 4.1) |
have | ||
| Monitoring | Netlink multicast (link + addr + neigh) | have | |
| Virtual iface state detection | have | ||
| 11 bus topics, JSON payloads | have | ||
| Interface statistics/counters | have | ||
| Persistent counter tracking | missing | medium | |
| Per-Interface Tuning | IPv4 forwarding | have | |
| ARP filter / ARP accept | have | ||
| IPv6 autoconf (SLAAC), netlink backend only | have | ||
| IPv6 accept-ra (0/1/2), netlink backend only | have | ||
| IPv6 forwarding | have | ||
| Proxy ARP | have | ||
| ARP announce / ARP ignore | have | ||
RPF / source validation (IPv4 rp_filter sysctl, plus VPP uRPF on backend vpp; IPv6 nftables fib lookup in ip6 ze_iface_rpf, netlink backend only) |
have | See "Reverse-path check" | |
| TCP MSS clamping (v4+v6) | missing | medium | |
| Directed broadcast | missing | lower | |
| IPv6 Extended | EUI-64 address generation | missing | medium |
| DAD configuration (messages, accept) | missing | medium | |
| Custom interface identifiers | missing | lower | |
| Router Advertisement | RA sender per unit (RFC 4861) | have | |
| Prefix Information options for SLAAC | have | ||
| RDNSS resolver option (RFC 8106) | have | ||
| Router Solicitation answers with rate limit | have | ||
| Route Information option (RFC 4191), for the delegated prefix of a unit that numbers from one | have | One option per delegated prefix, with the delegation's remaining valid lifetime; see "Send loop" | |
| A LAN /64 from a DHCPv6 delegated prefix, with lifetimes bounded by the delegation (RFC 7084 L-2, RFC 9915 Section 6.3) | have | ipv6 delegated-prefix <pppoe-client> { subnet-id; interface-identifier }; the source must be a pppoe-client whose dhcpv6 asks for a prefix ("PPPoE client IPv6") . docs/guide/pppoe.md |
|
| CE Router Lifetime: 0 without a WAN default route or a prefix, and an immediate zero-lifetime RA when either is lost (RFC 7084 G-4, G-5, L-4) | have | Applies only to a unit with delegated-prefix; see "Send loop" |
|
| Traffic Mirroring | Ingress/egress via tc mirred | have | |
| Idempotent setup/cleanup | have | ||
| Traffic redirect (vs mirror) | missing | lower | |
| Traffic Control | QoS / shaping | missing | lower |
| Policing | missing | lower | |
| Queuing disciplines | missing | lower | |
| Migration | Make-before-break 5-phase | have | |
| BGP readiness signaling | have | ||
| Per-phase rollback | have | ||
| BGP Integration | Reactor subscribes to addr events | have | |
| Listener start/stop on addr change | have | ||
| bgp/listener/ready publish | have | ||
| Bridge Features | Create and bring up | have | |
| STP | have | ||
| VLAN filtering | missing | medium | |
| Add/remove member ports | have | ||
| Multicast snooping | missing | lower | |
| Port isolation | missing | lower | |
| Ageing/forward delay/hello/max age | missing | lower | |
| Bonding | Mode selection | missing | high |
| Hash policy | missing | high | |
| LACP rate | missing | high | |
| MII monitoring | missing | high | |
| Min active links | missing | high | |
| Member management | missing | high | |
| VRF | YANG leaf exists | ∿ | high |
| Route table isolation | missing | high | |
| Per-VRF address assignment | missing | high | |
| VRF-aware DHCP | missing | medium | |
| Gateway Redundancy | VRRP / keepalived | have | |
| Virtual MAC | have | ||
| State monitoring/failover | missing | medium | |
| Physical Layer | Speed / duplex / autoneg | missing | medium |
| Hardware offload (GRO/GSO/TSO/LRO) | missing | medium | |
| Ring buffer sizing | missing | lower | |
| RPS / RFS | missing | lower | |
| ethtool integration | missing | medium | |
| Security | 802.1X / EAPoL | missing | lower |
| Storm control | missing | lower | |
| Configuration | YANG model (all types + units) | have | |
| Input validation (name, VLAN, MTU) | have | A pppoe-client key is the kernel device name, so it is 1 to 15 octets (IFNAMSIZ), starts with a letter, holds only letters, digits, _, . and -, and may not repeat the key of any other configured interface of any kind |
|
| Platform | Pluggable backend interface | have | |
| Linux netlink backend (default) | have | ||
YANG backend leaf (config-driven selection) |
have | backend vpp starts the VPP component with the interface plugin. A build without ze_vpp refuses it when the config is validated, committed or booted, naming the leaf and the tag, rather than at apply |
|
| VPP backend (ifacevpp, via GoVPP) | have | ResetCounters via sw_interface_clear_stats; ListKernelRoutes via ip_route_v2_dump; RouteLookup via IPRouteLookupV2 (VPP FIB is authoritative); ListNeighbors via ip_neighbor_dump; VXLAN/GRE/IPIP/LCP/stats-socket/mirror/STP pending third-party vendoring | |
| macOS / Darwin | missing | lower | |
| FreeBSD / OpenBSD | missing | lower | |
| systemd-networkd | missing | lower | |
| Quality | Context-wrapped errors | have | |
| Panic recovery | have | ||
| 14 test files (unit + integration) | have |
Creating a device
Each apply creates the devices the config names: dummy, veth, bridge, tunnel,
wireguard, xfrm, and the VLAN of each unit with a vlan-id. Whether a device
exists is read before its create, never after, and it is read from the link
alone (GetLink), as are the MTU and admin state the undo puts back: a failed
address read cannot make a present device read as absent. Only the backend's
"no such device" answer is absence: a link read that fails (a socket that does
not open, ENOBUFS, a dump interrupted on every retry) fails the commit, naming
the device and ze show interface <name>, rather than reading as absent and
creating a present device again. A device that existed is kept,
and the undo of a failed apply leaves it in place. A create that fails for a
device that did not exist fails the commit, even when the backend left a
partial device behind, and the undo deletes only the devices the apply created.
Unnumbered IPv4 units
A unit can borrow the IPv4 address of another interface instead of holding a
subnet of its own (Cisco ip unnumbered, Junos unnumbered-address):
interface {
loopback { unit 0 { ipv4 { address [ 192.0.2.1/32 ]; } } }
ethernet eth0 { unit 0 { ipv4 { unnumbered loopback; } } }
}
The value names the donor, any interface in the config, or loopback for the
loopback block. Ze installs the donor's address on the unit as a /32, the shape
FRR and the Linux kernel use for an unnumbered point-to-point link. When the
donor holds several IPv4 addresses, unnumbered-source <a.b.c.d> picks one;
without it ze takes the donor's first address, in unit name order and then in
address order (Junos preferred-source-address).
The commit is refused when the unit also sets address or enables dhcp, when
the donor is not in the config, is disabled, is itself unnumbered or holds no
IPv4 address, when unnumbered-source names an address the donor does not
hold, and when unnumbered-source is set without unnumbered.
The /32 is a copy of the donor's address, so Ze keeps it only while the donor holds that address on a device that is up. When the donor goes down or loses the address, Ze removes the /32 from the unnumbered unit; when the donor recovers, Ze installs it again. Every interface link or address event, and every config apply, re-checks this. A protocol on the unit sees the address go: OSPF takes the link out of service until the address returns.
Reverse-path check
Each apply builds the complete set of reverse-path checks the config asks for,
one per unit and address family, and hands it to the backend whole:
ReplaceIPv4RPF and ReplaceIPv6RPF. A check whose leaf, unit or interface
left the config is absent from the set, so the backend removes it, and the
apply journal restores the previous set when a later step fails.
nftables refuses even the removal of ip6 ze_iface_rpf to a process without
CAP_NET_ADMIN. When the config asks for no IPv6 check, the netlink backend
therefore accepts that refusal with a warning, so an unprivileged ze still
applies its interfaces. A config that asks for an IPv6 check still fails
without the capability.
The table name is reserved for the reverse-path check: a firewall table named
iface_rpf (kernel name ze_iface_rpf) is refused at verify, naming the
owner iface-rpf, because the firewall backend and the reverse-path check
would each delete and recreate the shared table.
| Family | netlink backend | vpp backend |
|---|---|---|
| IPv4 | net.ipv4.conf.<dev>.rp_filter, written by the sysctl phase; ReplaceIPv4RPF installs nothing more |
urpf_update on the receive path of the VPP interface (strict or loose), read back with urpf_interface_dump so a check no longer configured is turned off. The sysctl still reaches the Linux Control Plane tap, which judges only what VPP punts to the host |
| IPv6 | nftables table ip6 ze_iface_rpf |
refused at commit (ze:backend "netlink") and by the backend |
VPP's uRPF (urpf_plugin.so, always enabled in the startup.conf ze writes)
passes a packet to a multicast destination or to 255.255.255.255, so DHCP from
0.0.0.0 passes. Loose accepts a source whose route has a real path, a default
route included, and drops one that only VPP's drop default covers. IPv6 is not
offered on VPP because VPP's fe80::/10 entry carries an empty uRPF list:
measured on VPP 26.06 with the packet generator, strict and loose both drop a
unicast packet from a link-local source, which would break neighbor discovery,
DHCPv6 and link-local BGP. An external VPP without the plugin accepts a config
with no IPv4 rpf-check and refuses one that sets it, naming the plugin.
Architecture
Config (YANG: ze-iface-conf.yang, "backend" leaf selects backend)
|
v
iface component (register.go) -- OnConfigure() loads backend, starts monitor
|
v
Backend interface (backend.go) -- 45 methods: lifecycle, address, route, bridge, mirror, monitor
|
v
+------------------+--------------------+
| | |
netlink backend DHCP plugin (future: networkd, FreeBSD)
(ifacenetlink/) (ifacedhcp/)
| |
netlink calls lease negotiation
| |
v v
Bus topics -- interface/{created,deleted,up,down,addr/*,dhcp/*}
|
v
Subscribers -- BGP reactor (starts/stops listeners on address changes)
Interface Discovery
During ze init, Ze discovers OS network interfaces and generates initial configuration
entries. The DiscoverInterfaces function enumerates interfaces via ListInterfaces
(netlink on Linux, stdlib on other platforms) and classifies each by Ze type: ethernet,
bridge, veth, dummy, or loopback. On Linux, the netlink device type maps to ethernet
(except lo, which maps to loopback). Results are sorted by type then name.
A managed client (ze init --managed) and an appliance seed (ze init --seed) skip
discovery and write no initial config: the managed client takes its first config from
its hub, and the appliance discovers on the device at first boot.
The generated config uses descriptive names as YANG list keys (the OS interface name at
discovery time). The MAC address serves as the physical binding between configuration and
hardware. For ethernet, veth, and bridge interfaces, the MAC address (mac { address }) is
optional and, when set, must be unique within each list. Omit it to keep the
hardware-assigned MAC, or set it to override the address and pin the named config entry to
a specific physical device.
Discovery writes one selector per entry. An ethernet that reports a factory address
(IFLA_PERM_ADDRESS) is bound by mac { match } against that address, so the entry
follows the NIC across a kernel rename. Every other discovered kind records an os-name
hidden leaf holding the original OS interface name, which remains available for debugging
and internal binding after the user renames the config entry. A discovered ethernet gets no
mac { address } override: that leaf imposes an address on whatever device the entry
resolves to, which is a wrong write the moment the name reaches a different port.
Operator-facing interface operations resolve the logical name to its kernel
device through the shared resolver before touching the kernel, so a configured
name that differs from the OS device (via the os-name or mac/match selector)
is honored uniformly: the iface CLI ops (set MTU, add/remove address, admin
up/down, bridge, mirror, ...), the DHCP client socket binding, and the
routing/protocol consumers all act on the bound kernel device. The dispatch
layer performs this translation for the by-name backend ops and leaves
GetInterface and ListInterfaces raw because the resolver is built on them.
./le arch iface-resolution check keeps new consumers from resolving the kernel
directly instead of through the resolver.
The config apply path resolves separately, and on purpose. It takes ONE interface listing per apply and binds every ethernet entry from it, so each logical name resolves once and Phase 2, Phase 2c and the address reconcile all key their work by the same kernel device. Going through the dispatch wrappers instead would translate at each of about twenty call sites and still miss the sites that take a device string rather than make a backend call: the per-interface sysctl keys and the ethtool offload ioctls.
An entry whose selector names no present device is unbound: every phase skips it,
and none falls back to the logical name. That fallback is what made an aliased entry
configure whatever else carried its name. An entry whose mac/match names more than one
device refuses the apply, and a device only answers a mac/match when the address it is
matched on is its own: a VLAN inherits its parent's, and a bridge or bond wears a
member's, so neither is a candidate. ResolveDevice draws the same line for the by-name
dispatch ops: a name with no selector passes through unchanged, and a name WITH a
selector that fails to resolve returns an error rather than the name.
The two settings that name a SECOND interface resolve through the same map: a bridge member is enslaved by the kernel device its entry selects, and a mirror sends its copy to the capture port's kernel device.
Created kinds need no resolution, because Ze makes the device under its configured
name: dummy, veth, bridge, tunnel, WireGuard, XFRM, and the pppoe-client session
device. The PPPoE client creates its PPP unit under the list key on every session and
every reconnect, so pppoe-client pppoe0 is the kernel device pppoe0 and never a
pppN. A static route next { interface pppoe0 }, a firewall output-interface "pppoe0" match and a masquerade rule therefore reach the session. The device exists
only while a session is up. A static route through it waits for the device and follows
it across a reconnect (docs/guide/static-routes.md, "Interface-only next-hops").
A MAC address validator (ze:validate "mac-address") provides format checking (colon-separated
hex octets) and live OS autocomplete. The CompleteFn calls DiscoverInterfaces on each
tab press, returning MAC addresses from currently active OS interfaces. The os-name
selector carries the sibling validator ze:validate "os-device-name", which screens the
FORM of a kernel device name and completes to the device names present on the box. It
does not check that the device exists: the YANG promises a binding that defers until its
device appears, and a config is validated on machines that will never run it.
Key Design Decisions
- Descriptive names as keys, MAC as binding. Interface names in config are user-chosen descriptive labels. The MAC address ties the config entry to physical hardware. This separates the logical identity (name) from the physical identity (MAC).
- Pluggable backends. The iface component defines a
Backendinterface (34 methods). OS-specific operations live in backend plugins (ifacenetlinkfor Linux). The YANGbackendleaf selects the backend (default:netlink). DHCP is a separate plugin (ifacedhcp) that uses the backend for address operations. - Pure netlink, no shell-outs. The netlink backend uses
github.com/vishvananda/netlink. - Event-driven. Monitor publishes to bus; consumers subscribe and react. No polling.
- Per-family addresses. Addresses live inside
ipv4 { address [...]; }andipv6 { address [...]; }containers within each unit, following the Junos/Nokia model. - DAD-aware. IPv6 addresses with
IFA_F_TENTATIVEflag are held until DAD completes. - The config owns its addresses, not every address on the unit. Every commit, VPP connect and owned-address change reconciles each enabled unit: it adds the addresses the config and the owned-address registry name, and removes the other addresses on the unit, such as one an operator added by hand or one a dropped config line left behind. Three kinds stay in place. A DHCPv4 lease stays because its client registers it as leased while it holds it, finite or infinite: an infinite lease (RFC 2131 lease time 0xffffffff) is installed with the kernel's infinite lifetime, which makes it permanent like a static address, so only the registration tells the two apart. The reconcile never installs a leased address and never changes its lifetime, so a lease its client removed is not put back. A DHCPv6 lease stays the same way: its client registers every leased address, finite or infinite, and installs it with exactly the lifetimes the lease carries. An address with a finite valid lifetime stays: a SLAAC address. Its owner renews it and the kernel expires it. An IPv6 link-local address the replaced config did not name stays, because the kernel generates one on every IPv6 link (RFC 4291 Section 2.1). A link-local the operator configured and then dropped is removed.
The DHCPv6 client follows RFC 9915. It acquires a lease with Solicit, Advertise,
Request and Reply, or with a Rapid Commit Reply. It extends the lease with Renew to
the leasing server at the earliest T1 of its IAs and with Rebind to any server at
the earliest T2, for as long as a server answers. When every valid lifetime has run
out it removes the addresses, publishes one dhcp-expired per lease (a delegated
prefix included) and solicits again. Each exchange retransmits on the RFC 9915
Section 15 schedule, with Solicit backing off to SOL_MAX_RT (3600 s, or the value a
server sends between 60 and 86400) and never stopping, and at most 20 messages in
any 20 s per interface. One Warn reports an outage, and retransmissions log at
Debug. A valid lifetime of 0 removes the lease; an address with a preferred lifetime
above its valid lifetime, or an IA with T1 above T2, is discarded. With pd set,
every message carries an IA_PD, with a prefix-length hint when pd { length } is
set. Every message carries one stable DUID: the duid leaf when set, otherwise a
DUID-LL built from the device MAC. A DUID-LL is a known deviation from RFC 9915
Section 11.4, which advises against it when the client cannot tell the interface is
permanently attached; set duid where that matters. A duid is colon-separated
hex octets, the 2-octet type code then 1 to 128 octets of identifier (RFC 9915
Section 11.1), and a commit refuses any other form. Ze changed from a fresh
time-stamped DUID per Solicit to this stable one, so a server hands out one new
binding once after the upgrade. On stop the client removes its addresses, then sends
a Release and waits at most 3 s for the answer. A delegated prefix a LAN unit took a
/64 from is left out of that Release, and when nothing else is held no Release is
sent: RFC 9915 Section 6.3 forbids releasing a delegated prefix while any of its
space is outstanding.
The kernel runs duplicate address detection on every leased address the client
installs. The client reads the outcome while the address is tentative: when the
kernel marks the address dadfailed, or deletes it (what it does to an address of
finite lifetime whose DAD failed) while the link is up, the client removes it from
the lease, logs a Warn naming the address, and sends a Decline for it to the server
that leased it. The other bindings are kept, and when nothing is left the client
solicits again. Whether the link is up is read from the kernel, not only from link
events, because the kernel deletes the addresses of a device set down before the
client can receive the down event. The kernel backend reads the link before the
addresses, so an address found gone with the link up is declined only when a second
read, whose link state is read after those addresses, still reports the link up. A
device the kernel reports down deleted the address with the link, so nothing is
declined, and the next up counts as a move. When the backend reports a link state
that is neither up nor down, the client declines nothing, logs one Warn for each such
state naming it and the interface, and reads again at the next DAD poll. A read that
fails declines nothing either. The VPP backend reports interface addresses but not
their DAD outcome, because VPP's address dump carries no DAD state. On it the client
cannot read a DAD outcome, so it does not take an unflagged address as unique: it
declines nothing and logs one Warn saying so. When the bound device goes down and comes back up, the client may
have moved to a new link: with a delegated prefix held it sends a Rebind for every
lease, on the Confirm timers (RFC 9915 Section 18.2.12), at most once every 30 s,
and it installs every leased address again with what is left of its lifetimes,
because the kernel deletes them when the interface is set down, then runs DAD on
them again. A lease of addresses alone sends a Confirm instead, after a random
delay of up to 1 s and under the same 30 s limit, carrying every leased address with its
lifetimes set to 0. A Reply with Success, stated or implied by a missing Status Code,
keeps the lease, and so does no answer at all. When every Reply that arrives in a
listen window says NotOnLink, the client removes every leased address, publishes
one dhcp-expired for each, and solicits again.
- A VLAN unit carries its parent's MTU. A unit has no
mtuleaf, so every commit sets each VLAN sub-interface of an ethernet entry with anmtuto that MTU, and a VLAN already at it gets no call. Neither backend does it alone: Linux gives a VLAN its parent's MTU when it is created and keeps it when the parent's later rises (it caps it when the parent's falls), and VPP rewrites a sub-interface's MTU only when its parent's link MTU changes, not when the parent takes a protocol MTU alone. An entry with nomtuleaf leaves its VLANs alone; each backend still creates a VLAN at its parent's MTU. A commit that fails after changing a parent's MTU puts the parent's old MTU back first, then the MTU each existing VLAN held before the commit: Linux caps a VLAN when its parent falls and refuses a VLAN above its parent, so a VLAN restored first, or read after the parent changed, would stay capped. - An omitted
mtuleaves the device's MTU alone. The schema declares no MTU default, because each kind has its own: a device ze creates keeps the kernel's value, 1500 for a dummy, veth or bridge, 1420 for wireguard, and the underlay MTU less the encapsulation for a tunnel (1476 for gre).
- The first configured address is the primary. A unit's
addressleaf-list is installed in the order it is written, IPv4 before IPv6, so the first address is added first and the kernel lists it first. That address is the interface's primary: the DHCPv4 server takes the segment's home subnet from it, and source-address selection prefers it. Every install path keeps the order: the commit's reconcile, the reconcile after a VPP connect or a registry change, the additive pass a not-ready backend takes, and the transaction's add-address operations. Addresses a plugin registers follow the configured ones. A reconcile adds only the addresses missing from the device, so reordering the leaf-list of a live interface does not move an address that is already installed.
- Make-before-break. Migration adds new address, waits for BGP readiness, then removes old. Prevents session loss during address moves.
- Same-subnet renumbers keep working. Because the new address is added first, a renumber
inside one subnet (
10.0.0.1/24->10.0.0.2/24) briefly leaves the new address as a Linux IPv4 secondary of the old one, and Linux deletes every secondary of a subnet along with its primary. Before such a removal the netlink backend enablesnet.ipv4.conf.<device>.promote_secondariesso the kernel promotes a secondary instead of flushing the subnet, loggingenabled promote_secondarieswhen it changes the knob. If the knob cannot be set the removal fails, naming the addresses that would have been destroyed, rather than silently emptying the interface. The knob is left enabled: restoring it would re-arm the same hazard on the next removal. IPv6 has no primary/secondary distinction and is untouched, as is the VPP backend, which deletes exactly the requested address.
- Point-to-point address removal.
RemoveAddressselects by local CIDR and retains the installed peer metadata for deletion.RemoveAddressP2Prequires the exact installed local/peer pair before deletion. A successful dump proving that pair absent completes removal without a delete call. A failed dump or lookup remains an error. IPv4 deletion also accepts the kernel'sEADDRNOTAVAILwhen the pair disappeared after the dump. Requests and installed addresses use the same normalization, including equal IPv4 endpoints. The primary/secondary guard uses the selected pair and its peer subnet to protect other addresses before deletion.
- Virtual interface state. Dummy/bridge/veth report
OperUnknownnotOperUp; monitor checksIFF_UPflag as fallback. - Tunnel encapsulation as YANG choice/case. The
tunnellist at the iface level is one container with anencapsulationchoice that branches into one case per Linux netlink kind (gre, gretap, ip6gre, ip6gretap, ipip, sit, ip6tnl, ipip6). Per-case leaf sets are constrained by the schema:keyonly appears in GRE-family cases,hoplimit/tclass/encaplimitonly in v6-underlay cases. Local and remote endpoints use the samelocal { ip ... } remote { ip ... }shape as the BGP peer connection block, withlocal { interface ... }as an alternative when the source should be taken from another interface. The remote is mandatory and must be a specific address, on every tunnel kind: config verify refuses0.0.0.0and::(and a remote that is not an address), soze config validateand a commit refuse it on either backend, naming theset interface tunnel <name> encapsulation <kind> remote ip <address>command that fixes it. The decapsulator matches the outer source address against the remote, and an unspecified remote accepts packets from any source (RFC 4213 Section 3.6 requires the decapsulator to verify the tunnel source). The netlink and VPP backends run the same check again before they program a tunnel.
- Idempotent cleanup. Delete and mirror removal succeed even if already gone.
- The mirror shares its qdisc, so it owns only its filters. Mirroring attaches a
tc mirred filter at priority 1 to the qdisc at handle
ffff:on the source interface. That qdisc is a shared attachment point: flow-export sampling attaches its own filter at priority 100 to the same object, and both hooks, ingress and egress, hang off it. So mirror setup accepts a qdisc another subsystem created (EEXISTis not an error), and mirror teardown deletes its own filters and leaves the qdisc standing, empty. A teardown cannot know who created a shared qdisc, and the set of filters on it can change between the moment it is read and the moment it is acted on. An empty qdisc carries no filter and classifies no packet, and the next mirror or sampling setup adopts it. Only the rollback of a failed setup deletes a qdisc, because it created that qdisc moments earlier and knows so. Ze always creates clsact, never the older ingress qdisc, because clsact carries both hooks and can therefore serve a mirror with a different destination per direction.tc qdisc showon an interface that once mirrored reports a clsact qdisc with no filter on it, which is expected.
- A mirror the config drops is removed.
applyConfigcompares the mirrors the new config asks for against the mirrors the previous config installed, and removes every one that was dropped or changed before it installs the new set. A changed destination is a remove followed by an install, because tc filters are additive: installing the new destination would otherwise leave the old one duplicating traffic. - The reconcile reads the dataplane, so a restart retires a stranded mirror. A second pass runs inside the config reconcile. It compares the mirrors the dataplane carries against the mirrors the configuration asks for, so it needs no previous config. A mirror the operator deleted while ze was down is therefore removed on the next boot. So is a mirror an earlier apply failed to remove. A backend that cannot read its live mirrors removes nothing, because "the read failed" is not "no mirror is installed".
- The pass acts only on the interfaces the configuration names. A priority-1 matchall mirred filter is a shape, not a mark of ownership, and another tool can install the same one. So ze removes a mirror only on an interface named by the current config or the previous one. One case is out of reach: an interface whose whole stanza was deleted while ze was down keeps its mirror. Nothing then tells that mirror apart from a filter ze never installed. To retire that mirror, delete the interface from the configuration while ze runs, or clear the mirror leaves first.
Tunnel Configuration
interface {
tunnel gre0 {
encapsulation {
gre {
local { ip 192.0.2.1; }
remote { ip 198.51.100.1; }
key 42
}
}
unit default {
ipv4 {
address [ 10.0.0.1/30 ]
}
}
}
tunnel sixin4 {
encapsulation {
sit {
local { ip 192.0.2.1; }
remote { ip 198.51.100.1; }
}
}
unit default {
ipv6 {
address [ 2001:db8::1/64 ]
}
}
}
tunnel v6ov6 {
encapsulation {
ip6tnl {
local { ip 2001:db8::1; }
remote { ip 2001:db8::2; }
hoplimit 64
encaplimit 4
}
}
}
}
The nine supported encapsulation kinds map to Linux netlink kinds:
| Kind | Linux netlink | Underlay | Layer | Notes |
|---|---|---|---|---|
gre |
gre | IPv4 | L3 | RFC 2784, RFC 2890 key |
gretap |
gretap | IPv4 | L2 (bridgeable) | Ethernet over GRE |
ip6gre |
ip6gre | IPv6 | L3 | hoplimit/tclass per RFC 2473 |
ip6gretap |
ip6gretap | IPv6 | L2 (bridgeable) | |
ipip |
ipip | IPv4 | L3 | RFC 2003, no key |
sit |
sit | IPv4 | L3 | 6in4 per RFC 4213 |
ip6tnl |
ip6tnl | IPv6 | L3 | RFC 2473 with Proto=IPV6 |
ipip6 |
ip6tnl | IPv6 | L3 | RFC 2473 with Proto=IPIP |
vxlan |
vxlan | IPv4 (UDP) | L2 (bridgeable) | RFC 7348. Mandatory vni 1..16777215, port default 4789 |
ipip6 shares the kernel ip6tnl netdev with a different inner protocol byte (4 vs 41).
Both surface as distinct YANG cases so the schema and config are unambiguous.
L2 tunnel kinds (gretap, ip6gretap) support an optional mac container (with an
address leaf) inside the case container. L3 kinds do not carry a MAC address (the
kernel does not assign one).
Outer-header TTL
A tunnel that names no ttl carries an outer TTL of 64, and a tunnel that names no
hoplimit carries an outer hop limit of 64. The value 0 keeps its meaning: it tells
the kernel to copy the inner packet's TTL onto the outer header, and an operator who
wants that mode writes ttl 0.
64 is the default because inherit blackholes a multi-hop underlay. A locally originated packet can carry a small inner TTL, and every underlay router decrements the outer header, so an inherited value expires the encapsulated packet before it reaches the far endpoint. The tunnel then works between directly connected endpoints and drops everything else.
Each default is declared once, in the YANG leaf, and the parse materializes it into the tunnel spec before the backend reads the leaves. No Go constant repeats it, so the schema, the CLI completion, the config diff and the device cannot disagree.
The defaults reach netlink-backed tunnels. The VPP tunnel calls carry no outer-TTL
field, so the ttl leaf is declared ze:backend "netlink" and a commit that names
one on the vpp backend is refused by path:
/interface/tunnel/t0/encapsulation/gre/ttl: feature not supported by backend "vpp" (supported: netlink)
A vpp-backed tunnel that names no ttl still commits, because the gate reads what
the operator wrote and the schema default is materialized after it. sit and the
other netlink-only kinds are refused at the kind rather than at this leaf.
ERSPAN, GRE keepalives, VRF underlay/overlay leaves, and ignore-df on gretap are
out of scope for v1.
All nine kinds are creatable on the gokrazy appliance image. Nine kinds need six
kernel symbols plus the GRE demux gate, and the appliance kernel builds all seven
in: NET_IPGRE_DEMUX, NET_IPGRE, IPV6_GRE, NET_IPIP, IPV6_TUNNEL,
IPV6_SIT, VXLAN. runtime.require pins the same seven and the required-symbol
check accepts =y alone, so a Kconfig default m answer fails the image build
rather than shipping a kind nobody can create. Earlier images carried two of the
nine, so a config Ze accepted produced a device that never appeared. Measured cost
of the seven: vmlinuz grew 98304 bytes, 0.60%.
Tunnel Reload Behaviour
On config reload (SIGHUP or transaction commit), applyConfig compares each
tunnel's spec against the previously applied config, which indexTunnelSpecs
indexes by name. Tunnels
whose spec is unchanged are left alone; MTU, MAC, and addresses still reconcile
through later phases, so non-spec changes still take effect. Tunnels whose spec
changed (encapsulation kind, local, remote, key, ttl, hoplimit, and
the rest of the per-case leaves) are deleted and recreated, because Linux does
not support in-place modification of most tunnel kinds. The recreate briefly
drops traffic on the changed tunnel only; unrelated tunnels are not disturbed.
A third case applies when the previously applied config is not available. The comparison above reads the config this process applied before, and a netdev outlives the process that made it. A tunnel therefore has no previous spec at every plugin start, at every daemon start, and in the second apply of a reload that starts the interface plugin, so the apply tries to create a link that is already there. What holds the name decides the result:
| What holds the name | Result |
|---|---|
| Nothing | The create runs. A failure here is the kernel's own answer, and it fails the apply |
| A tunnel of the configured kind | Ze keeps it and logs a WARN. The link is not deleted and not rebuilt, so traffic crossing it is not interrupted. Its encapsulation parameters are not compared: the read-back carries the link type and no encapsulation field |
| Any other device: a dummy, a bridge, a physical NIC, or a tunnel of a different kind | The apply fails. Ze does not delete a device it did not create, and it does not push this tunnel's MTU, admin state and addresses onto a device that is not the tunnel |
The last row is also what an operator meets after editing encapsulation while
the daemon is down. There is no previous spec to compare against, so the edit
does not reach the delete-and-recreate path. Ze refuses the config instead of
running the old encapsulation under the new addresses. Delete the tunnel, or
give the new encapsulation a new interface name.
Tunnel Validation Scope
ze config validate, API pre-save validation, and CLI commit validation run
YANG schema checks plus registered side-effect-free in-process plugin verifiers.
They do not call live external plugin OnConfigVerify callbacks because those
callbacks are runtime transaction participants. Live external plugin verification
runs when the daemon loads, reloads, or commits config; failed API commits roll
the saved file back to the previous content before returning the reload error.
Interface validation that has an in-process verifier, such as tunnel case
consistency and backend feature gates, is visible in static validation. Any
third-party external plugin that only implements a live OnConfigVerify
callback is verified at daemon transaction time, not by ze config validate.
WireGuard Configuration
WireGuard interfaces are a top-level wireguard list under interface,
alongside ethernet, tunnel, and the other iface kinds. Each entry carries
interface-level parameters plus a nested peer list; unit-level addresses
ride the same interface-unit grouping used everywhere else.
interface {
wireguard wg0 {
listen-port 51820
fwmark 0
private-key "$9$ABCabc..." # $9$-encoded; see below
peer site2 {
public-key "YYYY..." # base64, plaintext
preshared-key "$9$DEF..." # optional, also $9$-encoded
endpoint {
ip 198.51.100.2
port 51820
}
allowed-ips [ 10.0.0.2/32 192.168.10.0/24 ]
persistent-keepalive 25
}
unit default {
ipv4 {
address [ 10.0.0.1/24 ]
}
}
}
}
Key material and $9$ encoding
private-key and peer preshared-key are marked ze:sensitive in YANG.
The config parser auto-decodes $9$-prefixed values on load
(internal/component/config/parser.go); ze config show / ze config dump always re-encodes them on output (internal/component/config/cli/cmd_dump.go),
so the plaintext base64 form never reaches the config file on disk. Public
keys are public and stored plaintext.
$9$ is JunOS-compatible obfuscation, not encryption. Anyone with
read access to the config file (or the database/ store) can trivially
recover the plaintext key via secret.Decode. The protection is on the
filesystem layer: chmod 600 /etc/ze/ze.conf (the store's tree is 0700 and
its files 0600, see storage-backends.md). This is the same posture ze uses for BGP
MD5 passwords, SSH secrets, MCP tokens, and API tokens.
Reconciliation
On reload, applyConfig compares the new spec to the previously
applied spec via wireguardSpecEqual. Unchanged entries are a no-op; the
kernel is not touched and peer handshake state is preserved. Changed
entries trigger a single ConfigureWireguardDevice call with
wgtypes.Config{ReplacePeers: true} -- the kernel matches unchanged peers
by public-key and preserves their handshake state, so "apply entire spec
on every change" is functionally equivalent to a per-peer diff at a tiny
fraction of the code. New wireguard entries get a CreateWireguardDevice
(netlink) before the Configure call. Wireguard list entries removed from
config are deleted by Phase 4 reconciliation, same as tunnels.
Peer names in config (peer site2 { ... }) are operator-chosen labels.
The kernel tracks peers only by public key, so the label can change
freely without affecting the handshake. ze init emits discovered peers
with synthetic names (peer0, peer1, ...) which operators typically
rename via ze config edit.
Port conflict detection
listen-port participates in the same conflict-detection mechanism as
TCP services (web, ssh, mcp, etc.) with one Phase-5 extension:
ListenerEndpoint gained a Protocol field so wireguard's UDP ports
never clash with a TCP service on the same port. Two wireguards with
the same listen-port are rejected at reload time.
Dependencies
WireGuard peer and key configuration uses
golang.zx2c4.com/wireguard/wgctrl,
the reference Go client maintained by the WireGuard authors (Donenfeld,
Layher). It is vendored under vendor/golang.zx2c4.com/wireguard/wgctrl
along with its transitive dependencies github.com/mdlayher/genetlink
and github.com/mdlayher/netlink. WireGuard has no RFC; reference
material is the original whitepaper
(https://www.wireguard.com/papers/wireguard.pdf), the Linux kernel
genetlink spec
(https://www.kernel.org/doc/html/latest/userspace-api/netlink/specs/wireguard.html),
and wg(8).
IPv6 Router Advertisements
Ze sends IPv6 Router Advertisements on a LAN unit, which is the job radvd does on
other systems. Hosts on the link build addresses by stateless address
autoconfiguration (SLAAC), learn a default router, and learn DNS resolvers. The
iface-ra plugin owns the socket, the timers, and the answers to Router
Solicitations (RFC 4861).
The router-advertisement container sits inside the per-unit ipv6 container.
It carries ze:os "linux" and ze:backend "netlink". The schema walker prunes
it on another platform, and a tree with backend vpp rejects it at config verify
with feature not supported by backend "vpp".
Send and accept are separate
An advertising interface tells hosts to send it their off-link traffic. Set
ipv6 forwarding true on that unit, or the kernel drops that traffic and every
host on the link loses connectivity. Linux forwards IPv6 only while
net.ipv6.conf.all.forwarding is 1 (or the receiving device's
force_forwarding is), so while any unit forwards Ze holds that switch at 1
and writes back each unit's own forwarding leaf, which the kernel overwrites
when the switch changes. Leave accept-ra at 0 unless the same
interface also learns from another router.
Config verify cannot catch this, because forwarding can arrive from a sysctl
profile after verify runs. Ze reports the state instead. The
doctor-iface-ra-forwarding check emits one warning for each advertising
interface whose net.ipv6.conf.<device>.forwarding is 0, and one for each whose
kernel forwards no IPv6 (all.forwarding 0 and the device's force_forwarding
not 1).
Receiving advertisements needs the netlink backend
The autoconf and accept-ra leaves of the per-unit ipv6 container each carry
ze:backend "netlink". The kernel builds a SLAAC address only when Router
Advertisements reach the device it owns. On the netlink backend that device is
the NIC. On the vpp backend VPP owns the NIC, the kernel sees the Linux Control
Plane tap, and no advertisement reaches it. Ze writes
net.ipv6.conf.<device>.autoconf and net.ipv6.conf.<device>.accept_ra on the
tap, the kernel waits, and no address appears.
A tree that sets either leaf with backend vpp is rejected at config verify with
feature not supported by backend "vpp" (supported: netlink), and the message
names the unit. This matches the refusal the dhcp, dhcpv6 and
router-advertisement containers already carry.
Container leaves
| Leaf | Type | Range | Unit | Default | Meaning |
|---|---|---|---|---|---|
enabled |
boolean | false |
Send advertisements on this unit. RFC 4861 Section 6.2.1 requires the default false, so a node never becomes a router by accident |
||
maximum-interval |
uint16 | 4..1800 | seconds | 600 | Longest wait between unsolicited advertisements (MaxRtrAdvInterval) |
minimum-interval |
uint16 | 3..1350 | seconds | 200 | Shortest wait between unsolicited advertisements (MinRtrAdvInterval) |
router-lifetime |
uint16 | 0..9000 | seconds | 1800 | How long hosts keep Ze in their default router list (AdvDefaultLifetime) |
hop-limit |
uint8 | 0..255 | 64 | Value hosts put in the Hop Limit field of their outgoing packets. 0 states no value | |
managed |
boolean | false |
The M flag: hosts get their addresses from DHCPv6 | ||
other-config |
boolean | false |
The O flag: hosts get other configuration, such as DNS, from DHCPv6 | ||
reachable-time |
uint32 | 0..3600000 | milliseconds | 0 | How long a host treats a neighbor as reachable after a confirmation. 0 states no value |
retransmit-timer |
uint32 | milliseconds | 0 | Time between retransmitted Neighbor Solicitations on this link. 0 states no value |
Ze picks each unsolicited interval at random between minimum-interval and
maximum-interval, which keeps two routers on one link from synchronizing
(RFC 4861 Section 6.2.4). The first three advertisements after a start wait
16 seconds at most, so a new router is found quickly.
Prefix list
Each prefix entry becomes one Prefix Information option (RFC 4861
Section 4.6.2). The list key is the prefix in CIDR notation. SLAAC needs a
64-bit prefix (RFC 4862 Section 5.5.3).
| Leaf | Type | Unit | Default | Meaning |
|---|---|---|---|---|
on-link |
boolean | true |
The L flag: hosts treat addresses in this prefix as on-link | |
autonomous |
boolean | true |
The A flag: hosts build addresses from this prefix by SLAAC | |
valid-lifetime |
uint32 | seconds | 2592000 | How long the prefix stays valid. 30 days. 4294967295 never expires |
preferred-lifetime |
uint32 | seconds | 604800 | How long addresses from the prefix stay preferred. 7 days |
Resolvers (RDNSS)
The rdnss container points a link at a resolver without a DHCPv6 server
(RFC 8106 Section 5.1). server is a leaf-list of up to 8 IPv6 addresses, and
all of them share one lifetime.
lifetime has no default, and the two zero-like cases stay apart because of
that.
| You write | Ze advertises | Hosts do |
|---|---|---|
| ✕ | 3 x maximum-interval |
Use the resolvers, and refresh them on each advertisement |
lifetime 0 |
0 | Stop using these resolvers |
lifetime 4294967295 |
4294967295 | Use the resolvers, and never expire them |
rdnss dns-source chooses where the advertised resolvers come from: the
dns-source list of docs/features/dns-resolver.md. configured is
server, received is every IPv6 resolver Ze learned (DHCPv6, an accepted
Router Advertisement, the host's resolv.conf), and self is the unit's own
global or unique-local address. radius is refused. With no dns-source
the list is source [ configured received ], mode first-usable: a unit
with server set advertises exactly that, and a unit with none advertises
the learned resolvers instead of no RDNSS option.
| Case | What the RDNSS option carries |
|---|---|
| a learned resolver | a lifetime no longer than what remains of the lease that supplied it, recomputed at every send |
a resolver the unit stops advertising (lease withdrawn, server committed) |
one advertisement, sent at once, with a second RDNSS option of lifetime 0 for it; past 8 withdrawals the rest go in the next advertisements, at the RA rate limit |
| withdrawals queued while nothing can be sent (the link is down) | at most 24, what the 3 advertisements of the initial burst carry; each older one dropped is logged at WARN, and hosts keep it until its last advertised lifetime ends |
| a loopback or zone-less link-local address | never advertised; each drop is logged at WARN with the path |
a link-local address learned on another link (on a LAN unit). fe80::1%wan0 |
never advertised: the option carries no zone, so a host would query it on its own link (RFC 8106 Section 5.1); each drop is logged at WARN. One learned on the unit's own device is advertised |
| more than 8 selected | the first 8 in list order; the rest are logged |
Validation
The YANG ranges bound each leaf on its own. Config verify applies the cross-leaf rules of RFC 4861 that no single range can express, and every one of them rejects the commit.
| Rule | Rejected input | Message |
|---|---|---|
minimum-interval is at most 0.75 x maximum-interval |
minimum-interval 500 with maximum-interval 600 |
above three quarters of maximum-interval |
router-lifetime is 0, or at least maximum-interval |
router-lifetime 300 with maximum-interval 600 |
below maximum-interval, and not 0 |
preferred-lifetime is at most valid-lifetime |
valid-lifetime 3600 alone |
preferred-lifetime 604800 is above valid-lifetime 3600 |
| The prefix carries no host bits | prefix 2001:db8:1::1/64 |
host bits set below the prefix length, write 2001:db8:1::/64 |
| The prefix is not link-local | prefix fe80::/64 |
the link-local prefix is not advertised |
Two of these rows are traps worth reading twice.
router-lifetime 0 is valid input, and it means that Ze advertises its prefixes
and its resolvers while it is not a default router (RFC 4861 Section 4.2). The
"at least maximum-interval" rule applies to every other value.
preferred-lifetime defaults to 604800, so lowering only valid-lifetime below
that number is rejected. Set both leaves together.
Ze rejects a prefix with host bits rather than masking them, because a masked prefix advertises something the operator did not write.
Example
A LAN unit that advertises one prefix, two resolvers, and itself as the default router:
interface {
backend netlink;
ethernet eth0 {
unit 0 {
ipv6 {
address [ 2001:db8:1::1/64 ];
forwarding true;
router-advertisement {
enabled true;
maximum-interval 900;
minimum-interval 300;
router-lifetime 1800;
hop-limit 64;
prefix 2001:db8:1::/64 {
on-link true;
autonomous true;
valid-lifetime 86400;
preferred-lifetime 43200;
}
rdnss {
server [ 2001:db8:1::53 2001:db8:1::54 ];
lifetime 3600;
}
}
}
}
}
}
Send loop
One goroutine owns the socket and every timer of one sender. It joins ff02::2
to receive Router Solicitations, and it sends to ff02::1. Every advertisement
leaves with Hop Limit 255, which RFC 4861 Section 6.1.2 makes a receiver check.
A solicitation draws an answer after a random wait of 500 milliseconds at most, and a burst of solicitations draws one answer, timed from the first of them. Consecutive multicast advertisements stay 3 seconds apart, so a flood of solicitations cannot become a flood of advertisements (RFC 4861 Section 6.2.6). A sender that stops sends ONE advertisement with a Router Lifetime of 0, so each host drops Ze from its default router list at once instead of waiting the lifetime out (RFC 4861 Section 6.2.5). The RFC permits up to three. Ze sends one because three leave in a single scheduler tick, so a receiver cannot tell them apart and a link that drops one drops all three; radvd, FRR and BIRD each send one as well.
Nothing leaves a link that is down. The next link-up event restarts the initial burst.
A config change or a lease renew updates a running sender in place. It is never restarted, because a restart would send the zero Router Lifetime above and withdraw Ze from every host for nothing.
A unit with ipv6 delegated-prefix <wan> is in the CE role. Its advertisement
carries the unit's /64 with the L and A flags, and a Route Information option
(RFC 4191) for the whole delegated prefix. Both carry the lifetimes that remain of
the delegation at the moment of each send, so they never outlive it (RFC 9915
Section 6.3). Its Router Lifetime is the configured value only while the main table
holds an IPv6 default route through the WAN device and the unit advertises a prefix;
otherwise it is 0 (RFC 7084 G-4 and L-4). When either is lost after a non-zero
Router Lifetime went out, an advertisement with Router Lifetime 0 leaves at once
rather than at the next periodic one (G-5), still carrying the prefix and route
while the delegation holds (L-3). The change back, when both hold again, leaves at
once the same way, and "at once" still keeps the 3 second spacing above. A /64 whose delegation ended is advertised once
with zero lifetimes, then dropped. A unit without delegated-prefix keeps its
configured Router Lifetime whatever the WAN does. A WAN whose device Ze cannot
resolve holds no default route, and Ze logs it at WARN, naming the LAN and the WAN,
when it starts failing, and at INFO when it resolves again.
Counters
| Metric | Type | Labels | Counts |
|---|---|---|---|
ze_iface_ra_sent_total |
CounterVec | interface |
Every advertisement put on the wire, unsolicited and solicited together |
ze_iface_ra_solicited_total |
CounterVec | interface |
The advertisements that answered a Router Solicitation |
A solicited advertisement increments both counters, so ze_iface_ra_sent_total
stays the total.
Plugin-Owned Devices (Macvlan)
Beyond the plugin-owned address registry (a plugin declares addresses ze should keep on an interface), iface offers a plugin-owned device registry for bridge-mode macvlan devices. A same-process plugin asks iface to maintain a macvlan carrying a caller-chosen unicast MAC on a parent interface; iface creates it, keeps it reconciled, and deletes it on release or after a crash. The one consumer today is VRRP (one macvlan per group carrying the RFC virtual MAC), but the mechanism is generic and carries zero VRRP knowledge.
An owned macvlan derives no IPv6 link-local of its own: iface sets its IPv6 address generation mode to none before bringing it up, because the MAC it carries is the caller's choice (for VRRP, a MAC every router of the group shares) and Linux would otherwise build the link-local's identifier from it. The owner installs whatever IPv6 address the device needs.
The API is a small value struct plus three calls (mirroring the address registry):
MacvlanSpec{Name, Parent, MAC}--Parentis the parent's OS device name; mode is always bridge (no field).ComposeOwnedDeviceName(prefix, parentIfindex, id)-- builds a deterministic, collision-free name<prefix>-<parentIfindex>-<id>that fits the 15-char IFNAMSIZ budget, rejecting (never truncating) an over-budget candidate.RegisterOwnedMacvlan(owner, spec),UnregisterOwnedMacvlan(owner, name),UnregisterOwnedMacvlans(owner).
Ownership marker. The device's kernel link alias (IFLA_IFALIAS) carries
ze:owned:<owner>. The kernel ignores the alias in the RTM_NEWLINK that
creates a device (Linux limitation LL-RTNL-NEWLINK-NO-ALIAS,
docs/architecture/linux-limitations.md), so CreateMacvlanDevice creates the
device with its MAC, then sets the alias with a second message, and deletes the
device if that fails, before it sets the IPv6 address generation mode to none
and brings the device up. A crash between the two messages can leave an
unmarked device under a registered name; the reconcile pass adopts and re-marks
it. The reconcile pass reads ownership back from the kernel rather than remembering
history, so orphan cleanup works across restarts and crashes -- the one
structural difference from the address registry (which must track its own stale
interfaces because an address carries no owner).
Reconcile ordering. The device pass runs inside the same reconcile as
addresses, BEFORE the address loops: registered devices are created (or
re-asserted on MAC/parent/MTU drift by delete + recreate) first, so a VIP
registered on the device name via the address registry lands on an existing
device in the same pass. Then the orphan scan deletes any macvlan carrying the
ze:owned: alias that has no registration; deletion requires BOTH kind macvlan
AND the alias, so an operator's own macvlan is never touched. Macvlans are not
in the Phase 4 prune set (zeManageable) -- they are alias-guarded, not
YANG-managed. A registered device whose parent is absent is retried when the
parent appears (monitor interface/created / interface/up re-triggers the
pass). Non-netlink backends (VPP, non-Linux) reject CreateMacvlanDevice
fail-closed (exact-or-reject).
A ze doctor check (doctor-iface-macvlan) probes kernel macvlan capability
(create + delete a throwaway device pair), and the gauge
ze_iface_owned_devices{owner} tracks the registry.
Bus Topics
| Topic | Trigger | Payload |
|---|---|---|
interface/created |
First RTM_NEWLINK for an index | name, unit, type, index, mtu |
interface/up |
OperUp or OperUnknown+IFF_UP | name, unit, index |
interface/down |
Other oper states or RTM_DELLINK | name, unit, index; deletion also carries type and mtu |
interface/addr-added |
Usable address added, initial netlink dump, or retained address replay | name, unit, index, address, prefix-length, family, origin |
interface/addr-removed |
Address removed, or its interface deleted | name, unit, index, address, prefix-length, family, origin |
interface/address-replay-request |
Consumer requests the active monitor's current usable addresses | none |
interface/dhcp-acquired |
DHCPv4 ACK | name, unit, address, prefix-length, router, dns, lease-time |
interface/dhcp-renewed |
Renewal success | name, unit, address, prefix-length, router, dns, lease-time |
interface/dhcp-expired |
Lease timeout | name, unit, address, prefix-length, router, dns, lease-time |
The netlink monitor starts after the boot-time apply has assigned the
configured addresses, and a netlink subscription delivers only changes. It
therefore asks the kernel for every existing address on the subscribed socket
when it starts, and announces each as interface/addr-added before any later
change. Without that dump no consumer heard of an address that predated the
monitor: connected published no prefix for it, so the system RIB could not
prove a static gateway inside it, and a multipath group that also names a
device-only next-hop dropped that gateway. A consumer can meet the same address
announced twice (the dump overlapping a live change, or the kernel announcing
an IPv6 address again when its lifetimes change), so a consumer that counts
addresses keys them by identity: connected keeps the set of addresses that
cover each prefix, not a count of events.
Links and addresses reach the monitor on two netlink subscriptions, and nothing
orders one against the other: an interface created and addressed straight away
(ip link add, then ip addr add) can deliver its address before the monitor
has read the link. The monitor then asks the kernel for that link by index and
announces the address under its name, so the address is not lost; the link's
own update still announces interface/created and its state afterwards. An
address whose link the kernel no longer holds is not announced, since its link
deletion follows. Any other failure of that lookup leaves the address
unannounced until it changes, so the monitor logs it at WARN and raises a
doctor-iface-address-unattributed error naming the address, the index and
the kernel's error, with the commands that add the address again.
A late subscriber emits interface/address-replay-request after subscribing
to the address events. The active monitor publishes its retained addresses as
ordinary addr-added events. Replay and live changes share one publication
order, so a removed address cannot return from an older snapshot. Requests
coalesce while a replay is pending. A request starts no backend and has no
answer when the interface provider is inactive.
The VPP monitor seeds its retained addresses from the interface and address API dumps. Monitor startup and complete address changes share a lock, so an address changed during startup cannot overwrite the newer state with an older dump. Stopping either monitor removes its replay subscription and joins its worker before returning.
Neighbor table changes are not a bus topic. These topics come from the
interface monitor, which starts only when an interface block configures a
backend, and BGP needs neighbor changes in every config. The interface
component registers the neighbor table in internal/core/network as a reader
and a watcher (network.RegisterNeighborTable). The BGP reactor starts the
watch when it starts, so a next hop naming another router on a shared subnet
carries that router's link-local (RFC 2545 Section 3) in the advertisements
made after the router's entry changes, with or without an interface block.
The watch loads the default backend when none is loaded, as the reader does. The netlink backend watches with its own netlink subscription to neighbor changes, which starts with a dump of the table, so an entry that existed before the watch started is known and its deletion signals. If the kernel flags that dump as interrupted, a deletion of an entry the watch never saw signals too. It signals only when an IPv6 entry (global or link-local) appears, goes away, changes its link-layer address, or crosses between resolved (reachable, stale, delay, probe, permanent) and unresolved. A flip inside one class signals nothing. Closing the backend, at a reload or at boot when the interface component loads its backend a second time, ends that backend's watch, and the interface component moves the watch to the backend that replaced it and signals once, so BGP reads the new table and logs nothing. A watch that ends any other way (its netlink subscription failed, or the backend was closed with none in its place) is reported to BGP, which logs that the table is not watched and starts the watch again after a delay that doubles from one second to one minute. The VPP backend refuses the watch, because it subscribes to no VPP neighbor event. While no watch runs, BGP logs a warning that names the reason, and a next hop sent without its router's link-local says that the table it read is not watched.
BGP does not read interface/addr-added or interface/addr-removed either,
for the same reason: it needs address changes with or without an interface
block. The netlink backend package registers a watcher of the kernel address
table in internal/core/network (network.RegisterAddressWatcher) at init,
whichever backend is active, with its own netlink subscription to address
changes. The BGP reactor opens the watch when it starts, re-reads the table on
each signal, starts a listener for a passive peer's local address that
appeared, on the port that peer listens on, stops it when the address goes
away, and re-settles RFC 2545 Section 3's
common-subnet condition. On a platform with no netlink backend the watch
cannot start, and BGP logs a warning that names the cause. Other consumers
(OSPF, the unnumbered donor, the connected-route plugin) still read the bus
topics.
Each RTM_NEWLINK also emits the current up or down state, including the first
update that emits created. An interface can exist before the monitor starts,
so its first observed update can already be a link failure.
Compound Commands (Auto-Ensure Parent)
When a command creates a sub-resource (VLAN unit, address) on an interface that may not exist yet, the compound form auto-creates the parent:
| Command | Behavior |
|---|---|
create interface dummy name <name> unit <vid> |
Creates dummy <name> if missing, then creates VLAN <name>.<vid> |
create interface dummy name <name> address <prefix> |
Creates dummy <name> if missing, then adds address |
create interface bridge name <name> unit <vid> |
Creates bridge <name> if missing, then creates VLAN |
create interface bridge name <name> address <prefix> |
Creates bridge <name> if missing, then adds address |
create interface <name> unit <vid> |
Direct form: parent must already exist |
create interface <name> address <prefix> |
Direct form: interface must already exist |
The type keyword (dummy, bridge) is required when the parent does not
exist, because the system needs to know what kind of interface to create.
If the parent already exists with a different type, the command fails
(e.g., create interface dummy name br0 unit 100 rejects if br0 is a bridge).
Rollback: if the sub-resource creation fails after the parent was auto-created, the parent is deleted. Pre-existing parents are never deleted on failure.
The mechanism is driven by the ze:ensure-exists YANG extension on the
type containers (dummy, bridge). The dispatch system builds an ensure
chain at registration time and wraps the leaf handler automatically.
Telling those two rollback cases apart requires the creation handler to
report whether it created the resource or found it already present. That
report is mandatory: if a handler on an ensure-exists path returns no
created flag, the command is refused with ensure-exists contract violation naming the handler, and the sub-resource step never runs.
Ze fails closed here rather than guessing, because each guess corrupts a
different case: assuming "created" would delete an interface the operator
already owned, and assuming "not created" would leave a half-built
interface behind.
veth has no compound form. Creating a veth creates a pair, so it needs
a peer name, and the ensure chain invokes parent handlers with no arguments
(it only knows the parent's name selector). There is nowhere in
create interface veth name <name> unit <vid> to put the peer, so veth is
deliberately absent from the table above; create the pair first, then use
the direct form on either end.
Backend Implementations
The Backend interface declares 34 methods. Three implementations ship in
tree; coverage varies per platform and dataplane. The table below lists
each method against the backend and whether it is wired or returns an
error. err means the method is implemented as a stub that rejects every
call with a descriptive error. real means the method drives the
underlying mechanism. Cells with a footnote carry a caveat.
| Category | Method | netlink (Linux) | VPP | stub (non-Linux) |
|---|---|---|---|---|
| Lifecycle | CreateDummy |
real | real (CreateLoopback) | err |
CreateVeth |
real | err (VPP uses memif/TAP) | err | |
CreateBridge |
real | real (BridgeDomainAddDelV2) | err | |
CreateVLAN |
real | real (CreateVlanSubif) | err | |
CreateTunnel |
real [1] | err (pending GoVPP tunnel API) | err | |
CreateWireguardDevice |
real (rtnetlink) | err (requires VPP wg plugin) | err | |
ConfigureWireguardDevice |
real (wgctrl) | err (requires VPP wg plugin) | err | |
GetWireguardDevice |
real (wgctrl) | err (requires VPP wg plugin) | err | |
CreateXFRM |
real (rtnetlink) | err (XFRM is Linux netlink only) | err | |
GetXFRMInfo |
real (rtnetlink+xfrm) | err (XFRM is Linux netlink only) | err | |
DeleteInterface |
real | real (DeleteLoopback/DeleteSubif) | err | |
| Address | AddAddress |
real | real (SwInterfaceAddDelAddress) | err |
RemoveAddress |
real | real (SwInterfaceAddDelAddress) | err | |
ReplaceAddressWithLifetime |
real | ∿ | err | |
AddAddressP2P |
real | err (PPP NCP not supported yet) | err | |
RemoveAddressP2P |
exact installed pair | err (PPP NCP not supported yet) | err | |
| Route | AddRoute |
real | err (use fib-vpp plugin) | err |
RemoveRoute |
real | err (use fib-vpp plugin) | err | |
ListRoutes |
real | err (use fib-vpp plugin) | err | |
RouteLookup |
real (netlink RouteGet) | real (IPRouteLookupV2 LPM) | err | |
ListKernelRoutes |
real | err [3] | err | |
| Link state | SetAdminUp |
real | real (SwInterfaceSetFlags) | err |
SetAdminDown |
real | real (SwInterfaceSetFlags) | err | |
| Properties | SetMTU |
real | real (HwInterfaceSetMtu, then SwInterfaceSetMtu) | err |
SetMACAddress |
real | real (SwInterfaceSetMacAddress) | err | |
GetMACAddress |
real | real (via SwInterfaceDump) | err | |
GetStats |
real | err (pending GoVPP stats API) | err | |
ResetCounters |
real [4] | err (pending sw_interface_clear_stats) | err | |
| Query | ListInterfaces |
real | real (SwInterfaceDump, IPAddressDump per family; no DAD state) | real (net.Interfaces) |
GetInterface |
real | real (SwInterfaceDump, IPAddressDump per family; no DAD state) | real (net.InterfaceByName) | |
GetLink |
real (no address dump; ENODEV is not-found) | real (SwInterfaceDump, no address dump) | real (net.Interfaces) | |
ListNeighbors |
real | err (pending ip_neighbor_dump) | err | |
| Bridge | BridgeAddPort |
real | real (SwInterfaceSetL2Bridge) | err |
BridgeDelPort |
real | real (SwInterfaceSetL2Bridge) | err | |
BridgeSetSTP |
real (rtnetlink IFLA_BR_STP_STATE; works with a read-only /sys) | err (VPP STP varies by version) | err | |
| Mirror | SetupMirror |
real (tc mirred) | real (device SPAN) | err |
RemoveMirror |
real | real (device SPAN) | err | |
ListMirrors |
real (tc filter dump) | real (SwInterfaceSpanDump) | err | |
| Monitor | StartMonitor |
real (netlink multicast) | real (WantInterfaceEvents) | err |
StopMonitor |
real | real | ✕ | |
Close |
real | real | ✕ |
[1] CreateTunnel rejects an unknown kind with unsupported tunnel kind <k>. Valid kinds are gre, gretap, ip6gre, ip6gretap, ipip, sit, ip6tnl,
ipip6.
[2] VPP has no kernel-style address lifetimes. The VPP backend ignores
the validLft/preferredLft arguments and installs the address without
an expiry, matching the exact-or-reject rule only when the operator does
not actually need DHCP lease-aware behaviour on VPP. DHCP runs against
the netlink backend today.
[3] On VPP the kernel FIB is not authoritative; the VPP backend rejects
ListKernelRoutes rather than return misleading data. A VPP FIB dump via
ip_route_v2_dump is the correct replacement and is not yet wired.
[4] Linux netlink has no generic counter-reset syscall. The netlink
backend returns iface.ErrCountersNotResettable and the dispatch layer
falls back to a per-interface baseline-delta model: the current values
become a baseline and GetStats / ListInterfaces / GetInterface
subtract the baseline before returning, so the operator sees
"since last clear" values.
Raw snapshots also carry counter-generation, an opaque process-local
continuity token. The Linux backend reads counters and link lifecycle events
on one socket subscribed to RTNLGRP_LINK, so a delete and recreate at the same
ifIndex changes the token even if the replacement's counters have already
overtaken the previous values. Renames and ordinary down/up transitions preserve
it. Decreases in exported raw 64-bit counters change it; truncation to a 32-bit
wire counter does not.
Socket errors discard the incomplete snapshot and invalidate notification history. The next successful read assigns fresh tokens rather than treating possibly replaced interfaces as continuous. Backends without this metadata report zero. Display-counter clearing only changes the baseline, not raw continuity.
The backend owns this subscribed socket. Close stops its monitor, waits for
any bounded counter read, closes the socket and rejects later reads. The iface
component calls Close when replacing or shutting down its backend; direct
backend users must do the same.
Linux requires statistics to persist “across routine operations like bringing the interface down and up” (networking statistics documentation). Its generic statistics API provides no driver-reset epoch. A driver reset whose new counters exceed the previous sample before the next read cannot be inferred from these values; the token does not claim to detect it.
Netlink (Linux, default)
Pure netlink via vishvananda/netlink, with WireGuard peer/key
operations via golang.zx2c4.com/wireguard/wgctrl. Every method is
implemented; the only caveats are CreateTunnel rejecting unknown kinds
and ResetCounters using baseline-delta (both noted above). No
iproute2 shell-outs. Non-Linux builds of this package compile into the
stub backend described below.
VPP (opt-in, via GoVPP)
Selected by the backend vpp leaf. 17 methods drive the VPP binary API
through GoVPP; 16 return errNotSupported with a string naming the
missing GoVPP call or the plugin gap. The channel to VPP is acquired
lazily on first method call; before the first successful acquire, every
method returns iface.ErrBackendNotReady.
Nothing makes the interface plugin wait for the GoVPP handshake, so its first
apply can run before VPP answers. An apply whose step meets that answer is
deferred whole rather than failed: it creates nothing, reports no error, logs
iface config: apply deferred, the vpp backend has not connected yet at info
level, and the plugin performs it in full (loopbacks and their LCP pairs, MTU,
admin state, addresses) when vppevents.EventConnected fires, then logs
iface config: deferred apply performed at info level. A commit that arrives
first replaces the deferred apply. Any other error still fails the apply. A
deferred apply that then fails is logged as a warning that names the error and
the fix, and stays deferred, so the next commit is measured against what the
dataplane was last told. Nothing retries it on a timer: only a VPP restart
(vppevents.EventReconnected) or the next commit does. After a VPP restart,
vppevents.EventReconnected reloads the backend to drop the dead channel and
its name map, then performs a deferred apply or reconciles the addresses.
An MTU on a NIC goes to its link first (hw_interface_set_mtu), which hands
the frame size to the driver, so a NIC that takes no larger frame refuses it and
the commit fails naming the interface, the asked MTU, the link MTU VPP reports
and the set interface ethernet <name> mtu <link MTU> command that fits it. VPP
then writes the new MTU on every sub-interface of that NIC. The protocol MTUs
(sw_interface_set_mtu) follow, with the per-protocol values left at 0 so each
tracks the L3 MTU. That message alone is never enough: VPP stores it without
reading the link, so it would accept mtu 9000 over a NIC capped at 1500. An
interface whose frame size VPP cannot change (a loopback, a tunnel) takes the
protocol MTU alone, up to the link MTU it reports, and a sub-interface is bounded
by its parent's link MTU.
Some DPDK drivers change the frame size of a stopped port only: i40e answers
EBUSY while the port runs, and VPP relays it as Busy. A raise that meets that
answer on an interface that is admin up is retried with the port stopped: Ze sets
the interface down, raises the link, and sets it up again, which interrupts the
link for that moment. An MTU the link already carries never stops the port, since
the protocol MTU serves it, and an interface left down because VPP would not set
it up again fails the commit saying so.
A VLAN sub-interface Ze creates in VPP takes its parent's MTU: VPP creates it with
an MTU of 0, which it reads as 9000, so without this a parent with no mtu leaf
at 1500 would carry a 9000 VLAN. A sub-interface whose MTU VPP refuses is deleted
and the create fails. So is one whose ingress-qos-map or egress-qos-map VPP
refuses to program: left in place, the existing VLAN would let the commit succeed
with its QoS maps not applied.
Ze sets no MTU on a linux-cp tap itself. With lcp sync true, the default, VPP's
linux-cp copies every MTU change of a paired interface to its tap, on the VPP
side and on the Linux side, and gives a sub-interface with no MTU its parent's
(lcp_itf_mtu_change and lcp_itf_pair_sync_state in
src/plugins/linux-cp/lcp_interface_sync.c, VPP 25.10). With lcp sync false
the tap keeps the MTU it was created with, which is the drift that leaf
describes.
Use the VPP backend when VPP owns the dataplane. Use netlink elsewhere. Routes, stats, counter reset, neighbour table, mirrors, and STP are the main gaps against feature parity.
Stub (non-Linux)
On darwin, macOS, BSD, and Windows the netlink backend package compiles
to a stub whose constructor succeeds. ListInterfaces, GetInterface and
GetLink read the host's interfaces through the Go standard library, so ze init
discovers them and writes an initial config: the loopback is typed by its
flag, and any other interface with a hardware address is classified as
ethernet. Every other method returns
"interface management not supported on <GOOS>". StopMonitor and
Close are no-ops. The stub exists so the rest of the daemon can load
and the binary remains testable on developer machines; real interface
management requires Linux.
The stub never installs itself as the default silently. A macOS daemon can list its interfaces, but any config-driven reconciliation that creates, addresses or changes one sees the explicit error and rejects under exact-or-reject.