Plugins
Ze uses a plugin architecture for all features beyond core BGP session management. Plugins handle RIB storage, route reflection, graceful restart, RPKI validation, NLRI encoding, and more.
Choosing plugins
| Use case | Plugins | Why |
|---|---|---|
| Announce your own routes to upstream | bgp-rib |
Stores routes, elects the best one and replays what this router announced. It does not pass a route learned from one peer on to another |
| Pass routes between peers (transit) | bgp-rib + bgp-rs + bgp-adj-rib-in, each attached to every peer |
bgp-rs forwards what one peer announced to the others, and bgp-adj-rib-in lets it replay to a peer that comes up later. Set session { next-hop self; } toward an external peer: a forwarded route otherwise keeps the next hop it arrived with. Why |
| Route server (IXP) | bgp-rib + bgp-rs + bgp-adj-rib-in |
Forward routes between clients, replay on reconnect |
| With RPKI validation | Add bgp-rpki + bgp-adj-rib-in |
Validate origin AS against ROA cache |
| With merged RPKI events | Add bgp-rpki-decorator (+ above) |
Receive UPDATE events pre-merged with RPKI state |
| With graceful restart | Add bgp-gr |
Hold routes across restarts (RFC 4724) |
| Service healthcheck | bgp-healthcheck + bgp-watchdog |
Monitor services, control route announcement via MED or withdraw. Guide |
| Monitor only (no route export) | ✕ | BGP loads bgp-rib; peer bindings determine which events it stores and whether it may send updates |
| Interface-aware BGP | iface + bgp-rib |
React to OS interface changes -- start/stop BGP listeners when addresses appear/disappear |
| Static routes | (auto-loaded) | Config-driven static routes with ECMP, weighted load balancing, BFD failover. A main-table route competes with other protocols on rib { distance { static } } and the FIB plugin programs the winner; a named-table route is programmed directly. Guide |
| Route policy library | policy (auto-loaded) |
Owns the top-level policy {} root, discovers its filter types from the ze:filter YANG marker, and holds the route field vocabulary filter chains are checked against. A policy {} section loads it, and so does every protocol with a filter attachment, because each lists it in Dependencies |
| OSPFv2 edge plugin | (auto-loaded) | ospf {} starts the native OSPFv2 edge plugin, validates router-id/area/interface config, opens raw IPv4 sockets for active links, runs the Interface and Neighbor State Machines, and handles LSDB flooding |
NLRI family plugins (bgp-nlri-evpn, bgp-nlri-vpn, etc.) are loaded automatically when you configure the corresponding address family. You don't need to declare them.
Config-Driven Loading
BGP itself is a config-driven plugin. If your config has a bgp { } section, BGP loads automatically. If it doesn't, ze starts without BGP (useful for interface-only or FIB-only deployments). Native OSPFv2 follows the same pattern: an ospf { } section auto-loads the ospf edge plugin through ConfigRoots ["ospf"].
A config root can also be a list path. Policy routing registers
ConfigRoots ["policy/route"], so it loads only when the config holds a
policy { route ... } entry; a policy {} block that holds only filters loads
the policy plugin and leaves policy routing (and the firewall it needs) out.
The config tree reports the path of every non-empty list beside its
containers, which is what lets a list path match.
Loading Plugins
Optional plugins are declared in the plugin { } block. Built-in plugins use internal, external processes use external. BGP loads bgp-rib automatically; an explicit RIB declaration is useful when naming that instance:
plugin {
internal rib {
use bgp-rib
}
internal adj-rib-in {
use bgp-adj-rib-in
}
internal gr {
use bgp-gr
}
}
For external processes (scripts, custom binaries):
plugin {
external collector {
run ./collector.py
encoder json
}
}
Plugin Block Settings
The system RIB uses the registration name rib. To run it externally, use
external system-rib { run "ze plugin rib"; }. The equivalent internal
block is internal system-rib { use rib; }. Keep the same route policy and
distance configuration with either placement. bgp-rib is a different plugin:
it stores BGP routes. See Route Policy
for the forwarding and recovery behavior.
| List | Setting | Description |
|---|---|---|
internal |
use |
Name of a built-in plugin to run in-process |
external |
run |
Command to start an external plugin process. Ze runs it as /bin/sh -c <run>, so quoting, pipes and variable expansion work |
external |
encoder |
Wire encoding: json (default) or text |
external |
respawn |
What you expect when the process exits. Read the section below before you write it |
external |
timeout |
Startup stall timeout for this plugin |
An external plugin needs a shell on the host, because the run string is
given to /bin/sh -c. A gokrazy appliance image carries none, so an appliance
runs its plugins from internal blocks. ze doctor reports the missing shell
under doctor-plugin-shell-missing, and a start that meets it names the shell
rather than the plugin.
An external name is 1 to 64 letters, digits and hyphens, the first a letter
or a digit: collector and looking-glass are accepted, my_plugin and
feed.v2 are not. The forked process authenticates to ze under that name, and
ze binds the process's token to it, so the commit refuses any other form with
an error that names the allowed form. A config written before that refusal
still fails its load, and ze doctor reports each such block under
doctor-plugin-connect-back-refused. A process that connects back under a
name the form refuses, or with a token ze did not issue for that name, is
refused too: the refusal names the claimed name, never the token, and show doctor lists it under the same code.
A plugin whose registration declares that it works only inside the daemon
cannot run from an external block: the forked process does not hold the
daemon state it reaches, so it would start and do nothing. ze doctor reports
such a block under doctor-plugin-external-unsupported, with the internal
block that runs it instead.
When a Plugin Fails
The plugin decides, not the configuration. Every plugin tells ze at startup what its own failure means, and ze does that.
| The plugin declares | What ze does when it fails |
|---|---|
restart |
starts the plugin again, up to 5 times in 60 seconds and 20 in the life of the daemon |
ignore |
logs it and carries on without the plugin |
fatal |
stops ze |
| nothing | logs it and carries on, the same as ignore |
fatal is open to any plugin, including one you wrote or one you downloaded.
Running a plugin is accepting its terms. If you disagree with a plugin's policy,
do not configure it, or change its code.
The respawn leaf states what YOU expect. It can ask for less than the plugin
permits and never for more:
| You write | The plugin declares | Result |
|---|---|---|
| nothing | anything | the plugin's declaration decides |
respawn false |
restart |
the plugin is left stopped when it exits |
respawn true |
restart |
the plugin is started again, which it already was |
respawn true |
ignore, fatal, or nothing |
ze does not start. The error names the plugin and the policy it declared |
The last row is deliberate. Honoring the leaf would restart a plugin whose
author forbade it, and honoring the declaration would leave what you wrote with
no effect. Ze picks neither in silence: remove the leaf, or run a plugin that
declares restart.
The leaf is spelled the way ExaBGP spells it, because an ExaBGP process block carries the same option and a migrated configuration keeps working.
Past the restart bound the plugin is disabled, show health reports
doctor-plugin-stopped for it, and ze carries on. A plugin left stopped this
way, or by respawn false, comes back with ze request plugin restart <name>
without restarting ze; the order lifts the restart bound
(command reference).
When a Reload Changes a Plugin's Config
A plugin that reads a section of the config is asked to check a reload's candidate before anything is applied, then to apply it, and to put its previous config back if the commit fails. A plugin that registers those handlers either takes the change at commit or refuses the reload at that check, naming the leaf and the restart steps. A plugin that registers no apply handler, and does not declare that it applies nothing, refuses at that check every reload that changes a section it reads: the commit fails, the running config stays in force, and the error names the plugin, the changed sections and the restart that delivers the change at startup. What each plugin does with a reload, and which leaves need a restart, is listed once, in config reload, under "Plugin config changed" and "What Requires Restart".
When a Reload Changes a Plugin's Capability
A plugin that adds a capability to ze's OPEN (bgp-softver, bgp-hostname,
bgp-role, bgp-llnh, bgp-gr, exabgp-bridge) answers each reload with
the capabilities the new config declares, and ze installs them when the reload
commits. A reload that does not commit puts the previous ones back.
The declarations are checked together after every affected plugin has answered. One commit may move a capability from one plugin to another when the old owner removes it in that same commit. A conflict with an unchanged owner, or two new owners claiming the same capability for the same peer, refuses the commit before any plugin applies it. Each OPEN built during replacement or rollback sees a complete declaration set, never a temporary gap between owners.
| Reload | Result |
|---|---|
| A peer is added that carries one of these capabilities | Accepted: its first OPEN carries them |
| A capability's value changes, or is added or removed, on a peer that stays configured | Accepted. The session restarts to send the new OPEN when the peer advertised that capability code in its own OPEN, because the negotiated result can change. Otherwise the session keeps running, and its next OPEN carries the new value |
A session family is added or removed on a graceful-restart peer with no graceful-restart family list |
Accepted, under the row above: the Graceful Restart payload follows the session families. An explicit family list keeps the payload fixed |
The exabgp bridge's route-refresh or add-path changes |
Accepted, for every peer, under the same rule |
| The exabgp bridge's families are edited | Refused by the bridge, naming the family leaf, with the restart steps: its scripts keep the families they started with |
| A peer is removed, or nothing these plugins declare changes | Accepted |
systemctl restart ze and ze signal stop end each session with a normal
teardown, so peers withdraw ze's routes until it is back. The appliance routes
(request reboot, ze signal reboot) write the Graceful Restart marker and
stop without a NOTIFICATION, so peers that negotiated Graceful Restart keep
ze's routes for the restart time. Under systemd, the restart that keeps the
marker is ze signal restart, a wait until the process has exited, then
systemctl start ze: ze signal restart alone leaves ze stopped, because it
exits cleanly and the unit restarts only on failure.
Binding Plugins to Peers
Each peer declares which plugins receive its events, in attach process blocks.
The process name must match the plugin's internal or external name in the
plugin { } block:
plugin {
internal rib { ... } # <-- this name
}
peer transit-a {
attach process rib { ... } # <-- must match
}
Plugins receive BGP events through the attach blocks on each peer:
peer transit-a {
...
attach process rib {
receive [ update state refresh ]
send [ update ]
}
attach process adj-rib-in {
receive [ update-received state ]
}
}
A peer that attaches nothing feeds nothing. The block is the whole statement of the relationship: a plugin that no peer attaches runs and is fed no peer event, and a peer that attaches one plugin feeds that plugin and no other. Loading a plugin and attaching it are two separate acts, and the second is the one that moves data.
Event Types
| Event | Description |
|---|---|
update |
Route announcements and withdrawals |
open |
OPEN message |
notification |
NOTIFICATION message |
keepalive |
KEEPALIVE message |
refresh |
Route refresh request |
state |
Peer state changes (up/down) |
negotiated |
Capability negotiation results |
eor |
End-of-RIB marker |
rpki |
RPKI validation results |
update-rpki |
Merged UPDATE + RPKI validation (from bgp-rpki-decorator) |
Plugins can register custom event types via the EventTypes field in their registration.
These become valid in receive config directives and subscribe-events RPCs.
Runtime subscriptions
The Go SDK can subscribe at startup with SetStartupSubscriptions or after
startup with SubscribeEvents. Each subscription carries its own namespace,
event list, peer selector, format, and envelope preference. An empty namespace
uses the protocol component's default namespace, normally bgp.
The event name "*" expands at registration time to every event type currently
registered in that namespace. This avoids a wildcard check on every delivered
event. Events registered later require a new subscription.
By default, OnEvent receives the original payload string. Call
SetEnvelope(true) before startup when one handler needs to distinguish events
from several namespaces or event types. Delivery then wraps the payload:
{
"namespace": "vpn-ipsec",
"event": "sa-up",
"payload": {
"peer": "branch-a"
}
}
The envelope sits inside the existing event string, so single-event and batch delivery use the same callback. Subscribers that do not opt in remain byte-compatible with earlier releases.
Directions
attach process my-plugin {
receive [ update-received ] # UPDATEs the peer sends to ze
send [ update ] # permission to announce toward the peer
}
receive names an event type and the direction it is fed in. A plain type means
both directions, so receive [ update ] feeds the program the UPDATEs ze sends
as well as the ones it gets. send carries no direction, because every send type
is sent.
The two lists are independent. A program with send and no receive announces
to the peer and is told nothing by it. A program with receive and no send
watches the peer and may not answer it.
Groups and dynamic peers
Write the block once on a group and every peer that group produces carries it. A member that restates the block REPLACES the group's list for itself, and the other members keep the group's.
plugin {
external looking-glass {
run ./looking-glass.py
encoder json
}
}
bgp {
group transit {
attach process looking-glass {
receive [ update-received state ]
}
peer transit-a { } # updates and state
peer transit-b { } # updates and state
peer transit-c {
attach process looking-glass {
receive [ state ] # state ONLY
}
}
}
}
transit-c is fed state and no update. Its own list stands in for the group's
rather than adding to it, which is how every leaf-list in a group merges. To
widen one member, restate the whole list you want it to have.
A peer a DYNAMIC group creates carries the group's blocks too. Such a peer is
named by no config: ze generates dyn-<address> when it accepts the
connection. Nothing needs to name it, because ze reads each peer's blocks after
the group merge, not from the text of the config file.
bgp {
router-id 192.0.2.1
session {
asn {
local 65000
}
}
group ix {
connection {
remote {
ip dynamic
connect false
range 198.51.100.0/24
}
local {
ip 198.51.100.1
accept true
}
}
attach process looking-glass {
receive [ update-received state ]
}
}
}
Every member that connects from 198.51.100.0/24 feeds looking-glass its
received UPDATEs and its session state. show event delivery lists them by the
address each one connected from.
The send permission
send is a permission, and ze enforces it on the peers a command resolves to.
A program that issues send bgp * update ... reaches the peers that attach it with
send [ update ], and no others. A peer the program is not attached to is
dropped from the command, and ze writes one WARN naming the peer, the process
and the message type. When the selector names ONLY peers that refuse it, the
whole command fails and the program is told why.
The three BASE message types are separate permissions. A plugin registers more,
and naming one auto-loads the plugin that enables it, enhanced-refresh from
bgp-route-refresh among them:
| Type | Permits | Commands |
|---|---|---|
update |
originating routes toward the peer | update text ... add, update text ... del, an End-of-RIB marker, a named commit |
refresh |
asking the peer to re-advertise | peer <sel> refresh, borr, eorr, clear soft |
raw |
writing a whole BGP message the program built itself | send bgp <sel> raw ... |
raw is the widest of the three, because the bytes can be any BGP message, an
OPEN or a NOTIFICATION included. send [ update ] does not imply it.
send [ * ] grants all three, and every send type registered later.
An operator at the CLI, over SSH or through the REST API is not a process and is not gated by this: their authority is the one AAA already checked.
Counter: ze_bgp_send_refused_total{process,type} counts messages the permission
refused. A non-zero value with no config change means a program is asking for a
peer it was never attached to.
Reading the delivery graph
show event delivery prints the peer-to-process edges the running config
produces: one block per peer, one row per attach process block on it, with the
event types that process is fed and the message types it may send toward that
peer. A peer with no attach process block shows no process.
The daemon builds this from the RESOLVED settings of each peer, so a block a
peer inherits from its group is shown on the peer. A token the event registry
does not know is listed under unresolved and carries no edge, which is what an
operator sees when a custom event type names a plugin that did not load.
These edges DECIDE delivery. A peer hands a program an event when the peer's
block grants the type and the program subscribed to it; a peer that attaches no
block for a program feeds it nothing, whatever the program asked for at startup.
That holds for ze's own plugins as well: a config that loads bgp-rib and
attaches it to no peer stores no route. When the two halves disagree, the daemon
names each peer, process and event type in the log rather than dropping the
event in silence.
The name in attach process <name> is the name the program runs under: the one
in its plugin { external <name> } or plugin { internal <name> } block, or the
registry name of a plugin loaded with --plugin.
Attached to a running peer
A reload can attach a plugin to a peer whose session is already up, through a
receive-only attach process block (config reload). The
plugin then receives a state up for that peer with reason attached. That up
does not start the session: the peer's initial routing update already went out,
so the up carries no initial-replay token and holds back no End-of-RIB. A plugin
that replays routes toward a peer when it comes up must skip that replay on an
attached up, or it sends the peer routes it already holds. bgp-rib,
bgp-rr, bgp-rs, bgp-watchdog, redistribute-orchestrator and
bgp-adj-rib-in skip it. Routes the peer announced before the attach reach the
plugin through the ROUTE-REFRESH Ze sends after the up.
A reload that detaches a plugin sends it a state down for that peer alone,
and the session stays up for every other plugin. The down names the
still-running session as its session-generation, as a real down names the
session it ended. The plugin drops what it holds
for the peer. When that plugin is bgp-rs, the other peers lose the routes it
forwarded from that peer, every one that session wrote included: each is replaced by the best remaining route or
withdrawn, although the peer's session never went down. A plugin whose grant
changed receives a down, then an attached up.
Invocation Modes
| Mode | Config Syntax | Description |
|---|---|---|
| Internal | internal rib { use bgp-rib; } |
Compiled-in plugin using net.Pipe for startup and DirectBridge for hot paths |
| External | external feed { run "/usr/local/bin/my-plugin"; } |
External binary or script using TLS connect-back |
Internal mode (use pluginname) runs a compiled-in plugin as a goroutine within the ze process. Startup still uses the same YANG RPC handshake as external plugins, then DirectBridge bypasses socket I/O for supported hot paths. External mode starts a separate process; that process connects back to the plugin hub over TLS and authenticates with its per-plugin token.
Built-In Plugins
List available plugins:
ze show plugin list
Reporting a Setup Outcome
A plugin that sets something up in its own init() records what happened, so
an operator can ask why a feature is absent. The outcome and reason columns
of show plugin list carry it:
ze show plugin list
The record is one call, made from the plugin's init() beside its
registration:
| Outcome | Meaning | Effect on the daemon |
|---|---|---|
registry.SetupSucceeded |
The setup completed | None |
registry.SetupFailedSoft |
The feature is absent and the daemon runs correctly without it | The daemon starts |
registry.SetupFailedHard |
The daemon cannot run without it | The daemon refuses to start, naming every failing plugin |
The two writes a plugin makes from init(), registry.Register and
registry.RecordSetup, are keyed by the plugin name and neither reads the
other, so their order does not matter. Go initializes the files of one package
in filename order and no plugin author has to know it.
Recording is optional at the call site, and a plugin that records nothing is
listed as unknown. That is the signal that the plugin owes a record, not that
it is absent. memlock is the worked example: it locks the executable in
init() and records a soft failure carrying the cause and the remedy when
RLIMIT_MEMLOCK is too small. Its memlock-rlimit doctor check answers the
question this record cannot: whether the host could lock the executable at all,
BEFORE ze runs (docs/architecture/doctor-and-health-checks.md).
A reason string reaches CLI output as data, so never put a secret in one.
Storage and Policy
| Plugin | Purpose | Typical Binding |
|---|---|---|
bgp-rib |
Mandatory Route Information Base with in-memory sent-history replay on reconnect | receive [ update state refresh ] send [ update refresh ] |
bgp-adj-rib-in |
Adj-RIB-In (raw hex replay, auto-replays on peer-up) | receive [ update-received state ] |
bgp-rs |
Route server (RFC 7947): forwards each UPDATE, and a client without ADD-PATH is sent the bgp-rib-elected route when sources contest a prefix |
receive [ update-received state open-received refresh ] send [ update ] |
bgp-rr |
Route reflector (RFC 4456) | receive [ update-received state open-received ] |
bgp-watchdog |
Deferred route announcement | receive [ state ] send [ update ] |
Migrating from bgp-persist
bgp-persist has been removed. Delete each plugin { internal <name> { use bgp-persist } } block, or its external invocation. Replace its peer
attach process <name> blocks with a binding to the mandatory RIB:
attach process bgp-rib {
receive [ update state refresh ]
send [ update ]
}
BGP loads bgp-rib automatically, so no replacement internal/use block
is required. If your configuration already names a RIB instance explicitly,
use that instance name in the binding and merge the grants into its existing
binding rather than creating a second instance.
Automatic loading does not grant replay or export authority. The binding must
deliver both received and sent UPDATEs (update), peer state and refresh
events, and permit UPDATE sends. Feed source peers' received UPDATEs and state
to the same RIB so it can validate forwarded history. A selection-only
received/state binding does not retain sent history or authorize replay.
There is no compatibility alias, replacement cache command, or separate
persistence store.
bgp-rib keeps sent history in process memory across a peer disconnect.
On peer-UP, its initial replay restores only eligible history. Forwarded routes
must still match their source-Peer incarnation, received path and message
revision in the authoritative RIB. Replay preserves native NLRI identity and
ADD-PATH identity rather than treating every route as a prefix. Locally
originated history is explicitly identified; missing source provenance does
not make a route local. Config-static routes are sent from the current
configuration by the reactor, not restored from old sent history.
Replay and its completion report use the initial-replay session token
captured from that peer-UP event. A stale or closed Initial replay session
cannot authorize a write on a replacement session, and restoration cannot
overwrite a route already accepted on the new session. A readiness report
means the replay attempts finished, not that every historical route was
delivered. See replay authority.
This is reconnect recovery, not durable storage: sent history does not survive
a daemon or RIB plugin restart. clear bgp rib out performs same-session
refresh; it does not grant Initial replay authority.
A peer selector given to bgp-rib by name matches the configured name the
session's state events carried (selectsPeer), since bgp-rib keys sessions by
address. A session whose events named it nothing is selected by no name.
Protocol
| Plugin | Purpose | Typical Binding |
|---|---|---|
bgp-gr |
Graceful Restart (RFC 4724) and Long-Lived GR (RFC 9494) | receive [ open-received state eor ] send [ update ] |
bgp-rpki |
RPKI origin validation (RFC 6811) | receive [ update-received state ] |
bgp-rpki-decorator |
Merged UPDATE+RPKI events | receive [ update-received rpki ] |
bgp-route-refresh |
Route Refresh (RFC 2918) | send [ refresh ] |
bgp-role |
BGP Role (RFC 9234) | -- |
bgp-hostname |
FQDN capability | -- |
bgp-softver |
Software version capability | -- |
bgp-llnh |
Link-local next-hop (RFC 2545) | -- |
bgp-bmp |
BMP receiver + sender (RFC 7854) | receive [ state update open notification keepalive refresh ] |
Infrastructure
| Plugin | Description | Process Binding |
|---|---|---|
iface |
OS interface orchestration: loads backend, dispatches operations | -- (Bus events, no peer binding) |
iface-netlink |
Netlink backend for iface: manage, monitor, bridge, sysctl, mirror | -- (registered as iface backend) |
iface-dhcp |
DHCP client: DHCPv4/DHCPv6 lease acquisition and renewal | -- (Bus events) |
iface-ra |
IPv6 Router Advertisement sender (RFC 4861): advertises prefixes, flags, and RDNSS resolvers (RFC 8106) on a LAN unit. Hosts autoconfigure addresses, learn a default router, and learn resolvers. Linux and netlink only. Guide | -- (Config-driven, raw ICMPv6 socket per unit) |
rib |
System RIB: resolves authoritative routes and publishes forwarding changes | Internal or external; no peer binding |
fib-kernel |
FIB kernel: programs OS routes from system RIB via netlink | -- (Bus events, no peer binding) |
fib-p4 |
FIB P4: programs P4 switch from system RIB via gRPC/P4Runtime (noop backend) | -- (Bus events, no peer binding) |
fib-vpp |
FIB VPP: programs VPP FIB from system RIB via GoVPP. MPLS label push (IPRouteAddDel with LabelStack), swap/pop (MplsRouteAddDel), interface enable (SwInterfaceSetMplsEnable). | -- (Bus events, no peer binding) |
firewall-vpp |
VPP ACL backend for firewall: translates ze Match/Action types to VPP ACL rules, read-merge-write bindings preserving foreign ACLs | -- (registered as firewall backend) |
policy-routes |
Policy-based routing via nftables packet marking and kernel ip rules. Steers traffic to alternate routing tables or next-hops based on L3/L4 match criteria. Guide | -- (Config-driven, depends on firewall) |
sysctl |
Kernel tunable management: three-layer precedence (config > transient > default), restore on stop. Named profiles (dsr, router, hardened, multihomed, proxy) for interface units. User-defined profiles. | -- (Bus events, CLI commands) |
dhcpserver |
DHCP server (RFC 2131/2132) with PXE boot support (RFC 4578): pool management, lease tracking, static mappings, PXE option injection (options 43/60/66/67/93) for BIOS/UEFI bootfile selection; option 6 from the subnet's dns-source list (default source [ configured received ]: the subnet's dns-server, else the learned resolvers) |
-- (Config-driven, UDP listener) |
tftpserver |
Read-only TFTP server (RFC 1350): serves bootloader files for PXE provisioning in 512-byte blocks with stop-and-wait ACK, concurrent transfer limiting, path traversal protection | -- (Config-driven, UDP listener on port 69) |
imageserver |
HTTP image server for PXE provisioning: serves gokrazy disk images, installer boot files, and pre-provisioned zefs databases with SSH credentials. Own HTTP listener, path traversal protection, Range request support | -- (Config-driven, HTTP listener) |
geodns |
GeoDNS server (RFC 1035): DNS answers selected by client source IP. Client IP from EDNS0 client-subnet (RFC 7871) or packet source; CIDR longest-prefix selects a named host-set (A/AAAA/SRV records); synthesizes SOA/NS/glue. show geodns, ze_geodns_* metrics, UDP+TCP listeners (default 127.0.0.1:5300); compile-out-able with the ze_geodns build tag |
-- (Config-driven, UDP+TCP listeners) |
as112 |
AS112 anycast DNS node (RFC 7534, RFC 7535): authoritative-only sink for misdirected RFC 1918 / link-local reverse-DNS queries plus the EMPTY.AS112.ARPA DNAME-redirection zone. Four fixed anycast host addresses (never operator-typed) registered via the iface address-ownership registry, IP_FREEBIND listeners, optional allow-from client-source access list (loopback always permitted). show as112, as112 health (one-shot query for the healthcheck probe), ze_as112_* metrics, UDP+TCP listeners on port 53. Now also a BGP redistribute source: redistribute { destination bgp { import as112 } } originates the four covering prefixes into BGP, with asn (origin AS, default 112), community, and a watchdog health-gate under service { as112 }. |
-- (Config-driven, UDP+TCP listeners) |
dnsresolver |
LAN DNS resolver (RFC 1035, RFC 2181), configured under service { dns-resolver }: UDP and TCP listeners on the addresses the operator names, an allow-from client prefix list, an ordered forward server list, and a forward dns-source list that chooses the upstreams from those servers and the resolvers Ze learned (default source [ configured received ], mode first-usable). A commit is refused for a wildcard listener address, for an enabled resolver with no allow-from prefix, and for one with no forward server whose forward dns-source names no received source. A query from an allowed client with RD set is answered from the cache or forwarded to the upstream servers in order, and SERVFAIL is returned when no upstream gives a usable answer within the timeout and the per-query work budget . docs/features/dns-resolver.md |
-- (Config-driven, UDP+TCP listeners) |
traffic-usage |
eBPF TCX per-(port, protocol) and opt-in per-IP byte accounting (IPv4, monitoring only). Pure-Go assembled eBPF (cilium/ebpf asm.Instructions, no C/clang). Prometheus ze_traffic_usage_* metrics, show traffic usage [name <interface>]. Linux >= 6.6. Guide |
-- (Config-driven, Bus events) |
L2TP
| Plugin | Description |
|---|---|
l2tp-auth-local |
Static user/password authentication for L2TP PPP sessions (PAP/CHAP-MD5/MS-CHAPv2) |
l2tp-auth-radius |
RADIUS authentication (Access-Request), accounting (Start/Stop/Interim-Update), CoA/DM listener, and Access-Accept attribute extraction (Framed-IP-Address, Framed-Pool, Session-Timeout, Idle-Timeout, Filter-Id, Acct-Interim-Interval) |
l2tp-pool |
Bitmap-backed IPv4 address pool with default and named pools; Framed-IP-Address bypasses pool (direct RADIUS IP assignment), Framed-Pool selects a named pool; IPCP DNS from the pool's dns-source list (default source [ radius configured received ]: the Access-Accept's MS-Primary-DNS-Server and MS-Secondary-DNS-Server, else dns-primary/dns-secondary, else the learned resolvers) |
l2tp-shaper |
TC traffic shaping (TBF/HTB) on pppN interfaces with configured default rates, RADIUS CoA rate updates, and initial rate from Filter-Id at session establishment |
These plugins register via the L2TP handler registry (RegisterAuthHandler,
RegisterPoolHandler) at init time. Only one auth handler is active at a
time (last registered wins). The pool and shaper plugins subscribe to
session lifecycle events via the EventBus.
See L2TP guide for configuration details.
The iface plugin defines a Backend interface and loads a backend by name (YANG
backend leaf, default netlink). The iface-netlink backend handles all Linux
interface operations. iface-dhcp is a separate plugin for DHCP client lifecycle.
BGP reacts to address events by starting/stopping listeners. Uses a JunOS-style
two-layer model: physical interfaces + logical units (VLANs).
Bus topics published:
| Topic | When |
|---|---|
interface/created |
Interface appeared |
interface/deleted |
Interface removed |
interface/up |
Link state to up |
interface/down |
Link state to down |
interface/addr/added |
IP assigned |
interface/addr/removed |
IP removed |
The rib plugin consumes the daemon Loc-RIB's selected routes, after import
policy and administrative-distance selection. It resolves next hops, retains
equal-cost paths, and publishes forwarding changes through its authenticated
output lease. show rib candidates queries the daemon's full candidate table,
including hidden and standby paths, in either placement.
An external placement uses the same rib implementation:
plugin {
external sysrib-owner {
run "ze plugin rib";
encoder json;
}
}
For internal placement, use internal sysrib-owner { use rib; } in the
plugin block. Both placements apply their distance declaration at the daemon;
an external process never changes a disconnected local copy. Replay requests
subscribe at startup and wait for the consumer's complete view before output.
The fib-kernel plugin programs OS routes from the system RIB into the kernel
via netlink (Linux). Uses a custom rtm_protocol ID (RTPROT_ZE=250) to identify
ze-installed routes. Crash recovery marks existing ze routes as stale at startup
and sweeps them after reconvergence. A kernel route monitor restores externally
deleted Ze routes with their accepted forwarding metadata. Recovery requires
matching ownership of the prefix, table and priority, so withdrawn routes stay
withdrawn. Conflicting foreign replacements are retained and reported.
Repair replays reassert each unchanged, successfully installed IPv4 route after a fresh check of its exact kernel identity. The write cannot create a route or replace a preferred foreign alias. Missing or modified routes use the full ownership check and repair path. IPv6, startup routes, changed forwarding and pending failures also retain that full path.
An IPv4 unicast route whose entire forwarding group names devices without gateways uses link scope. It can therefore resolve a recursive gateway outside the interface's configured subnet. A gateway in any primary, equal-cost or backup path keeps universe scope, including gateways from another address family. IPv6 and discard routes retain universe scope.
Every removal of a Ze route (a withdraw, the stale sweep, the stop flush) asks the kernel for Ze's route on that prefix and table at any scope. Linux deletes an IPv4 route only when the request names its scope or names none, so a delete that named universe scope would miss a link-scope route and leave it installed.
Shutdown closes IP forwarding admission before joining the route monitor. An
already admitted backend operation can finish, but a replay admits no later
entries. The writer joins that operation before closing the backend, including
when forwarding is retained. With flush-on-stop true, it then runs the owned
route cleanup. Ordinary repair replays still reassert installed routes.
The cleanup removes every route the writer installed, from the table it was
installed in, and every route a previous run left that startup retained and
the sweep has not yet removed, from each table startup listed it in. It then
lists the kernel's Ze routes in every table. A delete the kernel refused, or a
flushed prefix the kernel still lists in any table, raises one
doctor-fib-flush-incomplete error (subject flush) and one WARN log line under the same code,
both carrying the counts, the first prefix and its table, and the command that removes what
is left: ip route flush proto 250 table all; ip -6 route flush proto 250 table all.
When the listing itself fails, the error says the kernel could not be read and
carries the listing's error. It reports no count of routes left, because none
was taken.
Bus topics in the FIB pipeline:
| Topic | Publisher | Subscriber | Payload |
|---|---|---|---|
system-rib/best-change |
rib, through its daemon output lease |
FIB plugins | Batch of system-wide forwarding changes |
fib/external-change |
fib-kernel |
monitoring | External route change on ze-managed prefix |
Bus topics in the sysctl pipeline:
| Topic | Publisher | Subscriber | Payload |
|---|---|---|---|
sysctl/default |
fib-kernel, iface |
sysctl |
Plugin-required kernel default (key, value, source) |
sysctl/set |
CLI | sysctl |
Transient value from user (key, value) |
sysctl/applied |
sysctl |
any | Notification after kernel write (key, value, source) |
sysctl/show-request |
CLI | sysctl |
Request active keys table (request-id) |
sysctl/show-result |
sysctl |
requester | Active keys JSON (request-id, entries) |
sysctl/list-request |
CLI | sysctl |
Request known keys table (request-id) |
sysctl/list-result |
sysctl |
requester | Known keys JSON (request-id, entries) |
sysctl/clear-profile-defaults |
iface |
sysctl |
Clear stale profile defaults for an interface before re-emission (interface) |
Route Filters
Plugins can declare named filters at stage 1 for import and/or export filtering. A filter declares the route fields it reads, and the engine sends it exactly those fields. See The Route Fields of filter-update for the names and their value shapes. Filters respond accept, reject, or modify (the fields they change). See Route Filters and Redistribution for configuration.
A single plugin can offer multiple named filters. Config references them as
<plugin>:<filter> (e.g., rpki:validate, community:scrub). <plugin> is
the plugin's own name, not the name of the plugin { } block that runs it: a
block internal gr { use bgp-gr; } serves bgp-gr:<filter>, and two blocks
running the same filter plugin are refused.
| Category | Behavior | Example |
|---|---|---|
| Mandatory | Always on, cannot be overridden | rfc:otc |
| Default | On by default, overridable per-peer | rfc:no-self-as |
| User | Explicit in filter {} config |
rpki:validate |
Filters can declare overrides to remove default filters from the chain
(e.g., allow-own-as:relaxed overrides rfc:no-self-as for a specific peer).
Route Evaluators at the Loc-RIB
The Loc-RIB import chains (isis, ospf, static, connected and rib
filter { import { ... } }) run in the engine on every route insert, so they
never call a plugin process. A filter type works there only when its package
registers an in-process route evaluator with policy.MustRegisterRouteEvaluator
from its init(). prefix-list and family-filter register one, so their
instances under policy {} can be named in those chains. A chain that names
a type with no evaluator is refused at load, and the message lists the types
that have one. A plugin whose config holds a Loc-RIB attachment declares it
with policy.MustRegisterRIBImport. See Route Policy at the Loc-RIB.
Cross-Protocol Redistribute (redistribute-orchestrator)
redistribute-orchestrator is the single subscriber that dispatches non-consumer
protocol route-change events to registered RedistConsumer implementations.
Unlike the route filter chain above (which gates intra-BGP traffic),
the orchestrator lets operators redistribute locally-originated routes from
other protocols (L2TP sessions, connected interface prefixes, static routes,
future OSPF / ISIS) into destination protocols (BGP, future OSPF/ISIS).
Config:
policy {
match redistribute-bgp-l2tp { family [ ipv4/unicast ipv6/unicast ]; }
}
redistribute {
destination bgp {
import connected;
import static;
import l2tp { filters [ redistribute-bgp-l2tp ]; final accept; }
}
}
Each destination <protocol> names a registered consumer. Under it,
import <source> enables one non-consumer protocol. The import rule's
source is the protocol's canonical name registered via
redistribute.RegisterSource. Per-source family lists narrow which
address families are redistributed; an empty list means "all families".
An import is scoped to its enclosing destination: an import under
destination bgp feeds only BGP, not OSPF/IS-IS.
The orchestrator auto-loads when redistribute {} appears in the
config. No plugin { internal redistribute-orchestrator { use redistribute-orchestrator; } }
block is required.
The rules need no attach process plumbing either. Ze derives two bindings from
the redistribute root, and they are the only bindings in the delivery index Ze
writes for you. A peer whose config already names the process keeps its own
binding unchanged. When that grant is narrower than what the plugin declared,
for example attach process bgp-rib { receive [ state ]; } beside an
import bgp rule, the daemon logs event delivery: the plugin declared an event the peer does not grant it for each type left out, naming the peer and
the process, as it does for any plugin fed less than it declared.
| The rule | The derived binding |
|---|---|
import bgp |
receive [ update state refresh ] toward bgp-rib, because the Loc-RIB is the source and a plugin sees a peer's UPDATEs only through a grant |
destination bgp |
receive [ state ] and send [ update ] toward redistribute-orchestrator, because that plugin puts the route on the peer's wire |
Reactor per-peer NEXT_HOP substitution applies: when the producer leaves
NextHop zero, the reactor stamps each peer's local session address as the
NEXT_HOP. Producers that have an explicit address pass it through verbatim.
Late-join replay: a producer emits once, so whoever arrives after that emit
holds nothing. Two arrivals are late, and each fires a replay request carrying an
opaque ReplayID token that every producer echoes on a re-emit of its CURRENT
set. Producers stay target-agnostic: the orchestrator alone holds the
ReplayID -> target mapping, and it holds it for a TTL because an
out-of-process producer re-emits asynchronously.
| The late arrival | What the replay reaches |
|---|---|
| a BGP peer's down-to-up edge | that ONE peer, through the BGP consumer. This closes the gap for a dynamic or inbound peer not in the reactor map at injection time |
| a destination protocol's consumer becoming registered | that ONE consumer, with the ordinary all-peers fan-out. Nothing orders the plugin startup tiers, so a producer can emit before the consumer exists |
Counters: ze_bgp_redistribute_events_received, _announcements,
_withdrawals, _filtered_protocol_total,
ze_redistribute_chain_rejected_total{destination,source} (routes an import
rule's chain rejected), ze_bgp_redistribute_replay_total{source} (routes replayed, to a newly
established peer or to a consumer that registered late).
Prefix-List Filter (bgp-filter-prefix)
bgp-filter-prefix is a built-in filter plugin that matches IPv4 and IPv6
routes against ordered prefix lists defined in policy { prefix-list NAME { ... } }. Each list is a sequence of entry <CIDR> blocks with
optional ge / le bounds and an action of accept or reject; the
first matching entry wins and no match is an implicit deny.
A BGP peer chain reaches a list through a match instance:
policy { match FROM-CUSTOMERS { prefix-list CUSTOMERS; } }, then
filters [ FROM-CUSTOMERS ]. The daemon judges the match in process over the
UPDATE, so the plugin answers no filter-update. A BGP chain that names the
list itself, as CUSTOMERS or prefix-list:CUSTOMERS, is refused at commit:
no BGP filter serves the prefix-list type. The plugin-qualified
bgp-filter-prefix:CUSTOMERS is refused at config load, as is any
<plugin>:<name> naming a compiled-in plugin that declares no filter.
A prefix-list can also run in a Loc-RIB import chain, named there by its bare instance name. It judges one route at a time with the same entry matching: the first matching entry decides and no match rejects the route (Route Policy at the Loc-RIB).
The match judges each prefix of the UPDATE on its own, in the legacy NLRI and
Withdrawn Routes and in each MP_REACH_NLRI and MP_UNREACH_NLRI, so an IPv6
UPDATE or IPv4 unicast carried in MP_REACH_NLRI is judged the same way. An
UPDATE whose prefixes are all accepted passes, one whose prefixes are all
denied is rejected, and a mixed one passes with its denied prefixes dropped. A FlowSpec UPDATE with a
denied destination is rejected whole, because the engine does not rewrite a
FlowSpec NLRI (Route Policy at the Loc-RIB, "The match
conditions").
AS-Path Filter (bgp-filter-aspath)
bgp-filter-aspath matches the UPDATE's AS-path against ordered regex
entries defined in policy { as-path-list NAME { entry REGEX { action accept|reject; } } }. Each entry's regex is matched against the AS-path
string using Go's RE2 engine (linear time, inherently ReDoS-safe). First
match wins; no match is implicit deny.
Config authors should use [0-9] instead of \d in regex strings because
ze's config parser interprets backslash as an escape character.
A BGP chain reaches a list only through the as-path condition of a match
instance, which the plugin registers; a chain naming the list itself is
refused at commit:
policy { match FROM-65001 { as-path TRANSIT; } } passes an UPDATE the
TRANSIT list accepts and rejects any other, the same regexes over the same
AS-path string the list judges as a chain member. Only a BGP chain judges it:
a Loc-RIB route carries no AS path, so a match naming as-path there is
refused. The editor completes the configured lists, and a commit naming any
other is refused.
The AS-path string every pattern matches against. One reader produces it,
the route view's ASPathText, and the as-path condition takes it from
there. It is the
space-separated decimal ASNs with no brackets, for example 65001 65002 65003.
Two properties of it are load-bearing for a pattern you write:
- Every segment type is flattened into one list with no marker. AS_SEQUENCE, AS_SET, AS_CONFED_SEQUENCE and AS_CONFED_SET all contribute their ASNs in order. A pattern therefore cannot ask which segment an ASN came from, and an ASN inside an AS_SET is matched like any other.
- It is the path the route traversed, not the AS_PATH attribute as encoded. A peer that did not negotiate the four-octet AS capability puts AS_TRANS (23456) in AS_PATH wherever a four-octet AS number belongs and sends the real numbers in AS4_PATH. The reactor reconstructs one path from the two before any filter sees the text (RFC 6793 Section 4.2.3), so a pattern and a hop count read the real AS numbers and never AS_TRANS.
- A route with no AS_PATH gives the empty string, and a route with one ASN gives that ASN alone, with no brackets around it.
Reject-ASN Filter (bgp-filter-path-asn)
bgp-filter-path-asn drops a route whose AS_PATH carries a listed ASN at a
listed position, and leaves the session up. It is the cheap answer to a peer
leaking its transit, which the max-prefix limit otherwise stops by dropping the
whole session.
policy {
reject-asn NO-TRANSIT {
indirect [ 174 3356 ]
}
match reject-asn-NO-TRANSIT {
reject-asn NO-TRANSIT
}
}
bgp {
router-id 192.0.2.1
session {
asn {
local 65000
}
}
peer peer-a {
filter {
import {
filters [ reject-asn-NO-TRANSIT ];
final accept;
}
export {
filters [ reject-asn-NO-TRANSIT ];
final accept;
}
}
}
}
A list is an unordered reject SET. A route matching any leaf-list is rejected,
and there is no first-match-wins, which is what keeps the type different from
as-path-list. Both types stay and can sit in one chain.
A list carries seven keywords, and the one an ASN is written under says WHERE in the path it is unacceptable.
| Keyword | Rejects the ASN when it is |
|---|---|
direct |
the peer you are talking to, prepends collapsed |
indirect |
anywhere it is NOT that peer: transit or origin |
transit |
between the peer and the route's origin |
origin |
the last ASN of the path |
anywhere |
anywhere in the path |
nth <n> |
at collapsed position n, counted from you, 1-based |
regex |
not an ASN at all. The values are Go RE2 patterns matched against the whole AS-path string |
Six are plain leaf-lists. nth takes a number, so it is written
nth 2 [ 3491 ];, and it counts RUNS rather than tokens: a run of consecutive
identical ASNs advances the count once, so a peer cannot move your rule by
prepending.
indirect is the everyday keyword, and it is worth the sentence it says: do not
give me anything you reached through a transit provider, and peering with that
provider directly is still fine. direct is the sending PEER's ASN, not the
first ASN in the path: a path [3356 65001] from AS65001 carries 3356 at index
zero and is still a leak, which is the route-server case RFC 7454 Section 9
names. nth 1 asks the positional question instead, and catches that same 3356.
On an export chain the filter is told the DESTINATION peer, not who sent the
route, so nothing is direct there and indirect covers the whole path. That is
the export half of RFC 7454 Section 9: do not advertise a path through your
upstream to a peer you do not sell transit to.
The same ASN written under two keywords unions, so indirect plus direct is
anywhere.
A list that names nothing at all, whether its leaf-lists are absent or written
empty, is REFUSED at load: an empty reject set accepts every route while reading
in the config file like a safety filter. So is an nth entry with no ASN. A
pattern that does not compile and one longer than 512 characters are refused too,
naming the list. The ASN leaf-lists are uint32, an nth index is bounded
1..255, and the keywords are the schema's own, so a word where a number belongs
and a keyword nobody declared are both refused by the config parser.
Every reject logs the offending ASN or pattern, the position it matched at, the list, the peer and the direction.
A BGP chain reaches a list through the reject-asn condition of a match
instance: policy { match NO-TRANSIT-IN { reject-asn NO-TRANSIT; } }, then
filters [ NO-TRANSIT-IN ]. A chain naming the list itself is refused at
commit.
Every reject also increments ze_filter_path_asn_rejects_total, labeled with
the direction, the position that matched and the reason. The peer stays in the
log line: a peer address in a label would grow the series count with the session
count.
Ze ships no ASN set. The operator lists the ASNs, and the list means exactly
what it says. show bgp reject-asn known transit-free prints the well-known
transit-free ASNs as a indirect [ ... ]; block to paste, with the sources and the
curated date as comments, and after the paste the config holds the numbers.
show bgp reject-asn lists what each list holds, and
show bgp reject-asn name <name> answers for one of them.
Community Match Filter (bgp-filter-community-match)
bgp-filter-community-match checks for presence of a specific community
value in the route's standard, large, or extended community attributes.
Defined in policy { community-match NAME { entry COMMUNITY { type standard|large|extended; action accept|reject; } } }. First match wins;
no match is implicit deny.
Separate from the tag/strip community plugin (bgp-filter-community) because
intent differs: this plugin filters (accept/reject), that one modifies
(tag/strip). They can coexist in the same deployment.
A BGP chain reaches a list through the community NAME condition of a
match instance, which the package registers: the daemon judges it over the
attributes' octets, with the list's own first-match order and implicit deny
(docs/guide/route-policy.md, "The match conditions"). A chain naming the
list itself is refused at commit.
Well-known community names (no-export, no-advertise, blackhole, etc.)
work as match values.
Route Attribute Modifier (bgp-filter-modify)
bgp-filter-modify applies declared operations to every route that reaches it
in the filter chain. A definition that states a match container applies them
only to the routes that meet it. The operations are:
Set (absolute value): set { local-preference 200; med 50; origin igp; next-hop 10.0.0.1; as-path-prepend 3; }. Only present leaves are applied.
next-hop writes the NEXT_HOP attribute, which carries an IPv4 address only,
so an IPv6 address there is refused when the config loads. as-path-prepend
takes 1 to 32.
Increment/Decrement (relative adjustment): increment { local-preference 50; }
or decrement { med 30; }. Supported attributes: local-preference, med, aigp.
Increment saturates at uint32 max (4294967295). Decrement floors at 0.
Set and increment/decrement for the same attribute are mutually exclusive.
The arithmetic reads the attribute the route carries. When the route carries
none, med and local-preference start from a declared default and aigp is
left alone:
| Attribute | Absent on the route | Why |
|---|---|---|
med |
starts from 0, and the result is written, so a route that had no metric gains one | RFC 4271 Section 9.1.2.2 names 0 as the value of an absent MULTI_EXIT_DISC |
local-preference |
starts from 100 | RFC 4271 Section 9.1.1 leaves the value to local policy. 100 is the value FRR and BIRD use |
aigp |
nothing is written, and the route keeps no AIGP attribute | RFC 7311 Section 4.1 removes a route with no AIGP TLV from consideration rather than scoring it, and Section 3.4.1 forbids adding the attribute outside the AIGP administrative domain |
The first two values are configurable, and the table shows what they are when
nothing is written: bgp { defaults { attribute { med 0; local-preference 100; } } }. Both leaves take 0 to 4294967295. They govern this arithmetic alone, so
a changed med does NOT move the Decision Process, which compares an absent
MULTI_EXIT_DISC as 0 whatever the leaf holds. There is no aigp leaf, for the
reason the third row gives.
decrement { med 30; } on a route that carried no metric therefore leaves
med 0 on the route. An RFC-default receiver reads an absent MED and a MED of 0
the same way, so this changes nothing for it. A receiver configured to treat a
missing MED as the worst value reads them differently, and for that peer the
route is promoted.
Community Add/Remove: set { community-add [ 65000:200 ]; community-remove [ 65000:100 ]; large-community-add [ 65000:100:200 ]; }. Adds or removes
individual community values (standard, large, extended) without replacing the
entire attribute. The engine maps these to AttrModAdd/AttrModRemove operations.
MED removal: del { med; }. This directive removes MULTI_EXIT_DISC (RFC
4271 type 4) from the route. It is the mechanism RFC 4271 Section 5.1.4 requires
a speaker to implement. It is honored on an import chain only, because that
section also requires the removal before Decision Process phases 1 and 2. The
import chain is the only chain that runs there.
On an export chain, the removal is ignored with a logged warning, and the
definition's other operations still apply. RFC 4271 Section
9.1.2.2 states that comparing on a MULTI_EXIT_DISC and then advertising the
route without it causes route loops. The directive is mutually exclusive with
set med, increment med, and decrement med. A metric received from one
neighboring AS is already kept off a session toward another with no
configuration at all (Section 5.1.4). This directive is for the operator who
wants the metric gone from the route itself.
Match (the condition the operations apply under): match { community [ 65535:666 ]; large-community [ 65001:100:200 ]; extended-community [ target:65001:1 ]; }. The three leaf-lists hold alternatives, so any one value
present in the route satisfies the condition. A route that matches none passes
through UNCHANGED rather than rejected. An absent match container applies the
operations to every route, which is what every definition written before this
container did.
A match container and an earlier match filter answer different questions. A
filter chain is a pipe in which a reject DROPS the route. A match filter placed
earlier therefore expresses "modify these and discard everything else", and the
route that must keep flowing untouched has nowhere to go. Use an earlier filter
when you want the rest dropped:
policy {
match CUSTOMERS-IN {
prefix-list CUSTOMERS;
}
}
filter {
import {
filters [ CUSTOMERS-IN modify:PREFER-LOCAL ];
final accept;
}
}
Chain references: bgp-filter-modify:NAME or modify:NAME.
AS-Path Length Filter (bgp-filter-aspath-length)
bgp-filter-aspath-length accepts or rejects routes based on AS_PATH hop count.
Configure named filters with min and/or max bounds:
policy { as-path-length REJECT-LONG { max 30; } }.
Routes outside the configured range are rejected. At least one of max or min
is required. Path length counts every ASN of the path: a member of an AS_SET
and an ASN of a confederation segment each count as one hop. That is the
filter's own count, not the RFC 4271 Section 9.1.2.2 path length best-path
selection uses, which counts an AS_SET as 1.
A BGP chain reaches an instance only through the as-path-length condition
of a match instance, which the plugin registers; a chain naming the
instance itself is refused at commit: policy { match NOT-LONG { as-path-length REJECT-LONG; } } passes
an UPDATE whose hop count lies within REJECT-LONG's bounds and rejects any
other, counting the hops the filter counts. Only a BGP chain judges it: a
Loc-RIB route carries no AS path, so a match naming as-path-length there is
refused. The editor completes the configured instances, and a commit naming
any other is refused.
Remove Private AS (bgp-filter-remove-private-as)
bgp-filter-remove-private-as removes RFC 6996 Private Use ASNs from AS path
attributes in an import or export policy chain. Define named actions in
policy { remove-private-as NAME { ... } } and reference them from a
peer, group, or global filter chain by their unique name.
Default mode strips private ASNs from AS_PATH and AS4_PATH. The optional
replace-with peer-as mode replaces each private ASN with the destination peer
AS on export, or the source peer AS on import.
policy {
remove-private-as STRIP {
}
remove-private-as REPLACE {
replace-with peer-as
}
}
bgp {
router-id 192.0.2.1
session {
asn {
local 65000
}
}
peer transit-a {
filter {
export {
filters [ STRIP ];
final accept;
}
}
}
}
Filter instance names are globally unique under the policy root. When a name is
unique, reference it directly (e.g. STRIP). The prefixed forms
remove-private-as:STRIP and bgp-filter-remove-private-as:STRIP remain
accepted for disambiguation or advanced use.
The plugin emits policy intent only: for a route whose AS_PATH or AS4_PATH
carries a Private Use ASN it answers the as-path remove intent (private,
or private-peer-as in peer-as mode), read from the route's declared
as-path field and raw body, through one evaluator whether it runs internal
or forked. A chain naming it runs on the in-process library chain, which
applies the intent. The reactor performs the wire rewrite so
AS_SEQUENCE, AS_SET, and confederation segment structure is preserved. On export
to EBGP peers, private-AS removal runs before the normal local-AS prepend. On
export to an IBGP peer (peer AS equals local AS, or equals the configured
migration AS, which RFC 7705 Section 4.2 makes internal) the removal is not applied:
RFC 4271 Section 5.1.2 forbids modifying the AS_PATH of a route advertised to
an internal peer, so the route leaves with its AS_PATH unchanged.
To test what a filter would do without sending traffic, use show policy test:
ze show policy test peer upstream1 export filter STRIP update <BGP-UPDATE-HEX>
This returns per-filter trace output showing accept/reject/modify decisions and changed attributes. See the command reference for full syntax.
NLRI Encoders/Decoders
NLRI plugins register address family support at init time via family.MustRegister(afi, safi, afiStr, safiStr). The four base families (ipv4/unicast, ipv6/unicast, ipv4/multicast, ipv6/multicast) live in internal/core/family/registry.go itself; everything else is owned by its plugin's types.go. Plugins are loaded automatically when the corresponding family is configured.
| Plugin | Families |
|---|---|
bgp-nlri-vpn |
ipv4/mpls-vpn, ipv6/mpls-vpn |
bgp-nlri-evpn |
l2vpn/evpn |
bgp-nlri-vpls |
l2vpn/vpls |
bgp-nlri-flowspec |
ipv4/flow, ipv6/flow, ipv4/flow-vpn, ipv6/flow-vpn |
bgp-nlri-labeled |
ipv4/mpls-label, ipv6/mpls-label |
bgp-nlri-mup |
ipv4/mup, ipv6/mup |
bgp-nlri-mvpn |
ipv4/mvpn, ipv6/mvpn |
bgp-nlri-rtc |
ipv4/rtc |
bgp-nlri-ls |
bgp-ls/bgp-ls, bgp-ls/bgp-ls-vpn |
Hub Configuration
For external plugins that connect over TLS (non-internal mode), configure the hub:
plugin {
hub {
server local {
host 127.0.0.1;
port 0; # auto-assign port
secret change-this-token-to-at-least-32-chars; # TLS auth token
}
}
}
Writing External Plugins
External plugins communicate with Ze through the same newline-framed YANG RPC
protocol as internal plugins: #<id> <verb> [json]. External processes connect
to the plugin hub over TLS with the ZE_PLUGIN_* environment supplied by the
engine. That environment carries the hub address, a per-plugin token, and in
ZE_PLUGIN_CA_PEM the certificate authority root that issued the hub's
certificate. A plugin validates the hub's chain against that root and dials no
hub without it. The Go SDK in pkg/plugin/sdk is the reference implementation. A
plugin written in another language implements the same documented wire
protocol; no first-party Python launcher or helper is required.
See plugin-development/protocol.md for the
protocol and examples/plugin/go for a complete
plugin.
Answering a command with rows
A command handler returns (status string, data any, error). When data is a
built value, the operator gets that value. When the command walks a large
collection, return an sdk.Records instead. The SDK then writes one line for
each row past the first 256, and neither the plugin nor the engine holds the
whole answer. A walk that ends inside those 256 rows collapses to the one
document the command answered with before it produced rows at all.
p.OnExecuteCommand(func(serial, command string, args []string, peer string) (string, any, error) {
return "done", sdk.Records{
Key: "sessions",
Rows: sessionRows(),
}, nil
})
Key names the envelope the rows belong under. Rows is an
iter.Seq[sdk.Record], and each record carries one Item the command produced
or one Fault it rejected. A Row appends its own JSON into the buffer the
writer owns, so a walk of a million rows allocates for none of them. The row is
appended before the yield that carried it returns and nothing keeps a reference
to it, so a producer can hand back one row value for every row of the walk and
refill it in place.
Fields names the columns when every row of the walk shares one schema. Declare
them and each row is a JSON array of values in that order, so the names travel
once on the head instead of on every row:
return "done", sdk.Records{
Key: "sessions",
Fields: []string{"peer", "state", "uptime"},
Rows: sessionColumnRows(),
}, nil
An operator sees the same objects either way. The engine reads the names off the head and puts each value back under its own name. Declaring a schema therefore changes what the wire carries and never what the command answers.
A row MUST then carry exactly one value for each name, in the same order. The two are read against each other by POSITION. A short row would gain a column it never carried, and a long one would lose a value, so such a row is refused rather than repaired.
Three rules bind a handler that answers this way.
| Rule | Why |
|---|---|
| The walk is read before the handler's call returns. Do not store the sequence | It is the answer being written, not a collection that can be read again |
| Keep whatever the walk reads alive until that call returns | The SDK pulls a row at a time, so a released buffer reaches the operator as zeros |
Do not name the envelope errors |
That name holds the rejected rows, and both producers refuse the collision |
A row that no wire message can carry is reported as a rejected row and the walk
continues. The operator then reads the rows that were applied under Key, and
the row that was refused under errors beside them. A short walk whose rows
each fit and whose one collapsed document does not is refused the same way, and
the operator reads that rejection alone.
A plugin also READS a record answer. Plugin.DispatchCommandAnswer runs an
engine command and yields each row as it arrives, which is what bounds the memory
of a walk over a large table. Plugin.DispatchCommand reads the same answer as
one document for a caller that wants the whole payload.
Declaring what your command's answer holds
Each CommandDecl carries an optional Shape, Columns and AddressFields.
Declare them and the CLI publishes the pipe operators your command supports, and
refuses the others BY NAME before your handler runs. Declare nothing and the CLI
waits for the answer, then refuses from what it has in hand.
Commands: []sdk.CommandDecl{{
Name: "my-plugin peers",
ShortHelp: "Show the sessions",
Shape: "tab",
Columns: []string{"address", "state", "up"},
AddressFields: []string{"address"},
}},
The shape MUST be doc, map, tab or blob, and a column or address-field list MUST
come with a shape. One command path carries one declaration, so make every
branch of your handler answer the same shape whatever argument it takes. The
field list, the bounds and the four refusals are in
plugin-development/commands.
Naming a pipe alias for your own command
A plugin declares a CLI pipe alias in the Pipes list of the same
Registration. An alias is the word an operator types after the pipe character,
and it stands for an operator chain. The BGP RPKI plugin declares summary on
show bgp rpki, so show bgp rpki | summary answers the counters without the
cache server rows.
Pipes: []sdk.PipeDecl{{
Command: "my-plugin status",
Name: "totals",
Description: "The counters, without the per-session rows",
Expansion: "display sessions-total sessions-established",
}},
The pipe layer selects and re-sequences. It renames no key, adds no numbers and counts no matching rows. So your handler MUST emit the aggregate fields beside the detail rows, as siblings at one level. A command whose second view needs computed data stays a subcommand, and so does one that takes a value.
Command MUST be one of the commands you declare in the same message. Three
holders refuse the name:
- a built-in pipe operator that carries it.
- a pipe filter on an overlapping command path that carries it.
- an alias on the exact same command path that carries it.
One refusal fails your whole startup, and the daemon log names the plugin, the path and the name.
A declared alias resolves over ze cli -c "<command>" and in the interactive
session a plain ssh client reaches. It does NOT resolve in ze cli with no
command argument, which expands the chain in the client process and answers
pipe error: unknown pipe operator: totals.
Dependencies
Plugins can declare dependencies on other plugins. The engine starts plugins in dependency order and delivers state/EOR events to dependents first.
# bgp-gr depends on bgp-rib and policy
# bgp-rpki depends on bgp-adj-rib-in
# bgp-rs optionally uses bgp-adj-rib-in
Dependencies are declared in the plugin's registration, not in config. The engine resolves them automatically. Two kinds:
| Kind | Field | Behavior if missing |
|---|---|---|
| Hard | Dependencies |
Ze refuses to start, before any plugin runs, at boot, at a config reload and for an explicit plugin {} block. ze config validate and a commit refuse the same config with the same message, so a config that validates is one the daemon can start. The message names the plugin, the dependency and the build tags to rebuild with (or says the name is unknown to every build), and ends with the fix on its own line: rebuild with the tags, or remove the configuration that loads the plugin. A dependency an external program declares that nothing loads is refused at its startup, with the plugin { internal ... } or plugin { external ... } block to add as the fix. |
| Optional | OptionalDependencies |
Silently skipped. Plugin owner handles runtime absence (typically a one-shot WARN + feature disabled). |
A block whose whole plugin was compiled out of the binary, such as mrt { }
or bgp { } in a build without ze_bgp, fails at parse, because the build has
no schema for it. The refusal says the keyword belongs to a plugin this build
lacks and names the build tag to rebuild with, on a fix: line. The same holds
for a nested block a compiled-out plugin owns, such as fib { vpp { } } in a
build without ze_vpp. A keyword no build knows (a typo) keeps the
closest-keyword hint.
The policy dependency
Every protocol that attaches filter chains lists policy in its hard
Dependencies: bgp, isis, ospf, static, connected, rib and the
redistribute orchestrator. Each filter-type plugin lists it too. Loading any of
them loads the policy library, and a build without it refuses the config with
the missing-dependency message rather than ignoring a filter block.
TestFilterAttachmentOwnersDependOnPolicy derives the set from the attachment
registrations and the registered filter types, so a new attachment without the
edge fails by name.
The BGP filter-type plugins (bgp-filter-prefix, bgp-filter-family,
bgp-filter-modify, bgp-filter-aspath, bgp-filter-aspath-length,
bgp-filter-community-match, bgp-filter-remove-private-as,
bgp-filter-path-asn and bgp-filter-irr) own the bgp root
(ConfigRoots ["bgp"]) and read the policy root (ConfigReads ["policy"]),
where their named instances live. They serve BGP chains only, so a bgp { }
block loads them, and a config holding policy with no bgp block loads
none of them: policy routing, or an IS-IS or OSPF import chain, starts without
BGP. Their configure and reload deliveries carry both sections, which
bgp-filter-family, bgp-filter-modify and bgp-filter-path-asn need for
their BGP settings. A Loc-RIB chain naming a prefix-list, modify or
family-filter does not need these plugins running: its evaluators are linked
in and run inside the policy plugin (see Route policy).
TestPolicyOnlyConfigLoadsNoBGP derives the plugins a policy-only config
selects from the registry and refuses one that depends on bgp.
StartAfter is separate from either kind of dependency. It orders two plugins
only when both are already selected and never activates its target. BFD starts
after interface when interface configuration is present, so its logical names
bind before sessions start; a BFD-only configuration does not load an interface
backend. Cycles among selected plugins are refused for order-only edges too.
bgp-rs uses bgp-adj-rib-in optionally: when both are loaded, replay-on-peer-up works; when bgp-adj-rib-in is absent, forwarding still works and a single WARN log announces that replay is disabled. bgp-rs forwards via the typed Plugin.ForwardCached / ReleaseCached fast path (rs-fastpath-3) instead of the legacy text-RPC send bgp <sel> cached <id> pipeline. See architecture/api/commands for the full SDK surface.
Exclusive Roles
When two plugins both implement a behavior but only one should run it, the plugin that takes over declares the role in its static registration (Claims), and the other stands down. The engine unions the claims of every plugin in the startup set and delivers the union on each plugin's Stage-2 configure callback; the standing-down plugin reads it with sdk.Plugin.ClaimActive(role) from its OnConfigure handler.
Stage 2 is part of the sequential handshake, so the decision is recorded before any plugin sends Stage-5 ready and therefore before the engine starts peers. A handler reading it during a runtime event always sees the final answer.
bgp-rs claims bgp-peer-up-replay; bgp-adj-rib-in stands its own replay down when that claim is active. The decision must not be re-derived from OnAllPluginsReady: that callback is fanned out on detached goroutines that race session establishment, and when the ownership decision was taken there both plugins replayed, so a peer received a byte-identical duplicate UPDATE. An unclaimed or unresolvable role reads false, which is the fail-closed direction: nobody promised to do this, so keep doing it yourself.
A claim is daemon-wide, and delivery is per-peer
A claim says a role has an owner in this daemon. It cannot say the owner will act on a given peer, because Stage 2 runs before any session exists. Two things make the claim wrong for one peer, and the plugin that stood down can see neither: the claimant takes no delivery of that peer's events, because the peer's attach process blocks do not name it, or the claimant never reached Running at all.
So the engine RETRACTS the claim, per event, for the peers it does not cover. Each peer-scoped event carries the claimed roles that no process being fed this event holds: StructuredEvent.UnheldRoles for a plugin on the direct bridge, and the unheld-roles member of the state event for a JSON one. A plugin that stood a role down MUST run its own default behavior for an event that names it, because nothing else will. The list is absent whenever every claim holds, which is the common case and costs no bytes.
bgp-adj-rib-in reads it at peer-up: it replays a peer that bgp-rs is not fed, and stands down for the peers bgp-rs drives. Without the retraction such a peer was served by nobody, because bgp-rs replays and forwards only peers it takes state delivery of.
Peer-Up Barrier
A plugin that decides on the peer-up event whether a peer may receive traffic declares PeerUpBarrier: true. The engine then holds that peer's initial-sync End-of-RIB until every barrier-declaring plugin subscribed to state events has taken delivery of the peer-up event, so "End-of-RIB sent" means "every barrier plugin has registered this peer".
bgp-rs declares it: it registers the peer as a forward target on that event, and an UPDATE arriving before that is forwarded nowhere. The engine delivers the peer-up event to one plugin at a time and waits for each delivery's result before the peer's initial sync starts. A plugin whose delivery fails or is refused does not delay the End-of-RIB: the engine lowers the count to the acknowledgements it received and logs a WARN naming the peer and the shortfall. A barrier the sync still finds shut is bounded by a 2 second timeout, which logs its own WARN. The expected count is taken over the plugins the event is actually delivered to, so declaring the field without subscribing to state events does not stall anything. It is separate from the session-ready report below: merging them would let a route sender's report satisfy a registrar's obligation.
Session-Ready Report
A plugin that reports completion of its peer-up route replay declares SignalsSessionReady: true and dispatches request peer <addr> plugin session ready session <captured-token> when its replay attempts finish, including an empty replay. Capture the nonzero uint64 initial-replay token from the original peer-UP event before beginning that replay; never fetch a later session's token to finish old work. This report does not assert successful delivery and does not delay the peer's End-of-RIB marker.
The declaration is voluntary. A plugin that pushes routes on its own schedule need not declare it; binding that plugin with send [ update ] alone does not make it a session-ready reporter.
An external plugin has the same declaration under a different name. It is registered nowhere in this tree, so it declares signals-session-ready in its Stage-1 declare-registration instead, and the engine reads it off the running process. Declaring nothing stays the default there too.
The name your report is credited to is the one the operator wrote in attach process <name>, whatever your plugin is called. plugin { internal rs { use bgp-rs } } runs the process as rs, and the engine resolves that alias back to the bgp-rs registration before it asks whether you declared. Your report carries the process name, so it is credited to rs as well. The spelling of the implementation does not change the answer: use bgp-rs, use ze.bgp-rs, run ze.bgp-rs and run ze plugin bgp-rs all reach the same registration, because one function (plugin.RegistryNames) answers which registry row a process configuration names and every caller derives from it.
Three facts have to hold before a peer names your process in its barrier, and each one is something you can check in your own config. The peer grants the route-push rail, with send [ update ] or send [ raw ]. The plugin declares the field. The peer grants receive [ state ], because the report answers the peer-up event: a process the peer never tells about the session cannot push into that session's initial update, so it is never named.
Report once per establishment, from your peer-up handler, and report even when you had nothing to replay: completion accounting must distinguish "finished with nothing to send" from "still working".
Your report does not hold the peer's End-of-RIB (owner ruling, 2026-09-18). A process that creates routes counts as a peer when the marker is decided, and Ze does not wait for a peer's marker before sending its own. The marker follows the routes Ze owns, after the separate peer-up registration barrier. Routes a process pushes after the marker are ordinary updates.
Replay ordering is a separate obligation. A reporter whose registration also declares FencesLiveForwards holds live forwarding behind its peer-up replay. Once all fence owners report, the engine releases those live forwards. A reporter without that declaration does not hold the forwarding fence, but a fence owner that never reports can leave live forwards held; neither case holds End-of-RIB.
Startup Timing
Each handshake stage is bounded by progress, not by wall clock. A stage fails only when the whole startup tier goes ze.plugin.stage.timeout (default 5s) without any plugin completing a stage; every completion re-arms the window. A BGP config puts twenty or more plugins in one tier (bgp, bgp-bmp, bgp-rpki, every bgp-filter-*), and a flat per-tier budget meant a CPU-starved or slow-disk host could lose all of them at once and come up with its BGP plugins missing.
The wait stays bounded three ways: a repeated stage completion is ignored, so a looping plugin cannot hold the window open; a wedged tier trips within one timeout of the last progress; and shutdown ends the wait. Worst case is (plugins + 1) x timeout per stage.
Debugging Plugins
The plugin debug shell lets you manually interact with the engine using the plugin protocol. This is useful when debugging plugin code -- you can send individual commands and inspect responses.
ze bgp plugin cli
The debug shell:
- Asks about handshake parameters (plugin name, families) with defaults -- hit Enter to accept
- Connects to the daemon via SSH
- Runs the 5-stage plugin handshake over the SSH channel
- Enters interactive command mode
Available post-handshake commands:
| Command | Description |
|---|---|
dispatch-command <cmd> |
Dispatch an engine command |
subscribe-events <events> |
Subscribe to events |
unsubscribe-events |
Unsubscribe from events |
decode-nlri <family> <hex> |
Decode NLRI from hex. The text command carries no ADD-PATH token, so the hex holds no Path Identifier |
encode-nlri <family> <args> |
Encode NLRI |
bye |
Disconnect |
Use --name <name> to set a custom plugin name for the session.