Plugins

Choosing plugins

NLRI family plugins (bgp-nlri-evpn, bgp-nlri-vpn, etc.) are loaded automatically when you configure the corresponding address family. You don't need to declare them.

Config-Driven Loading

BGP itself is a config-driven plugin. If your config has a bgp { } section, BGP loads automatically. If it doesn't, ze starts without BGP (useful for interface-only or FIB-only deployments). Native OSPFv2 follows the same pattern: an ospf { } section auto-loads the ospf edge plugin through ConfigRoots ["ospf"].

Loading Plugins

Plugins are declared in the plugin { } block. Built-in plugins use internal, external processes use external:

plugin {
    internal rib {
        use bgp-rib
    }
    internal adj-rib-in {
        use bgp-adj-rib-in
    }
    internal gr {
        use bgp-gr
    }
}

For external processes (scripts, custom binaries):

plugin {
    external collector {
        run ./collector.py
        encoder json
    }
}

Plugin Block Settings

An external plugin needs a shell on the host, because the run string is given to /bin/sh -c. A gokrazy appliance image carries none, so an appliance runs its plugins from internal blocks. ze doctor reports the missing shell under doctor-plugin-shell-missing, and a start that meets it names the shell rather than the plugin.

When a Plugin Fails

The plugin decides, not the configuration. Every plugin tells ze at startup what its own failure means, and ze does that.

fatal is open to any plugin, including one you wrote or one you downloaded. Running a plugin is accepting its terms. If you disagree with a plugin's policy, do not configure it, or change its code.

The respawn leaf states what YOU expect. It can ask for less than the plugin permits and never for more:

The last row is deliberate. Honoring the leaf would restart a plugin whose author forbade it, and honoring the declaration would leave what you wrote with no effect. Ze picks neither in silence: remove the leaf, or run a plugin that declares restart.

The leaf is spelled the way ExaBGP spells it, because an ExaBGP process block carries the same option and a migrated configuration keeps working.

Past the restart bound the plugin is disabled, show health reports plugin-down for it, and ze carries on.

Binding Plugins to Peers

Each peer declares which plugins receive its events, in attach process blocks. The process name must match the plugin's internal or external name in the plugin { } block:

plugin {
    internal rib { ... }       # <-- this name
}
peer transit-a {
    attach process rib { ... }        # <-- must match
}

Plugins receive BGP events through the attach blocks on each peer:

peer transit-a {
    ...
    attach process rib {
        receive [ update state refresh ]
        send [ update ]
    }
    attach process adj-rib-in {
        receive [ update-received state ]
    }
}

A peer that attaches nothing feeds nothing. The block is the whole statement of the relationship: a plugin that no peer attaches runs and is fed no peer event, and a peer that attaches one plugin feeds that plugin and no other. Loading a plugin and attaching it are two separate acts, and the second is the one that moves data.

Event Types

Plugins can register custom event types via the EventTypes field in their registration. These become valid in receive config directives and subscribe-events RPCs.

Runtime subscriptions

The Go SDK can subscribe at startup with SetStartupSubscriptions or after startup with SubscribeEvents. Each subscription carries its own namespace, event list, peer selector, format, and envelope preference. An empty namespace uses the protocol component's default namespace, normally bgp.

The event name "*" expands at registration time to every event type currently registered in that namespace. This avoids a wildcard check on every delivered event. Events registered later require a new subscription.

By default, OnEvent receives the original payload string. Call SetEnvelope(true) before startup when one handler needs to distinguish events from several namespaces or event types. Delivery then wraps the payload:

{
  "namespace": "vpn-ipsec",
  "event": "sa-up",
  "payload": {
    "peer": "branch-a"
  }
}

The envelope sits inside the existing event string, so single-event and batch delivery use the same callback. Subscribers that do not opt in remain byte-compatible with earlier releases.

Directions

attach process my-plugin {
    receive [ update-received ]   # UPDATEs the peer sends to ze
    send [ update ]               # permission to announce toward the peer
}

receive names an event type and the direction it is fed in. A plain type means both directions, so receive [ update ] feeds the program the UPDATEs ze sends as well as the ones it gets. send carries no direction, because every send type is sent.

The two lists are independent. A program with send and no receive announces to the peer and is told nothing by it. A program with receive and no send watches the peer and may not answer it.

Groups and dynamic peers

Write the block once on a group and every peer that group produces carries it. A member that restates the block REPLACES the group's list for itself, and the other members keep the group's.

plugin {
    external looking-glass {
        run ./looking-glass.py
        encoder json
    }
}

bgp {
    group transit {
        attach process looking-glass {
            receive [ update-received state ]
        }

        peer transit-a { }                    # updates and state
        peer transit-b { }                    # updates and state
        peer transit-c {
            attach process looking-glass {
                receive [ state ]             # state ONLY
            }
        }
    }
}

transit-c is fed state and no update. Its own list stands in for the group's rather than adding to it, which is how every leaf-list in a group merges. To widen one member, restate the whole list you want it to have.

A peer a DYNAMIC group creates carries the group's blocks too. Such a peer is named by no config: ze generates dyn-<address> when it accepts the connection. Nothing needs to name it, because ze reads each peer's blocks after the group merge, not from the text of the config file.

bgp {
    group ix {
        connection {
            remote {
                ip dynamic
                connect false
                range 198.51.100.0/24
            }
            local {
                ip 198.51.100.1
                accept true
            }
        }
        attach process looking-glass {
            receive [ update-received state ]
        }
    }
}

Every member that connects from 198.51.100.0/24 feeds looking-glass its received UPDATEs and its session state. show event delivery lists them by the address each one connected from.

The send permission

send is a permission, and ze enforces it on the peers a command resolves to. A program that issues send bgp * update ... reaches the peers that attach it with send [ update ], and no others. A peer the program is not attached to is dropped from the command, and ze writes one WARN naming the peer, the process and the message type. When the selector names ONLY peers that refuse it, the whole command fails and the program is told why.

The three BASE message types are separate permissions. A plugin registers more, and naming one auto-loads the plugin that enables it, enhanced-refresh from bgp-route-refresh among them:

raw is the widest of the three, because the bytes can be any BGP message, an OPEN or a NOTIFICATION included. send [ update ] does not imply it.

send [ * ] grants all three, and every send type registered later.

An operator at the CLI, over SSH or through the REST API is not a process and is not gated by this: their authority is the one AAA already checked.

Counter: ze_bgp_send_refused_total{process,type} counts messages the permission refused. A non-zero value with no config change means a program is asking for a peer it was never attached to.

Reading the delivery graph

show event delivery prints the peer-to-process edges the running config produces: one block per peer, one row per attach process block on it, with the event types that process is fed and the message types it may send toward that peer. A peer with no attach process block shows no process.

The daemon builds this from the RESOLVED settings of each peer, so a block a peer inherits from its group is shown on the peer. A token the event registry does not know is listed under unresolved and carries no edge, which is what an operator sees when a custom event type names a plugin that did not load.

These edges DECIDE delivery. A peer hands a program an event when the peer's block grants the type and the program subscribed to it; a peer that attaches no block for a program feeds it nothing, whatever the program asked for at startup. That holds for ze's own plugins as well: a config that loads bgp-rib and attaches it to no peer stores no route. When the two halves disagree, the daemon names each peer, process and event type in the log rather than dropping the event in silence.

The name in attach process <name> is the name the program runs under: the one in its plugin { external <name> } or plugin { internal <name> } block, or the registry name of a plugin loaded with --plugin.

Invocation Modes

Internal mode (use pluginname) runs a compiled-in plugin as a goroutine within the ze process. Startup still uses the same YANG RPC handshake as external plugins, then DirectBridge bypasses socket I/O for supported hot paths. External mode starts a separate process; that process connects back to the plugin hub over TLS and authenticates with its per-plugin token.

Built-In Plugins

List available plugins:

ze show plugin list

Reporting a Setup Outcome

A plugin that sets something up in its own init() records what happened, so an operator can ask why a feature is absent. The outcome and reason columns of show plugin list carry it:

ze show plugin list

The record is one call, made from the plugin's init() beside its registration:

The two writes a plugin makes from init(), registry.Register and registry.RecordSetup, are keyed by the plugin name and neither reads the other, so their order does not matter. Go initializes the files of one package in filename order and no plugin author has to know it.

Recording is optional at the call site, and a plugin that records nothing is listed as unknown. That is the signal that the plugin owes a record, not that it is absent. memlock is the worked example: it locks the executable in init() and records a soft failure carrying the cause and the remedy when RLIMIT_MEMLOCK is too small. Its memlock-rlimit doctor check answers the question this record cannot: whether the host could lock the executable at all, BEFORE ze runs (docs/architecture/doctor-and-health-checks.md).

A reason string reaches CLI output as data, so never put a secret in one.

Storage and Policy

Protocol

Infrastructure

L2TP

These plugins register via the L2TP handler registry (RegisterAuthHandler, RegisterPoolHandler) at init time. Only one auth handler is active at a time (last registered wins). The pool and shaper plugins subscribe to session lifecycle events via the EventBus.

See L2TP guide for configuration details.

The iface plugin defines a Backend interface and loads a backend by name (YANG backend leaf, default netlink). The iface-netlink backend handles all Linux interface operations. iface-dhcp is a separate plugin for DHCP client lifecycle. BGP reacts to address events by starting/stopping listeners. Uses a JunOS-style two-layer model: physical interfaces + logical units (VLANs).

Bus topics published:

The rib plugin aggregates best routes from all protocol RIBs and selects the system-wide best per prefix by administrative distance (lower wins). Subscribes to bgp-rib/best-change/ Bus topic prefix, publishes system-rib/best-change.

The fib-kernel plugin programs OS routes from the system RIB into the kernel via netlink (Linux). Uses a custom rtm_protocol ID (RTPROT_ZE=250) to identify ze-installed routes. Crash recovery marks existing ze routes as stale at startup and sweeps them after reconvergence. A kernel route monitor detects external changes and re-asserts ze routes when overwritten.

Bus topics in the FIB pipeline:

Bus topics in the sysctl pipeline:

Route Filters

Plugins can declare named filters at stage 1 for import and/or export filtering. A filter declares which attributes it reads. That declaration does not narrow what it receives. The engine builds one subject for the whole chain, and the subject names every attribute the UPDATE carries. See The Attribute Names in the Filter Text Protocol for the names and their value shapes. Filters respond accept, reject, or modify (delta-only). See Route Filters and Redistribution for configuration.

A single plugin can offer multiple named filters. Config references them as <plugin>:<filter> (e.g., rpki:validate, community:scrub).

Filters can declare overrides to remove default filters from the chain (e.g., allow-own-as:relaxed overrides rfc:no-self-as for a specific peer).

Cross-Protocol Redistribute (redistribute-orchestrator)

redistribute-orchestrator is the single subscriber that dispatches non-consumer protocol route-change events to registered RedistConsumer implementations. Unlike the route filter chain above (which gates intra-BGP traffic), the orchestrator lets operators redistribute locally-originated routes from other protocols (L2TP sessions, connected interface prefixes, static routes, future OSPF / ISIS) into destination protocols (BGP, future OSPF/ISIS).

Config:

redistribute {
    destination bgp {
        import connected;
        import static;
        import l2tp { family [ ipv4/unicast ipv6/unicast ]; }
    }
}

Each destination <protocol> names a registered consumer. Under it, import <source> enables one non-consumer protocol. The import rule's source is the protocol's canonical name registered via redistribute.RegisterSource. Per-source family lists narrow which address families are redistributed; an empty list means "all families". An import is scoped to its enclosing destination: an import under destination bgp feeds only BGP, not OSPF/IS-IS.

The orchestrator auto-loads when redistribute {} appears in the config. No plugin { internal redistribute-orchestrator { use redistribute-orchestrator; } } block is required.

The rules need no attach process plumbing either. Ze derives two bindings from the redistribute root, and they are the only bindings in the delivery index Ze writes for you. A peer whose config already names the process keeps its own binding unchanged.

Reactor per-peer NEXT_HOP substitution applies: when the producer leaves NextHop zero, the reactor stamps each peer's local session address as the NEXT_HOP. Producers that have an explicit address pass it through verbatim.

Late-join replay: a producer emits once, so whoever arrives after that emit holds nothing. Two arrivals are late, and each fires a replay request carrying an opaque ReplayID token that every producer echoes on a re-emit of its CURRENT set. Producers stay target-agnostic: the orchestrator alone holds the ReplayID -> target mapping, and it holds it for a TTL because an out-of-process producer re-emits asynchronously.

Counters: ze_bgp_redistribute_events_received, _announcements, _withdrawals, _filtered_protocol_total, _filtered_rule_total, ze_bgp_redistribute_replay_total{source} (routes replayed, to a newly established peer or to a consumer that registered late).

Prefix-List Filter (bgp-filter-prefix)

bgp-filter-prefix is a built-in filter plugin that matches IPv4 and IPv6 routes against ordered prefix lists defined in bgp { policy { prefix-list NAME { ... } } }. Each list is a sequence of entry <CIDR> blocks with optional ge / le bounds and an action of accept or reject; the first matching entry wins and no match is an implicit deny.

Filter chain references use the standard <plugin>:<filter> form: bgp-filter-prefix:CUSTOMERS. The shorter form prefix-list:CUSTOMERS also resolves to the same plugin via the filter-type registration, and a bare CUSTOMERS resolves if no other filter plugin claims a filter of that name.

The mixed case supports only IPv4 unicast legacy NLRI in v1. For multiprotocol families (MP_REACH_NLRI), the plugin falls back to whole- update accept when any prefix passes -- implementing per-NLRI rewriting for MP_REACH requires declaring raw=true on the filter registration and rewriting the attribute value directly.

AS-Path Filter (bgp-filter-aspath)

bgp-filter-aspath matches the UPDATE's AS-path against ordered regex entries defined in bgp { policy { as-path-list NAME { entry REGEX { action accept|reject; } } } }. Each entry's regex is matched against the AS-path string using Go's RE2 engine (linear time, inherently ReDoS-safe). First match wins; no match is implicit deny.

Chain references: bgp-filter-aspath:NAME, as-path-list:NAME, or bare NAME. Config authors should use [0-9] instead of \d in regex strings because ze's config parser interprets backslash as an escape character.

The AS-path string every filter matches against. One reader produces it, filtertext.ASPath, and every AS-path filter takes it from there. It is the space-separated decimal ASNs with no brackets, for example 65001 65002 65003. Two properties of it are load-bearing for a pattern you write:

• Every segment type is flattened into one list with no marker. AS_SEQUENCE, AS_SET, AS_CONFED_SEQUENCE and AS_CONFED_SET all contribute their ASNs in order. A pattern therefore cannot ask which segment an ASN came from, and an ASN inside an AS_SET is matched like any other.
• It is the path the route traversed, not the AS_PATH attribute as encoded. A peer that did not negotiate the four-octet AS capability puts AS_TRANS (23456) in AS_PATH wherever a four-octet AS number belongs and sends the real numbers in AS4_PATH. The reactor reconstructs one path from the two before any filter sees the text (RFC 6793 Section 4.2.3), so a pattern and a hop count read the real AS numbers and never AS_TRANS.
• A route with no AS_PATH gives the empty string, and a route with one ASN gives that ASN alone. The producer writes one ASN unbracketed (as-path 65001) and several in brackets (as-path [65001 65002]), and the reader strips the brackets, so a pattern never sees one. A plugin that rewrites the AS path can pad the inside of the brackets, as-path [ 65001 65002 ], and the reader trims that padding too. A pattern and a hop count therefore read the same list whether the path came off the wire or out of a plugin.

Reject-ASN Filter (bgp-filter-path-asn)

bgp-filter-path-asn drops a route whose AS_PATH carries a listed ASN at a listed position, and leaves the session up. It is the cheap answer to a peer leaking its transit, which the max-prefix limit otherwise stops by dropping the whole session.

bgp {
    policy {
        reject-asn NO-TRANSIT {
            indirect [ 174 3356 ]
        }
    }
    peer peer-a {
        filter {
            import [ NO-TRANSIT ]
            export [ NO-TRANSIT ]
        }
    }
}

A list is an unordered reject SET. A route matching any leaf-list is rejected, and there is no first-match-wins, which is what keeps the type different from as-path-list. Both types stay and can sit in one chain.

A list carries seven keywords, and the one an ASN is written under says WHERE in the path it is unacceptable.

Six are plain leaf-lists. nth takes a number, so it is written nth 2 [ 3491 ];, and it counts RUNS rather than tokens: a run of consecutive identical ASNs advances the count once, so a peer cannot move your rule by prepending.

indirect is the everyday keyword, and it is worth the sentence it says: do not give me anything you reached through a transit provider, and peering with that provider directly is still fine. direct is the sending PEER's ASN, not the first ASN in the path: a path [3356 65001] from AS65001 carries 3356 at index zero and is still a leak, which is the route-server case RFC 7454 Section 9 names. nth 1 asks the positional question instead, and catches that same 3356.

On an export chain the filter is told the DESTINATION peer, not who sent the route, so nothing is direct there and indirect covers the whole path. That is the export half of RFC 7454 Section 9: do not advertise a path through your upstream to a peer you do not sell transit to.

The same ASN written under two keywords unions, so indirect plus direct is anywhere. A list that names nothing at all, whether its leaf-lists are absent or written empty, is REFUSED at load: an empty reject set accepts every route while reading in the config file like a safety filter. So is an nth entry with no ASN. A pattern that does not compile and one longer than 512 characters are refused too, naming the list. The ASN leaf-lists are uint32, an nth index is bounded 1..255, and the keywords are the schema's own, so a word where a number belongs and a keyword nobody declared are both refused by the config parser.

Every reject logs the offending ASN or pattern, the position it matched at, the list, the peer and the direction.

Chain references: bare NAME, reject-asn:NAME, or bgp-filter-path-asn:NAME.

Every reject also increments ze_filter_path_asn_rejects_total, labeled with the direction, the position that matched and the reason. The peer stays in the log line: a peer address in a label would grow the series count with the session count.

Ze ships no ASN set. The operator lists the ASNs, and the list means exactly what it says. show bgp reject-asn known transit-free prints the well-known transit-free ASNs as a indirect [ ... ]; block to paste, with the sources and the curated date as comments, and after the paste the config holds the numbers. show bgp reject-asn lists what each list holds, and show bgp reject-asn name <name> answers for one of them.

Community Match Filter (bgp-filter-community-match)

bgp-filter-community-match checks for presence of a specific community value in the route's standard, large, or extended community attributes. Defined in bgp { policy { community-match NAME { entry COMMUNITY { type standard|large|extended; action accept|reject; } } } }. First match wins; no match is implicit deny.

Separate from the tag/strip community plugin (bgp-filter-community) because intent differs: this plugin filters (accept/reject), that one modifies (tag/strip). They can coexist in the same deployment.

Chain references: bgp-filter-community-match:NAME or community-match:NAME.

Well-known community names (no-export, no-advertise, blackhole, etc.) work as match values because the filter text format renders them as names.

Route Attribute Modifier (bgp-filter-modify)

bgp-filter-modify applies declared operations to every route that reaches it in the filter chain. A definition that states a match container applies them only to the routes that meet it. The operations are:

Set (absolute value): set { local-preference 200; med 50; origin igp; next-hop 10.0.0.1; as-path-prepend 3; }. Only present leaves are applied.

Increment/Decrement (relative adjustment): increment { local-preference 50; } or decrement { med 30; }. Supported attributes: local-preference, med, aigp. Increment saturates at uint32 max (4294967295). Decrement floors at 0. Set and increment/decrement for the same attribute are mutually exclusive.

The arithmetic reads the attribute the route carries. When the route carries none, med and local-preference start from a declared default and aigp is left alone:

The first two values are configurable, and the table shows what they are when nothing is written: bgp { defaults { attribute { med 0; local-preference 100; } } }. Both leaves take 0 to 4294967295. They govern this arithmetic alone, so a changed med does NOT move the Decision Process, which compares an absent MULTI_EXIT_DISC as 0 whatever the leaf holds. There is no aigp leaf, for the reason the third row gives.

decrement { med 30; } on a route that carried no metric therefore leaves med 0 on the route. An RFC-default receiver reads an absent MED and a MED of 0 the same way, so this changes nothing for it. A receiver configured to treat a missing MED as the worst value reads them differently, and for that peer the route is promoted.

Community Add/Remove: set { community-add [ 65000:200 ]; community-remove [ 65000:100 ]; large-community-add [ 65000:100:200 ]; }. Adds or removes individual community values (standard, large, extended) without replacing the entire attribute. The engine maps these to AttrModAdd/AttrModRemove operations.

MED removal: del { med; }. This directive removes MULTI_EXIT_DISC (RFC 4271 type 4) from the route. It is the mechanism RFC 4271 Section 5.1.4 requires a speaker to implement. It is honored on an import chain only, because that section also requires the removal before Decision Process phases 1 and 2. The import chain is the only chain that runs there.

On an export chain, the directive is refused and logged. RFC 4271 Section 9.1.2.2 states that comparing on a MULTI_EXIT_DISC and then advertising the route without it causes route loops. The directive is mutually exclusive with set med, increment med, and decrement med. A metric received from one neighboring AS is already kept off a session toward another with no configuration at all (Section 5.1.4). This directive is for the operator who wants the metric gone from the route itself.

Match (the condition the operations apply under): match { community [ 65535:666 ]; large-community [ 65001:100:200 ]; extended-community [ target:65001:1 ]; }. The three leaf-lists hold alternatives, so any one value present in the route satisfies the condition. A route that matches none passes through UNCHANGED rather than rejected. An absent match container applies the operations to every route, which is what every definition written before this container did.

A match container and an earlier match filter answer different questions. A filter chain is a pipe in which a reject DROPS the route. A match filter placed earlier therefore expresses "modify these and discard everything else", and the route that must keep flowing untouched has nowhere to go. Use an earlier filter when you want the rest dropped: filter import [ prefix-list:CUSTOMERS modify:PREFER-LOCAL ].

Chain references: bgp-filter-modify:NAME or modify:NAME.

AS-Path Length Filter (bgp-filter-aspath-length)

bgp-filter-aspath-length accepts or rejects routes based on AS_PATH hop count. Configure named filters with min and/or max bounds: bgp { policy { as-path-length REJECT-LONG { max 30; } } }. Routes outside the configured range are rejected. At least one of max or min is required. Path length counts AS_SEQUENCE entries individually and AS_SET as 1, following RFC 4271 Section 9.1.2.2.

Chain references: bgp-filter-aspath-length:NAME or as-path-length:NAME.

Remove Private AS (bgp-filter-remove-private-as)

bgp-filter-remove-private-as removes RFC 6996 Private Use ASNs from AS path attributes in an import or export policy chain. Define named actions in bgp { policy { remove-private-as NAME { ... } } } and reference them from a peer, group, or global filter chain by their unique name.

Default mode strips private ASNs from AS_PATH and AS4_PATH. The optional replace-with peer-as mode replaces each private ASN with the destination peer AS on export, or the source peer AS on import.

bgp {
    policy {
        remove-private-as STRIP {
        }
        remove-private-as REPLACE {
            replace-with peer-as
        }
    }
    peer transit-a {
        filter {
            export [ STRIP ]
        }
    }
}

Filter instance names are globally unique under bgp policy. When a name is unique, reference it directly (e.g. STRIP). The prefixed forms remove-private-as:STRIP and bgp-filter-remove-private-as:STRIP remain accepted for disambiguation or advanced use.

The plugin emits policy intent only. The reactor performs the wire rewrite so AS_SEQUENCE, AS_SET, and confederation segment structure is preserved. On export to EBGP peers, private-AS removal runs before the normal local-AS prepend.

To test what a filter would do without sending traffic, use show policy test:

ze show policy test peer upstream1 export filter STRIP update <BGP-UPDATE-HEX>

This returns per-filter trace output showing accept/reject/modify decisions and changed attributes. See the command reference for full syntax.

NLRI Encoders/Decoders

NLRI plugins register address family support at init time via family.MustRegister(afi, safi, afiStr, safiStr). The four base families (ipv4/unicast, ipv6/unicast, ipv4/multicast, ipv6/multicast) live in internal/core/family/registry.go itself; everything else is owned by its plugin's types.go. Plugins are loaded automatically when the corresponding family is configured.

Hub Configuration

For external plugins that connect over TLS (non-internal mode), configure the hub:

plugin {
    hub {
        server local {
            host 127.0.0.1;
            port 0;                               # auto-assign port
            secret change-this-token-to-at-least-32-chars;  # TLS auth token
        }
    }
}

Writing External Plugins

External plugins communicate with Ze through the same newline-framed YANG RPC protocol as internal plugins: #<id> <verb> [json]. External processes connect to the plugin hub over TLS with the ZE_PLUGIN_* environment supplied by the engine. That environment carries the hub address, a per-plugin token, and in ZE_PLUGIN_CA_PEM the certificate authority root that issued the hub's certificate. A plugin validates the hub's chain against that root and dials no hub without it. The Go SDK in pkg/plugin/sdk is the reference implementation. A plugin written in another language implements the same documented wire protocol; no first-party Python launcher or helper is required.

See plugin-development/protocol.md for the protocol and examples/plugin/go for a complete plugin.

Answering a command with rows

A command handler returns (status string, data any, error). When data is a built value, the operator gets that value. When the command walks a large collection, return an sdk.Records instead. The SDK then writes one line for each row past the first 256, and neither the plugin nor the engine holds the whole answer. A walk that ends inside those 256 rows collapses to the one document the command answered with before it produced rows at all.

p.OnExecuteCommand(func(serial, command string, args []string, peer string) (string, any, error) {
	return "done", sdk.Records{
		Key:  "sessions",
		Rows: sessionRows(),
	}, nil
})

Key names the envelope the rows belong under. Rows is an iter.Seq[sdk.Record], and each record carries one Item the command produced or one Fault it rejected. A Row appends its own JSON into the buffer the writer owns, so a walk of a million rows allocates for none of them. The row is appended before the yield that carried it returns and nothing keeps a reference to it, so a producer can hand back one row value for every row of the walk and refill it in place.

Fields names the columns when every row of the walk shares one schema. Declare them and each row is a JSON array of values in that order, so the names travel once on the head instead of on every row:

return "done", sdk.Records{
	Key:    "sessions",
	Fields: []string{"peer", "state", "uptime"},
	Rows:   sessionColumnRows(),
}, nil

An operator sees the same objects either way. The engine reads the names off the head and puts each value back under its own name. Declaring a schema therefore changes what the wire carries and never what the command answers.

A row MUST then carry exactly one value for each name, in the same order. The two are read against each other by POSITION. A short row would gain a column it never carried, and a long one would lose a value, so such a row is refused rather than repaired.

Three rules bind a handler that answers this way.

A row that no wire message can carry is reported as a rejected row and the walk continues. The operator then reads the rows that were applied under Key, and the row that was refused under errors beside them. A short walk whose rows each fit and whose one collapsed document does not is refused the same way, and the operator reads that rejection alone.

A plugin also READS a record answer. Plugin.DispatchCommandAnswer runs an engine command and yields each row as it arrives, which is what bounds the memory of a walk over a large table. Plugin.DispatchCommand reads the same answer as one document for a caller that wants the whole payload.

Declaring what your command's answer holds

Each CommandDecl carries an optional Shape, Columns and AddressFields. Declare them and the CLI publishes the pipe operators your command supports, and refuses the others BY NAME before your handler runs. Declare nothing and the CLI waits for the answer, then refuses from what it has in hand.

Commands: []sdk.CommandDecl{{
	Name:          "my-plugin peers",
	Description:   "Show the sessions",
	Shape:         "tab",
	Columns:       []string{"address", "state", "up"},
	AddressFields: []string{"address"},
}},

The shape MUST be doc, map or tab, and a column or address-field list MUST come with a shape. One command path carries one declaration, so make every branch of your handler answer the same shape whatever argument it takes. The field list, the bounds and the four refusals are in plugin-development/commands.

Naming a pipe alias for your own command

A plugin declares a CLI pipe alias in the Pipes list of the same Registration. An alias is the word an operator types after the pipe character, and it stands for an operator chain. The BGP RPKI plugin declares summary on show bgp rpki, so show bgp rpki | summary answers the counters without the cache server rows.

Pipes: []sdk.PipeDecl{{
	Command:     "my-plugin status",
	Name:        "totals",
	Description: "The counters, without the per-session rows",
	Expansion:   "display sessions-total sessions-established",
}},

The pipe layer selects and re-sequences. It renames no key, adds no numbers and counts no matching rows. So your handler MUST emit the aggregate fields beside the detail rows, as siblings at one level. A command whose second view needs computed data stays a subcommand, and so does one that takes a value.

Command MUST be one of the commands you declare in the same message. Three holders refuse the name:

• a built-in pipe operator that carries it.
• a pipe filter on an overlapping command path that carries it.
• an alias on the exact same command path that carries it.

One refusal fails your whole startup, and the daemon log names the plugin, the path and the name.

A declared alias resolves over ze cli -c "<command>" and in the interactive session a plain ssh client reaches. It does NOT resolve in ze cli with no command argument, which expands the chain in the client process and answers pipe error: unknown pipe operator: totals.

Dependencies

Plugins can declare dependencies on other plugins. The engine starts plugins in dependency order and delivers state/EOR events to dependents first.

# bgp-gr depends on bgp-rib
# bgp-rpki depends on bgp-adj-rib-in
# bgp-rs optionally uses bgp-adj-rib-in

Dependencies are declared in the plugin's registration, not in config. The engine resolves them automatically. Two kinds:

bgp-rs uses bgp-adj-rib-in optionally: when both are loaded, replay-on-peer-up works; when bgp-adj-rib-in is absent, forwarding still works and a single WARN log announces that replay is disabled. bgp-rs forwards via the typed Plugin.ForwardCached / ReleaseCached fast path (rs-fastpath-3) instead of the legacy text-RPC send bgp <sel> cached <id> pipeline. See architecture/api/commands for the full SDK surface.

Exclusive Roles

When two plugins both implement a behavior but only one should run it, the plugin that takes over declares the role in its static registration (Claims), and the other stands down. The engine unions the claims of every plugin in the startup set and delivers the union on each plugin's Stage-2 configure callback; the standing-down plugin reads it with sdk.Plugin.ClaimActive(role) from its OnConfigure handler.

Stage 2 is part of the sequential handshake, so the decision is recorded before any plugin sends Stage-5 ready and therefore before the engine starts peers. A handler reading it during a runtime event always sees the final answer.

bgp-rs claims bgp-peer-up-replay; bgp-adj-rib-in stands its own replay down when that claim is active. The decision must not be re-derived from OnAllPluginsReady: that callback is fanned out on detached goroutines that race session establishment, and when the ownership decision was taken there both plugins replayed, so a peer received a byte-identical duplicate UPDATE. An unclaimed or unresolvable role reads false, which is the fail-closed direction: nobody promised to do this, so keep doing it yourself.

A claim is daemon-wide, and delivery is per-peer

A claim says a role has an owner in this daemon. It cannot say the owner will act on a given peer, because Stage 2 runs before any session exists. Two things make the claim wrong for one peer, and the plugin that stood down can see neither: the claimant takes no delivery of that peer's events, because the peer's attach process blocks do not name it, or the claimant never reached Running at all.

So the engine RETRACTS the claim, per event, for the peers it does not cover. Each peer-scoped event carries the claimed roles that no process being fed this event holds: StructuredEvent.UnheldRoles for a plugin on the direct bridge, and the unheld-roles member of the state event for a JSON one. A plugin that stood a role down MUST run its own default behavior for an event that names it, because nothing else will. The list is absent whenever every claim holds, which is the common case and costs no bytes.

bgp-adj-rib-in reads it at peer-up: it replays a peer that bgp-rs is not fed, and stands down for the peers bgp-rs drives. Without the retraction such a peer was served by nobody, because bgp-rs replays and forwards only peers it takes state delivery of.

Peer-Up Barrier

A plugin that decides on the peer-up event whether a peer may receive traffic declares PeerUpBarrier: true. The engine then holds that peer's initial-sync End-of-RIB until every barrier-declaring plugin subscribed to state events has taken delivery of the peer-up event, so "End-of-RIB sent" means "every barrier plugin has registered this peer".

bgp-rs declares it: it registers the peer as a forward target on that event, and an UPDATE arriving before that is forwarded nowhere. The wait is bounded and never blocks establishment. A plugin that does not acknowledge only delays the End-of-RIB to the timeout, which logs a WARN naming the peer and the shortfall. The expected count is taken over the plugins the event is actually delivered to, so declaring the field without subscribing to state events does not stall anything. It is separate from the session-ready wait below: merging them would let a route sender's report satisfy a registrar's obligation.

Session-Ready Report

A plugin whose routes belong to a peer's INITIAL routing update declares SignalsSessionReady: true and dispatches request peer <addr> plugin session ready once those routes are out. The engine holds that peer's End-of-RIB until the report arrives, so the marker means the initial routing update completed (RFC 4724 Section 4).

The declaration is voluntary. It says WHEN your routes belong, not what you may send, so a plugin that pushes routes on its own schedule declares nothing and is never waited for, and binding it with send [ update ] costs the peer no delay.

An external plugin has the same declaration under a different name. It is registered nowhere in this tree, so it declares signals-session-ready in its Stage-1 declare-registration instead, and the engine reads it off the running process. Declaring nothing stays the default there too.

The name you are waited for under is the one the operator wrote in attach process <name>, whatever your plugin is called. plugin { internal rs { use bgp-rs } } runs the process as rs, and the engine resolves that alias back to the bgp-rs registration before it asks whether you declared. Your report carries the process name, so it is credited to rs as well. The spelling of the implementation does not change the answer: use bgp-rs, use ze.bgp-rs, run ze.bgp-rs and run ze plugin bgp-rs all reach the same registration, because one function (plugin.RegistryNames) answers which registry row a process configuration names and every caller derives from it.

Three facts have to hold before a peer waits for your process, and each one is something you can check in your own config. The peer grants the route-push rail, with send [ update ] or send [ raw ]. The plugin declares the field. The peer grants receive [ state ], because the report answers the peer-up event: a process the peer never tells about the session cannot push into that session's initial update, so it is not waited for. A binding with send [ update ] and no receive [ state ] is therefore free of the wait rather than stalled by it.

Report once per establishment, from your peer-up handler, and report even when you had nothing to replay: the barrier cannot tell "finished with nothing to send" from "still working". A process that never reports only delays that peer's End-of-RIB to apiSyncTimeout (2s), which logs a WARN naming the peer and the silent processes.

Startup Timing

Each handshake stage is bounded by progress, not by wall clock. A stage fails only when the whole startup tier goes ze.plugin.stage.timeout (default 5s) without any plugin completing a stage; every completion re-arms the window. A BGP config puts twenty or more plugins in one tier (bgp, bgp-bmp, bgp-rpki, every bgp-filter-*), and a flat per-tier budget meant a CPU-starved or slow-disk host could lose all of them at once and come up with its BGP plugins missing.

The wait stays bounded three ways: a repeated stage completion is ignored, so a looping plugin cannot hold the window open; a wedged tier trips within one timeout of the last progress; and shutdown ends the wait. Worst case is (plugins + 1) x timeout per stage.

Debugging Plugins

The plugin debug shell lets you manually interact with the engine using the plugin protocol. This is useful when debugging plugin code -- you can send individual commands and inspect responses.

ze bgp plugin cli

The debug shell:

1. Asks about handshake parameters (plugin name, families) with defaults -- hit Enter to accept
2. Connects to the daemon via SSH
3. Runs the 5-stage plugin handshake over the SSH channel
4. Enters interactive command mode

Available post-handshake commands:

Use --name <name> to set a custom plugin name for the session.