Plugin Metrics
Register metrics in the process that performs the work. Plugins whose engines
produce the observations use the existing ConfigureMetrics callback in the
Registration struct. Code that runs in the daemon independently of its plugin
process uses the deferred daemon hook described below.
Implementation Pattern
Each plugin follows the same three-step pattern used by bgp-rib and bgp-gr:
-
Registration: Set
ConfigureMetricsin theRegistrationstruct insideinit(). The callback receives a typedmetrics.Registry; no assertion is needed. -
Storage: Declare a package-level
atomic.Pointerholding a metrics struct. The struct contains the counters and gauges created from the registry. -
Usage: At metric update points, load the pointer. If nil (metrics disabled), skip. Otherwise update the metric.
Naming Policy
Every metric name is built from four parts in fixed order:
ze_{scope}_{subject}_{detail}
| Part | Rule | Examples |
|---|---|---|
ze |
Always. Global prefix for all ze metrics. | |
| scope | Plugin name, hyphens stripped, lowercase. Reactor uses functional area. | rib, gr, fibkernel, systemrib, rpki, persist, watchdog |
| subject | Singular noun: the type of thing being measured. | route, peer, timer, session, vrp |
| detail | What aspect. Depends on metric type (see below). | inserts, active, expired |
Type-Specific Rules
| Type | Pattern | Suffix | Examples |
|---|---|---|---|
| Counter | ze_{scope}_{subject}_{event}_total |
Always _total |
ze_rib_route_inserts_total, ze_gr_timer_expired_total |
| Gauge | ze_{scope}_{subject}_{qualifier} |
Never _total |
ze_gr_active_peers, ze_fibkernel_routes_installed |
| Histogram | ze_{scope}_{subject}_{action}_seconds |
Always unit suffix | ze_peer_dial_seconds |
| CounterVec | Same as Counter, labels for dimensions | _total |
ze_rpki_validation_outcomes_total with {result} label |
| GaugeVec | Same as Gauge, labels for dimensions | ✕ | ze_rib_routes_in with {peer} label |
Subject Rules
The subject is always a singular noun describing the category of thing measured. When the metric counts a quantity of that noun, the grammatical plural appears naturally in the detail.
| Subject (singular) | Gauge (current count) | Counter (cumulative events) |
|---|---|---|
| route | routes_installed |
route_installs_total |
| peer | peers_up |
peer_flaps_total |
| vrp | vrps_cached |
|
| session | sessions_active |
session_teardowns_total |
Detail Rules
For counters, the detail is the event that happened, as a noun or past participle:
inserts, withdrawals, expired, installs, removals, errors.
For gauges, the detail is a qualifier distinguishing this gauge from others
in the same scope: installed, active, up, cached, in, out.
Label Rules
Use labels for runtime dimensions. Never encode variable data in metric names.
| Label | When to use | Example |
|---|---|---|
peer |
Per-peer breakdown | ze_rib_routes_in{peer="10.0.0.1"} |
family |
Per-address-family breakdown | ze_rib_route_inserts_total{family="ipv4/unicast"} |
operation |
Error categorization | ze_fibkernel_errors_total{operation="add"} |
result |
Outcome categorization | ze_rpki_validation_outcomes_total{result="valid"} |
Scope-to-Prefix Mapping
| Plugin (Registration Name) | Metric scope | Rationale |
|---|---|---|
bgp-rib |
rib |
Established; bgp prefix would be redundant |
bgp-gr |
gr |
Established |
fib-kernel |
fibkernel |
Hyphen stripped |
rib (sysrib) |
systemrib |
Registration name is rib but ze_rib_ is taken by bgp-rib; systemrib reads naturally |
bgp-watchdog |
watchdog |
bgp prefix redundant |
bgp-rpki |
rpki |
bgp prefix redundant |
bgp-role |
role |
bgp prefix redundant |
bgp-filter-path-asn |
filter_path_asn |
The package name. The bgp prefix is redundant on a BGP filter, and the hyphen-stripped bgpfilterpathasn cannot be read |
bgp-filter-community |
filter_community |
The package name, for the same reason |
Full Inventory
| Metric Name | Type | Labels | Owner |
|---|---|---|---|
ze_task_panics_total |
CounterVec | component, task | every component with a network-input task, through its taskguard.Guard: one per panic recovered at the task boundary |
ze_rib_routes_in_total |
Gauge | bgp-rib | |
ze_rib_routes_out_total |
Gauge | bgp-rib | |
ze_rib_routes_in |
GaugeVec | peer | bgp-rib |
ze_rib_routes_out |
GaugeVec | peer | bgp-rib |
ze_rib_route_inserts_total |
CounterVec | peer, family | bgp-rib |
ze_rib_route_withdrawals_total |
CounterVec | peer, family | bgp-rib |
ze_attr_pool_intern_total |
GaugeVec | pool | bgp-rib |
ze_attr_pool_dedup_hits_total |
GaugeVec | pool | bgp-rib |
ze_attr_pool_slots_used |
GaugeVec | pool | bgp-rib |
ze_bgp_paths_limit_refused_total |
CounterVec | peer, family | bgp reactor (receive ceiling): one per received path refused over Ze's advertised PATHS-LIMIT |
ze_bgp_paths_limit_election_failed_total |
CounterVec | peer, family | bgp reactor (send admission): one per contested prefix whose paths no RIB could rank under the peer's PATHS-LIMIT, so arrival order stood |
ze_bgp_attr_mod_remove_buffer_refused_total |
CounterVec | attribute | bgp reactor (filterapi) |
ze_bgp_rs_prepend_applied_total |
Counter | bgp reactor (route-server forward) | |
ze_bgp_rs_large_control_community_stripped_total |
Counter | bgp reactor (route-server forward) | |
ze_bgp_redistribute_events_received |
Counter | redistribute-orchestrator | |
ze_bgp_redistribute_announcements |
Counter | redistribute-orchestrator | |
ze_bgp_redistribute_withdrawals |
Counter | redistribute-orchestrator | |
ze_bgp_redistribute_filtered_protocol_total |
Counter | redistribute-orchestrator | |
ze_redistribute_chain_rejected_total |
CounterVec | destination, source | redistribute-orchestrator |
ze_bgp_redistribute_replay_total |
CounterVec | source | redistribute-orchestrator |
ze_gr_active_peers |
Gauge | bgp-gr | |
ze_gr_stale_routes |
GaugeVec | peer | bgp-gr |
ze_gr_timer_expired_total |
CounterVec | peer | bgp-gr |
ze_fibkernel_routes_installed |
Gauge | fib-kernel | |
ze_fibkernel_route_installs_total |
Counter | fib-kernel | |
ze_fibkernel_route_updates_total |
Counter | fib-kernel | |
ze_fibkernel_route_removals_total |
Counter | fib-kernel | |
ze_fibkernel_errors_total |
CounterVec | operation | fib-kernel |
ze_systemrib_routes_best |
Gauge | rib (sysrib) | |
ze_systemrib_route_changes_total |
CounterVec | action | rib (sysrib) |
ze_systemrib_events_received_total |
Counter | rib (sysrib) | |
ze_watchdog_peers_up |
Gauge | bgp-watchdog | |
ze_watchdog_route_announcements_total |
Counter | bgp-watchdog | |
ze_watchdog_route_withdrawals_total |
Counter | bgp-watchdog | |
ze_rpki_vrps_cached |
Gauge | bgp-rpki | |
ze_rpki_sessions_active |
Gauge | bgp-rpki | |
ze_rpki_validation_outcomes_total |
CounterVec | result | bgp-rpki |
ze_rpki_updates_refused_total |
CounterVec | reason | bgp-rpki: JSON UPDATEs refused before validation, with unknown-peer-as or malformed-peer-as |
ze_role_route_rejects_total |
CounterVec | reason | bgp-role |
ze_role_route_suppressions_total |
CounterVec | reason | bgp-role |
ze_filter_path_asn_rejects_total |
CounterVec | direction, position, reason | bgp-filter-path-asn |
ze_filter_community_value_scrubbed_total |
CounterVec | width | bgp-filter-community |
ze_filter_community_relation_tag_written_total |
Counter | bgp-filter-community | |
ze_filter_community_blackhole_guard_added_total |
CounterVec | community | bgp-filter-community |
ze_isis_adjacencies_up |
GaugeVec | level, interface | isis |
ze_isis_adjacencies_total |
GaugeVec | level | isis |
ze_isis_lsps |
GaugeVec | level | isis (lsdb) |
ze_isis_lsp_fragments |
GaugeVec | level | isis (lsdb) |
ze_isis_lsp_originations_total |
CounterVec | level | isis (lsdb) |
ze_isis_sequence_wraps_total |
CounterVec | level | isis (lsdb) |
ze_isis_purges_total |
CounterVec | level | isis (lsdb) |
ze_isis_lsps_received_total |
CounterVec | level | isis (flooding) |
ze_isis_lsps_transmitted_total |
CounterVec | level | isis (flooding) |
ze_isis_csnp_sent_total |
CounterVec | level | isis (flooding) |
ze_isis_csnp_received_total |
CounterVec | level | isis (flooding) |
ze_isis_psnp_sent_total |
CounterVec | level | isis (flooding) |
ze_isis_psnp_received_total |
CounterVec | level | isis (flooding) |
ze_isis_srm_resends_total |
CounterVec | level | isis (flooding) |
ze_isis_lsps_dropped_total |
CounterVec | level, reason | isis (flooding) |
ze_isis_dis_elections_total |
CounterVec | level | isis (dis) |
ze_isis_pseudonode_lsps |
GaugeVec | level | isis (dis) |
ze_isis_auth_failures_total |
CounterVec | level, interface | isis (auth) |
ze_isis_redist_injected_total |
CounterVec | source, afi | isis (redistribute) |
ze_isis_redist_withdrawn_total |
CounterVec | source, afi | isis (redistribute) |
ze_isis_redist_inject_failures_total |
CounterVec | source | isis (redistribute) |
ze_isis_lsp_reoriginations_total |
CounterVec | level | isis (redistribute) |
ze_isis_spf_runs_total |
CounterVec | level | isis (spf) |
ze_isis_spf_duration_seconds |
HistogramVec | level | isis (spf) |
ze_isis_spf_nodes |
GaugeVec | level | isis (spf) |
ze_isis_routes_installed |
GaugeVec | level, afi | isis (spf) |
ze_isis_frames_sent_total |
CounterVec | interface | isis (transport) |
ze_isis_frames_received_total |
CounterVec | interface | isis (transport) |
ze_isis_frames_dropped_total |
CounterVec | interface, reason | isis (transport) |
ze_isis_sockets_open |
Gauge | isis (transport) | |
ze_ospf_packets_sent_total |
CounterVec | interface, type | ospf, ospfv3 (transport) |
ze_ospf_packets_received_total |
CounterVec | interface, type | ospf, ospfv3 (transport) |
ze_ospf_packets_dropped_total |
CounterVec | interface, reason | ospf, ospfv3 (transport) |
ze_ospf_sockets_open |
Gauge | ospf, ospfv3 (transport) | |
ze_ospf_interface_up |
GaugeVec | area, interface | ospf (iface) |
ze_ospf_dr_elections_total |
CounterVec | interface | ospf (iface) |
ze_ospf_neighbors |
GaugeVec | area, interface, state | ospf (neighbor) |
ze_ospf_adjacencies_full |
GaugeVec | area | ospf (neighbor) |
ze_ospf_nsm_events_total |
CounterVec | event | ospf (neighbor) |
ze_ospf_lsdb_lsas |
GaugeVec | area, type | ospf (lsdb) |
ze_ospf_lsa_originations_total |
CounterVec | type | ospf (lsdb) |
ze_ospf_lsa_refreshes_total |
CounterVec | type | ospf (lsdb) |
ze_ospf_lsa_purges_total |
CounterVec | type | ospf (lsdb) |
ze_ospf_lsupdates_sent_total |
CounterVec | interface | ospf (lsdb) |
ze_ospf_lsupdates_received_total |
CounterVec | interface | ospf (lsdb) |
ze_ospf_lsacks_sent_total |
CounterVec | interface | ospf (lsdb) |
ze_ospf_retransmissions_total |
CounterVec | area | ospf (lsdb) |
ze_ospf_spf_runs_total |
CounterVec | area | ospf (spf) |
ze_ospf_spf_duration_seconds |
HistogramVec | area | ospf (spf) |
ze_ospf_routes_installed |
GaugeVec | type, af | ospf (spf) |
ze_ospf_abr |
Gauge | ospf (spf) | |
ze_ospf_summary_lsas |
GaugeVec | area | ospf (spf) |
ze_ospf_asbr |
Gauge | ospf (engine) | |
ze_ospf_external_lsas |
Gauge | ospf (engine) | |
ze_ospf_redist_injected_total |
CounterVec | source | ospf (redistribute) |
ze_ospf_redist_withdrawn_total |
CounterVec | source | ospf (redistribute) |
ze_ospf_nssa_translations_total |
CounterVec | area | ospf (engine) |
ze_ospf_auth_failures_total |
CounterVec | interface, reason | ospf (engine) |
ze_ospf_af_instances |
GaugeVec | af | ospf (RFC 5838 multi-AF) |
ze_ospf_af_bit_mismatch_total |
CounterVec | af | ospf (RFC 5838 multi-AF) |
ze_traffic_usage_ingress_port_bytes_total |
GaugeVec | interface, dst_port, protocol | traffic-usage |
ze_traffic_usage_egress_port_bytes_total |
GaugeVec | interface, src_port, protocol | traffic-usage |
ze_traffic_usage_ingress_bytes_total |
GaugeVec | interface, src_ip | traffic-usage (track-ip) |
ze_traffic_usage_egress_bytes_total |
GaugeVec | interface, dst_ip | traffic-usage (track-ip) |
ze_traffic_usage_map_entries |
GaugeVec | interface, map | traffic-usage |
ze_managed_clients_connected |
Gauge | managed (hub server) | |
ze_managed_config_fetch_total |
CounterVec | result | managed (hub server) |
ze_managed_config_changed_pushed_total |
Counter | managed (hub server) | |
ze_plugin_write_watchdog_total |
CounterVec | transport | plugin (server) |
ze_plugin_delivery_failures_total |
CounterVec | plugin | plugin (process): event batches sent with no caller waiting that failed to reach the plugin |
ze_plugin_delivery_slow_batches_total |
CounterVec | plugin | plugin (process): event batches that took over 500 ms to deliver, on any transport |
ze_plugin_filter_batch_routes |
Histogram | plugin (ipc): the routes one filter-update exchange with a forked plugin carried, buckets 1 to 64 (rpc.FilterUpdateBatchMax); a plugin running internal is called through its filter slot and observes nothing |
|
ze_plugin_filter_slot_routes_total |
Counter | plugin (ipc): routes for which the typed in-process filter slot returned a non-nil verdict without an error; closed slots and evaluator errors do not count. The batch histogram's _sum counts routes sent through the encoded transport instead |
|
ze_plugin_filter_route_enqueues_total |
Counter | plugin (ipc): routes successfully sent to the encoded filter-update queue, including routes whose later exchange fails. Closed connections and canceled waits that enqueue nothing do not count. Unlike the batch histogram, this advances while an earlier exchange is still in flight | |
ze_policy_route_set_refusals_total |
CounterVec | filter | bgp library chains (policychain): one per route a plugin filter failed by answering a set Ze refused (undeclared, not settable at the attachment, unknown, or of the wrong shape), whichever on-error then decided the route. filter is the chain reference, <plugin>:<filter> |
ze_iface_owned_devices |
GaugeVec | owner | iface (owned-device registry) |
ze_iface_link_events_coalesced_total |
CounterVec | name | iface (link event queue) |
ze_iface_carrier_resyncs_total |
CounterVec | name | iface (carrier resync) |
ze_iface_resolver_events_dropped_total |
CounterVec | name | iface (resolver fan-out) |
ze_iface_link_worker_blocked_total |
CounterVec | name | iface (link event worker) |
ze_iface_link_events_queued_while_blocked_total |
CounterVec | name | iface (link event queue) |
ze_iface_config_apply_started_total |
Counter | iface (config apply) | |
ze_iface_ra_sent_total |
CounterVec | interface | iface-ra |
ze_iface_ra_solicited_total |
CounterVec | interface | iface-ra |
ze_iface_delegated_prefixes_held |
GaugeVec | name | iface (delegated-prefix table) |
ze_pppoe_client_renegotiations_total |
CounterVec | name | l2tp-pppoe-client |
ze_pppoe_client_ipv6cp_opened |
GaugeVec | name | l2tp-pppoe-client |
ze_radius_unsigned_eap_discards_total |
Counter | radius (client) | |
ze_radius_unsigned_response_discards_total |
CounterVec | server | radius (client): one per Access answer discarded for carrying no Message-Authenticator |
ze_dns_resolver_queries_total |
CounterVec | transport, rcode | dnsresolver: one per query answered |
ze_dns_resolver_refused_total |
CounterVec | reason | dnsresolver: one per REFUSED answer, acl or rd-clear |
ze_dns_resolver_rate_limited_total |
Counter | dnsresolver: one per query dropped by limit per-client-rate |
|
ze_dns_resolver_dropped_total |
Counter | dnsresolver: one per query dropped at limit outstanding |
|
ze_dns_resolver_outstanding |
Gauge | dnsresolver: resolutions in progress | |
ze_dns_resolver_cache_entries |
GaugeVec | cache | dnsresolver: entries in the answer and negative caches |
ze_dns_resolver_cache_hits_total |
Counter | dnsresolver: one per query answered from the cache | |
ze_dns_resolver_upstream_queries_total |
CounterVec | result | dnsresolver: one per upstream exchange, answered, timeout, error, failure or truncated |
ze_dns_resolver_listener_up |
GaugeVec | protocol, address | dnsresolver: 1 while a listener socket is bound, by protocol and address with port |
ze_dns_tcp_connection_refused_total |
Counter | DNS server harness: one per TCP connection closed at accept because its listener held Options.TCPConnMax (the LAN resolver sets it) |
|
ze_dns_source_snapshots_rejected_total |
CounterVec | consumer | dnsresolver (dns-resolver) and dhcpserver (dhcpserver): one per system-dns snapshot the consumer received and could not apply; it keeps its previous learned resolvers, logs the cause at WARN and goes on to the next snapshot |
ze_dns_source_records |
GaugeVec | origin | system-dns registry (config/system Resolvers.SetMetrics): the learned resolver records each origin holds, zero once its leases are withdrawn |
ze_dns_source_selection_changes_total |
CounterVec | consumer | system-dns registry: one per change of a consumer's published dns-source selection after its first, the router's under consumer system |
The three iface link-event counters answer the same question from different
points on the path, and none of them reports a lost final state. A rising
ze_iface_link_events_coalesced_total says the queue's worker fell behind the
kernel, which a config commit causes because it holds the lock that worker
takes. Nothing is lost when it rises: the queue keeps the state each interface
ENDED in. A rising ze_iface_carrier_resyncs_total says the recorded route
metric contradicted live carrier and ze moved the route to agree, which happens
when a route install failed or a device was already down at start. A rising
ze_iface_resolver_events_dropped_total names a LOGICAL interface whose
subscriber was too slow, so the fan-out discarded the OLDEST event buffered for
it to make room for the newest. That subscriber has lost the middle of a burst
and still sees the state the interface ended in.
Only the first two are read together. ze_iface_link_events_coalesced_total
rising while ze_iface_carrier_resyncs_total stays flat is the healthy shape
under load: the worker was behind and the event path still carried every
interface to its final state. The resync counter moving is the one that says a
transition never arrived at all.
The two iface-ra counters carry one interface label each.
ze_iface_ra_sent_total counts every Router Advertisement put on the wire.
ze_iface_ra_solicited_total counts the subset that answered a Router
Solicitation, and each of those increments both counters, so the first one stays
the total. A rise in the solicited counter with a flat send rate means that
hosts join the link.
ze_iface_delegated_prefixes_held{name} counts the delegated prefixes the
DHCPv6 lease of the pppoe-client name holds. It is set each time the client
binds, renews or loses a prefix, so 0 means no LAN unit that takes a /64 from
that client has one. ze_pppoe_client_ipv6cp_opened{name} is 1 while that
client's IPv6CP is Opened and IPv6 passes on the session, and 0 before it
opens, after the access concentrator rejects or closes it, during an LCP
restart and once the session ends. Read together: a client at 1 on IPv6CP and
0 on delegated prefixes has IPv6 on the WAN but no delegation for the LAN.
ze_pppoe_client_renegotiations_total{name} counts network-phase LCP
renegotiations the client started.
The LAN resolver's ze_dns_resolver_queries_total counts answered queries
only: a query dropped by the rate or the outstanding bound gets no reply and
is counted by ze_dns_resolver_rate_limited_total or
ze_dns_resolver_dropped_total instead. ze_dns_resolver_cache_entries is
published by the resolver's sweeper every 10 seconds and by a cache clear, so
it can lag the cache by that interval. What each label value means is in
the LAN resolver design.
ze_plugin_write_watchdog_total increments when a plugin RPC write on a
transport that does not support SetWriteDeadline (SSH channels, io.Pipe)
stalls past the write-watchdog window (default 30s); the connection is then
closed (fail-fast). Deadline-capable transports (net.Conn) use write
deadlines instead and never contribute to this counter. The transport label
is a low-cardinality bucket (stream, pipe, file).
ze_ospf_packets_dropped_total counts transport drops before packet dispatch or
after a failed send. The Linux receive path currently emits reason
malformed-ipv4 when a raw datagram is too short for an IPv4 header or its IHL
overruns the received bytes; SendPacket emits reason send-error when the
backend transmit call fails.
The two ospf (iface) series track the OSPFv2 Interface State Machine:
ze_ospf_interface_up{area,interface} is 1 for active point-to-point or
broadcast interfaces and 0 for Down, passive, or loopback records;
ze_ospf_dr_elections_total{interface} increments when a broadcast interface's
elected DR or BDR changes. The three ospf (neighbor) series track the Neighbor
State Machine: ze_ospf_neighbors{area,interface,state} gauges neighbours by
NSM state, ze_ospf_adjacencies_full{area} gauges Full adjacencies, and
ze_ospf_nsm_events_total{event} counts state-machine events.
The eight ospf (lsdb) series track the RFC 2328 Section 13 flooding and
Section 14 aging machinery: ze_ospf_lsdb_lsas{area,type} gauges current LSDB
entries, ze_ospf_lsa_originations_total{type},
ze_ospf_lsa_refreshes_total{type}, and ze_ospf_lsa_purges_total{type} count
self-LSA lifecycle events, ze_ospf_lsupdates_sent_total{interface} and
ze_ospf_lsupdates_received_total{interface} count flooding packets,
ze_ospf_lsacks_sent_total{interface} counts explicit acknowledgements, and
ze_ospf_retransmissions_total{area} counts retransmit-list resends.
ReceiveUpdate increments the received-packet counter on entry, before interface
lookup and per-LSA validation. A packet whose LSAs all fail checksum, opaque-body
alignment, or the registered receive validator still increments this counter.
Those LSAs are skipped before installation, acknowledgement, or reflooding, so
the received count is not an accepted-LSA count.
The five ospf (spf) series track route computation and install state:
ze_ospf_spf_runs_total{area} and ze_ospf_spf_duration_seconds{area} report
SPF scheduling and runtime, ze_ospf_routes_installed{type,af} reports the current
Loc-RIB-owned route count by OSPF path type and address family (RFC 5838), ze_ospf_abr is 1 only while this
router has an active backbone attachment and at least one other active area, and
ze_ospf_summary_lsas{area} reports current self-originated Summary-LSA count by
area.
The two isis (dis) series are the Designated IS / pseudo-node metrics (broadcast
LANs): ze_isis_dis_elections_total counts DIS elections that changed the elected
DIS per level, and ze_isis_pseudonode_lsps gauges the pseudo-node LSPs this node
currently originates as the elected DIS per level.
The isis (auth) series ze_isis_auth_failures_total counts IS-IS PDUs rejected
because authentication verification failed, by level and receiving interface. It
is incremented at the receive verify site (a PDU with no, wrong, or misordered
authentication TLV under configured auth) and is the operator's signal for a
key mismatch, a downgrade attempt, or a spoofed purge.
The four isis (redistribute) series track redistribution into IS-IS:
ze_isis_redist_injected_total{source,afi} and
ze_isis_redist_withdrawn_total{source,afi} count routes imported into IS-IS LSPs
(as TLV 135 IPv4 reachability or TLV 236 IPv6 reachability) and withdrawn, by
originating source (connected, static, bgp) and address family
(afi=ipv4|ipv6); ze_isis_redist_inject_failures_total{source} counts
injections whose LSP re-origination failed; and
ze_isis_lsp_reoriginations_total{level} counts the own-LSP re-originations that
redistribution drove, by level. Exporting IS-IS routes out to BGP uses the
generic redistribute-orchestrator counters, not an IS-IS-specific series.
The six redistribute-orchestrator series track the generic dispatch every
protocol shares. ze_bgp_redistribute_events_received counts route-change
batches taken off the bus. _announcements and _withdrawals count the entries
dispatched to a consumer. _filtered_protocol_total counts batches the loop
guard skipped. ze_redistribute_chain_rejected_total{destination,source}
counts the routes an import rule covers and its policy chain rejected, on the
incremental and the replay path; a route no import rule names into that
destination is not a chain's verdict and is not counted.
ze_bgp_redistribute_replay_total{source} counts routes replayed to a party
that arrived after a producer emitted. That party is either a BGP peer whose
session just established, or a destination protocol's consumer that just
registered. One counter serves both, labeled by the producing source rather than
by which of the two triggered it.
The four isis (spf) series track the shortest-path computation and its Loc-RIB
install: ze_isis_spf_runs_total{level} counts Dijkstra runs per level,
ze_isis_spf_duration_seconds{level} is a histogram of each run's wall-clock
duration (buckets span 50us..500ms), ze_isis_spf_nodes{level} gauges the node
count of the last run's graph, and ze_isis_routes_installed{level,afi} gauges
the prefixes currently installed into the shared Loc-RIB by level and address
family (afi=ipv4|ipv6). The IPv6 install pass runs over the same shared SPF
tree, so the gauge is the only SPF series carrying an afi label.
The four isis (transport) series track the raw Layer-2 socket I/O:
ze_isis_frames_sent_total{interface} and
ze_isis_frames_received_total{interface} count frames transmitted and received
per circuit, ze_isis_frames_dropped_total{interface,reason} counts frames
dropped on receive by reason, and ze_isis_sockets_open gauges the number of
open IS-IS raw L2 sockets process-wide (unlabelled, a single process-level gauge).
The four ospf (transport) series track raw IPv4 protocol 89 socket I/O:
ze_ospf_packets_sent_total{interface,type} and
ze_ospf_packets_received_total{interface,type} count OSPF packets transmitted
and received per interface, ze_ospf_packets_dropped_total{interface,reason}
counts receive rejects and send failures, and ze_ospf_sockets_open gauges open
OSPF raw sockets process-wide. The ospf (iface) and ospf (neighbor) series
track interface state, DR/BDR election changes, NSM states, Full adjacencies,
and NSM events.
Dual-stack IPv6 (RFC 5308) adds no new IS-IS series: it sets afi=ipv6 on the
labelled series above and on the SPF route-install gauge
ze_isis_routes_installed{level,afi}. IPv4 uses afi=ipv4; the IPv6 install pass
runs over the same shared SPF tree.
When ConfigureMetrics Runs
The hook does not fire while the plugin is being spawned. runPluginPhase
starts every process of a startup phase before it runs the tier-ordered
handshake, and the Prometheus registry is not created until the bgp plugin's
stage-2 OnConfigure builds the reactor, so at spawn time there is usually no
registry yet. registry.InjectPluginMetrics therefore defers the hook and
registry.SetMetricsRegistry drains the deferred set the moment a registry
exists; whichever of the two happens second performs the injection, exactly once
per plugin.
Two consequences for a plugin author:
- Do not read your metrics struct during
init()or at the top ofRunEngine; load the atomic pointer at each metric update point, as step 3 above says. - Declaring a
Dependencyonbgpdoes not order the injection. Dependencies order the 5-stage handshake, not the spawn the hook used to be bound to.
Metrics owned by daemon callbacks
A callback that runs in the daemon registers its metrics with
registry.InjectPluginMetrics from its owner's init(), as the crash-capture
plugin does. The same deferred injection creates its counters when the daemon's
registry becomes available; without a registry, no exporter is created.
The reject-asn match condition uses this form: its list is compiled into the
daemon's BGP chain even when the plugin's command process runs forked.
bgp-filter-path-asn therefore declares no process ConfigureMetrics callback
for these counters. The command process must not publish a second copy of
daemon-owned observations.
A plugin run in its own process
A plugin the daemon forks (plugin { external <name> { run ...; } }) cannot
reach the daemon's registry. Its runner, cli.RunPlugin, calls the same
ConfigureMetrics before RunEngine, with an sdk.MetricsPublisher: a
metrics.Registry that creates each metric in the daemon under the same name
and shape, over the metrics-publish RPC. The plugin's code does not change,
and a scrape of the daemon shows the same names and values as in-process.
The plugin decides how often the daemon learns of changes:
- a declaration is published at once, as soon as the plugin is running, so every name exists in the daemon from the start;
- updates accumulate in the plugin and are published every interval:
MetricsIntervalDefault(5 seconds), or what the plugin sets withPlugin.SetMetricsIntervalbeforeRun(100 ms to one hour); Plugin.FlushMetricspublishes at once, and the last updates are published before the plugin acknowledges its removal (bye). A daemon stop closes the connection with nobye, so the changes of its last interval are not sent; the daemon is stopping, and nothing scrapes it.
What the plugin holds is bounded whatever the update rate: a counter's sum
since the last publish, a gauge's value, and one count and one sum per
histogram bucket. Deletion tracking holds only the labeled series the daemon
acknowledged, at most rpc.MetricsSeriesMax; repeated creation and deletion
between publishes does not accumulate a deletion log. Recreating a deleted
label replaces its previous value, and handles kept from the deleted series
cannot change its replacement.
A flush takes one snapshot and splits it into finite requests when necessary
to fit the RPC frame limit, applying declarations before deletions before
updates. A request the daemon did not take (an error, or no registry yet)
keeps its changes and the unsent tail for the next flush; requests already
acknowledged are not replayed. An increment made during startup is not lost.
Per plugin the daemon holds at most rpc.MetricsDeclaredMax names and
rpc.MetricsSeriesMax labeled series. A series past the limit, a label value
longer than rpc.MetricLabelValueMax octets, a name without the ze_ prefix,
a name the daemon holds with another shape, a declaration too wide to fit an
RPC frame even alone, and a value that is not finite are not forwarded, each
with one WARN line. A name the daemon refuses gives back its labeled series and
its declared slot, and its handles update nothing. The plugin remembers
rpc.MetricsDeclaredMax refused names, so a repeat stays one WARN line. Past
that, a name the daemon refuses is remembered in the declared slot it keeps, and
a name the plugin itself refuses is not held: one further WARN line says later
refused names go unnamed. A name longer than rpc.MetricNameMax is never held
either. One WARN line covers all such names, showing the name cut at the limit.
Declaration admission uses its encoded size, not an
arbitrary label-name length limit; a fitting singleton is left to the actual
request writer. The label-value boundary is inclusive: 256 octets are accepted.
A restarted plugin declares its metrics again and carries on from the daemon's
values. A Go plugin outside this repository gets the same registry from
Plugin.MetricsRegistry.
Label values MUST be valid UTF-8: the publisher refuses malformed bytes before
JSON can replace them and change a series' identity. A valid empty-string
label and the Unicode replacement character are accepted normally. Delete
returns false for malformed values or a label-value count that does not match
the vector; it cannot delete another valid label by resolving the same key.
The publisher also refuses an addition or observation if its resulting pending
aggregate cannot be represented as a finite float64. It logs the refusal
without changing the previously accepted value or histogram count. While a
request is unacknowledged, admission includes the values that request would
restore if it failed. Those retained changes remain bounded; an overflowing
operation cannot poison other labels or metrics. Once the request is
acknowledged, its values no longer constrain later deltas against a possible
restoration. A gauge can replace its value with Set. This is a transport
boundary, not parity with native Prometheus's support for infinite aggregates:
metrics-publish carries no non-finite JSON number.
System RIB routing-view metrics
The existing ze_systemrib_events_received_total counter counts each accepted
nonempty authoritative selected-change batch, including a duplicate input that
produces no FIB change. Malformed batches and empty selected-feed idle polls do
not count. Route-change counters count the emitted forwarding effects separately.
Both placements retain their existing metrics registration and transport.
The system RIB's routing-view monitor creates three scalar metrics through
Plugin.MetricsRegistry before Run. The SDK's existing publisher carries
them in both placements; the monitor starts no publisher of its own.
| Metric | Meaning |
|---|---|
ze_rib_view_resync_total |
Reconstruction attempts after the initial consumer starts, including selected-view reset and local capture overflow. |
ze_rib_view_pending_prefixes |
Distinct prefixes held in an incomplete selected snapshot or collected forwarding work awaiting reconciliation. |
ze_rib_view_ready |
One after the view and its forwarding output synchronize; zero during initialization, reconstruction, failure or shutdown. |
These names have no labels, so route prefixes and session tokens cannot grow
their series. The ze_rib_view_ names describe the shared routing-view
transport; the existing system RIB metrics retain ze_systemrib_.
NopRegistry Fallback
When telemetry is disabled in config, plugins receive a NopRegistry that returns
no-op metric implementations. The nil-check on the atomic pointer handles the case
where ConfigureMetrics is never called at all (plugin loaded without a metrics
registry). A forked plugin always receives its publisher; with telemetry
disabled the daemon creates the published metrics in the NopRegistry, so
they vanish there exactly as an in-process plugin's do.
The Task Panic Counter
A plugin with a task that handles network input (a receive goroutine, a
session read loop, a request handler) guards it with internal/core/taskguard,
and the guard owns ze_task_panics_total{component,task}. The plugin does not
declare the metric itself:
- Build one
taskguard.Guardper component withtaskguard.New(component, log), and declare each task once withGuard.Task(name, restart).restartsays in plain words what brings the unit back. - Call
Guard.SetMetricsRegistry(reg)fromConfigureMetrics. A panic recovered before the registry arrived is added then, so none is lost. - Defer
task.Contain(describe, end)directly at the top of the task. It logs, counts and writes the crash record, then callsendto tear the unit down.
The name and labels are one declaration (taskguard.MetricName), so every
component registering it shares one vector. The log line and the crash record
are described in docs/architecture/diagnostics/crash-capture.md, "Recovered
panics".
Which Plugins Need Metrics
Not every plugin needs metrics. NLRI codec plugins (flowspec, evpn, labeled, etc.) are stateless encoders/decoders with no runtime state to observe. Only plugins with state machines, caches, or I/O operations benefit from metrics.
| Has metrics | No metrics needed |
|---|---|
| Plugins with route state (rib, sysrib, persist) | NLRI codecs (bgp-nlri-*) |
| Plugins with external I/O (fib-kernel, rpki) | Pure capability-only plugins |
| Plugins with timers/state machines (gr, watchdog) | Format/encoding plugins |
| Plugins that drop or withhold routes (bgp-role, bgp-filter-path-asn) | |
| Plugins that rewrite received routes (bgp-filter-community) |
A plugin that can refuse a route needs metrics even if it holds no state: a
suppression is invisible to the operator otherwise. bgp-role was once listed
as capability-only, but it filters every UPDATE on ingress and egress, so each
of its refusal paths carries a reason-labeled counter.
A route filter is the same case. bgp-filter-path-asn refuses a route whose
AS_PATH carries a listed ASN. A match naming a list that is not defined is
refused at config load, so no fail-closed reject reaches the counter, and its
reason label has the one value listed-asn. A position label says where in
the path the match was, so an operator reads which rule is eating the routes
rather than only how many. The peer is in the log line: a peer address in a label would grow the
series count with the session count.
A filter that rewrites a received route is the same case again: the route is
the only trace of the rewrite. bgp-filter-community counts the values its
own-GA scrub removes (RFC 7454 Section 11), labelled width (standard or
large); the routes its RFC 8195 relation tag is written to; and the BLACKHOLE
routes its RFC 7999 Section 3.2 guard adds a propagation community to, labelled
community with the YANG keyword (no-export or no-advertise). Each is
counted after the rebuild that applies it succeeded, never where the change is
only computed. The registry arrives through the plugin's
Registration.ConfigureMetrics hook (filter_community/register.go), before
the plugin's engine runs.
The route server counts the control-community edits it makes for its clients.
ze_bgp_rs_prepend_applied_total counts the UPDATEs written to a client that
carry the AS_PATH prepend a route's large control community requested for that
client (RFC 7947 Section 2.2.2.1), and
ze_bgp_rs_large_control_community_stripped_total counts the LARGE_COMMUNITY
values in the route server's own namespace removed from them. Both rails count
each UPDATE once it is queued for the client: a client the egress filters
refused, a failed rebuild and a withdrawal count nothing, and a received UPDATE
split into several sections counts once per section, because each section
carries the edit.
Reference Implementation
The bgp-rib plugin (internal/component/bgp/plugins/rib/) is the reference
implementation. It demonstrates all aspects: registration callback, metrics struct,
atomic pointer, periodic update loop, and per-peer label cleanup.