Monitoring

Live Peer Dashboard

ze cli -c "monitor bgp"

The dashboard refreshes every 2 seconds and shows router identity plus a sortable, colour-coded peer table with update rates. Keys: j/k moves, s/S sorts, Enter opens detail, and Esc exits.

Event Streaming

ze cli -c "monitor event"

Filters

Combine filters:

ze cli -c "monitor event peer upstream1 include update direction received"

Event Types

Output Formats

Pipe the output through format operators:

ze cli -c "monitor event | json"      # Full JSON envelope
ze cli -c "monitor event | table"     # Tabular format
ze cli -c "monitor event | match rx"  # Regex filter on output

JSON Event Format

All events follow the ze-bgp JSON envelope:

{
  "type": "bgp",
  "bgp": {
    "peer": {
      "address": "10.0.0.1",
      "local": {"address": "10.0.0.2", "as": 65000},
      "remote": {"address": "10.0.0.1", "as": 65001}
    },
    "message": {
      "id": 42,
      "direction": "received",
      "type": "update"
    }
  }
}

UPDATE Event

{
  "type": "bgp",
  "bgp": {
    "peer": {"address": "10.0.0.1", "local": {"address": "10.0.0.2", "as": 65000}, "remote": {"address": "10.0.0.1", "as": 65001}},
    "message": {"id": 1, "direction": "received", "type": "update"},
    "update": {
      "ipv4/unicast": [
        {
          "next-hop": "10.0.0.1",
          "action": "add",
          "nlri": ["10.0.0.0/24", "10.0.1.0/24"]
        }
      ]
    },
    "origin": "igp",
    "as-path": [65001, 65002],
    "local-preference": 100
  }
}

State Event

{
  "type": "bgp",
  "bgp": {
    "peer": {"address": "10.0.0.1", "local": {"address": "10.0.0.2", "as": 65000}, "remote": {"address": "10.0.0.1", "as": 65001}},
    "message": {"type": "state"},
    "state": "up"
  }
}

Programmatic Access

Plugins can subscribe to events via the SDK:

attach process my-plugin {
    receive [ update state ]
}

The plugin receives events through its OnEvent callback. See Plugins guide for details.

Prometheus Metrics

Ze exposes Prometheus metrics when telemetry { prometheus { ... } } is configured. BGP metrics are refreshed every 10 seconds. By default the HTTP listener binds to 127.0.0.1:9273; configure an explicit server address to expose it to remote scrapers.

The netdata block only controls Netdata-compatible OS collector metrics. It does not rename Ze-native metrics such as ze_bgp_*, ze_bfd_*, or ze_l2tp_*.

telemetry {
    prometheus {
        enabled true;
        server main {
            ip 0.0.0.0;
            port 9273;
        }
        path /metrics;
        basic-auth {
            enabled true;
            username prometheus;
            plaintext-password "secret";
        }
        netdata {
            enabled true;
            prefix netdata;
            interval 1;
            collector diskspace {
                enabled false;
            }
            collector snmp6 {
                interval 10;
            }
        }
    }
}

Deprecated compatibility aliases remain accepted: prefix, interval, and collector directly under prometheus. Prefer netdata/prefix, netdata/interval, and netdata/collector in new config.

HTTP Basic Authentication

When basic-auth/enabled is true, Ze requires HTTP Basic Authentication for every handler on the Prometheus service, including both /metrics and /health. The password is stored as a bcrypt hash in the persisted config. Use plaintext-password when you edit the config and Ze replaces it with password at commit. If automation already has a hash from ze passwd, set password directly.

A config file loaded at daemon start or at SIGHUP gets the same transform. Ze hashes the leaf in memory and leaves your file as you wrote it. It warns that the plaintext is still on disk.

Prometheus scrape configuration:

scrape_configs:
  - job_name: ze
    static_configs:
      - targets: ["router.example.net:9273"]
    basic_auth:
      username: prometheus
      password: secret

Basic Auth does not provide transport encryption. Keep the listener on loopback, use a trusted management network, or put TLS in front of the service if the scrape crosses an untrusted network.

Per-collector overrides:

netdata {
    collector diskspace { enabled false; }
    collector snmp6 { interval 10; }
}

OS Metrics (Netdata-compatible)

Ze exports 138 OS metrics matching Netdata's Prometheus format exactly (same names, labels, values), acting as a drop-in replacement for Netdata's /api/v1/allmetrics?format=prometheus endpoint. Existing Grafana dashboards built against Netdata continue to work unchanged.

Metric name format: {prefix}_{context}_{units}_average{chart="...",dimension="...",family="..."}, where {prefix} is telemetry.prometheus.netdata.prefix.

Collectors whose data sources are absent (no ZFS loaded, no btrfs mounts, no wireless NICs, etc.) skip silently.

Side-by-side validation against Netdata:

curl -s http://localhost:9273/api/v1/allmetrics?format=prometheus | grep "^netdata_" | sed 's/ .*//' | sort -u > nd.txt
curl -s http://localhost:9274/metrics | grep "^netdata_" | sed 's/ .*//' | sort -u > ze.txt
diff nd.txt ze.txt

Host Inventory Metrics

Host metrics are refreshed on a configurable interval (default 60 seconds). Linux only; on other platforms no host metrics are registered.

BGP Metrics

Instance

Per-Peer

Session Lifecycle

A non-zero ze_bgp_open_in_established_total names a peer that tried to re-negotiate mid-session. Ze answers it with a Cease and closes the connection.

ze_bgp_connect_retry_counter is RFC 4271 Section 8.1.1 mandatory session attribute 2, "the number of times a BGP peer has tried to establish a peer session". The BGP FSM raises it by one on each teardown RFC 4271 Section 8.2.2 gives an increment clause: a hold-timer expiry, a header or OPEN error, a NOTIFICATION, an UPDATE error, a mid-session TCP failure, and any event the state does not expect. Two events set it back to zero, and only these two: the operator starts the peer, and the operator stops it. A reconnect does not, which is what makes the value a history rather than a flag.

It is a GAUGE, not a counter. Those operator resets make the value go down, and a Prometheus counter that goes down reads as a counter reset to rate(). Use the value directly, not a rate: it already IS a count.

Read the same number per peer with show bgp peer <address> detail, field connect-retry-counter.

Startup and Connection Timing

Forward Pool / Congestion

Attribute Index

Ze indexes the path attributes of every received UPDATE once, on the receive goroutine, and holds the first 8 spans inline. A 9th attribute puts the remainder on the heap and increments this counter. A public-internet corpus of 112M routes has a maximum of 10 attributes and 99.9% at 8 or fewer, so a steady rate here means either an unusual peer or an attribute set worth raising the inline size for.

Egress Modification Failures

A non-zero value means a configured modification (a next-hop rewrite, a community strip, a private-ASN removal) did not fit the route it applied to. The reason label set is closed: malformed, overflow, attr-length-range, withdrawn-size. Two further values exist and both indicate a defect rather than peer input: no-failure must never be emitted, and unclassified means a reason reached the counter that no constant produced.

Treat any increment as a policy that did not take effect. Ze counts the failure at the point the modification is built, on all five paths that build one: the forward rail, the route-server rail, the ingress and egress policy chains, and the RFC 9494 stale re-advertise rail.

Well-Known Community Suppressions

An increment means a route ze received carrying NO_EXPORT, NO_ADVERTISE or NO_EXPORT_SUBCONFED was not advertised to a peer that community forbids. One observation is one destination, so a route withheld from 20 peers counts 20. The suppression is not configurable and is never logged per route, so this counter is the only place an operator sees it.

The community label set is closed and holds three values: no-advertise, no-export-subconfed, no-export. A route carrying more than one is counted under the strictest, so one suppressed route is always one observation.

The counter says nothing about withdrawals. RFC 1997 forbids ADVERTISING such a route, so a suppressed destination still receives the withdrawals of the same UPDATE and keeps no route ze cannot take back.

Announces Refused For Size

An increment means a route never reached one peer. Ze queries the encoded size before it writes, so no truncated UPDATE goes out: the announce is abandoned whole. Nothing else reports this to the operator, because the peer is not notified and the session is not disturbed.

Both label sets are closed. rail is batch for an announce built from the API and queued for one built from the RIB. stage is nlri when the prefix block did not fit and attributes when the path-attribute block did not.

A batch increment also reaches the caller: AnnounceNLRIBatch returns errAnnounceTooLarge, which the dispatcher turns into a StatusError response, so a plugin sees its own announce refused. A queued increment has no such channel and the counter is the only signal besides the log line. Act on either by sending fewer prefixes per announce, or by reducing the attributes on the route.

Sends Refused By The Attach Permission

A peer feeds and is fed by the processes its attach process <name> blocks name, and the send list inside a block is the permission to originate toward that peer. An increment means a program addressed a peer it was never attached to: the peer was dropped from that command and nothing went on its wire. type is update for a route, a withdrawal or an End-of-RIB marker, and refresh for a route refresh or a soft clear.

A steady non-zero rate with no config change is usually a program that widened its selector to peer * and is now reaching for peers the operator never linked it to. A step at reload is usually the opposite: an attach block was removed while the program kept announcing. The WARN line beside each increment names the peer, which this counter deliberately does not (an error path should not carry peer-driven cardinality), and it prints the exact block to add.

An operator at the CLI, over SSH or through the REST API never appears here: their authority is checked by AAA, and send grants authority to a program.

Prefix Limits (RFC 4486)

Single Command

For scripting, use -c to execute a single command and exit:

ze cli -c "show bgp summary"
ze cli -c "show bgp rib received"
ze cli -c "show bgp rpki status"