MRT Analysis
Ze includes ./le mrt, a development command for analysing real-world BGP data
from public route collectors (RIPE RIS, RouteViews). It processes MRT dump files
to extract statistics that inform ze's internal buffer sizing, caching strategies,
and congestion handling.
Running
./le mrt <subcommand> [options] runs one subcommand. A bare ./le mrt lists
every subcommand with its description, as data, so ./le mrt '|' json works.
le builds itself, so no separate binary is needed.
Quick Start
Download BGP data from public collectors and run an analysis:
./le mrt download # fetch latest data
./le mrt density test/internet/ripe-updates.*.gz # UPDATE density + burst patterns
./le mrt attributes test/internet/latest-bview.gz # attribute repetition analysis
Data Sources
./le mrt download fetches MRT files from two public BGP collectors:
| Source | Type | Interval | Size |
|---|---|---|---|
| RIPE RIS rrc00 (Amsterdam) | BGP4MP updates | 5 min | ~5 MB per file |
| RIPE RIS rrc00 | TABLE_DUMP_V2 RIB | Latest | ~400 MB |
| RouteViews route-views2 | BGP4MP updates | 15 min | ~2 MB per file |
| RouteViews route-views2 | TABLE_DUMP_V2 RIB | 2-hour intervals | ~100 MB |
Files are saved to test/internet/ (gitignored). RouteViews bz2 files are
converted to gzip on download for Go stdlib compatibility.
./le mrt download # today's data at 00:00 UTC
./le mrt download 20260324 # specific date
./le mrt download 20260324 1200 # specific date and time
./le mrt download -o /tmp/mrt # custom output directory
Commands
density
Measures how many NLRIs each UPDATE carries and how many UPDATEs arrive per second. Separates traffic into setup (table dumps, convergence) and maintenance (steady-state churn) using per-source-peer burst detection.
./le mrt density test/internet/ripe-updates.*.gz
Output sections:
- NLRIs per UPDATE distribution (announced, withdrawn, total)
- UPDATEs per active second distribution
- Setup vs maintenance classification per source peer
- Per-peer maintenance rate distribution
- Channel sizing recommendation with empirical P50/P95/P99
Used for: per-peer forward pool channel sizing. Results documented in Update Density Analysis.
attributes
Analyses attribute repetition across routes to guide caching decisions. Measures per-attribute cache hit rates, bundle deduplication effectiveness, and temporal locality (consecutive identical bundles).
./le mrt attributes test/internet/latest-bview.gz 2>/dev/null | jq . # JSON
./le mrt attributes test/internet/latest-bview.gz >/dev/null # summary
Output: JSON to stdout, human summary to stderr.
Used for: attribute pool sizing, cache strategy decisions. Results documented in mrt-attribute-caching.md.
communities
Identifies per-ASN community defaults: communities that appear in nearly every route from a given ASN. These defaults can be assumed present in a cache, encoding only exceptions (absent defaults) to save wire bytes.
./le mrt communities test/internet/latest-bview.gz
./le mrt communities --threshold 0.90 --format json test/internet/latest-bview.gz
./le mrt communities --post-policy test/internet/latest-bview.gz
Options:
--threshold(default 0.95): minimum frequency to be considered a default--min-routes(default 1000): minimum routes from an ASN to generate defaults--format(yaml or json)--post-policy: strip action communities (simulates route server post-policy view)
count-attrs
Counts how many path attributes each route carries. Produces a distribution table showing the typical attribute set size.
./le mrt count-attrs test/internet/latest-bview.gz
mrt-dump
Dumps MRT records as BGP UPDATE hex, one per line. Useful for piping into
ze bgp decode or other tools.
./le mrt mrt-dump test/internet/ripe-updates.*.gz | head -5
./le mrt mrt-dump test/internet/latest-bview.gz | ./ze bgp decode -
show
Human-readable MRT dump (like bgpdump). Displays record headers, peer info, and decoded BGP message contents including attributes, AS paths, and prefixes.
./le mrt show test/internet/ripe-updates.*.gz | head -50
./le mrt show test/internet/latest-bview.gz
routes
Extracts a prefix table from TABLE_DUMP_V2 files as JSON. Each entry includes prefix, next-hop, AS path, origin, local-pref, MED, and communities.
./le mrt routes test/internet/latest-bview.gz | jq '.[] | select(.prefix == "1.0.0.0/24")'
inject
Opens a BGP session to a remote peer and sends routes from an MRT file. Supports both TABLE_DUMP_V2 (RIB entries) and BGP4MP (UPDATE messages).
./le mrt inject --local-as 65000 test/internet/latest-bview.gz 10.0.0.1:179
replay
Replays BGP4MP messages over a BGP session preserving original inter-message timing. Configurable speed multiplier.
./le mrt replay --local-as 65000 --speed 10 test/internet/ripe-updates.*.gz 10.0.0.1:179
convert
Converts MRT records to other formats.
./le mrt convert pcap test/internet/ripe-updates.*.gz output.pcap # BGP4MP to pcap (IPv4 only)
./le mrt convert json test/internet/ripe-updates.*.gz | jq . # record headers as JSON
export
Send MRT data to network targets.
./le mrt export bmp --target 10.0.0.1:4321 test/internet/ripe-updates.*.gz
./le mrt export bmp --target collector:4321 --peer-ip 10.0.0.1 test/internet/ripe-updates.*.gz
Connects to a BMP collector and sends each BGP4MP message as a BMP Route
Monitoring message. Optional --peer-ip filters to a single peer.
record
Record incoming protocol streams to MRT files.
./le mrt record bmp --listen :4321 output.mrt
Listens for incoming BMP (RFC 7854) connections. Received Route Monitoring messages are written as BGP4MP_MESSAGE_AS4 MRT records. Peer Up/Down notifications are written as BGP4MP_STATE_CHANGE_AS4. Multiple concurrent BMP connections are supported (writes are serialized).
serve
Passive BGP server that sends MRT file contents to any peer that connects. Useful for IXP traffic replay testing: blast an entire routing table at a router and observe its behavior.
./le mrt serve --local-as 65000 --listen :1179 test/internet/latest-bview.gz
./le mrt serve --local-as 65000 --per-peer test/internet/ripe-updates.*.gz
With --per-peer, only records matching the connecting peer's ASN are sent.
Multiple MRT files can be specified; all are sent sequentially.
statistics
Per-type/subtype counts, AFI breakdown, peer summary, timestamp range, and BGP message type distribution.
./le mrt statistics test/internet/ripe-updates.*.gz
filter
Select records by peer IP, peer ASN, prefix, AS-path regex, community regex, MRT type, or timestamp range. Writes matching records verbatim (no re-encoding) to a new MRT file. Multiple filters are AND-composed.
./le mrt filter --peer-asn 13335 test/internet/latest-bview.gz cloudflare.mrt
./le mrt filter --prefix 1.0.0.0/24 --after 1780272000 test/internet/ripe-updates.*.gz filtered.mrt
./le mrt filter --as-path "174 .* 13335" test/internet/latest-bview.gz transit.mrt
./le mrt filter --community "13335:" test/internet/latest-bview.gz tagged.mrt
AS-path regex matches against space-separated ASNs (e.g. "174 1916 52888").
AS_SET segments are rendered as {asn,asn}. Community regex matches per-community
strings: standard high:low, large global:local1:local2, extended high:low.
Damaged Input
A dump from a public collector can carry a truncated or corrupt record. The tool reports the damage instead of printing a short result as fact.
| What you get | Where |
|---|---|
warning: N malformed MRT record(s) skipped or partially decoded; results are incomplete |
stderr, from density, attributes, aspath, communities, count-attrs, mrt-dump, routes and show. A report that scrolls prints it before the numbers it qualifies. A clean file prints nothing. |
HH:MM:SS <peer> [unparseable: truncated] <error> |
show, for a record that decoded to nothing. The tag is truncated, unsupported-afi, or damaged. |
A=3+ or W=12+ |
show, for a count that is partial. The + says "at least this many"; a count with no + is exact. A damaged record prints the field even at zero, so a missing count and a zero count stay distinct. |
mrt: truncated record 41902 (type 13 subtype 2, timestamp 1780272000): unexpected EOF |
stderr, when the file itself stops early. The record ordinal is 1-based over the stream, so the failing record can be found in a multi-gigabyte dump. |
Damage never discards what already decoded. A record with 500 good prefixes and one truncated prefix reports the 500 and counts one damaged record.
BGP message parsing requires exactly one complete frame. Trailing bytes beyond the declared BGP length and truncated attribute headers, lengths or values produce errors even when the file contains both directional OPENs. Semantic commands report these errors, and BGP4MP inject, replay and serve refuse the damaged UPDATE before writing it to the peer.
AS Width and Add-Path
Two properties of an MRT record decide how its BGP payload reads, and neither can be inferred from the payload bytes:
- AS width comes from the record type (RFC 6396). TABLE_DUMP is 2-byte (Section 4.2), TABLE_DUMP_V2 is 4-byte (Section 4.3.4), BGP4MP_MESSAGE is 2-byte (Section 4.4.2), and BGP4MP_MESSAGE_AS4 is 4-byte (Section 4.4.3). A 2-byte and a 4-byte AS_PATH can occupy the same number of octets, so a wrong width yields fictitious ASNs rather than an error. Every subcommand derives the width from the record it is reading.
- BGP4MP Add-path modes come from both captured OPENs. Negotiation is per family and direction. Ordinary subtypes and unambiguous single-family ADD-PATH messages can also be decoded without OPENs, but mixed-family ADD-PATH messages cannot. An empty MP field still counts as family evidence because it can select the recorder's subtype. Missing context produces an error instead of guessed prefixes or Path Identifiers. TABLE_DUMP_V2 subtypes 8 to 11 carry explicit identifiers in each RIB entry; subtype 12 carries them in its raw NLRI.
Inject, replay and serve do not negotiate ADD-PATH. They refuse Path-ID-bearing or ambiguous UPDATEs; ordinary UPDATEs and standalone empty EOR messages remain supported. An updates-only capture or a rotated file opened without its OPEN exchange cannot recover missing negotiation from another file.
MRT File Formats
The tool handles two MRT record types (RFC 6396):
| Type | Records | Used By |
|---|---|---|
| TABLE_DUMP_V2 | RIB snapshots (one entry per route per peer) | attributes, communities, count-attrs, mrt-dump, show, routes, inject, filter |
| BGP4MP | Live UPDATE messages with timestamps | density, attributes, communities, mrt-dump, show, inject, replay, convert, filter |
Both .gz and .bz2 compressed files are supported. HTTP and HTTPS URLs are
accepted anywhere a file path is expected; compression is auto-detected from
the URL suffix.
./le mrt statistics https://data.ris.ripe.net/rrc00/2026.06/updates.20260607.0000.gz
Daemon MRT Recording
Ze produces MRT dumps from live BGP sessions via the mrt component. Configure
under mrt {} in the YANG config:
Three independent streams (following FRR's model):
- Updates -- BGP4MP records for UPDATE messages only
- All -- BGP4MP records for all BGP messages + state changes
- Routes -- periodic TABLE_DUMP_V2 RIB snapshots
Features: per-peer filtering, direction filtering (received/sent), extended
timestamps (BGP4MP_ET), add-path aware, on-demand CLI dump via
request mrt dump-rib, strftime filename rotation, async non-blocking writes.
Related
- MRT Architecture -- package structure and design
- MRT Implementation Comparison -- feature comparison across BGP implementations
- Update Density Analysis -- empirical findings that inform forward pool channel sizing
- Forward Congestion Pool -- the design that consumes these measurements
- Congestion Industry Survey -- how other BGP implementations handle similar problems