Operations

SSH Server

Ze exposes its CLI over SSH. ze cli, ze cli -c, and ze signal commands connect to this server. Most ze show subcommands do too, except local ones (version, bgp decode/encode, env, schema, yang, completion) which run in-process.

Setup

ze init                        # interactive: prompts for username, password, host, port

Defaults: 127.0.0.1:2222, ED25519 host key auto-generated.

Credentials are stored in the ze database (database.zefs) with bcrypt-hashed passwords.

Reinitializing

Running ze init a second time fails with error: database already exists. This is intentional -- it prevents accidental credential loss.

To reinitialize:

ze signal stop                 # daemon must be stopped first
ze init --force                # prompts for confirmation interactively

--force moves the old database to database.zefs.replaced-<date> as a backup before creating a new one. The backup contains your previous SSH credentials and any stored configs. Non-interactive use (piped stdin) is rejected for safety -- --force requires interactive confirmation.

Connection

ze cli                         # interactive CLI
ze cli -c "show bgp peer list"  # single command
ze show bgp peer list                # read-only shorthand
ze cli -c "request peer transit teardown 2" # one-shot command

Override Host/Port

# Environment variables
export ZE_SSH_HOST=10.0.0.1
export ZE_SSH_PORT=2222

# Per-command flags
ze signal reload --host 10.0.0.1 --port 2222

Signals

Via SSH (preferred)

Via Unix Signals

Reload writes the edited config as a candidate version. The active pointer moves only after verification, apply, and subsystem reload succeed. A failed reload clears the candidate and keeps the previous active config.

Exit Codes (signal command)

Health Checks

Liveness

ze status                      # exit 0 = running, exit 1 = not running

This dials the SSH port without completing a handshake. Suitable for systemd watchdog or load balancer TCP check.

Scripting

if ze status --host 127.0.0.1 --port 2222; then
    echo "ze is running"
else
    echo "ze is down"
fi

Peer Health

ze cli -c "show bgp peer list"  # brief peer list with state
ze cli -c "show bgp summary"   # summary table with uptime and prefix counts

Environment Variables

Ze environment variables use dot notation (ze.log.bgp) and are case-insensitive. All forms are equivalent: ze.log.bgp, ZE_LOG_BGP, ze_log_bgp.

List all registered variables:

ze env list                    # all vars with types and defaults
ze env list -v                 # include current values
ze env get ze.log              # details for one var

Key Variables

CLI Flags

systemd

Use ze install systemd on standard Linux hosts:

sudo ze init
sudo ze install systemd --start

This writes /etc/systemd/system/ze.service, creates the ze user/group if missing, assigns ownership of the config directory and database.zefs, reloads systemd, enables the service, and starts it when --start is present.

Inspect the generated unit without writing anything:

ze install systemd --dry-run --config /etc/ze

Run doctor after installation to verify the service account and binary path:

ze doctor

When /etc/systemd/system/ze.service exists, doctor checks that the unit's ExecStart binary exists and is executable, and that the configured User and Group exist on the host.

The service runs with XDG_RUNTIME_DIR=/run/ze, so the daemon socket is /run/ze/ze.socket. Configure the same socket in ze config or export XDG_RUNTIME_DIR=/run/ze before running local operator CLI commands.

Remove only the service unit:

sudo ze uninstall systemd

Generated unit shape:

[Unit]
Description=Ze Network OS
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=ze
Group=ze
ExecStart=/usr/local/bin/ze start
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
WorkingDirectory=/etc/ze
Environment=ZE_CONFIG_DIR=/etc/ze
Environment=XDG_RUNTIME_DIR=/run/ze
AmbientCapabilities=CAP_NET_ADMIN CAP_NET_RAW CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_RAW CAP_NET_BIND_SERVICE
NoNewPrivileges=true
ProtectHome=true
RuntimeDirectory=ze

[Install]
WantedBy=multi-user.target

Troubleshooting

Peer Won't Connect

Symptom: peer stays in Connect/Active state

1. Check connectivity: Can you reach the peer IP on port 179? bash nc -zv 10.0.0.2 179

2. Check config: Validate before debugging network issues bash ze config validate config.conf

3. Enable debug logging: bash ze cli -c "request log level bgp.fsm debug" ze cli -c "bgp monitor event state"

4. Common causes: - Firewall blocking TCP 179 - Peer not configured for your address - AS number mismatch (check local { as } and remote { as }) - Hold time rejected (must be 0 or >= 3)

Peer Drops After OPEN

Symptom: session establishes briefly then NOTIFICATION received

1. Check the NOTIFICATION code in monitor output:

2. Role mismatch: If using role { import customer; strict true; }, both peers must advertise compatible roles.

3. Capability issues: Enable debug to see negotiated vs offered capabilities: bash ze cli -c "request log level bgp.reactor debug"

Daemon Won't Start

1. Config parse error: Run ze config validate config.conf first
2. Port in use: Check if another ze instance or BGP daemon holds port 179
3. SSH port conflict: Default SSH is 2222. Check with netstat -tlnp | grep 2222
4. Missing credentials: Run ze init before starting

Plugin Not Working

1. Check plugin is loaded: bash ze --plugins # list compiled-in plugins

2. Check plugin is bound to peer: Config must have process <name> { receive [...] } on the peer

3. Check plugin logs: Set ZE_LOG_PLUGIN_RELAY=debug to see plugin stderr output

4. Plugin not reaching Ready state: Enable ZE_LOG_PLUGIN=debug and look for startup stage failures

No Routes in RIB

1. Check RIB plugin is loaded and bound: bash ze cli -c "show bgp rib status"

2. Check peer is sending updates: bash ze cli -c "bgp monitor peer transit-a event update direction received"

3. Check RPKI validation: If bgp-rpki is loaded, routes may be pending validation. Check: bash ze cli -c "show bgp rpki status"

Crash Capture

Ze automatically captures panic stack traces and forwards them to syslog. On restart after a crash, show crashes displays saved crash reports with ring buffer context (the last 64 log entries before the panic).

Configuration (environment variables):

Crash dir autodetection probes in order: /perm/ze/crash/ (gokrazy), <config-dir>/crash/, /var/lib/ze/crash/, /tmp/ze-crash/. The resolved path is printed at startup.

Syslog forwarding uses ze.log.destination (no separate config). Crash messages use LOG_CRIT facility for filtering.

CLI (online, daemon running):

show crashes           # list crash files with timestamps
show crashes latest    # display most recent crash report

CLI (works with or without a daemon; falls back to in-process read when the daemon is down):

ze show crashes          # list crash files (JSON)
ze show crashes latest   # display most recent crash report

Collecting Debug Information

For bug reports, collect:

ze --version                   # ze version
ze config validate config.conf        # config validity
ze status                      # daemon state
ze cli -c "show bgp peer list"  # peer states
ze cli -c "show bgp summary"   # session summary
ze env list -v                 # effective configuration
ze signal quit                 # goroutine dump (kills daemon!)