Documentation Testing

Quick start

./le doc check verify              # Run all documentation tests
./le doc wiring    # Run changed-file-aware wiring/doc/inventory gate

./le doc check verify runs every documentation checker and returns non-zero if any of them report drift. Run it after editing documentation files, after adding or removing plugins, or as part of review. ./le doc wiring selects the checks needed for the current diff and is included in ./le verify current mode full.

What gets checked

./le doc check verify combines documentation drift, command validation, the two command help gates, and source-anchor validation. ./le doc wiring is the changed-file-aware pre-commit gate.

When to run

The full ./le doc check verify remains the explicit documentation review target. ./le verify current mode full runs ./le doc wiring, which invokes the relevant doc, command, inventory, and wiring checks for changed files.

How to interpret output

./le docvalid doc-drift

  Documentation drift detected (N issues)

  x docs/DESIGN.md:708: claims 19 interop scenarios, actual is 32
  x docs/DESIGN.md:0: plugin "bgp-nlri-vpn" registered but missing from Shipped Plugins table
  ...

Each issue points at a file, a line number (0 = file-level), and a description. Most fixes are mechanical: update a count, add a missing table row, remove a stale entry.

./le docvalid command-contract

# Command Validation

YANG commands: 97
Registered handlers: 69

## YANG commands with no handler (30)

  ze-show:bgp-decode  (show > bgp > decode in ze-bgp-tools-cmd)
  ...

## Handlers with no YANG command (0)

Two-direction check. Both directions are contract bugs:

• YANG declares a command but no Go code registered an RPC or local handler -> dead command
• RPC handler registered but YANG doesn't declare it -> command unreachable from CLI

./le docvalid help-shape

# Command Help Shape

Command tree nodes: 611
Nodes that run a command: 395
Nodes with a summary: 611
Nodes with a long help: 396
RPCs: 211
RPCs with a summary: 211
RPCs with a long help: 87
Offline local commands: 19
Offline local commands with a summary: 19
Offline local commands with a long help: 12
Config nodes: 3174
Config nodes with a summary: 3109
Config nodes with a long help: 1459
Long help judged over: declarations this working tree added or changed against HEAD

Nodes with a broken summary: 1
RPCs with a broken summary: 0
Offline local commands with a broken summary: 0
Config nodes with a broken summary: 11

## Broken rules (23)

  missing-long-help 12
  char-cap          6
  word-cap          5

  command show bgp reject-asn known
    rule:    missing-long-help
    problem: the declaration carries a summary and no long text beside it
    summary: ...
  schema ze-bgp-conf:bgp/policy/reject-asn/name
    rule:    char-cap
    problem: the summary is 110 characters (a one-line row renders 96)
    summary: ...

The gate walks FOUR corpora. A -cmd.yang node declares the CLI path an operator types. An -api.yang rpc declares the wire method that path reaches, and the plugin IPC modules in internal/core/ipc/yang/ declare 22 more. Every loaded module is walked, so a module whose name carries no -api suffix is judged with the rest. An offline local command declares its help in a registry.Meta beside its handler and reaches no YANG module at all, and ze help command --json merges it with the tree (cmd/ze/help_command.go, collectCommands). A CONFIG node declares its help in a -conf module, and entryDescription (internal/component/cli/completer.go) puts it on the row under the completion menu.

The first three corpora are held to the eight summary rules. The config corpus is held to the two caps and to the pair rules, and to nothing else: a YANG description is written over as many lines as its author needed, and entryDescription collapses the whitespace before it renders, so a newline in one is the normal spelling rather than a defect.

What the config corpus IS

The population is the RESOLVED entry tree of the -conf modules, which is the tree the completer itself walks. Deriving it settles three questions no rule over statement keywords answers correctly.

A false refusal here is expensive rather than noisy. Given a brief with no population rule, three agents shortened exactly the module and revision descriptions and moved the prose into // comments. A YANG description is schema that standard tooling reads and the schema output publishes, and a comment is neither, so all three passes were reverted.

The pair rules

Every one of the three is absolute, and so is missing-summary. A declaration an operator can reach owes both texts wherever it was written.

missing-long-help was scoped to what the working tree added or changed against HEAD while 193 declarations in the corpus carried a summary alone. Those are written, so the rule holds over the whole tree and the HEAD baseline that scoped it is gone. There is no file to append a path to, and no scope line in the report: a summary with no long text beside it is refused wherever it sits.

The third corpus is read two ways, because Go forbids importing a main package. The registrations this binary links are read from the registry. The four cmd/ze declares in package main are read from its source, and a registration there whose path is not a literal STOPS the gate rather than being skipped. A path the command tree also holds is left to the command half: the catalog publishes the node's description for such a path and never the registration's. Development commands under le are left out, for the reason the published catalog leaves them out.

The coverage counts say how much of each corpus is written. A node with no summary and a node with no long help are both counted, so an unwritten command is visible rather than silent. Each refusal opens with its surface, command, rpc, local or schema, then the name that finds it. That name is the CLI path for a command node and for a local command, <module>:<rpc-name> for an rpc, and <module>:<node>/<node> for a config node. A summary that breaks two rules is reported twice. The report therefore states the number of nodes, of RPCs and of local commands as well as the number of refusals.

Fix the summary in the YANG description of that node or that rpc, or in the Description of that registration. Prose that does not fit one short sentence belongs in the ze:help beside it, or in the LongHelp beside it, which no one-line surface reads.

The write hook, an edit before the gate

./le hook-check pretool-writeedit reads the PROPOSED text of any .yang file an agent writes or edits, and warns with exit 1. It loads no YANG model. It reads the file it was handed, which is not on disk yet. It never speaks for the tree: ./le docvalid help-shape is the gate.

The three summary rules hold one population: the statements whose text reaches a one-line operator surface. Those are container, list, leaf, leaf-list, choice, case, action, rpc and notification. The owner is the nearest enclosing statement, so a leaf inside a grouping is judged and the grouping is not.

A module, submodule, revision, import, include, grouping, typedef, identity, feature or extension description is schema documentation that no row renders. The hook passes it whatever its length. Capping one invites the repair that moves the prose into a // comment. Standard YANG tooling cannot read a comment, and the schema output does not publish one.

A leaf and a leaf-list are judged in a config module and passed in a -cmd.yang or an -api.yang one, because two producers read them. extractArgDefs walks every child of a command container and calls argDefFor, which admits any entry that carries a type. Both statements carry one, so each becomes a command.ArgDef built from the type and the mandatory flag. ArgDef declares nine fields and none of them holds text, so the description is dropped at the tree boundary. entryDescription puts a config leaf's description on the completion row. The file name is what separates the two at the level the hook reads.

Each finding names the file, the statement that encloses the text, the rule, and the measured value beside its bound. The text is joined across its concatenated parts and its lines, and its whitespace is collapsed. The author is therefore told what an operator will read, and not how the module was wrapped.

The hook reads one Edit region as readily as one whole file. That is where its two silences are reported rather than hidden. A description the region carries with no statement around it is counted as NOT judged, because nothing says which surface renders it. A string or a comment the scan cannot close is reported as text that does not read as YANG. No description in that file is judged. A scan that stopped early must not read as a file that broke no rule.

How to fix common issues

How the tools find drift

internal/le/docvalid.Answer imports internal/component/plugin/all so all plugins register themselves, then queries registry.All() and registry.FamilyMap(). It walks the .ci files and reads the native functional suite catalog from internal/le/functional. It compares those facts with claims in docs/DESIGN.md, docs/comparison.md, README.md, docs/features.md, and docs/functional-tests.md.

internal/le/docvalid.Answer imports the same set plus the BGP cmd plugin schema/handler packages, loads the YANG modules, and walks the schema tree looking for ze:command extensions. For each extension it checks registry.CollectRPCHandlers() for a matching method name.

./le docs-to-code check scans every Markdown source anchor under docs/ and refuses a missing source path or symbol. ./le docs-to-code update regenerates the two documentation indexes.

One walk of docs/ feeds the path and symbol checks. An anchor is <!-- source: <path> -- Sym1, Sym2 -->. internal/le/docstocode.CheckCodeIndex keeps an identifier or dotted method chain from the claim and compares it with the declarations in the anchored Go file. The scan is independent of build tags, so a Linux declaration remains visible on a macOS host.

The same walk answers the opposite question. internal/le/docstocode.ClaimsByPath maps a code path to the claims written about it, each with the symbols its anchor names, and the doc-drift check in internal/le/doc/wiring refuses a commit that changed one of those symbols and left its page alone.

The check reads the diff hunks, maps each to the declaration it lands in, and compares those names with the anchor's claim. So an edit elsewhere in the same file reports nothing, and a page counts for itself alone. A test file is out of scope. A claim naming no resolvable symbol blocks nothing and is counted in the verdict line, because silence and "nothing to say" must not read alike. ai/rules/documentation.md states the obligation this enforces: the page edit belongs in the same work as the code edit.

internal/le/doc/check.Answer walks the instruction corpus for paths, // Design: targets and hook names. Its last two checks do not use that corpus. sweep_tracked reads every file git ls-files names, one time, and check_ignore_reasons and check_tracked_citations share that one walk. Check 4 reports a doc-links: ignore marker that states no reason, because a marker outside the walked corpus suppresses nothing and nobody audits it. Check 5 reports a path reference that does not resolve, in any tracked file. Two moves remove a finding: repair the reference, or mark its line with a marker that states why the path cannot resolve.

The references that predate check 5 are grandfathered in internal/le/doc/check/testdata/doc_citation_baseline.txt, one citing file<TAB>dead target pair per line. It records pairs rather than bare targets, so a new file that cites an already-dead target is reported. The check compares the file with its version at HEAD and refuses every pair that HEAD does not hold, so the baseline only shrinks.

Repair a citation and delete its pair from the baseline in the same change. ./le doc check links validates the result against the working tree and warns when a baseline pair is no longer needed. The command does not rewrite the shared baseline, so one session cannot absorb another session's unfinished edits.

The three roots outside the tracked citation scan are vendor/, third_party/, and plan/handover/: the first two hold another repository's files, while the last records an earlier tree.

internal/le/consistency parses // Design:, // Detail:, // Overview:, and // Related: comments, checks their symmetry, and reports references to packages that no longer exist.

Adding a new documentation check

1. Add callable Go behavior to the owning package under internal/le/.
2. Add one action to that package's table and register its area through leroot.Register. A related check joins an existing area rather than opening another root name.
3. Register the action with internal/le/leroot and expose it as ./le <area> <action>.
4. Add the callable action to internal/le/doc/wiring when changed files should trigger it during pre-commit verification.
5. Add its producer and exact command to this page and ai/INDEX.md.
6. If agent rules need the command, edit the canonical rule point and render the rule corpus through ./le rules render-update.

Repository tooling is compiled Go under internal/le; package-owned fixtures belong in that package's testdata/ directory.

See also

• ai/rules/writing.md -- canonical documentation rules, including the BLOCKING Documentation Update Checklist for specs
• ai/rules/repo-maintenance.md -- required discovery updates when new checks, tools, or verification gates are added
• ai/rules/repo-maintenance.md -- which native gates enforce which rules
• internal/le/doc/check and internal/le/doc/wiring -- documentation checks, command validation, and changed-file wiring