Documentation Testing

Quick start

./le doc check verify              # Run all documentation tests
./le doc wiring    # Run changed-file-aware wiring/doc/inventory gate

./le doc check verify runs every documentation checker and returns non-zero if any of them report drift. Run it after editing documentation files, after adding or removing plugins, or as part of review. ./le doc wiring selects the checks needed for the current diff and is included in ./le verify current mode full.

What gets checked

./le doc check verify combines documentation drift, command validation, the two command help gates, and source-anchor validation. ./le doc wiring is the changed-file-aware pre-commit gate.

When to run

The full ./le doc check verify remains the explicit documentation review target. ./le verify current mode full runs ./le doc wiring, which invokes the relevant doc, command, inventory, and wiring checks for changed files.

The change is the unpushed range. The base is the merge base of HEAD and the branch upstream, or of HEAD and origin/main when the branch tracks none, and the range runs from that base to the working tree. The gate is owed before a push, so a change committed without a run is still judged by the next run. The file set, the changed lines and every read of a file's earlier content come from that one base, so a change in the range is never missing from the file set. A file the change deleted is in the file set. A pure rename and a mode-only change are in the file set with no changed line, because the file set lists a rename as two paths and the line diff detects it. Neither hides a claim: a mode-only change touches no symbol, and a claim on a renamed file names the old path, which the change deleted, so that claim is unanswerable and counted. A failed read of a file's earlier content fails the run: it is never read as an empty file. A file named with changed-file <path> that holds no hunk in the range answers every line. The base reads refs only, so the detached worktree that ./le verify worktree makes judges the same range. When no base resolves, the gate exits 2 and names why, and no check runs.

How to interpret output

./le doc yang-contract doc-drift

  Documentation drift detected (N issues)

  x docs/DESIGN.md:708: claims 19 interop scenarios, actual is 32
  x docs/DESIGN.md:0: plugin "bgp-nlri-vpn" registered but missing from Shipped Plugins table
  ...

Each issue points at a file, a line number (0 = file-level), and a description. Most fixes are mechanical: update a count, add a missing table row, remove a stale entry.

./le doc yang-contract command-contract

# Command Validation

YANG commands: 97
Registered handlers: 69

## YANG commands with no handler (30)

  ze-show:bgp-decode  (show > bgp > decode in ze-bgp-tools-cmd)
  ...

## Handlers with no YANG command (0)

Two-direction check. Both directions are contract bugs:

• YANG declares a command but no Go code registered an RPC or local handler -> dead command
• RPC handler registered but YANG doesn't declare it -> command unreachable from CLI

./le doc yang-contract help-shape

# Command Help Shape

Command tree nodes: 611
Nodes that run a command: 395
Nodes with a summary: 611
Nodes with a long help: 396
RPCs: 211
RPCs with a summary: 211
RPCs with a long help: 87
Offline local commands: 19
Offline local commands with a summary: 19
Offline local commands with a long help: 12
Config nodes: 3174
Config nodes with a summary: 3109
Config nodes with a long help: 1459
Long help judged over: declarations this working tree added or changed against HEAD

Nodes with a broken summary: 1
RPCs with a broken summary: 0
Offline local commands with a broken summary: 0
Config nodes with a broken summary: 11

## Broken rules (23)

  missing-description 12
  char-cap          6
  word-cap          5

  command show bgp reject-asn known
    rule:    missing-description
    problem: the declaration carries a summary and no long text beside it
    summary: ...
  schema ze-bgp-conf:bgp/policy/reject-asn/name
    rule:    char-cap
    problem: the summary is 110 characters (a one-line row renders 96)
    summary: ...

The gate walks FOUR corpora. A -cmd.yang node declares the CLI path an operator types. An -api.yang rpc declares the wire method that path reaches, and the plugin IPC modules in internal/core/ipc/yang/ declare 22 more. Every loaded module is walked, so a module whose name carries no -api suffix is judged with the rest. An offline local command declares its help in a registry.Meta beside its handler and reaches no YANG module at all, and ze help command --json merges it with the tree (cmd/ze/help_command.go, collectCommands). A CONFIG node declares its help in a -conf module, and entryShortHelp (internal/component/cli/completer.go) puts it on the row under the completion menu.

The first three corpora are held to the eight summary rules. The config corpus is held to the two caps and to the pair rules, and to nothing else: the summary of a config node is written over as many lines as its author needed, and entryShortHelp collapses the whitespace before it renders, so a newline in one is the normal spelling rather than a defect.

What the config corpus IS

The population is the RESOLVED entry tree of the -conf modules, which is the tree the completer itself walks. Deriving it settles three questions no rule over statement keywords answers correctly.

A false refusal here is expensive rather than noisy. Given a brief with no population rule, three agents shortened exactly the module and revision descriptions and moved the prose into // comments. A YANG description is schema that standard tooling reads and the schema output publishes, and a comment is neither, so all three passes were reverted.

The pair rules

Every one of the three is absolute, and so is missing-short-help. A declaration an operator can reach owes both texts wherever it was written.

missing-description was scoped to what the working tree added or changed against HEAD while 193 declarations in the corpus carried a summary alone. Those are written, so the rule holds over the whole tree and the HEAD baseline that scoped it is gone. There is no file to append a path to, and no scope line in the report: a summary with no long text beside it is refused wherever it sits.

The third corpus is read two ways, because Go forbids importing a main package. The registrations this binary links are read from the registry. The four cmd/ze declares in package main are read from its source, and a registration there whose path is not a literal STOPS the gate rather than being skipped. A path the command tree also holds is left to the command half: the catalog publishes the node's description for such a path and never the registration's. Development commands under le are left out, for the reason the published catalog leaves them out.

The coverage counts say how much of each corpus is written. A node with no summary and a node with no long help are both counted, so an unwritten command is visible rather than silent. Each refusal opens with its surface, command, rpc, local or schema, then the name that finds it. That name is the CLI path for a command node and for a local command, <module>:<rpc-name> for an rpc, and <module>:<node>/<node> for a config node. A summary that breaks two rules is reported twice. The report therefore states the number of nodes, of RPCs and of local commands as well as the number of refusals.

Fix the summary in the YANG ze:help of that node or that rpc, or in the Description of that registration. Prose that does not fit one short sentence belongs in the description beside it, or in the Description beside it, which no one-line surface reads.

The write hook, an edit before the gate

./le ai hooks pretool-writeedit reads the PROPOSED text of any .yang file an agent writes or edits, and warns with exit 1. It loads no YANG model. It reads the file it was handed, which is not on disk yet. It never speaks for the tree: ./le doc yang-contract help-shape is the gate.

The three summary rules hold one population: the statements whose text reaches a one-line operator surface. Those are container, list, leaf, leaf-list, choice, case, action, rpc and notification. The owner is the nearest enclosing statement, so a leaf inside a grouping is judged and the grouping is not.

A module, submodule, revision, import, include, grouping, typedef, identity, feature or extension description is schema documentation that no row renders. The hook passes it whatever its length. Capping one invites the repair that moves the prose into a // comment. Standard YANG tooling cannot read a comment, and the schema output does not publish one.

A leaf and a leaf-list are judged under the pair rules in a config module and under the two caps alone in a -cmd.yang or an -api.yang one, because different producers read them. extractArgDefs walks every child of a command container and calls argDefFor, which copies the leaf's ze:help into ArgDef.ShortHelp and its description into ArgDef.Description, and every command surface prints the one it has. entryShortHelp puts a config leaf's ze:help summary on the completion row. The file name is what separates the two at the level the hook reads.

Each finding names the file, the statement that encloses the text, the rule, and the measured value beside its bound. The text is joined across its concatenated parts and its lines, and its whitespace is collapsed. The author is therefore told what an operator will read, and not how the module was wrapped.

The hook reads one Edit region as readily as one whole file. That is where its two silences are reported rather than hidden. A ze:help the region carries with no statement around it is counted as NOT judged, because nothing says which surface renders it. A string or a comment the scan cannot close is reported as text that does not read as YANG. No text in that file is judged. A scan that stopped early must not read as a file that broke no rule.

How to fix common issues

How the tools find drift

internal/le/doc/yangcontract.Answer imports internal/component/plugin/all so all plugins register themselves, then queries registry.All() and registry.FamilyMap(). It walks the .ci files and reads the native functional suite catalog from internal/le/test/functional. It compares those facts with claims in docs/DESIGN.md, docs/comparison.md, README.md, docs/features.md, and docs/functional-tests.md.

internal/le/doc/yangcontract.Answer imports the same set plus the BGP cmd plugin schema/handler packages, loads the YANG modules, and walks the schema tree looking for ze:command extensions. For each extension it checks registry.CollectRPCHandlers() for a matching method name.

./le doc index check scans every Markdown source anchor under docs/ and refuses a missing source path or symbol. ./le doc index write regenerates the two documentation indexes.

One walk of docs/ feeds the path and symbol checks. An anchor is <!-- source: <path> -- Sym1, Sym2 -->. internal/le/doc/index.CheckCodeIndex keeps an identifier or dotted method chain from the claim and compares it with the declarations in the anchored Go file. The scan is independent of build tags, so a Linux declaration remains visible on a macOS host.

The same walk answers the opposite question. internal/le/doc/index.ClaimsByPath maps a code path to the claims written about it, each with the symbols its anchor names, and the doc-drift check in internal/le/doc/wiring refuses a commit that changed one of those symbols and left its page alone.

The check reads the diff hunks, maps each to the declaration it lands in, and compares those names with the anchor's claim. So an edit elsewhere in the same file reports nothing, and a page counts for itself alone. A test file is out of scope. A claim naming no resolvable symbol blocks nothing and is counted in the verdict line, because silence and "nothing to say" must not read alike. ai/rules/documentation.md states the obligation this enforces: the page edit belongs in the same work as the code edit.

A change that preserves behavior leaves every claim about its symbol as true as it was. A branch swap, an early return and a split guard are examples. A page edit made only to satisfy the check is banned, so the reader who compared the claim with the change records a reviewed claim instead. The ledger is plan/doc-reviewed/<session>.md, one shard for each commit session, named after the eight-hex id ./le commit session prints. Each shard holds one table:

The key is Doc, Source and Symbol, never a line number. Commit is the object id, 7 to 40 lowercase hex digits: a ref such as HEAD moves, so the check refuses it. The check reads every shard, so a claim another session reviewed stays accepted. A flagged claim is covered only when both of these hold:

• Every unpushed commit (between the upstream base and HEAD) that changed a line of Symbol has a row for the claim's key.
• The working tree does not change Symbol after HEAD.

A claim that fails either stays a finding, and the finding names each commit with no row (447c5c24da changed Documented and has no reviewed row) or the working-tree edit. Each of these is a finding of its own, with the shard and the line: a shard with no table, a row with the wrong cell count or an empty cell, a Commit that is not an object id or that the repository does not hold, a key that matches no claim anchor, and an unpushed commit that did not change the row's Symbol. A row naming a pushed commit covers nothing. A merge commit that changed Symbol against its first parent needs a row of its own. When an unpushed commit renamed or copied a file into Source, no row covers any claim on that Source, because an edit made before the rename is not at Source and no row can name it. That claim needs a page edit. The ledger is judged on every run, including one with no changed Go. When rows cover claims, the verdict line counts them.

A reviewed claim is only for a change that preserves behavior. A change to the behavior a claim states MUST edit the page, in the same work as the code (ai/rules/documentation.md). A row that accepts a false claim hides exactly the defect this check exists to find.

internal/le/doc/check.Answer walks the instruction corpus for paths, // Design: targets and hook names. Its last two checks do not use that corpus. sweep_tracked reads every file git ls-files names, one time, and check_ignore_reasons and check_tracked_citations share that one walk. Check 4 reports a doc-links: ignore marker that states no reason, because a marker outside the walked corpus suppresses nothing and nobody audits it. Check 5 reports a path reference that does not resolve, in any tracked file. Two moves remove a finding: repair the reference, or mark its line with a marker that states why the path cannot resolve.

The references that predate check 5 are grandfathered in internal/le/doc/check/testdata/doc_citation_baseline.txt, one citing file<TAB>dead target pair per line. It records pairs rather than bare targets, so a new file that cites an already-dead target is reported. The check compares the file with its version at HEAD and refuses every pair that HEAD does not hold, so the baseline only shrinks.

Repair a citation and delete its pair from the baseline in the same change. ./le doc check links validates the result against the working tree and warns when a baseline pair is no longer needed. The command does not rewrite the shared baseline, so one session cannot absorb another session's unfinished edits.

The three roots outside the tracked citation scan are vendor/, third_party/, and plan/handover/: the first two hold another repository's files, while the last records an earlier tree.

internal/le/doc/consistency parses // Design:, // Detail:, // Overview:, and // Related: comments, checks their symmetry, and reports references to packages that no longer exist.

Adding a new documentation check

1. Add callable Go behavior to the owning package under internal/le/.
2. Add one action to that package's table and register its area through leroot.Register. A related check joins an existing area rather than opening another root name.
3. Register the action with internal/le/le/root and expose it as ./le <area> <action>.
4. Add the callable action to internal/le/doc/wiring when changed files should trigger it during pre-commit verification.
5. Add its producer and exact command to this page and ai/INDEX.md.
6. If agent rules need the command, edit the canonical rule point and render the rule corpus through ./le ai rules render-update.

Repository tooling is compiled Go under internal/le; package-owned fixtures belong in that package's testdata/ directory.

See also

• ai/rules/writing.md -- canonical documentation rules, including the BLOCKING Documentation Update Checklist for specs
• ai/rules/repo-maintenance.md -- required discovery updates when new checks, tools, or verification gates are added
• ai/rules/repo-maintenance.md -- which native gates enforce which rules
• internal/le/doc/check and internal/le/doc/wiring -- documentation checks, command validation, and changed-file wiring