ADR-0011 — Discovered and overlay files

Status: proposed

Marrow separates network discovery into two layers: a machine-owned base file (<name>.discovered.yaml) regenerated from hardware configs, and a human-owned overlay file (<name>.yaml) that extends it using extends:.

WhyCopy Anchor Link

Network diagrams decay under two distinct operational pressures:

1. Manual diagrams rot: Engineers fail to update visual layouts following maintenance windows.
2. Automated diagrams lose business context: Re-running automated discovery overwrites human annotations (tier, zone, operational notes, and maintenance histories).

Import adapters (ADR-0010) exacerbate this loss: re-running discovery would overwrite every manual edit previously made to the topology.

DecisionCopy Anchor Link

1. marrow import writes exclusively to <name>.discovered.yaml. This file is machine-owned and regenerated wholesale without manual preservation.
2. The schema supports an optional root key, extends: <path>, allowing human-authored files to reference and layer over discovered files.
3. Merge rules evaluate by id for devices and zones, and by endpoint pairs plus lag for links:
   • Overlays may append new devices, zones, and links.
   • Overlays may override any discovered property (e.g., custom tier or zone).
   • Discovered elements are hidden explicitly using x-hide: true.
   • Overlay definitions that reference non-existent base IDs raise validation errors.
4. extends operates at a single depth: discovered files cannot extend other files.
5. validate --json surfaces all active overrides for auditing.

ConsequencesCopy Anchor Link

• Positive: CI/CD pipelines can regenerate discovered topologies from config backups (RANCID, Oxidized) without destroying human documentation.
• Positive: Human-owned overlay files remain concise, containing only organizational context.
• Negative: Schema validation and error reporting must track source provenance across multiple physical files.
• Negative: Re-cabled ports change endpoint identities, requiring corresponding updates in overlay link references.

Considered and rejectedCopy Anchor Link

• Three-way in-place file merges. Rejected: merge conflicts corrupt YAML syntax and require manual conflict resolution.
• In-place round-tripping. Rejected: preserving comments, custom formatting, and deletions inside automated output introduces brittle state management.