Cumulus Linux (NVUE) — Fault finding

Telemetry-driven troubleshooting and health assertions for Cumulus Linux (NVUE) data center fabrics (VXLAN/EVPN). Part of the multi-vendor Data Center automation guide.

Repository

Cumulus

NVUE

REST API

VXLAN / EVPN

Pull structured state, then assert health

Check Cumulus EVPN peers, VTEPs and interfaces

Flag faults automatically — don't eyeball CLI

Setup, key commands & run examples

evpn_check.sh

EVPN health

Confirm the overlay is up on Cumulus Linux (NVUE): every EVPN/BGP peer should be Established.

evpn_check.sh

bash

nv show vrf default router bgp neighbor
# any neighbor not "Established" / Up is a fault

faultfind.py

Assert peers

Collect BGP/EVPN neighbors from Cumulus and print any peer that is down.

faultfind.py

python

# parse "nv show vrf default router bgp neighbor" structured output
for nbr in get_bgp_neighbors():   # via NVUE REST
    if nbr["state"] != "established":
        print(f"[FAULT] Cumulus EVPN peer {nbr['peer']} {nbr['state']}")

underlay_check.sh

Underlay first

Most overlay faults hide in the underlay — verify routes and interfaces on Cumulus before blaming EVPN.

underlay_check.sh

bash

nv show interface
nv show vrf default router rib ipv4

Best practices

Assert, don't eyeball
Codify the checks — Cumulus EVPN peers up, all VTEPs reachable, no climbing error counters.
Underlay before overlay
Verify OSPF/eBGP and MTU first; VXLAN/EVPN faults usually trace back to the underlay.
Compare against a baseline
Snapshot a healthy fabric state and diff against it so you spot what changed.
Both members of a pair
Check MLAG/ESI consistency on both leaves of a redundant pair, not just one.

Hands-on exercises

Guided practice tasks — copy the starter snippet, run it against a lab device, and extend it to meet the goal.

Exercise 1

Cumulus EVPN fault sweep

Goal: Loop over leaves, collect EVPN neighbors and print any peer that is not up.

exercise_1

python

for dev in leaves:
    # TODO: collect EVPN neighbors and report any down peer

Exercise 2

VTEP reachability

Goal: For each remote VTEP, confirm it is present in the overlay and reachable in the underlay.

exercise_2

python

expected_vteps = {"10.255.0.21", "10.255.0.22", "10.255.0.23"}
# TODO: compare against learned VTEPs and ping in the underlay VRF

Curated videos & guides

Checklist — pitfalls, gotchas & verification

Common pitfalls

•Blaming the overlay when the underlay (routing/MTU) is the real fault.
•Reading Cumulus CLI by eye instead of asserting on structured state.
•Checking only one leaf of an MLAG/ESI pair.

Gotchas to watch

•An empty diff/return is not the same as healthy — assert peers are explicitly up.
•Cumulus counters reset on reload — compare deltas, not absolutes.

Verify it works

•Re-read EVPN neighbors and confirm every peer is Established.
•Confirm all expected VTEPs appear and L2/L3 VNIs are programmed.