NAPT
Data Center guide
Management
Fault finding
Deployment
Back to Learn

Cisco NX-OS / ACI — Fault finding

Telemetry-driven troubleshooting and health assertions for Cisco NX-OS / ACI data center fabrics (VXLAN/EVPN). Part of the multi-vendor Data Center automation guide.

Repository
Cisco
NX-OS
ACI / APIC
VXLAN / EVPN
Pull structured state, then assert health
Check Cisco EVPN peers, VTEPs and interfaces
Flag faults automatically — don't eyeball CLI

Setup, key commands & run examples

evpn_check.sh
EVPN health

Confirm the overlay is up on Cisco NX-OS / ACI: every EVPN/BGP peer should be Established.

evpn_check.sh
bash
show bgp l2vpn evpn summary
# any neighbor not "Established" / Up is a fault
faultfind.py
Assert peers

Collect BGP/EVPN neighbors from Cisco and print any peer that is down.

faultfind.py
python
bgp = dev.get_bgp_neighbors()
for vrf, data in bgp.items():
    for ip, peer in data["peers"].items():
        if not peer["is_up"]:
            print(f"[FAULT] Cisco EVPN peer {ip} DOWN in {vrf}")
underlay_check.sh
Underlay first

Most overlay faults hide in the underlay — verify routes and interfaces on Cisco before blaming EVPN.

underlay_check.sh
bash
show interface status
show ip route vrf all

Best practices

  • Assert, don't eyeball

    Codify the checks — Cisco EVPN peers up, all VTEPs reachable, no climbing error counters.

  • Underlay before overlay

    Verify OSPF/eBGP and MTU first; VXLAN/EVPN faults usually trace back to the underlay.

  • Compare against a baseline

    Snapshot a healthy fabric state and diff against it so you spot what changed.

  • Both members of a pair

    Check MLAG/ESI consistency on both leaves of a redundant pair, not just one.

Hands-on exercises

Guided practice tasks — copy the starter snippet, run it against a lab device, and extend it to meet the goal.

Exercise 1
Cisco EVPN fault sweep

Goal: Loop over leaves, collect EVPN neighbors and print any peer that is not up.

exercise_1
python
for dev in leaves:
    # TODO: collect EVPN neighbors and report any down peer
Exercise 2
VTEP reachability

Goal: For each remote VTEP, confirm it is present in the overlay and reachable in the underlay.

exercise_2
python
expected_vteps = {"10.255.0.21", "10.255.0.22", "10.255.0.23"}
# TODO: compare against learned VTEPs and ping in the underlay VRF

Curated videos & guides

Checklist — pitfalls, gotchas & verification

Common pitfalls

  • •Blaming the overlay when the underlay (routing/MTU) is the real fault.
  • •Reading Cisco CLI by eye instead of asserting on structured state.
  • •Checking only one leaf of an MLAG/ESI pair.

Gotchas to watch

  • •An empty diff/return is not the same as healthy — assert peers are explicitly up.
  • •ACI faults live in the APIC fault MO tree, not in leaf CLI.

Verify it works

  • •Re-read EVPN neighbors and confirm every peer is Established.
  • •Confirm all expected VTEPs appear and L2/L3 VNIs are programmed.