Chaos engineering

From 'what if' to 'let's actually try it'. Inject controlled faults in the Lab to empirically prove resilience โ€” not just calculate it.

Chaos experiments run against:
Safe default

Production chaos testing requires ALL of:

  • Explicit environment override (not the default)
  • 2 approvers
  • Active maintenance window
  • Auto-abort safety net configured and tested

Experiment library

Link Failure Injection

Shuts down an interface, observes failover, and measures convergence time. Auto-restores after the timer or on successful convergence, whichever is first.

Auto-restore after 60s or on convergenceConfigure

BGP Session Kill

Sends a TCP RST to a BGP session and observes re-establishment behavior. Auto-restores if the session does not recover in time.

Auto-restore if no recovery within 120sConfigure

Latency / Packet Loss Injection

Introduces artificial latency or loss via tc/netem to test QoS and application resilience. Auto-removes after the experiment duration.

Auto-remove after 5 min (default)Configure
Lab-only

Multi-Failure ("Game Day")

Combines two or more simultaneous failures. Requires explicit confirmation and is Lab-only by default.

Explicit confirmation required ยท Lab-onlyConfigure