#sre
2 posts
GCP europe-west2-b Outage: Why the Generators Never Started
A PLC read voltage on a disconnected feed and never started the generators, so a Google Cloud London zone went dark. Check your zonal blast radius.
SLO Burn-Rate Alerts: Stop Paging 54 Minutes After Recovery
A 1h burn-rate alert kept firing 54 minutes after errors stopped. Add a 5m short window in Prometheus, prove it with promtool, and page only while budget burns.