onotify: A scalable, flexible Alertmanager

While Prometheus has quickly become the defacto standard for collecting, storing, and alerting on time series metrics. However, adopting Prometheus locks you into the Prometheus ecosystem – exporters, middleware boxes, and critically, Alertmanager. Despite this tight coupling, Alertmanager has traditionally avoided implementing the business logic necessary to function as a fully featured Alerting platform, instead delegating that responsibility to downstream alerting sinks such as Pagerduty or Slack. In this talk, I will highlight these limitations and contextualize them within Prometheus’s historical development. I will then introduce onotify, a drop in serverless replacement for Alertmanager built on top of Cloudflare workers. onotify addresses many of Alertmanager’s business logic challenges, including multi-tenancy, rate limiting, scaling, and Alert meta-observability, while scaling almost infinitely and providing all the tools that platform teams need to manage their alerting system as they scale to thousands, or even tens of thousands of alerts.