Skip to content
Dan Ochoa.All notes

Cronradar · architecture

Who watches the watcher?

Building a cron monitor that has to survive its own outage — and why I deleted half the infrastructure to get there.

Cronradar

The problem

Cronradar exists to tell you the moment a scheduled job silently fails. That inverts the usual reliability question: the product is only useful when something else is down, and it must never be the thing that goes quiet without anyone noticing. You can't alert on an error you received — there is no error. You alert on a ping that never arrived, which means the whole system is built around detecting absence on a deadline.

Every monitor carries an expected next-ping time and a grace period. A background sweep looks for monitors whose deadline has passed, moves them Warning then Critical, and escalates. A second sweep handles the monitor that was created but never pinged at all — a job that was supposed to start reporting and never did. Absence has two shapes, and both have to be caught on a clock.

Auto-discovery, and the drift problem

The five-line integration doesn't just send pings — the framework SDKs (Hangfire, Quartz) reconcile the jobs your app actually declares into monitors. You don't hand-register anything; the schedule in your code is the source of truth.

That creates a subtler failure than a missed ping: a job that gets deleted from the schedule entirely. It will never ping again, so a naive monitor would alert forever. The drift sweep tracks when each job was last declared, and auto-pauses monitors whose jobs have vanished from the schedule — deliberately behind an off-by-default flag, because silently pausing a monitor is exactly the kind of helpful behavior that can hide a real outage if you're wrong about intent.

Why I deleted Redis, Typesense, and the bundled database

The backend is .NET (Aspire-orchestrated) over Postgres, with Hangfire running the schedule math and the sweeps. It didn't start that lean. Early on it also ran Redis, Typesense, nginx, and a bundled Postgres. In one pass I tore all of that out and routed everything through Traefik against a single shared, hosted Postgres.

The reasoning is the whole thesis of the product: every additional moving part is another thing that can fail on the one service whose entire job is to still be working when other things fail. A cache I have to keep coherent, a search box's index I have to keep in sync, a database I have to operate myself — each is a new failure mode I'd be introducing into a reliability tool. Postgres already does durable, transactional state well; a cron monitor's state is small and relational. So the answer to "why not a queue / a cache / a search cluster" is that I ran them, and then chose to have fewer things that can break.

Watching the watcher

Ping ingestion runs as its own service, separate from the dashboard. That separation is the point: the plane that receives your jobs' heartbeats doesn't share a fate with the plane that renders charts or processes billing, so a dashboard problem can't cause you to miss a ping. The alerting, ping-ingest, and dashboard-mutation paths were hardened for durable execution so an in-flight alert survives a restart rather than being lost.

cron.life is the heartbeat endpoint jobs ping on success; crontab.space is the plain-English schedule builder — companions that also let Cronradar's own liveness be exercised the same way a customer's job is.

What the numbers actually say

Measured at the edge (Traefik access logs) over roughly three weeks: median HTTP response 14 ms, p95 around 97 ms, p99 around 658 ms across the API, heartbeat-ingest and dashboard endpoints. That is endpoint response latency measured at the proxy — how fast Cronradar answers a request.

It is deliberately not the number a cron monitor's marketing wants to quote, because it is not alert-delivery time. An alert is a background job that never traverses the proxy; measuring "how fast we notice and notify" means timing ping-received to notification-sent, which is a different measurement I'd report separately rather than let a fast response time stand in for it.

14 msp50median response
97 msp95
658 msp99

What I'd do differently

The honest gap today is test coverage: the SDKs are tested, but the server's sweep logic — the exact code that decides whether your job is late — leans on manual confidence more than I'd like for the component that matters most. For a product whose promise is "we will notice," the miss-detection path is precisely where I'd want property-based tests around clock edges, grace-period boundaries, and daylight-saving transitions before I'd call it done.

All notes