Skip to main content

Failover & circuit breakers

Providers fail. The point of Nervly is that your code does not have to care.

Provider failover order​

ChannelTried firstTried second
SMSTermiiInfobip, then Sendchamp
Email (default)ResendZeptoMail
Email (postmark primary)Postmark for CRITICAL/HIGH, SendGrid for NORMAL/LOWthe other email provider
Email (cloudflare primary)Cloudflare EmailResend, then ZeptoMail
VoiceTermiiInfobip

With the default email configuration the order is Resend → ZeptoMail for every priority; the Postmark/SendGrid split applies only when the email primary provider is postmark. A cloudflare primary runs the three-leg chain Cloudflare → Resend → ZeptoMail for every priority. See Channels & providers.

If the first provider fails, the second is tried immediately, within the same dispatch. You see one event and one delivery receipt, from whichever provider succeeded.

If every provider on a channel fails, the worker moves to the next eligible channel rather than giving up — a NORMAL event that could not be emailed is attempted by SMS. Only when every provider on every eligible channel fails is the event recorded as failed and eventually dead-lettered. The channel the gateway queued is therefore not necessarily the channel that carried the message; read channel from the delivery receipt for what actually happened.

One breaker per provider​

Each provider has its own circuit breaker, so an outage is contained:

  • Termii down → the Termii breaker opens → SMS continues via Infobip
  • Postmark down (when Postmark is the email primary provider) → the Postmark breaker opens → email continues via SendGrid, including for CRITICAL mail
  • Cloudflare Email down (when Cloudflare is the email primary provider) → the Cloudflare breaker opens → email continues via Resend, then ZeptoMail
  • Termii down does not affect email at all

When a breaker opens​

A breaker tracks the ratio of failed to total attempts. It opens when:

  • more than 10 attempts have been recorded, and
  • the failure ratio has reached the configured threshold (CB_THRESHOLD_RATIO, default 0.2 — one failure in five)

The ten-attempt floor is what stops a single early failure from pulling a healthy provider out of rotation.

When it closes again​

An open breaker holds its provider out for a cool-off period (CB_COOLDOWN_SECONDS, default 30 seconds). The next request after that is allowed through and the counters reset: a success closes the breaker, a failure re-opens it for another cool-off.

While a breaker is open, calls to that provider are refused without a network round trip, so a dead provider costs you nothing but the failover hop.

A rejection counts as a failure​

The breaker treats two things as failures:

  • a transport error — timeout, DNS, connection refused
  • a successful HTTP call that the provider rejected

Termii, for instance, signals failure by omitting message_id from a 200 response. That counts. A provider that cheerfully returns 200 while dropping every message will still trip its breaker.

What you should do​

Very little. Failover and breaking are entirely server-side.

What is worth doing:

  • Watch delivery receipts, not just accept codes. A 202 tells you nothing about provider health. See Delivery receipts.
  • Do not implement your own provider retry. Retrying a 202-ed event creates a duplicate; Nervly has already tried both providers.
  • Do retry on 5xx from the gateway itself, with the same Idempotency-Key — that is a Nervly-side failure before the event was durable.

Next​