Notification Service

An internal router and bookkeeper: services hand it a push, email or SMS, it decides when and whether to send, and the providers do the last mile. Watch a one-time code stay fast while a million-user campaign drains beside it.

  • Daily volume: 10M/day · ~100/s sustained
  • Campaign: 1M users in 5 min · >3,000/s
  • Ceiling: ~5,000/s surge across channels
  • OTP target: <= 5 s after acceptance

Upstream services

Engineering services that need to reach a user: auth sends OTPs and security alerts, marketing schedules campaigns, and the mobile apps register device tokens. Every send names a priority.

API gateway

Authenticates the internal API key, applies per-service rate limits, and sheds standard traffic before high-priority traffic when the system is saturated.

Notification service

Validates the send, looks up the user's addresses in its user data slice, prechecks preferences, and publishes ONE durable queue message first, then returns the id.

Postgres

The source of truth for status: one row per notification (one user, one channel), campaign definitions, SCHEDULED rows, preferences, and a synced slice of user contacts and device tokens.

User service sync

Keeps the service's slice of user email and phone in sync with the user service, so a send never makes a synchronous call to another team.

Scheduler cron

Every minute it publishes SCHEDULED rows that are due and starts campaign fanouts. It never sits on the OTP path; immediate sends do not wait for it.

Segments

A named set of users. The fanout reads a snapshot of membership at fanout start so the audience is fixed for this campaign.

SQS high queue

A separate durable queue for high-priority sends only, so an OTP never waits behind a million promos.

SQS standard queue

Holds standard sends and campaign fanout. Bursts are expected here; minutes of delay are acceptable.

High workers

An independent pool that consumes only the high queue: claim the dedup key, recheck preferences, call the provider within the high budget, record status, delete the message.

Standard workers

Consumes the standard queue within the standard provider budget, applying quiet hours and opt-outs at dispatch time.

Dedup store

Holds per-key leased claims and ack records so redelivery, POST retries and cron re-fanout rarely become a second send. Best effort, not exactly-once.

Preferences

Checked just before the provider call: channel opt-outs always apply; quiet hours apply to standard only; high priority bypasses quiet hours but not channel opt-outs. Blocked sends are SUPPRESSED with a reason.

Provider budget

Splits the shared ~5,000/s provider ceiling into a high reserve and a standard share, so campaigns cannot consume the capacity OTPs need.

SMS provider

Delivers SMS over carriers. Twilio p99 is normally under 1 s; in a brownout it can reach 30 s.

Email provider

Delivers email; returns bounces and complaints asynchronously.

Push: APNs / FCM

Delivers push to device tokens registered by the apps; invalid-token responses retire the token.