Architecture

notifkit is eight services talking through Redis Streams, with PostgreSQL as the system of record. Every service is stateless and scales horizontally; all coordination happens in Redis. This page is the contract underneath.

The streams

Priority is separate physical streams, not a score, so a low-priority backlog can't sit in front of a critical message.

Each consumer reads through a named group: notifkit:group:enricher, notifkit:group:engine, notifkit:group:delivery, and so on, with consumer set to {service}-{pid}. Each new process joins the group and Redis divides the work automatically.

Sorted sets and locks

Delivery guarantees

Delivery is at-least-once. A message is acknowledged only after process() succeeds, so if a worker dies mid-flight, the message stays in the group's pending list for another consumer to claim.

The cost is the duplicate case. If a provider accepts a send but its response never arrives (a timeout on the way back), notifkit records a failure and retries, and the user gets it twice. No internal deduplication can prevent this; it's inherent to calling a remote API without two-phase commit.

Ordering isn't guaranteed either. Messages are processed concurrently, up to WORKER_CONCURRENCY per worker, so two sends to one user may arrive in either order. If order matters, use a workflow, which is serialised per instance.

Where deduplication happens

Marks are provisional: a stage that fails after claiming a key releases it, so a retry isn't swallowed as a duplicate.

Retries and the dead-letter queue

There's no exponential backoff. Recovery uses Redis pending-entry reclamation on a fixed interval, so timing is predictable and the mechanism crash-safe:

Failure classes

Backpressure

Streams are trimmed to QUEUE_MAX_LEN (10 million entries by default) as they're written. It's a safety valve against unbounded memory growth, not flow control: past the cap, the oldest entries are discarded.

Real backpressure comes from consumers. Workers read a bounded batch and process at most WORKER_CONCURRENCY messages at once, so when a provider slows, Delivery pulls fewer messages, its pending list grows, and upstream stages build depth instead of overrunning anything. Alert on stream depth.

Deployment topologies

Single process

new NotifkitServer({ services: ["all"] });

Everything in one process: right for development, and fine in production until one service's load starts affecting another's.

Split by role

Only the services array differs between deployments. Point them at the same Redis and Postgres and they form one system.

// Deployment 1 — behind your load balancer, scaled on request rate.
new NotifkitServer({
  services: ["api"],
  redisUrl: process.env.REDIS_URL,
  databaseUrl: process.env.DATABASE_URL,
  nodeEnv: "production",
});

// Deployment 2 — the pipeline, scaled on stream depth.
new NotifkitServer({
  services: ["enricher", "engine", "scheduler", "events"],
  redisUrl: process.env.REDIS_URL,
  databaseUrl: process.env.DATABASE_URL,
  nodeEnv: "production",
  autoMigrate: false,           // exactly one deployment should own migrations
  workerConcurrency: 25,
});

// Deployment 3 — the part that talks to providers, scaled on their limits.
new NotifkitServer({
  services: ["delivery"],
  redisUrl: process.env.REDIS_URL,
  databaseUrl: process.env.DATABASE_URL,
  nodeEnv: "production",
  autoMigrate: false,
  providers: [new ResendTransport({ apiKey: RESEND_KEY, from: "[email protected]" })],
});

What scales on what

The data model

Migrations run on startup unless you pass autoMigrate: false. With several deployments on one database, let exactly one own migrations.

These tables live in public by default. When notifkit shares a database with another app, set DB_SCHEMA to give them their own schema; see Installing into a custom PostgreSQL schema.

Multi-tenancy

Every query is scoped by projectId, derived from the API key, not the request body, so changing a field can't reach another tenant's data. Keys are hashed with SHA-256 and cached for 60 seconds; revoking one publishes an invalidation so every API process drops it at once rather than waiting out the TTL.

The admin key is the exception: it can act for any project, but must name one with an x-project-id header or projectId query parameter. Requests without it are rejected, not defaulted.