Temporal

Self-hosted orchestrator for notification delivery: retries, quiet hours, escalation, cancel-on-read. Admin Workflows are product definitions (escalation ladder). Temporal workflow types are the two functions below.

1. How it works
One durable run per subscriber + product workflow + idempotency key.

Ingest

Producer
API key
ingestion-api
POST /v1/trigger
Redis
trigger-buffer
bridge
signalWithStart

Temporal · namespace outreach

NotificationWorkflow
queue: orchestration · id: notif-<workflowId>-<subscriberId>-<idempotencyKey>
Resolve ladder
DB definition or code registry
Wait → gate
quiet hours / delay · evaluateChannelGate

Channel activity (isolated worker, 5 retries)

email
inapp
telegram
push
webhook
Success
writeAudit → notification_log
Fail after retries
child DeadLetterWorkflow

Signals in

triggerstart / payloadpref-changedre-read prefsreadcancel ladder

Terminal

Delivery log
sent / skipped / read
DeadLetterWorkflow
queue: dead-letter · status dead_lettered
Text outline
Producer
  → POST /v1/trigger  (Bearer producer key)
  → Redis BullMQ trigger-buffer
  → ingestion-bridge
  → signalWithStart NotificationWorkflow
       taskQueue: orchestration
       signal: trigger { payload }
       Temporal id: notif-<workflowId>-<subscriberId>-<idempotencyKey>

NotificationWorkflow (fresh run)
  → resolve escalation definition (DB or code registry)
  → sequential ladder: wait (quiet hours / delay) → gate → channel Activity
  → writeAuditActivity (notification_log)
  → on channel failure after 5 retries: child DeadLetterWorkflow
       taskQueue: dead-letter
       id: dlq-…:{channel}
2. Workflow types that exist
Only these two are registered on worker-core. Channel workers have Activities, not workflow code.
TypeQueueWhat it does
NotificationWorkfloworchestrationDelivery spine: preference gate, timers, five channels (email, inapp, telegram, push, webhook), signals, audit.
DeadLetterWorkflowdead-letterAbandoned child after Activity retries exhaust. Only writer of dead_lettered on the delivery log.

Visibility list query: WorkflowType = "NotificationWorkflow" AND ExecutionStatus = "Running"

3. Workers & task queues
Isolated processes. Retry on every Activity: 5 attempts, exponential backoff.
QueueProcessRole
orchestrationtemporal-worker-coreRuns NotificationWorkflow; prefs / audit / gate / wait
dead-lettertemporal-worker-core (same process)Runs DeadLetterWorkflow; writes status=dead_lettered
emailtemporal-worker-emailemailSendActivity (stub or DashaMail)
inapptemporal-worker-inappinappSendActivity → Centrifugo publish
telegramtemporal-worker-telegramtelegramSendActivity
pushtemporal-worker-pushpushSendActivity (FCM)
webhooktemporal-worker-webhookwebhookSendActivity
4. Signals
Named Temporal signals on NotificationWorkflow.
SignalWhenEffect
triggersignalWithStart from ingestion-bridge after POST /v1/triggerCarries payload; starts or attaches to NotificationWorkflow
pref-changedPUT preferences (best-effort list of Running executions)Re-read prefs before the next ladder send
readIn-app mark-read / producer readCancel pending escalation; audit status=read
5. How to connect
UI is HTTP. SDK is gRPC to the frontend service. Namespace must be outreach.
UI   http://localhost:8088/namespaces/outreach/workflows
     compose maps host 8088 → temporal-ui 8080
     namespace: outreach

gRPC localhost:7233
     workers + bridge + createTemporalClient()
Open Temporal UI
import { createTemporalClient, TASK_QUEUES } from "@finamx/outreach-temporal-shared";

const client = await createTemporalClient();
// TEMPORAL_ADDRESS + TEMPORAL_NAMESPACE (default: localhost:7233 / outreach)

const handle = client.workflow.getHandle("<temporal workflow id>");
const desc = await handle.describe();

In-app channel after a successful ladder step: Centrifugo. Delivery outcomes: Delivery log.

6. Env
Same values for workers, ingestion-bridge, and local SDK scripts.
TEMPORAL_ADDRESS=localhost:7233          # compose DNS: temporal:7233
TEMPORAL_NAMESPACE=outreach
TEMPORAL_UI_PORT=8088

# DEFAULT_NAMESPACE=outreach is set on the temporal auto-setup container