Orion — Event-Driven Notification System
Transactional email notifications for the Orion field service portal, triggered by GCP Pub/Sub push subscriptions from the Orion BFF — zero polling, publish and forget.
01Purpose & Scope
Orion currently sends no outbound notifications. Job assignments go unannounced, invoice creation is silent, and technicians learn of new work only by checking the portal manually. This document specifies an event-driven notification system for the Orion field service portal whose primary channel is transactional email, built without any polling — every notification is triggered by a GCP Pub/Sub event pushed from the Orion BFF to a dedicated microservice.
notification-service FastAPI microservice; three Pub/Sub topics published exclusively by the Orion BFF (platform.jobs, platform.invoices, platform.auth); push subscription wiring; email delivery via Resend; per-user notification preferences; delivery tracking (sent, bounced, opened).02Current State
Orion has no outbound notification capability. The table below documents the current situation for each domain within the portal.
| Domain | Current behaviour | Gap |
|---|---|---|
| Job assignment | No outbound notification; technician must check the portal | No job-assigned or job-status-changed email sent |
| Job completion / cancellation | Status updates visible in portal; no notification to partner contact | Partner unaware of field status changes without manual refresh |
| Invoice creation | Invoice PDF visible in portal; no email to partner billing contact | Invoice created / sent / paid events go unnotified |
| Auth events (Orion users) | Password reset handled ad-hoc via a direct SMTP call; no invite email for new technicians | No consistent template or delivery tracking; technician onboarding requires manual comms |
| GCP Pub/Sub | Scaffolded in Orion; local EventLog driver active in production; Pub/Sub not wired | Event bus exists but is unused — this design activates it |
03Event Architecture
All Orion notification triggers flow through GCP Pub/Sub using push subscriptions. The Orion BFF publishes a lightweight JSON envelope and returns immediately — no knowledge of notification logic, no polling, no callbacks. Pub/Sub delivers the message to the notification service endpoint over HTTPS.
3.1 — Topic taxonomy
Three topics are in scope for this document — all published exclusively by the Orion BFF. Topics are created once in the metiss-dev GCP project and can gain additional subscribers later without changes to Orion or the notification service.
| Topic | Published by | Event types carried |
|---|---|---|
platform.jobs | Orion BFF | job.assigned, job.status_changed, job.completed, job.cancelled |
platform.invoices | Orion BFF | invoice.created, invoice.sent, invoice.paid, invoice.overdue |
platform.auth | Orion BFF | auth.invite_sent, auth.password_reset, auth.account_locked |
3.2 — Event envelope schema
All events share the same envelope. Source-specific data goes in payload. The notification service acts on the envelope fields; it never assumes payload structure beyond what is documented in the event catalog (Section 07).
{
"event_id": "evt_01J3KQZ...", // ULID — used as idempotency key
"event_type": "job.assigned", // dot-namespaced string
"source": "orion-bff", // publishing service slug
"created_at": "2026-07-10T14:23:00Z",
"payload": {
"job_id": "uuid-...", // resource ID — no PII in envelope
"actor_id": "uuid-...", // user who triggered the event
"recipient_ids": ["uuid-..."] // user IDs who should be notified
}
}users table at delivery time. This keeps events safe to log, audit, and replay without exposing personal data in the message bus.3.3 — Why push, not pull
Pub/Sub supports two subscription modes. This design uses push exclusively.
| Mode | Behaviour | Why we avoid it |
|---|---|---|
| Pull (polling) | Subscriber calls subscriber.pull() on a loop or schedule to fetch messages | Polling defeats the event-driven goal. A sleeping loop introduces latency proportional to the poll interval; a tight loop wastes CPU and creates cost. |
| Push this design | Pub/Sub calls the subscriber's HTTP endpoint the moment a message is available | Zero latency gap; no background thread; Cloud Run scales to zero between bursts with no idle polling cost |
04Notification Service
A new notification-service FastAPI microservice is introduced. It has a single responsibility: receive Pub/Sub push messages, decide who to notify and how, render the appropriate email template, and dispatch via Resend. It shares the existing Cloud SQL instance and deploys to Cloud Run alongside the other Metiss microservices.
4.1 — API surface
| Method | Path | Purpose | Auth |
|---|---|---|---|
| POST | /events/push | Pub/Sub push subscription target — receives all platform events | GCP OIDC bearer token (verified against Google public keys) |
| POST | /events/webhook | Resend delivery webhooks — updates notification_log with delivery status | Resend webhook signing secret (HMAC-SHA256) |
| GET | /preferences/{user_id} | Fetch a user's notification preference record | Internal service token (not exposed via API Gateway) |
| PUT | /preferences/{user_id} | Update preferences — called by Orion settings UI via the Orion BFF | Internal service token |
| GET | /unsubscribe | One-click unsubscribe link target — verifies HMAC token and opts user out | HMAC-signed query params (no user session required) |
| GET | /health | Cloud Run health check | None |
4.2 — Request lifecycle (push endpoint)
Authorization: Bearer <oidc-token>header signed by Google. The service validates the token against Google's public key endpoint and confirms the audience matches the Cloud Run service URL. Rejects with HTTP 401 if invalid.notification_log for a row where idempotency_key = event_id. If found, return HTTP 200 immediately — the notification was already sent; this is a Pub/Sub redelivery.recipient_ids to email addresses via the users table. For each recipient, check user_notification_preference; skip users who have opted out of this event type.event_type. Fetch any additional payload data needed for rendering (e.g. job details from Orion). Render subject + HTML body + plain-text fallback. Call the Resend API to send; capture the returned email_id.notification_log with status sent and the Resend email_id. Return HTTP 200 to acknowledge the Pub/Sub message. Pub/Sub will not redeliver.notification_log row is written, so the idempotency check on the next delivery attempt will not short-circuit.05Email Delivery
5.1 — Email provider: Resend
Resend is selected as the transactional email provider for the following reasons:
| Criterion | Resend | Alternative (SendGrid) |
|---|---|---|
| Python SDK | Official SDK; async-native | Official SDK; older API surface |
| Webhook events | Svix-signed webhooks; delivery, bounce, open, click | Event webhook with SendGrid signing key |
| Custom domain sending | DNS setup; DKIM + SPF auto-managed | Same — manual DNS steps |
| Pricing | Free up to 3,000 emails/month; $20/month thereafter | Free up to 100/day; complex pricing tiers |
| Sending domain | Requires custom domain — use notifications@metiss.ai | Same |
5.2 — Template system
Email templates are Jinja2 files stored in notification-service/templates/. Each event type has two template files: an HTML file and a plain-text fallback. Template names follow the pattern {event_type}.html and {event_type}.txt (dots replaced by underscores in filenames).
templates/
job_assigned.html job_assigned.txt
job_status_changed.html job_status_changed.txt
job_completed.html job_completed.txt
job_cancelled.html job_cancelled.txt
invoice_created.html invoice_created.txt
invoice_sent.html invoice_sent.txt
invoice_paid.html invoice_paid.txt
auth_invite_sent.html auth_invite_sent.txt
auth_password_reset.html auth_password_reset.txt
auth_account_locked.html auth_account_locked.txt5.3 — Unsubscribe
Every notification email includes an RFC 8058 List-Unsubscribeheader and a visible “Unsubscribe” link in the footer. The link target is:
GET /unsubscribe
?user_id=<uuid>
&event_type=<type>
&nonce=<ulid>
&sig=<hmac-sha256-hex>The HMAC is computed server-side at send time using a secret key. Verification on click confirms the link was issued by the service and has not been tampered with. No user session is required — a single click opts the user out without a login wall.
event_typeonly, not all notifications. A “manage all preferences” deep-link in the email footer directs users to the Vista Portal settings page for full control.06Data Model
Three tables are added to the shared Cloud SQL instance (same database as Orion). No new database or Cloud SQL instance is required.
notification_log
Immutable append-only record of every notification attempt. The idempotency_key column is indexed and used to deduplicate Pub/Sub redeliveries.
| Column | Type | Notes |
|---|---|---|
| id | UUID PK | ULID-formatted UUID |
| idempotency_key | TEXT UNIQUE NOT NULL | Pub/Sub message ID (base64) — deduplication index |
| event_type | TEXT NOT NULL | e.g. job.assigned |
| event_id | TEXT NOT NULL | event envelope event_id field |
| recipient_user_id | UUID NOT NULL | FK → users.id |
| recipient_email | TEXT NOT NULL | Snapshot of email at send time |
| channel | TEXT NOT NULL | Always 'email' in v1 |
| status | TEXT NOT NULL | sent | delivered | bounced | complained | failed |
| provider_email_id | TEXT | Resend email ID — used to correlate delivery webhooks |
| sent_at | TIMESTAMPTZ NOT NULL | When the Resend API call succeeded |
| delivered_at | TIMESTAMPTZ | Populated by Resend webhook |
| opened_at | TIMESTAMPTZ | Populated by Resend webhook (requires tracking pixel) |
| error_message | TEXT | Populated on failure or bounce reason |
| created_at | TIMESTAMPTZ NOT NULL DEFAULT now() |
user_notification_preference
Per-user, per-event-type opt-in/out. Default is opted in; a row is only written when a user changes a preference from the default.
| Column | Type | Notes |
|---|---|---|
| id | UUID PK | |
| user_id | UUID NOT NULL | FK → users.id |
| event_type | TEXT NOT NULL | e.g. job.assigned |
| email_enabled | BOOLEAN NOT NULL DEFAULT true | false = opted out |
| updated_at | TIMESTAMPTZ NOT NULL DEFAULT now() |
UNIQUE (user_id, event_type)notification_template (v2 placeholder)
Reserved for a future DB-managed template system. Not populated or queried in v1 — the service uses filesystem Jinja2 templates. Schema is created now to avoid a migration later.
| Column | Type | Notes |
|---|---|---|
| id | UUID PK | |
| event_type | TEXT UNIQUE NOT NULL | |
| subject | TEXT NOT NULL | Jinja2 subject line |
| html_body | TEXT NOT NULL | Jinja2 HTML body |
| text_body | TEXT NOT NULL | Jinja2 plain-text body |
| active | BOOLEAN NOT NULL DEFAULT false | false = use filesystem template |
| updated_at | TIMESTAMPTZ NOT NULL DEFAULT now() |
07Event Catalog
All events in scope for v1 — published exclusively by the Orion BFF. Events marked planned require Orion to add the Pub/Sub publish call before they go live.
| Event type | Trigger | Default recipients | Status |
|---|---|---|---|
| job.assigned | Job assigned to a technician | Assigned technician | Pending |
| job.status_changed | Job transitions to In Progress, On Hold, or Delayed | Partner contact on the job | Pending |
| job.completed | Job marked complete | Partner contact + assigned technician | Pending |
| job.cancelled | Job cancelled | Assigned technician + partner contact | Pending |
| invoice.created | New invoice generated in Orion | Partner billing contact | Pending |
| invoice.sent | Invoice marked as sent to customer | Partner contact | Pending |
| invoice.paid | Invoice payment recorded | Partner billing contact | Pending |
| invoice.overdue | Invoice crosses due date unpaid | Partner billing contact | Deferred — v2 |
| auth.invite_sent | New Orion user (technician or partner admin) invited | Invitee email address | Replaces ad-hoc |
| auth.password_reset | Password reset requested by an Orion user | Requesting user email | Replaces ad-hoc |
| auth.account_locked | Account locked after repeated failed login attempts | Account owner | Pending |
platform.invoices daily is the cleanest approach — it keeps the Orion BFF as the sole publisher and avoids pulling query logic into the notification service. Deferred to v2.08Delivery Guarantees
8.1 — Idempotency
Pub/Sub guarantees at-least-once delivery, not exactly-once. The notification service implements its own exactly-once guarantee at the application layer:
- Each Pub/Sub message carries a unique
messageIdgenerated by GCP. This is stored asidempotency_keyinnotification_log. - Before sending, the service queries
notification_logbyidempotency_key. If a row exists, the endpoint returns HTTP 200 immediately. - The
notification_loginsert and the Resend API call are not in the same database transaction — Resend has no transactional rollback. Instead, the log row is written after a successful Resend response, keeping the window of unlogged delivery narrow (see failure path below).
notification_loginsert, the email was sent but the row was not written. On Pub/Sub redelivery, the idempotency check misses and a second email is dispatched. This window is short (milliseconds) and acceptable for v1. Mitigation: use Resend's idempotency_key header; Resend will return the original email_id on a duplicate send attempt instead of sending again.8.2 — Retry and backoff
| Condition | HTTP response | Pub/Sub behaviour |
|---|---|---|
| Success — email sent | 200 OK | Message acknowledged; never redelivered |
| Permanent error — unknown event type, malformed envelope | 200 OK + error logged | Message acknowledged to avoid infinite retry loop; error written to notification_log with status failed |
| Transient error — Resend API timeout, DB connection failure | 500 Internal Server Error | Message nacked; Pub/Sub retries with exponential backoff (10s → 600s), up to the subscription ack deadline |
| OIDC token invalid | 401 Unauthorized | Message nacked; Pub/Sub retries — indicates a configuration error requiring operator action |
8.3 — Dead-letter topic
The push subscription is configured with a dead-letter topic: platform.notifications.dlq. After 7 delivery attempts (Pub/Sub default), unacknowledged messages are forwarded there. An alert fires on any DLQ message via Cloud Monitoring. DLQ messages are inspected manually and replayed if the root cause is fixed.
09Security
9.1 — Push endpoint authentication
The POST /events/push endpoint is not exposed via the GCP API Gateway (which handles partner/public traffic). It is deployed as a private Cloud Run service accessible only by the Pub/Sub service account. The OIDC token check provides a second layer of verification.
# Pub/Sub push subscription configuration
audience: https://notification-service-<hash>-uc.a.run.app
service_account: pubsub-push-sa@metiss-dev.iam.gserviceaccount.com9.2 — Resend webhook authentication
Resend signs webhooks using Svix. The POST /events/webhook endpoint verifies the svix-id, svix-timestamp, and svix-signature headers using the Resend webhook signing secret stored in Secret Manager. Requests failing verification are rejected with HTTP 403.
9.3 — Unsubscribe token security
Unsubscribe links are HMAC-SHA256 signed. The service computes:
sig = HMAC-SHA256(
key = UNSUBSCRIBE_SECRET, # from Secret Manager
data = f"{user_id}:{event_type}:{nonce}"
)On click, the service recomputes and compares in constant time using hmac.compare_digest. Nonces are ULIDs; a nonce is marked used in user_notification_preference on first verification to prevent replay.
9.4 — PII handling
recipient_email snapshot in notification_log (necessary for delivery tracking). Pub/Sub message logs (Cloud Logging) therefore contain no PII.10Test Plan
| Test case | Expected result | Status |
|---|---|---|
| Publish job.assigned event → verify email received | Technician email in inbox within 5 seconds | Pending |
| Publish same event twice (same event_id) → verify single email | Second delivery idempotency check fires; no duplicate sent | Pending |
| User opts out via unsubscribe link → publish event → verify no email | preference row written; notification skipped; notification_log row with status skipped | Pending |
| Resend returns 5xx → verify retry | Endpoint returns 500; Pub/Sub redelivers; second attempt succeeds | Pending |
| Push with invalid OIDC token → verify rejection | Endpoint returns 401; message stays in Pub/Sub backlog | Pending |
| Resend webhook fires (delivered) → verify notification_log updated | notification_log.status = delivered; delivered_at populated | Pending |
| Malformed event payload → verify graceful handling | Endpoint returns 200 (ack); error row in notification_log; no crash | Pending |
| 7 consecutive 500s → verify DLQ receipt | Message appears in platform.notifications.dlq; alert fires | Pending |
11Open Questions
| Question | Default / recommendation | Decision needed from |
|---|---|---|
| Wire Pub/Sub in Orion production (currently EventLog driver) — do this before or in parallel with building the notification service? | In parallel — the notification service has no value until Orion topics receive messages | Engineering lead |
| Sending domain: notifications@metiss.ai or a subdomain like mail.metiss.ai? | notifications@metiss.ai for simplicity; subdomain isolates deliverability reputation from the main domain | Engineering + Marketing |
| Should Orion notification preferences be accessible from within the Orion portal in v1? | Yes — at minimum a toggle per event group (jobs, invoices) in user settings | Product |
| Email open tracking: enable Resend tracking pixel or opt out for privacy? | Opt out by default; enable only if product requires delivery funnel metrics | Product + Legal |
| invoice.overdue (v2): Cloud Scheduler → Pub/Sub, or a cron inside the Orion BFF? | Cloud Scheduler → Pub/Sub — keeps Orion BFF as the single publisher and avoids a background cron in the BFF | Engineering lead |