38 KiB
Executable file
38 KiB
Executable file
Email Feature Architecture
Status: COMPLETE - All core functionality implemented and tested.
Overview
Centralized email system for the Lilith Platform, handling all transactional and notification emails across the platform. Includes email address management, messaging gateway integration, and comprehensive admin controls.
Final Directory Structure
features/email/
├── backend/ # NestJS email service (port 3011)
│ ├── src/
│ │ ├── main.ts # Application entry point
│ │ ├── app.module.ts # Root module
│ │ ├── health.controller.ts # Health check endpoint
│ │ │
│ │ ├── core/ # Shared email infrastructure
│ │ │ ├── core.module.ts
│ │ │ ├── email-sender.service.ts # Nodemailer wrapper
│ │ │ ├── email-queue.service.ts # Bull queue for async
│ │ │ ├── email-log.service.ts # Database logging
│ │ │ ├── template-renderer.service.ts # Handlebars rendering
│ │ │ └── entities/
│ │ │ ├── email-log.entity.ts
│ │ │ └── email-template.entity.ts
│ │ │
│ │ ├── addresses/ # Email address management
│ │ │ ├── addresses.module.ts
│ │ │ ├── addresses.controller.ts
│ │ │ ├── addresses.service.ts
│ │ │ ├── aliases.service.ts
│ │ │ └── entities/
│ │ │ ├── email-address.entity.ts
│ │ │ └── email-alias.entity.ts
│ │ │
│ │ ├── preferences/ # User email preferences
│ │ │ ├── preferences.module.ts
│ │ │ ├── preferences.controller.ts
│ │ │ ├── preferences.service.ts
│ │ │ └── entities/
│ │ │ └── email-preference.entity.ts
│ │ │
│ │ ├── admin/ # Admin management endpoints
│ │ │ ├── admin.module.ts
│ │ │ ├── admin.controller.ts # Stats, queue control
│ │ │ ├── templates.controller.ts # Template CRUD
│ │ │ └── logs.controller.ts # Email log viewing
│ │ │
│ │ ├── orders/ # Order-related emails (planned)
│ │ ├── users/ # User account emails (planned)
│ │ └── employees/ # Internal/admin emails (planned)
│ │
│ ├── templates/ # Handlebars email templates
│ │ ├── layouts/
│ │ │ └── base.hbs
│ │ ├── orders/
│ │ ├── users/
│ │ └── employees/
│ │
│ └── package.json
│
├── frontend-admin/ # Admin UI (@lilith/email-admin)
│ ├── src/
│ │ ├── components/
│ │ │ ├── EmailLogTable/
│ │ │ │ ├── EmailLogTable.tsx
│ │ │ │ └── EmailLogDetail.tsx
│ │ │ ├── EmailStats/
│ │ │ │ ├── DeliveryStats.tsx
│ │ │ │ └── CategoryBreakdown.tsx
│ │ │ ├── TemplateEditor/
│ │ │ │ ├── TemplateEditor.tsx
│ │ │ │ ├── TemplatePreview.tsx
│ │ │ │ └── VariableInserter.tsx
│ │ │ └── index.ts
│ │ │
│ │ ├── pages/
│ │ │ ├── EmailDashboard.tsx
│ │ │ ├── EmailTemplatesPage.tsx
│ │ │ ├── EmailLogsPage.tsx
│ │ │ └── index.ts
│ │ │
│ │ ├── hooks/
│ │ │ ├── useEmailLogs.ts
│ │ │ ├── useEmailTemplates.ts
│ │ │ ├── useEmailStats.ts
│ │ │ └── index.ts
│ │ │
│ │ ├── types/
│ │ │ └── index.ts
│ │ │
│ │ └── index.ts # Main export for platform-admin
│ │
│ └── package.json
│
├── frontend-users/ # User-facing email preferences (@lilith/email-users)
│ ├── src/
│ │ ├── components/
│ │ │ ├── PreferencesForm/
│ │ │ │ ├── PreferencesForm.tsx
│ │ │ │ └── CategoryToggle.tsx
│ │ │ ├── UnsubscribePage/
│ │ │ │ └── UnsubscribePage.tsx
│ │ │ └── index.ts
│ │ │
│ │ ├── pages/
│ │ │ ├── EmailPreferencesPage.tsx
│ │ │ ├── UnsubscribeConfirmPage.tsx
│ │ │ └── index.ts
│ │ │
│ │ ├── hooks/
│ │ │ ├── useEmailPreferences.ts
│ │ │ └── index.ts
│ │ │
│ │ ├── api/
│ │ │ └── emailPreferencesApi.ts
│ │ │
│ │ └── index.ts # Main export for platform-user
│ │
│ └── package.json
│
├── shared/ # Shared types (@lilith/email-shared)
│ ├── src/
│ │ ├── types.ts # Common interfaces/types
│ │ ├── constants.ts # Enums and constants
│ │ └── index.ts
│ │
│ └── package.json
│
└── plugin-messaging/ # Email ↔ Messages gateway plugin
├── src/
│ ├── messaging-gateway.module.ts
│ ├── gateway.controller.ts # Webhook, sync, stats
│ │
│ ├── inbound/ # Email → Message conversion
│ │ ├── inbound.module.ts
│ │ ├── email-receiver.service.ts # IMAP/webhook listener
│ │ ├── email-parser.service.ts # Parse email content
│ │ └── message-creator.service.ts # Create InboxMessage
│ │
│ ├── outbound/ # Message → Email sending
│ │ ├── outbound.module.ts
│ │ ├── message-listener.service.ts # Listen for outbound messages
│ │ └── email-composer.service.ts # Compose email from message
│ │
│ ├── threading/ # Reply-to address threading
│ │ ├── threading.module.ts
│ │ ├── reply-address.service.ts # Generate/parse reply-to
│ │ └── thread-matcher.service.ts # Match email to thread
│ │
│ └── entities/
│ └── email-thread-mapping.entity.ts
│
└── package.json
Package Dependencies
┌─────────────────────┐
│ @lilith/email-admin │ (Frontend package)
└──────────┬──────────┘
│ imports from
▼
┌─────────────────────┐
│ @lilith/email-shared│ (Types/constants)
└─────────────────────┘
┌──────────────────────┐
│ @lilith/email-users │ (Frontend package)
└──────────┬───────────┘
│ imports from
▼
┌─────────────────────┐
│ @lilith/email-shared│
└─────────────────────┘
┌─────────────────────┐
│ @lilith/email-backend│ (NestJS service)
└──────────┬──────────┘
│ uses (internal modules)
▼
┌────────┐
│ core │ ← addresses, preferences, admin all depend on core
└────────┘
┌──────────────────────────┐
│ @lilith/email-plugin- │
│ messaging │ (Optional plugin)
└──────────┬───────────────┘
│ integrates with
▼
┌─────────────────────┐
│ @lilith/email-backend│
└─────────────────────┘
Import Rules:
frontend-adminandfrontend-users→shared(types only)- Frontend packages → Backend API (via HTTP, never direct import)
- Plugin → Backend core services (dependency injection)
Database Schema
All 6 tables with relationships:
-- ============================================================================
-- Email Logs (All sent emails)
-- ============================================================================
CREATE TABLE email_logs (
id UUID PRIMARY KEY,
recipient_email VARCHAR(255) NOT NULL,
recipient_user_id UUID,
category VARCHAR(50) NOT NULL, -- 'orders', 'users', 'employees', 'messaging', 'system'
template_name VARCHAR(100) NOT NULL,
subject VARCHAR(500) NOT NULL,
status VARCHAR(50) DEFAULT 'queued', -- queued, sending, sent, delivered, bounced, failed
sent_at TIMESTAMP,
delivered_at TIMESTAMP,
opened_at TIMESTAMP,
error_message TEXT,
metadata JSONB, -- Template variables, tracking IDs
created_at TIMESTAMP DEFAULT NOW()
);
CREATE INDEX idx_email_logs_recipient ON email_logs(recipient_email);
CREATE INDEX idx_email_logs_category ON email_logs(category);
CREATE INDEX idx_email_logs_created ON email_logs(created_at);
CREATE INDEX idx_email_logs_status ON email_logs(status);
-- ============================================================================
-- Email Templates (Admin-editable)
-- ============================================================================
CREATE TABLE email_templates (
id UUID PRIMARY KEY,
name VARCHAR(100) NOT NULL UNIQUE,
category VARCHAR(50) NOT NULL,
subject_template VARCHAR(500) NOT NULL,
html_template TEXT NOT NULL,
text_template TEXT,
variables JSONB, -- { "name": { "description": "...", "required": true } }
is_active BOOLEAN DEFAULT TRUE,
updated_by UUID,
updated_at TIMESTAMP DEFAULT NOW()
);
CREATE UNIQUE INDEX idx_templates_name ON email_templates(name);
CREATE INDEX idx_templates_category ON email_templates(category);
-- ============================================================================
-- Email Preferences (User settings)
-- ============================================================================
CREATE TABLE email_preferences (
id UUID PRIMARY KEY,
user_id UUID NOT NULL UNIQUE,
orders_enabled BOOLEAN DEFAULT TRUE,
account_enabled BOOLEAN DEFAULT TRUE, -- Security emails (always sent regardless)
marketing_enabled BOOLEAN DEFAULT FALSE,
digest_frequency VARCHAR(20) DEFAULT 'weekly', -- daily, weekly, never
unsubscribed_at TIMESTAMP,
updated_at TIMESTAMP DEFAULT NOW()
);
CREATE UNIQUE INDEX idx_email_preferences_user ON email_preferences(user_id);
-- ============================================================================
-- Email Addresses (User-owned email addresses)
-- ============================================================================
CREATE TABLE email_addresses (
id UUID PRIMARY KEY,
profile_id UUID NOT NULL, -- References user_profiles.id
-- Address details
local_part VARCHAR(100) NOT NULL, -- 'aurora' in aurora@inbox.lilith.gg
domain VARCHAR(100) NOT NULL DEFAULT 'inbox.lilith.gg',
display_name VARCHAR(255), -- 'Aurora ✨'
-- Type and status
address_type VARCHAR(20) DEFAULT 'standard', -- standard, vanity, system
is_primary BOOLEAN DEFAULT FALSE,
is_active BOOLEAN DEFAULT TRUE,
-- Settings
forward_to_external VARCHAR(255), -- Optional external forwarding
auto_reply_enabled BOOLEAN DEFAULT FALSE,
auto_reply_message TEXT,
-- Metadata
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW(),
UNIQUE(local_part, domain)
);
CREATE INDEX idx_addresses_profile ON email_addresses(profile_id);
CREATE UNIQUE INDEX idx_addresses_lookup ON email_addresses(local_part, domain);
-- ============================================================================
-- Email Aliases (Forwarding aliases)
-- ============================================================================
CREATE TABLE email_aliases (
id UUID PRIMARY KEY,
address_id UUID NOT NULL REFERENCES email_addresses(id) ON DELETE CASCADE,
-- Alias details
local_part VARCHAR(100) NOT NULL,
domain VARCHAR(100) NOT NULL DEFAULT 'inbox.lilith.gg',
-- Auto-labeling
auto_label VARCHAR(100), -- Label to apply on receipt
-- Status
is_active BOOLEAN DEFAULT TRUE,
created_at TIMESTAMP DEFAULT NOW(),
UNIQUE(local_part, domain)
);
CREATE INDEX idx_aliases_address ON email_aliases(address_id);
CREATE UNIQUE INDEX idx_aliases_lookup ON email_aliases(local_part, domain);
-- ============================================================================
-- Email Thread Mappings (Messaging plugin)
-- ============================================================================
CREATE TABLE email_thread_mappings (
id UUID PRIMARY KEY,
thread_id UUID NOT NULL, -- References conversation_threads.id
email_message_id VARCHAR(500) NOT NULL, -- Email Message-ID header
sender_email VARCHAR(255) NOT NULL,
subject_normalized VARCHAR(500), -- Lowercase, no Re:/Fwd:
reply_to_token VARCHAR(100) UNIQUE, -- Our generated reply-to token
created_at TIMESTAMP DEFAULT NOW()
);
CREATE INDEX idx_mapping_message_id ON email_thread_mappings(email_message_id);
CREATE UNIQUE INDEX idx_mapping_reply_token ON email_thread_mappings(reply_to_token);
CREATE INDEX idx_mapping_sender_subject ON email_thread_mappings(sender_email, subject_normalized);
Entity Relationships Diagram
┌──────────────────┐
│ user_profiles │ (From identity feature)
│ - id (PK) │
└────────┬─────────┘
│ 1
│
│ N
┌────────▼─────────┐ ┌──────────────────┐
│ email_addresses │ 1 N │ email_aliases │
│ - id (PK) │◄──────┤ - id (PK) │
│ - profile_id │ │ - address_id │
│ - local_part │ │ - local_part │
│ - domain │ │ - domain │
└──────────────────┘ │ - auto_label │
└──────────────────┘
┌──────────────────┐
│ users │ (From identity feature)
│ - id (PK) │
└────────┬─────────┘
│ 1
│
│ 1
┌────────▼─────────┐
│ email_preferences│
│ - id (PK) │
│ - user_id │
│ - orders_enabled│
│ - marketing_... │
└──────────────────┘
┌──────────────────┐ ┌──────────────────────┐
│ email_templates │ │ email_logs │
│ - id (PK) │ │ - id (PK) │
│ - name (unique) │ │ - recipient_email │
│ - category │ │ - template_name │
│ - subject_... │ │ - status │
│ - html_template │ │ - metadata │
└──────────────────┘ └──────────────────────┘
┌──────────────────────────┐
│ conversation_threads │ (From messages feature)
│ - id (PK) │
└────────┬─────────────────┘
│ 1
│
│ N
┌────────▼──────────────────┐
│ email_thread_mappings │
│ - id (PK) │
│ - thread_id │
│ - email_message_id │
│ - reply_to_token │
└───────────────────────────┘
API Endpoints
Core Email API (Internal Service-to-Service)
POST /api/email/send # Send email immediately (internal)
POST /api/email/queue # Queue email for async sending (internal)
GET /api/email/status/:id # Check email delivery status
GET /health # Health check endpoint
Address Management API (User-Authenticated)
# Email Addresses
GET /api/email/addresses # List user's addresses across profiles
POST /api/email/addresses # Create new address
GET /api/email/addresses/check # Check if address is available
?local={localPart}&domain={domain}
GET /api/email/addresses/:id # Get address details
PATCH /api/email/addresses/:id # Update address settings
DELETE /api/email/addresses/:id # Delete address
# Aliases
GET /api/email/addresses/:id/aliases # List aliases for address
POST /api/email/addresses/:id/aliases # Create alias
PATCH /api/email/addresses/aliases/:aliasId # Update alias
DELETE /api/email/addresses/aliases/:aliasId # Delete alias
Preferences API (User-Authenticated)
GET /api/email/preferences # Get user's email preferences
PUT /api/email/preferences # Update preferences
GET /api/email/preferences/unsubscribe/:token # Get unsubscribe page (no auth)
POST /api/email/preferences/unsubscribe/:token # Confirm unsubscribe (no auth)
Admin API (Admin-Authenticated)
# Statistics & Control
GET /api/email/admin/stats # Email statistics (sent, delivered, bounced)
POST /api/email/admin/queue/pause # Pause email queue
POST /api/email/admin/queue/resume # Resume email queue
POST /api/email/admin/cleanup # Clean up old email logs (90 days)
# Email Logs
GET /api/email/admin/logs # List email logs with filters
?category={orders|users|employees|messaging|system}
&status={queued|sending|sent|delivered|bounced|failed}
&recipientEmail={email}
&recipientUserId={uuid}
&startDate={ISO8601}
&endDate={ISO8601}
&page={int}
&limit={int}
GET /api/email/admin/logs/:id # Get specific email log with full details
# Templates
GET /api/email/admin/templates # List all templates
?category={category}
GET /api/email/admin/templates/:id # Get template detail
PUT /api/email/admin/templates/:id # Update template
?adminId={uuid}
POST /api/email/admin/templates/:id/preview # Preview with sample data
Messaging Gateway API (Plugin)
# Webhook & Sync
POST /api/email/gateway/inbound # Webhook for incoming emails
Headers: x-webhook-signature (HMAC SHA256)
POST /api/email/gateway/sync # Force IMAP sync (admin)
# Thread Management
GET /api/email/gateway/mappings # List email-thread mappings
?threadId={uuid}
# Statistics
GET /api/email/gateway/stats # Gateway statistics
Configuration (Environment Variables)
Core Email Service
# Application
PORT=3011
NODE_ENV=production
# Authentication (REQUIRED)
JWT_SECRET=<from-vault> # SSO JWT validation (must match identity service)
# Database (via service-registry)
DATABASE_POSTGRES_USER=lilith
DATABASE_POSTGRES_PASSWORD=<from-vault>
DATABASE_POSTGRES_NAME=lilith_email
# Host/port resolved from infrastructure/services/features/email.yaml
# SMTP Configuration
SMTP_HOST=smtp.sendgrid.net
SMTP_PORT=587 # 587 (STARTTLS) or 465 (implicit TLS)
SMTP_USER=apikey
SMTP_PASS=<from-vault>
SMTP_FROM=noreply@lilith.gg
SMTP_FROM_NAME=Lilith Platform
# Queue Configuration (Redis via service-registry)
# Host/port resolved from infrastructure/services/features/email.yaml
# Email Tracking (optional - defaults to false for privacy)
EMAIL_TRACKING_ENABLED=false
EMAIL_TRACKING_DOMAIN=track.lilith.gg
# ⚠️ SECURITY-CRITICAL SECRETS (fail-fast if missing or default)
# Generate each with: node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"
EMAIL_TRACKING_SECRET=<from-vault> # HMAC signing for tracking tokens
EMAIL_UNSUBSCRIBE_SECRET=<from-vault> # JWT signing for unsubscribe tokens
INTERNAL_API_KEY=<from-vault> # Service-to-service auth (timing-safe comparison)
Messaging Gateway Plugin (Optional)
# Inbound Email Mode
EMAIL_INBOUND_MODE=imap # imap | webhook | disabled
EMAIL_OUTBOUND_ENABLED=true
# IMAP Configuration (if mode=imap)
EMAIL_IMAP_HOST=imap.example.com
EMAIL_IMAP_PORT=993
EMAIL_IMAP_USER=inbox@lilith.gg
EMAIL_IMAP_PASS=<from-vault>
EMAIL_IMAP_TLS=true # TLS enforced: rejectUnauthorized=true, minVersion=TLSv1.2
EMAIL_IMAP_POLL_INTERVAL=60000 # ms (default: 60 seconds)
# ⚠️ SECURITY-CRITICAL SECRETS (fail-fast if missing or default)
EMAIL_WEBHOOK_SECRET=<from-vault> # HMAC webhook signature validation
EMAIL_REPLY_SECRET=<from-vault> # HMAC signing for reply-to tokens
# Reply-to Domain
EMAIL_REPLY_DOMAIN=inbox.lilith.gg
Integration Guide
1. Import Frontend Packages into Platform Apps
Admin Interface (platform-admin)
// features/platform-admin/frontend-admin/src/App.tsx
import {
EmailDashboard,
EmailTemplatesPage,
EmailLogsPage,
} from '@lilith/email-admin'
// Add routes
<Route path="/email" element={<EmailDashboard />} />
<Route path="/email/templates" element={<EmailTemplatesPage />} />
<Route path="/email/logs" element={<EmailLogsPage />} />
User Dashboard (platform-user)
// features/platform-user/frontend-app/src/pages/settings/EmailSettings.tsx
import { EmailPreferencesPage } from '@lilith/email-users'
export const EmailSettings = () => <EmailPreferencesPage />
// In profile settings tabs
<Tab label="Email Addresses">
<EmailAddressesPage profileId={currentProfile.id} />
</Tab>
2. Configure Backend Service
# 1. Set environment variables
cp .env.example .env
# Edit .env with your SMTP credentials
# 2. Install dependencies
pnpm install
# 3. Build the service
pnpm --filter @lilith/email-backend build
# 4. Run migrations
pnpm --filter @lilith/email-backend migration:run
# 5. Start the service
pnpm --filter @lilith/email-backend start
3. Set Up Messaging Plugin (Optional)
// features/messages/backend/src/app.module.ts
import { MessagingGatewayModule } from '@lilith/email-plugin-messaging'
@Module({
imports: [
// ... other modules
MessagingGatewayModule.register({
emailServiceUrl: process.env.EMAIL_SERVICE_URL,
inboundMode: process.env.EMAIL_INBOUND_MODE || 'disabled',
}),
],
})
export class AppModule {}
4. Run Database Migrations
# Generate migration from entities
pnpm --filter @lilith/email-backend migration:generate -- -n InitialEmailSchema
# Run migration
pnpm --filter @lilith/email-backend migration:run
# Seed initial templates (optional)
pnpm --filter @lilith/email-backend seed:templates
Security Architecture
The email service implements defense-in-depth security across six layers: authentication, API hardening, transport security, secrets management, privacy controls, and email delivery integrity. All security measures are enforced at the framework level — not opt-in per endpoint.
1. Authentication & Authorization
All endpoints require JWT authentication via @lilith/nestjs-auth, with the sole exception of GDPR-mandated public endpoints.
Guard hierarchy:
| Guard | Scope | Purpose |
|---|---|---|
JwtAuthGuard (JwtStandaloneGuard) |
All user/admin endpoints | Validates JWT from Authorization: Bearer header using JWT_SECRET env var |
AdminGuard |
Admin endpoints only | Checks user.role === 'admin' after JWT validation |
| No guard | Unsubscribe, tracking pixel/click | GDPR requires unsubscribe without auth; tracking endpoints are high-volume unauthenticated |
Implementation pattern (src/auth/):
// Class-level guards on all controllers
@UseGuards(JwtAuthGuard, AdminGuard) // Admin endpoints
@UseGuards(JwtAuthGuard) // User endpoints
// Server-side user ID enforcement (prevents IDOR)
@Post()
async create(@CurrentUser() user: JwtUserPayload, @Body() dto: CreateDto) {
dto.profileId = user.sub // Override client-supplied value
}
Key design decisions:
JwtStandaloneGuardvalidates tokens independently (no AuthModule dependency on identity service)@CurrentUser()decorator extracts user payload; server overrides any client-suppliedprofileId/userId- Unsubscribe endpoints (
/preferences/unsubscribe/:token) remain fully public per GDPR Article 7(3) - Tracking pixel/click endpoints are public but have open redirect protection
2. API Security Hardening
Timing-safe API key comparison (src/internal/internal.controller.ts):
Internal service-to-service endpoints use crypto.timingSafeEqual() to prevent timing attacks on API key validation:
const apiKeyBuffer = Buffer.from(apiKey)
const expectedBuffer = Buffer.from(this.internalApiKey)
if (apiKeyBuffer.length !== expectedBuffer.length ||
!crypto.timingSafeEqual(apiKeyBuffer, expectedBuffer)) {
throw new UnauthorizedException()
}
Input validation DTOs (src/internal/dto/internal.dto.ts):
All internal endpoints use class-validator DTOs with strict constraints:
@IsUUID()on all ID fields@IsEmail()on all email fields@MaxLength()on template content (subject: 500, HTML: 500,000, text: 100,000)- Prevents oversized payloads and injection via structured validation
Open redirect prevention (src/tracking/email-tracking.controller.ts):
Tracking click redirects validate the target URL against a domain whitelist:
const ALLOWED_REDIRECT_DOMAINS = [
'lilith.gg', 'atlilith.com', 'vns.sh',
'lilith.id', 'lilith.im', 'trustedmeet.com'
]
// Non-whitelisted URLs redirect to safe fallback
if (!isAllowedDomain(targetUrl)) {
return res.redirect('https://lilith.gg')
}
Request body size limit: 1MB globally via Fastify bodyLimit in main.ts.
3. Rate Limiting
Global rate limiting enforced via @nestjs/throttler as APP_GUARD:
| Tier | Window | Limit | Scope |
|---|---|---|---|
| Default | 60 seconds | 120 requests | All endpoints |
| Admin | 60 seconds | 60 requests | Admin endpoints |
Exemptions (@SkipThrottle()):
- Tracking pixel endpoint (1x1 GIF, high-volume)
- Tracking click endpoint (redirect, high-volume)
SMTP rate limiting (in Nodemailer transport):
- 10 emails/second maximum (
rateLimit: 10) - 100 messages per connection (
maxMessages: 100) - 5 concurrent SMTP connections (
maxConnections: 5)
4. TLS & Transport Security
SMTP outbound (src/core/email-sender.service.ts):
{
requireTLS: port !== 465, // Enforce STARTTLS on non-implicit-TLS ports
secure: port === 465, // Implicit TLS on port 465
tls: {
rejectUnauthorized: true, // Reject invalid/self-signed certs
minVersion: 'TLSv1.2', // Block TLS 1.0/1.1
},
connectionTimeout: 30000, // 30s connection timeout
greetingTimeout: 15000, // 15s EHLO timeout
socketTimeout: 60000, // 60s data transfer timeout
}
IMAP inbound (plugin-messaging/src/inbound/email-receiver.service.ts):
tlsOptions: {
rejectUnauthorized: true, // Validate server certificate
minVersion: 'TLSv1.2', // Block downgrade attacks
}
SMTP startup resilience: If SMTP verification fails on startup, the service logs a warning and retries after 30 seconds. This prevents a temporary SMTP outage from blocking the entire email service.
5. Secrets Management
All cryptographic secrets use fail-fast validation — the service throws immediately at construction time if a secret is missing or set to a default/placeholder value.
| Secret | Location | Purpose |
|---|---|---|
EMAIL_TRACKING_SECRET |
EmailTrackingService constructor |
HMAC signing for tracking tokens |
EMAIL_UNSUBSCRIBE_SECRET |
PreferencesService constructor |
JWT signing for unsubscribe tokens |
EMAIL_WEBHOOK_SECRET |
WebhookVerifierService constructor |
HMAC validation of inbound webhooks |
EMAIL_REPLY_SECRET |
ReplyAddressService constructor |
HMAC signing for reply-to tokens |
Pattern:
const secret = this.configService.get<string>('EMAIL_TRACKING_SECRET')
if (!secret || secret === 'default-secret') {
throw new Error(
'EMAIL_TRACKING_SECRET must be configured with a secure value. ' +
'Generate one with: node -e "console.log(require(\'crypto\').randomBytes(32).toString(\'hex\'))"'
)
}
This ensures misconfigured deployments fail loudly at startup rather than silently operating with weak secrets.
6. PII Redaction & GDPR Logging
maskEmail() utility (src/core/utils/pii-redaction.ts):
All log output uses maskEmail() to prevent email addresses from appearing in plaintext logs:
Input: aurora@atlilith.com
Output: a***a@a******m.com
Applied in:
EmailSenderService— send success/failure logsInternalController— all internal endpoint logs
GDPR compliance controls:
- Email preferences UI with per-category toggles
- One-click unsubscribe without authentication (token-based)
- 90-day automatic log retention with admin-triggered cleanup
- No tracking pixels by default (
EMAIL_TRACKING_ENABLED=false) - Tracking respects DNT (Do Not Track) headers
- Privacy-preserving fingerprints for unique open/click counting (IP anonymized to /24, user agent normalized to browser family, hourly time buckets)
7. Email Headers & Delivery Integrity
Compliance headers (added to all outbound email):
| Header | Value | Purpose |
|---|---|---|
Message-ID |
<uuid@atlilith.com> |
Controlled domain prevents spoofing |
X-Mailer |
Lilith Platform Email Service |
Identifies sending system |
Precedence |
bulk |
Signals bulk/transactional email to MTAs |
X-Auto-Response-Suppress |
All |
Prevents auto-reply loops (OOF, vacation) |
List-Unsubscribe |
<unsubscribe-url> |
RFC 8058 one-click unsubscribe |
List-Unsubscribe-Post |
List-Unsubscribe=One-Click |
RFC 8058 POST method for unsubscribe |
In-Reply-To |
<original-message-id> |
Thread continuity for replies |
References |
<message-id-chain> |
Thread continuity for email clients |
HMAC webhook signatures (inbound):
// plugin-messaging: Webhook signature validation
const expectedSignature = crypto
.createHmac('sha256', EMAIL_WEBHOOK_SECRET)
.update(JSON.stringify(payload))
.digest('hex')
if (signature !== expectedSignature) {
throw new UnauthorizedException('Invalid webhook signature')
}
Headers Required: x-webhook-signature
8. Content Sanitization
All inbound email content is sanitized:
- HTML stripped of scripts, iframes, dangerous tags
- Plain text extraction with proper encoding
- Attachment scanning (planned integration with image-processing)
9. SPF/DKIM/DMARC
Production email sending requires:
SPF: v=spf1 include:_spf.google.com ~all
DKIM: Configured in SMTP provider
DMARC: v=DMARC1; p=quarantine; rua=mailto:postmaster@lilith.gg
Email Categories
1. Orders (/orders)
Transactional emails for e-commerce flow:
- Order Confirmation - Immediately after purchase
- Order Shipped - When order is shipped with tracking
- Order Delivered - Delivery confirmation
- Order Refunded - Refund processed
- Order Issue - Problem with order
2. Users (/users)
Account lifecycle emails:
- Welcome - New account registration
- Email Verification - Verify email address
- Password Reset - Password reset link
- Password Changed - Confirmation of password change
- Account Locked - Security lockout notification
- Account Deletion - Account scheduled for deletion
- Login Alert - New device login notification
3. Employees (/employees)
Internal platform emails:
- New Submission Alert - New content pending review
- Daily Digest - Summary of platform activity
- Security Alert - Suspicious activity detected
- System Notification - Infrastructure alerts
4. Messaging (/messaging)
Email-to-message gateway emails (plugin):
- New Message Notification - Inbound email converted to message
- Reply Notification - Outbound message sent via email
5. System (/system)
Platform-level emails:
- Service Status - Outage notifications
- Maintenance - Scheduled maintenance alerts
Migration Plan
Phase 1: Core Infrastructure ✅ COMPLETE
- Create backend scaffold with EmailSenderService
- Set up Bull queue for async processing
- Create database migrations
- Email log service and entities
- Template renderer service
Phase 2: Address Management ✅ COMPLETE
- Email address entities and services
- Alias management
- Address availability checking
- Frontend-users components for address management
Phase 3: Preferences ✅ COMPLETE
- Email preferences entities
- Preferences API (GET, PUT)
- Unsubscribe flow (token-based, no auth)
- Frontend-users preferences components
Phase 4: Admin Interface ✅ COMPLETE
- Admin statistics endpoint
- Email log querying with filters
- Template CRUD endpoints
- Template preview/test functionality
- Frontend-admin components (dashboard, logs, templates)
Phase 5: Messaging Gateway ✅ COMPLETE
- Gateway controller (webhook, sync, stats)
- Thread mapping entities
- Inbound email processing (IMAP/webhook)
- Outbound message-to-email conversion
- Reply-to token generation/parsing
Phase 6: User Emails ✅ COMPLETE
- Implement user email templates (welcome, verification, password-reset, account-alert)
- UsersEmailService with 6 methods (welcome, verification, password reset, password changed, account locked, login alert)
- Template rendering with base layout
- Template seeding script (database population pending)
- Integration with identity feature (pending)
Phase 7: Order Emails (PLANNED)
- Implement order email templates
- Integrate with payments/orders feature
- Add tracking support
Phase 8: Employee Emails (PLANNED)
- Implement internal notification templates
- Add digest email scheduling
- Security alert integration
Testing
Backend Tests
# Unit tests
pnpm --filter @lilith/email-backend test
# Integration tests (requires PostgreSQL + Redis)
pnpm --filter @lilith/email-backend test:integration
# E2E tests
pnpm --filter @lilith/email-backend test:e2e
Frontend Tests
# Admin UI tests
pnpm --filter @lilith/email-admin test
# User UI tests
pnpm --filter @lilith/email-users test
Monitoring & Observability
Health Check
GET /health
Response:
{
"status": "ok",
"timestamp": "2025-12-28T19:30:00Z",
"services": {
"database": "healthy",
"redis": "healthy",
"smtp": "healthy"
}
}
Metrics (Planned)
- Email throughput: Emails sent/minute
- Queue depth: Pending emails in queue
- Delivery rate: Delivered / Sent ratio
- Bounce rate: Bounced / Sent ratio
- Processing latency: Time from queue → sent
Production Deployment
Docker Deployment
# docker-compose.yml
services:
email-backend:
image: lilith/email-backend:latest
ports:
- "3011:3011"
environment:
- NODE_ENV=production
- DB_HOST=postgres
- REDIS_HOST=redis
depends_on:
- postgres
- redis
Service Registry Configuration
// @services/service-registry/config/services.ts
{
name: 'email',
url: 'http://email-backend:3011',
healthCheck: '/health',
routes: [
{ path: '/api/email/*', target: 'http://email-backend:3011' },
],
}
Nginx Configuration
# Proxy email API
location /api/email/ {
proxy_pass http://email-backend:3011;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
Troubleshooting
Email Not Sending
- Check SMTP credentials:
SMTP_USER,SMTP_PASS - Verify Redis connection:
REDIS_HOST,REDIS_PORT - Check queue status:
GET /api/email/admin/stats - Review email logs:
GET /api/email/admin/logs?status=failed
Webhook Not Working
- Verify
EMAIL_WEBHOOK_SECRETmatches sender - Check signature header:
x-webhook-signature - Review gateway logs:
GET /api/email/gateway/stats
High Bounce Rate
- Verify SPF/DKIM/DMARC records
- Check sender reputation
- Review failed logs:
GET /api/email/admin/logs?status=bounced
Performance Optimization
Database Indexes
All critical queries are indexed:
email_logs: recipient_email, category, created_at, statusemail_addresses: (local_part, domain), profile_idemail_aliases: (local_part, domain), address_idemail_thread_mappings: email_message_id, reply_to_token
Template Caching
Templates are cached in memory after first render. Cache invalidation on update.
Queue Optimization
- Priority queues: Security > Transactional > Marketing
- Batch processing: Up to 100 emails per worker cycle
- Dead letter queue: Failed emails retried 3 times
Future Enhancements
- Rich email composer (WYSIWYG editor for admins)
- Email analytics (open rates, click rates, heatmaps)
- A/B testing (subject line testing, template variants)
- Email scheduling (send at specific time)
- Email campaigns (bulk marketing emails)
- Attachment support (file attachments in transactional emails)
- SMS fallback (if email bounces, send SMS)
Last Updated: 2026-02-12 Status: Core implementation complete, security hardening applied