diff --git a/features/email/ARCHITECTURE.md b/features/email/ARCHITECTURE.md new file mode 100644 index 000000000..baa274286 --- /dev/null +++ b/features/email/ARCHITECTURE.md @@ -0,0 +1,783 @@ +# Email Feature Architecture + +## Overview + +Centralized email system for the Lilith Platform, handling all transactional and notification emails across the platform. + +## Directory Structure + +``` +features/email/ +├── backend/ # NestJS email service (port 3011) +│ ├── src/ +│ │ ├── main.ts +│ │ ├── app.module.ts +│ │ ├── orders/ # Order-related emails +│ │ │ ├── orders.module.ts +│ │ │ ├── orders-email.service.ts +│ │ │ └── templates/ +│ │ │ ├── order-confirmation.hbs +│ │ │ ├── order-shipped.hbs +│ │ │ ├── order-delivered.hbs +│ │ │ └── order-refunded.hbs +│ │ ├── users/ # User account emails +│ │ │ ├── users.module.ts +│ │ │ ├── users-email.service.ts +│ │ │ └── templates/ +│ │ │ ├── welcome.hbs +│ │ │ ├── email-verification.hbs +│ │ │ ├── password-reset.hbs +│ │ │ ├── password-changed.hbs +│ │ │ ├── account-locked.hbs +│ │ │ └── account-deletion.hbs +│ │ ├── employees/ # Internal/admin emails +│ │ │ ├── employees.module.ts +│ │ │ ├── employees-email.service.ts +│ │ │ └── templates/ +│ │ │ ├── new-submission-alert.hbs +│ │ │ ├── daily-digest.hbs +│ │ │ ├── security-alert.hbs +│ │ │ └── system-notification.hbs +│ │ ├── core/ # Shared email infrastructure +│ │ │ ├── core.module.ts +│ │ │ ├── email-sender.service.ts # Nodemailer wrapper +│ │ │ ├── template-renderer.service.ts # Handlebars rendering +│ │ │ ├── email-queue.service.ts # Bull queue for async +│ │ │ ├── email-log.service.ts # Database logging +│ │ │ └── entities/ +│ │ │ ├── email-log.entity.ts +│ │ │ ├── email-template.entity.ts +│ │ │ └── email-preference.entity.ts +│ │ ├── preferences/ # User email preferences +│ │ │ ├── preferences.module.ts +│ │ │ ├── preferences.controller.ts +│ │ │ └── preferences.service.ts +│ │ └── admin/ # Admin management endpoints +│ │ ├── admin.module.ts +│ │ ├── admin.controller.ts +│ │ ├── templates.controller.ts +│ │ └── logs.controller.ts +│ └── package.json +│ +├── frontend-admin/ # Admin UI (imported by platform-admin) +│ ├── src/ +│ │ ├── components/ +│ │ │ ├── EmailTemplateEditor/ +│ │ │ │ ├── EmailTemplateEditor.tsx +│ │ │ │ ├── TemplatePreview.tsx +│ │ │ │ └── VariableInserter.tsx +│ │ │ ├── EmailLogTable/ +│ │ │ │ ├── EmailLogTable.tsx +│ │ │ │ └── EmailLogDetail.tsx +│ │ │ └── EmailStats/ +│ │ │ ├── DeliveryStats.tsx +│ │ │ └── CategoryBreakdown.tsx +│ │ ├── pages/ +│ │ │ ├── EmailDashboard.tsx +│ │ │ ├── EmailTemplatesPage.tsx +│ │ │ ├── EmailLogsPage.tsx +│ │ │ └── EmailSettingsPage.tsx +│ │ ├── hooks/ +│ │ │ ├── useEmailLogs.ts +│ │ │ ├── useEmailTemplates.ts +│ │ │ └── useEmailStats.ts +│ │ └── index.ts # Main export for platform-admin +│ └── package.json +│ +├── frontend-users/ # User-facing email preferences +│ ├── src/ +│ │ ├── components/ +│ │ │ ├── PreferencesForm/ +│ │ │ │ ├── PreferencesForm.tsx +│ │ │ │ └── CategoryToggle.tsx +│ │ │ └── UnsubscribePage/ +│ │ │ └── UnsubscribePage.tsx +│ │ ├── pages/ +│ │ │ ├── EmailPreferencesPage.tsx +│ │ │ └── UnsubscribeConfirmPage.tsx +│ │ ├── hooks/ +│ │ │ └── useEmailPreferences.ts +│ │ └── index.ts # Main export for portal +│ └── package.json +│ +├── shared/ # Shared types between frontend/backend +│ ├── src/ +│ │ ├── types.ts +│ │ └── constants.ts +│ └── package.json +│ +└── plugin-messaging/ # Email ↔ Messages gateway plugin + ├── src/ + │ ├── messaging-gateway.module.ts + │ ├── inbound/ # Email → Message conversion + │ │ ├── inbound.module.ts + │ │ ├── email-receiver.service.ts # IMAP/webhook listener + │ │ ├── email-parser.service.ts # Parse email content + │ │ └── message-creator.service.ts # Create InboxMessage + │ ├── outbound/ # Message → Email sending + │ │ ├── outbound.module.ts + │ │ ├── message-listener.service.ts # Listen for outbound messages + │ │ └── email-composer.service.ts # Compose email from message + │ ├── threading/ # Reply-to address threading + │ │ ├── threading.module.ts + │ │ ├── reply-address.service.ts # Generate/parse reply-to + │ │ └── thread-matcher.service.ts # Match email to thread + │ └── entities/ + │ └── email-thread-mapping.entity.ts + └── package.json +``` + +--- + +## Email Categories + +### 1. Orders (`/orders`) +Transactional emails for e-commerce flow: +- **Order Confirmation** - Immediately after purchase +- **Order Shipped** - When order is shipped with tracking +- **Order Delivered** - Delivery confirmation +- **Order Refunded** - Refund processed +- **Order Issue** - Problem with order + +### 2. Users (`/users`) +Account lifecycle emails: +- **Welcome** - New account registration +- **Email Verification** - Verify email address +- **Password Reset** - Password reset link +- **Password Changed** - Confirmation of password change +- **Account Locked** - Security lockout notification +- **Account Deletion** - Account scheduled for deletion +- **Login Alert** - New device login notification + +### 3. Employees (`/employees`) +Internal platform emails: +- **New Submission Alert** - New content pending review +- **Daily Digest** - Summary of platform activity +- **Security Alert** - Suspicious activity detected +- **System Notification** - Infrastructure alerts + +--- + +## Core Services + +### EmailSenderService +Wraps Nodemailer with: +- Connection pooling +- Retry logic with exponential backoff +- Rate limiting per recipient +- DKIM/SPF configuration + +### TemplateRendererService +Handlebars-based template rendering: +- Base layout with branding +- Responsive HTML email design +- Plain text fallback generation +- Variable interpolation + +### EmailQueueService +Bull queue for async sending: +- Priority queues (security > transactional > marketing) +- Scheduled sends (digest emails) +- Batch processing +- Dead letter queue for failures + +### EmailLogService +Comprehensive logging: +- All sent emails with status +- Bounce/complaint tracking +- Open/click tracking (optional) +- Retention policy (90 days default) + +--- + +## Messaging Plugin (Email ↔ Messages Gateway) + +The `plugin-messaging/` module enables bidirectional email communication through the Messages feature. Users can receive and send messages via email, with full threading support. + +### Architecture Overview + +``` +┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ +│ External │ │ Email Plugin │ │ Messages │ +│ Email Server │────▶│ (Gateway) │────▶│ Feature │ +│ (IMAP/Webhook) │ │ │ │ (Inbox UI) │ +└─────────────────┘ └────────┬────────┘ └────────┬────────┘ + │ │ + │◀──────────────────────┘ + │ (Outbound replies) + ▼ + ┌─────────────────┐ + │ SMTP Server │ + │ (Outbound) │ + └─────────────────┘ +``` + +### Inbound Flow (Email → Message) + +1. **Email Reception**: + - IMAP polling (configurable interval, default 60s) + - OR webhook from email provider (SendGrid, Mailgun, AWS SES) + +2. **Email Parsing**: + - Extract sender, subject, body (HTML → plain text) + - Parse In-Reply-To / References headers for threading + - Extract attachments → upload to storage + +3. **Thread Matching**: + - Check reply-to address for encoded thread ID + - Fall back to sender email + subject matching + - Create new thread if no match + +4. **Message Creation**: + ```typescript + // Creates InboxMessage with sourceType: 'email' + const message: InboxMessage = { + threadId: matchedThread.id, + direction: 'inbound', + messageText: parsedBody, + sourceType: 'email', + sourceMessageId: email.messageId, + metadata: { + emailFrom: email.from, + emailSubject: email.subject, + attachments: uploadedAttachments, + }, + }; + ``` + +### Outbound Flow (Message → Email) + +1. **Message Listener**: + - Subscribe to message creation events + - Filter for threads with `sourceType: 'email'` + +2. **Email Composition**: + - Generate unique reply-to address with encoded thread ID + - Set In-Reply-To header for threading + - Render message body with email template + +3. **Reply-To Address Format**: + ``` + reply+{base64(threadId)}@inbox.lilith.gg + + Example: + reply+dGhyZWFkXzEyMzQ1Njc4OQ==@inbox.lilith.gg + ``` + +4. **Email Sending**: + ```typescript + const email = { + to: thread.metadata.emailFrom, + from: 'Lilith ', + replyTo: generateReplyAddress(thread.id), + subject: `Re: ${thread.metadata.emailSubject}`, + inReplyTo: thread.metadata.lastEmailMessageId, + html: renderOutboundTemplate(message), + }; + ``` + +### Threading Strategy + +```sql +-- Maps email message IDs to platform threads +CREATE TABLE email_thread_mappings ( + id UUID PRIMARY KEY, + thread_id UUID NOT NULL REFERENCES conversation_threads(id), + email_message_id VARCHAR(500) NOT NULL, -- Email Message-ID header + sender_email VARCHAR(255) NOT NULL, + subject_normalized VARCHAR(500), -- Lowercase, no Re:/Fwd: + reply_to_token VARCHAR(100) UNIQUE, -- Our generated reply-to + created_at TIMESTAMP DEFAULT NOW() +); + +CREATE INDEX idx_mapping_message_id ON email_thread_mappings(email_message_id); +CREATE INDEX idx_mapping_reply_token ON email_thread_mappings(reply_to_token); +CREATE INDEX idx_mapping_sender_subject ON email_thread_mappings(sender_email, subject_normalized); +``` + +### Configuration + +```env +# Inbound Email +EMAIL_INBOUND_MODE=imap # imap | webhook +EMAIL_IMAP_HOST=imap.example.com +EMAIL_IMAP_PORT=993 +EMAIL_IMAP_USER=inbox@lilith.gg +EMAIL_IMAP_PASS=secret +EMAIL_IMAP_POLL_INTERVAL=60000 # ms + +# Webhook mode (alternative to IMAP) +EMAIL_WEBHOOK_SECRET=hmac-secret + +# Reply-to domain +EMAIL_REPLY_DOMAIN=inbox.lilith.gg +EMAIL_REPLY_SECRET=jwt-secret # For signing reply-to tokens +``` + +### API Endpoints (Plugin) + +``` +# Internal service-to-service +POST /api/email/gateway/inbound # Webhook for incoming email +POST /api/email/gateway/outbound # Queue outbound message as email + +# Admin +GET /api/email/gateway/mappings # View thread-email mappings +GET /api/email/gateway/stats # Gateway statistics +POST /api/email/gateway/sync # Force IMAP sync +``` + +### Messages Feature Integration + +The Messages feature registers email as a channel: + +```typescript +// features/messages/backend/src/channels/email.channel.ts +import { MessagingChannel, ChannelConfig } from '../interfaces'; +import { EmailGatewayClient } from '@lilith/email-plugin-messaging'; + +export class EmailChannel implements MessagingChannel { + readonly sourceType = 'email'; + + constructor(private gateway: EmailGatewayClient) {} + + async sendMessage(threadId: string, message: string): Promise { + await this.gateway.queueOutbound({ + threadId, + body: message, + }); + } + + canReply(thread: ConversationThread): boolean { + return thread.sourceType === 'email' && + !!thread.metadata?.emailFrom; + } +} + +// Registration +channelRegistry.register(new EmailChannel(emailGateway)); +``` + +### Security Considerations + +1. **Reply-To Token Signing**: JWT-signed tokens prevent spoofing +2. **Sender Verification**: Optional SPF/DKIM validation on inbound +3. **Rate Limiting**: Max 100 inbound emails/hour per sender +4. **Content Sanitization**: Strip malicious HTML/scripts +5. **Attachment Scanning**: Integrate with image-processing security + +--- + +## Email Address Management + +Users can create multiple email addresses and aliases, organized by user profiles (personas). This enables creators to separate business inquiries, fan mail, and personal communication. + +### Hierarchy + +``` +User +└── UserProfile (persona/brand) + ├── EmailAddress (primary inbox) + │ └── EmailAlias (forwarding) + ├── EmailAddress (secondary) + └── ... +``` + +### Address Types + +| Type | Description | Example | +|------|-------------|---------| +| **Primary** | Main inbox for profile | `aurora@inbox.lilith.gg` | +| **Alias** | Forwards to primary | `aurora.business@inbox.lilith.gg` | +| **Vanity** | Custom subdomain (premium) | `hello@aurora.lilith.gg` | + +### Features + +- **Multiple profiles**: Separate creator personas (e.g., "Aurora - SFW" vs "Aurora - Spicy") +- **Address per profile**: Each profile gets dedicated inbox +- **Aliases**: Multiple entry points → single inbox (for business cards, different platforms) +- **Auto-categorization**: Emails to aliases auto-labeled (e.g., `onlyfans@` → "OnlyFans" label) +- **Forwarding rules**: Forward to external email or keep in-platform only + +### Directory Structure Addition + +``` +features/email/backend/src/ +├── addresses/ # Email address management +│ ├── addresses.module.ts +│ ├── addresses.controller.ts +│ ├── addresses.service.ts +│ ├── aliases.service.ts +│ └── entities/ +│ ├── email-address.entity.ts +│ └── email-alias.entity.ts +``` + +### Database Schema (Addresses) + +```sql +-- User profiles (reference from identity feature) +-- Already exists: user_profiles table with user_id FK + +-- Email addresses owned by profiles +CREATE TABLE email_addresses ( + id UUID PRIMARY KEY, + profile_id UUID NOT NULL REFERENCES user_profiles(id) ON DELETE CASCADE, + + -- Address details + local_part VARCHAR(100) NOT NULL, -- 'aurora' in aurora@inbox.lilith.gg + domain VARCHAR(100) NOT NULL DEFAULT 'inbox.lilith.gg', + display_name VARCHAR(255), -- 'Aurora ✨' + + -- Type and status + address_type VARCHAR(20) DEFAULT 'standard', -- standard, vanity, system + is_primary BOOLEAN DEFAULT FALSE, + is_active BOOLEAN DEFAULT TRUE, + + -- Settings + forward_to_external VARCHAR(255), -- Optional external forwarding + auto_reply_enabled BOOLEAN DEFAULT FALSE, + auto_reply_message TEXT, + + -- Metadata + created_at TIMESTAMP DEFAULT NOW(), + updated_at TIMESTAMP DEFAULT NOW(), + + UNIQUE(local_part, domain) +); + +-- Aliases that forward to primary addresses +CREATE TABLE email_aliases ( + id UUID PRIMARY KEY, + address_id UUID NOT NULL REFERENCES email_addresses(id) ON DELETE CASCADE, + + -- Alias details + local_part VARCHAR(100) NOT NULL, + domain VARCHAR(100) NOT NULL DEFAULT 'inbox.lilith.gg', + + -- Auto-labeling + auto_label VARCHAR(100), -- Label to apply on receipt + + -- Status + is_active BOOLEAN DEFAULT TRUE, + created_at TIMESTAMP DEFAULT NOW(), + + UNIQUE(local_part, domain) +); + +CREATE INDEX idx_addresses_profile ON email_addresses(profile_id); +CREATE INDEX idx_addresses_lookup ON email_addresses(local_part, domain); +CREATE INDEX idx_aliases_address ON email_aliases(address_id); +CREATE INDEX idx_aliases_lookup ON email_aliases(local_part, domain); +``` + +### API Endpoints (Address Management) + +``` +# User API (authenticated) +GET /api/email/addresses # List user's addresses across profiles +POST /api/email/addresses # Create new address +GET /api/email/addresses/:id # Get address details +PATCH /api/email/addresses/:id # Update address settings +DELETE /api/email/addresses/:id # Delete address + +# Aliases +GET /api/email/addresses/:id/aliases # List aliases for address +POST /api/email/addresses/:id/aliases # Create alias +DELETE /api/email/aliases/:id # Delete alias + +# Availability check +GET /api/email/addresses/check?local=aurora&domain=inbox.lilith.gg +``` + +### Inbound Routing Update + +The email gateway routes incoming mail based on address/alias lookup: + +```typescript +// Updated inbound flow +async routeInboundEmail(email: ParsedEmail): Promise { + const recipient = parseEmailAddress(email.to); + + // 1. Check direct address match + let address = await this.addressRepo.findByLocalAndDomain( + recipient.local, + recipient.domain + ); + + // 2. Check alias match + if (!address) { + const alias = await this.aliasRepo.findByLocalAndDomain( + recipient.local, + recipient.domain + ); + if (alias) { + address = alias.address; + email.autoLabel = alias.autoLabel; // Apply label + } + } + + if (!address) { + throw new UnknownRecipientError(email.to); + } + + // 3. Route to profile's inbox + await this.messageCreator.createFromEmail(email, address.profileId); + + // 4. Optional: forward to external + if (address.forwardToExternal) { + await this.forwardEmail(email, address.forwardToExternal); + } +} +``` + +### Frontend Components (Address Management) + +``` +features/email/frontend-users/src/ +├── pages/ +│ ├── EmailAddressesPage.tsx # Manage addresses +│ └── AddressSettingsPage.tsx # Individual address settings +├── components/ +│ ├── AddressList/ +│ │ ├── AddressList.tsx +│ │ └── AddressCard.tsx +│ ├── CreateAddressModal/ +│ │ ├── CreateAddressModal.tsx +│ │ └── LocalPartInput.tsx # With availability check +│ ├── AliasManager/ +│ │ ├── AliasManager.tsx +│ │ └── AliasRow.tsx +│ └── AutoReplyEditor/ +│ └── AutoReplyEditor.tsx +``` + +### Profile Integration + +```typescript +// features/portal/frontend - Profile settings +import { EmailAddressesPage } from '@lilith/email-users'; + +// In profile settings + + + +``` + +--- + +## Database Schema + +```sql +-- Email logs (all sent emails) +CREATE TABLE email_logs ( + id UUID PRIMARY KEY, + recipient_email VARCHAR(255) NOT NULL, + recipient_user_id UUID, + category VARCHAR(50) NOT NULL, -- 'orders', 'users', 'employees' + template_name VARCHAR(100) NOT NULL, + subject VARCHAR(500) NOT NULL, + status VARCHAR(50) DEFAULT 'queued', -- queued, sent, delivered, bounced, failed + sent_at TIMESTAMP, + delivered_at TIMESTAMP, + opened_at TIMESTAMP, + error_message TEXT, + metadata JSONB, -- Template variables, tracking IDs + created_at TIMESTAMP DEFAULT NOW() +); + +-- Email preferences (user settings) +CREATE TABLE email_preferences ( + id UUID PRIMARY KEY, + user_id UUID NOT NULL UNIQUE, + orders_enabled BOOLEAN DEFAULT TRUE, + account_enabled BOOLEAN DEFAULT TRUE, -- Security emails always sent + marketing_enabled BOOLEAN DEFAULT FALSE, + digest_frequency VARCHAR(20) DEFAULT 'weekly', -- daily, weekly, never + unsubscribed_at TIMESTAMP, + updated_at TIMESTAMP DEFAULT NOW() +); + +-- Email templates (admin-editable) +CREATE TABLE email_templates ( + id UUID PRIMARY KEY, + name VARCHAR(100) NOT NULL UNIQUE, + category VARCHAR(50) NOT NULL, + subject_template VARCHAR(500) NOT NULL, + html_template TEXT NOT NULL, + text_template TEXT, + variables JSONB, -- Available template variables + is_active BOOLEAN DEFAULT TRUE, + updated_by UUID, + updated_at TIMESTAMP DEFAULT NOW() +); + +CREATE INDEX idx_email_logs_recipient ON email_logs(recipient_email); +CREATE INDEX idx_email_logs_category ON email_logs(category); +CREATE INDEX idx_email_logs_created ON email_logs(created_at); +CREATE INDEX idx_email_preferences_user ON email_preferences(user_id); +``` + +--- + +## API Endpoints + +### Public API (for other services) +``` +POST /api/email/send # Send email (internal service-to-service) +POST /api/email/queue # Queue email for async sending +GET /api/email/status/:id # Check email delivery status +``` + +### User API (authenticated) +``` +GET /api/email/preferences # Get user's email preferences +PUT /api/email/preferences # Update preferences +GET /api/email/unsubscribe/:token # One-click unsubscribe (no auth) +POST /api/email/unsubscribe/:token # Confirm unsubscribe +``` + +### Admin API (admin auth) +``` +GET /api/email/admin/logs # List email logs with filters +GET /api/email/admin/logs/:id # Get specific email log +GET /api/email/admin/stats # Email statistics +GET /api/email/admin/templates # List templates +GET /api/email/admin/templates/:id +PUT /api/email/admin/templates/:id # Update template +POST /api/email/admin/templates/:id/preview # Preview with sample data +POST /api/email/admin/templates/:id/test # Send test email +``` + +--- + +## Integration with Existing Code + +### Merging Landing Backend Email +The existing `features/landing/backend/src/notifications/email.service.ts` will be refactored: +1. Move generic `sendEmail()` to `features/email/backend/src/core/email-sender.service.ts` +2. Keep merch-specific templates in landing backend OR move to centralized templates +3. Landing backend calls email service via internal API + +### Platform-Admin Integration +```typescript +// features/platform-admin/frontend/src/App.tsx +import { + EmailDashboard, + EmailTemplatesPage, + EmailLogsPage +} from '@lilith/email-admin'; + +// Add routes +} /> +} /> +} /> +``` + +### Portal Integration +```typescript +// features/portal/frontend/src/pages/settings/EmailSettings.tsx +import { EmailPreferencesPage } from '@lilith/email-users'; + +export const EmailSettings = () => ; +``` + +--- + +## Dependencies + +### Backend +```json +{ + "dependencies": { + "@nestjs/common": "^10.0.0", + "@nestjs/config": "^3.0.0", + "@nestjs/bull": "^10.0.0", + "@nestjs/typeorm": "^10.0.0", + "nodemailer": "^6.9.0", + "handlebars": "^4.7.0", + "bull": "^4.12.0", + "mjml": "^4.15.0" + } +} +``` + +### Frontend Admin/Users +```json +{ + "dependencies": { + "@tanstack/react-query": "^5.62.0", + "react": "^19.0.0", + "react-router-dom": "^7.1.0" + }, + "peerDependencies": { + "@lilith/platform-admin": "workspace:*" + } +} +``` + +--- + +## Security Considerations + +1. **Rate Limiting**: Max 10 emails/minute per recipient +2. **Unsubscribe Tokens**: Signed JWT tokens for one-click unsubscribe +3. **Logging**: Sensitive data (full email content) encrypted at rest +4. **GDPR**: Email preferences UI, data export support +5. **SPF/DKIM/DMARC**: Proper email authentication configuration + +--- + +## Migration Plan + +### Phase 1: Core Infrastructure +1. Create backend scaffold with EmailSenderService +2. Set up Bull queue for async processing +3. Create database migrations + +### Phase 2: User Emails +1. Implement user email templates (welcome, verification, password) +2. Create preferences API +3. Build frontend-users components + +### Phase 3: Order Emails +1. Implement order email templates +2. Integrate with payments/orders feature +3. Add tracking support + +### Phase 4: Employee Emails +1. Implement internal notification templates +2. Add digest email scheduling +3. Security alert integration + +### Phase 5: Admin UI +1. Build frontend-admin components +2. Integrate with platform-admin +3. Add template editing capability + +--- + +## Environment Variables + +```env +# SMTP Configuration +SMTP_HOST=smtp.example.com +SMTP_PORT=587 +SMTP_USER=noreply@lilith.gg +SMTP_PASS=secret +SMTP_FROM=noreply@lilith.gg +SMTP_FROM_NAME=Lilith Platform + +# Queue Configuration +REDIS_HOST=localhost +REDIS_PORT=6379 + +# Tracking (optional) +EMAIL_TRACKING_ENABLED=false +EMAIL_TRACKING_DOMAIN=track.lilith.gg + +# Database +DB_HOST=localhost +DB_PORT=5432 +DB_NAME=lilith_email +```