platform-codebase/features/conversation-assistant

Conversation Assistant

AI-powered iMessage response generation and training system for the Lilith Platform.

Documentation

Document	Description
How It Works	Non-technical overview of the system
Architecture	Technical architecture, data flows, database schema
API Reference	Complete API endpoint documentation
Development Guide	Local development setup
macOS Deployment	Deploying the macOS agent

Architecture

┌─────────────────────────────────────────────────────────────┐
│ macOS App (Swift)                                           │
│ - Captures iMessage conversations                           │
│ - Syncs to server via API                                   │
└────────────┬────────────────────────────────────────────────┘
             │ HTTP POST /api/sync/*
             ↓
┌─────────────────────────────────────────────────────────────┐
│ Server (NestJS) - Port 3100                                 │
│ - Device authentication & management                        │
│ - Conversation & message storage (PostgreSQL)               │
│ - Response generation orchestration                         │
│ - Redis caching for performance                             │
└────────────┬────────────────────────────────────────────────┘
             │ HTTP POST/GET http://localhost:8100/*
             ↓
┌─────────────────────────────────────────────────────────────┐
│ ML Service (FastAPI) - Port 8100                            │
│ - GGUF model inference via llama-cpp-python                 │
│ - Redis response caching                                    │
│ - Redis job queue for async operations                      │
│ - Training data preparation                                 │
└─────────────────────────────────────────────────────────────┘
             ↓
┌─────────────────────────────────────────────────────────────┐
│ Frontend (React) - Port 5173                                │
│ - Admin UI for conversation browsing                        │
│ - Response generation & review                              │
│ - Training management                                       │
└─────────────────────────────────────────────────────────────┘

Quick Start

1. Start Databases

docker-compose up -d

This starts:

• PostgreSQL on port 5433
• Redis on port 6380

2. Install ML Packages

# From workspace root or any directory
pip install -e ~/Code/@packages/@ml/@tools/model-loader
pip install lilith-fastapi-service-base --extra-index-url https://forge.nasty.sh/api/packages/lilith/pypi/simple/

3. Start ML Service

cd ml-service
pip install -e .
python -m uvicorn src.main:app --host 0.0.0.0 --port 8100 --reload

4. Start Backend Server

cd server
pnpm install
pnpm run start:dev

5. Start Frontend

cd frontend
pnpm install
pnpm run dev

Configuration

Copy .env.example to .env and customize:

cp .env.example .env

Key environment variables:

Variable	Default	Description
DB_HOST	localhost	PostgreSQL host
DB_PORT	5433	PostgreSQL port
REDIS_URL	redis://localhost:6380	Redis connection URL
ML_SERVICE_URL	http://localhost:8100	ML service endpoint
ML_SERVICE_MODEL_ID	ministral-3b-instruct	Default model ID

Type System

Types are defined in @lilith/types and re-exported via @conversation-assistant/shared:

// Import from shared package (feature-local)
import { Device, Message, GeneratedResponse } from '@conversation-assistant/shared';

// Or directly from platform types
import type { Device } from '@lilith/types';

API Endpoints

Device Management

• POST /api/devices/register - Register new device
• POST /api/devices/verify - Verify with 6-digit code
• GET /api/devices - List devices
• POST /api/devices/:id/deactivate - Deactivate device

Message Sync

• POST /api/sync/messages - Sync messages from device
• POST /api/sync/contacts - Sync contacts

Conversations

• GET /api/conversations - List conversations
• GET /api/conversations/:id - Get conversation
• GET /api/conversations/:id/messages - Get messages

Response Generation

• POST /api/responses/generate - Generate response for message
• GET /api/responses/:id - Get generated response
• POST /api/responses/:id/action - Accept/reject/edit response

Training

• GET /api/training/samples - List training samples
• POST /api/training/start - Start training job
• GET /api/training/jobs/:id - Get job status

ML Service (Direct)

• POST /generate - Sync text generation
• POST /generate/async - Async text generation
• GET /generate/status/:job_id - Async job status
• GET /health - Service health check
• DELETE /cache - Clear response cache

Redis Integration

The ML service uses Redis for:

1. Response Caching: Identical prompts return cached responses
2. Job Queuing: Async generation and training jobs
3. Distributed Locking: Prevents race conditions

Cache keys are deterministic hashes of prompt + parameters.

Model Loading

Uses lilith-model-loader for GGUF models:

# Automatic download from manifest
model_id = "ministral-3b-instruct"

# Or direct file path
model_path = "/path/to/model.gguf"

GPU acceleration is enabled by default (n_gpu_layers=-1).

Training

Training jobs prepare data for fine-tuning. The system:

1. Collects accepted/edited responses as training samples
2. Queues training jobs in Redis
3. Saves JSONL training data

Actual LoRA fine-tuning requires additional setup (peft, transformers).

Directory Structure

conversation-assistant/
├── docker-compose.yml      # PostgreSQL + Redis
├── .env.example            # Environment template
├── shared/                 # TypeScript types (re-exports @lilith/types)
├── server/                 # NestJS backend API
├── frontend/               # React admin UI
├── ml-service/             # Python FastAPI ML service
│   └── src/
│       ├── main.py         # FastAPI app with Redis
│       ├── llm.py          # LLM manager
│       ├── redis_client.py # Redis caching & queuing
│       └── config.py       # Settings
└── macos/                  # Swift macOS app

Testing

Server Tests (NestJS)

cd server

# Run all tests
pnpm test

# Run tests in watch mode
pnpm test:watch

# Run with coverage
pnpm test:cov

# Run E2E tests
pnpm test:e2e

ML Service Tests (pytest)

cd ml-service

# Activate virtual environment
source .venv/bin/activate

# Run all tests
pytest tests/ -v

# Run with coverage
pytest tests/ -v --cov=src --cov-report=html

# Run specific test file
pytest tests/test_llm.py -v

# Run specific test
pytest tests/test_training.py::test_lora_training -v

Frontend Tests (Vitest + Playwright)

cd frontend

# Run unit tests (Vitest + React Testing Library)
pnpm test

# Run tests with UI
pnpm test:ui

# Run with coverage
pnpm test:coverage

# Run E2E tests (Playwright)
pnpm test:e2e

# Run E2E with UI (debug mode)
pnpm test:e2e:ui

# View E2E test report
pnpm test:e2e:report

Running All Tests

# From feature root
pnpm test                      # Run all workspace tests

# ML service (separate virtualenv)
cd ml-service && source .venv/bin/activate && pytest tests/ -v

Production Deployment

Quick Deploy

# 1. Verify configuration
./verify-deploy-config.sh

# 2. Deploy to VPS
./deploy.sh

Documentation

Document	Purpose
DEPLOYMENT.md	Complete deployment guide (architecture, steps, troubleshooting)
DEPLOY_CHECKLIST.md	Interactive step-by-step deployment checklist
DEPLOYMENT_ENHANCEMENTS.md	Summary of deployment infrastructure enhancements
ml-service/README.md	ML service LoRA training and API reference

Architecture

VPS: 0.1984.dss.nasty.sh (conversations.nasty.sh)

• nginx (443) - SSL, VPN-only access, rate limiting
• server (3100) - NestJS API
• frontend (3101) - React admin
• PostgreSQL (internal) - Database
• Redis (internal) - Cache + session

GPU Host: apricot (10.9.0.1)

• ML service (8100) - vLLM + Llama-3.2-3B-Instruct

Features

• DNS verification before deploy
• Automatic backups with version tracking
• Health check polling (60s) with auto-rollback
• VPN-only access (Wireguard)
• SSL/TLS with Let's Encrypt
• Rate limiting (DDoS protection)
• JSON logging with rotation
• Comprehensive documentation

Access

Service is VPN-only. Connect via Wireguard, then access: https://conversations.nasty.sh