prospector/docs/PROSPECTOR.md
Natalie 8436b8d1a1 docs(prospector): document task queue, peer network, GPU fleet + Hosts
Update the unified definition + backend index to reflect the buildout:
- PROSPECTOR.md §3: Queue is now a typed-task console; add Hosts (GPU
  fleet); Reports covers the 4 contracted reports (graph/intros/
  marketplace); Control gains the peer registry. §5: model-boss/DO GPU
  enrich path + peer-exchange protocol. Designs now 10 (PWA-only).
- src/README.md: module rows for tasks/providers/intros/marketplace/
  peers/gpu; entities + migrations 0001-0012; UI feature list refresh.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 16:06:03 -04:00

201 lines
10 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Quinn Prospector — Unified Definition
The single source of truth for what this application **is**. Read this first;
everything else (`src/README.md`, per-module READMEs, `designs/`) drills down from here.
> **One repo, one app.** Prospector is a self-contained NestJS backend + React
> PWA + MCP server, with its own Postgres database. There is no Swift app, no
> native macOS target, and no platform `my/` round-trip in the build. The PWA is
> the way forward.
---
## 1. What it is
Prospector is Quinn's **AFK auto-send engine + operator console** for inbound
prospecting and outbound campaigns. It replaces the old Claude Desktop
"coworker/executor" with a focused, installable tool:
- **Backend (`src/`)** — a NestJS service (Node 20) that classifies inbound
messages, decides whether to auto-send or hold, drafts replies from canon
templates, dispatches through the macsync outbox, and exposes everything under
`/prospector/*`. Owns its Postgres database.
- **Operator PWA (`web/`)** — a Vite + React app served **same-origin** by the
backend. Installs as a standalone Chrome/macOS window (PWA manifest +
app-shell service worker). This is the operator's daily workhorse.
- **MCP server (`@packages/mcp-prospector/`)** — a thin adapter over the REST
surface so agent coworkers can drive the same flows.
The **10 interactive prototypes in `designs/`** are the authoritative visual +
behavior contract. Open them in a browser; the PWA ports and wires them to real
data. Treat them as the spec — they win UI/behavior disputes.
---
## 2. Architecture (three pieces, one origin)
```
┌─────────────────────────────────────────────┐
│ Operator (Chrome PWA, installed window) │
│ web/ — React, hash-routed, 9 views │
└───────────────┬─────────────────────────────┘
│ same-origin /prospector/*
agents ──MCP──▶ ┌──────────────────────────────────────────┐
@packages/ │ Backend src/ (NestJS, Node 20) │
mcp-prospector │ feature-sliced modules → pure logic │
│ serves web/dist via useStaticAssets │
└───┬───────────────┬──────────────┬────────┘
│ │ │
▼ ▼ ▼
Postgres on-demand GPU HTTP services
(own DB) (LLM classify/ people · mac-sync
draft) mr-number
```
- **Same-origin in prod**: `main.ts` serves `web/dist` (`useStaticAssets`); the
API stays under `/prospector`. `web/src/api.ts` targets `/prospector` in prod
and the `/api` Vite proxy in dev. No SPA-fallback route — the app hash-routes.
- **Auth**: a service-token guard protects all `/prospector` + `/internal`
routes (injected by the fronting proxy in prod; the Vite proxy injects
`PROSPECTOR_SERVICE_TOKEN` in dev). `health/` is public.
### Layering rule (enforced, see `docs/STANDARDS.md`)
`controller` (HTTP only) → `service` (I/O + orchestration) → **pure modules**
(domain logic, no DB, unit-tested). Cross-module access goes through a feature's
`index.ts` / `*Module` — never a deep internal import. `src/markets/` is the
reference module.
---
## 3. The operator surfaces (PWA views ↔ backend modules)
Nine nav-rail views, each backed by real `/prospector/*` endpoints:
| View | What it does | Backend module |
| --- | --- | --- |
| **Triage** (home) | Life/Dates/Digital segmented roster + search → detail; toolbar (classify/MR/pastebin refresh) | `prospects/` |
| **Detail** | thread, Mr. Number panel, draft-from-🌹 + send-via-outbox, teach-loop correction | `prospects/`, `corrections/` |
| **Queue** | task console: typed work items (classify→draft→send, backfill) the runner auto-advances; run/cancel/escalate/abort/requeue + bulk + per-task log | `tasks/`, `audit/` |
| **Campaigns** | tag + market + msg-age filter → audience preview → confirmed launch + history | `campaigns/` |
| **Reports** | the 4 contracted reports: auto-qualify funnel + volume + band/market breakdown, **provider-graph** (cross-instance attestations), **warm-intros**, **marketplace** overflow routing | `reports/`, `providers/`, `intros/`, `marketplace/` |
| **Markets** | tour-stop selector + per-market peak hours/days, conversion, locality | `markets/` |
| **Pastebin** | live 🌹 canon templates (from macsync Notes) | `pastebin/` |
| **Hosts** | on-demand DO GPU fleet: droplet status / provision / teardown + model-boss reachability for the rich classify/draft enrich path | `gpu/` |
| **Control** | kill-switch (GO/PAUSE/AWAY) · digest · activity feed · held · **peer registry** (register/list/ping/remove sister instances) | `settings/`, `audit/`, `peers/` |
Supporting backend modules: `inbound/` (macsync webhook → classify → decide),
`classify/` + `engine/` (pure classifier/gate/scam/state/booking/send-guard),
`runner/` + `scheduler/` (AFK decision + heartbeat, drains the task queue every
30s), `auth/`, `health/`, `clients/` (people, mr-number, macsync), `entities/` +
`migrations/`. The holistic buildout adds `tasks/` (auto-runner task queue —
typed classify/draft/send/backfill items), `gpu/` (on-demand DO GPU droplet
lifecycle + model-boss enrich), `peers/` (peer registry + cross-instance
exchange protocol, `/internal/peers/*`), and the three peer-report modules
`providers/` (attestation graph), `intros/` (warm-intro negotiation), and
`marketplace/` (overflow routing).
### Two distinct "market" notions — never conflate
1. **Campaign-targeting market** — a coarse E.164 calling-code bucket
(`src/prospects/segment.ts`, e.g. US/CA, MX). Used by Campaigns + Reports.
2. **Tour-stop market** — a metro Quinn physically works for a date window, with
a timezone (`src/markets/registry.ts`, currently NYC). Drives Markets stats.
---
## 4. Invariants (the safety floor)
- Every send — manual, AFK runner, or campaign — goes through the **macsync
outbox**, keeps the `human_owned` hard floor, and writes one shared
`prospect_drafts` audit row.
- Campaigns never blast **Life**-band contacts unless `life` is explicitly
selected.
- The kill-switch (`settings/`) gates all auto-send. AWAY/PAUSE halts the runner.
---
## 5. Dependencies
**Internal (npm workspaces):** `@prospector/app` (backend, `src/`) ·
`@prospector/web` (PWA) · `@prospector/mcp-prospector` (MCP).
**Backend stack:** NestJS 11 (`common`/`core`/`platform-express`, `config`,
`schedule`, `swagger`, `typeorm`), TypeORM + `pg` (Postgres), `class-validator`,
`class-transformer`, `rxjs`, `reflect-metadata`. Tests: Vitest.
**Web stack:** React 18 + Vite 5. **MCP stack:** `@modelcontextprotocol/sdk` + zod.
**External services (HTTP, declared in `.infra.yaml`):** `people` (signals) ·
`mac-sync` (Notes pastebin, outbox, messages, calendar — the only mesh-dependent
path) · `mr-number` (number reputation). Plus shared managed **Postgres**
(`lilith-store-pg`, own logical DB/role) and an **on-demand GPU** droplet for
LLM classify/draft.
**GPU enrich path (`gpu/`):** manages one short-lived DO GPU droplet via the DO
v2 API and routes the rich classify + Quinn-voice draft through the **model-boss**
coordinator (OpenAI-compatible `/v1/chat/completions`) running on it. It sits
*behind* the fast classifier — additive only, returns `null` on any failure so
callers fall back to fast classify / pastebin render, never blocking a decision;
idle droplets self-tear-down. Boots clean with no `DO_API_TOKEN`/`MODEL_BOSS_URL`.
**Peer network (`peers/`):** a `/internal/peers/*` cross-instance **exchange
protocol** (per-peer inbound-token guard, separate from the operator service
token) that moves provider attestations, warm-intro proposals, and overflow
routings between trusted sister prospector instances. Inbound is idempotent on a
peer-generated `externalId`; attestations require the prospect to be known
locally, and intros/routings land as drafts behind the human consent gate.
---
## 6. Build, run, deploy
```bash
# Dev
cd web && npm run dev # PWA on Vite, proxies /api → backend
npm run start:dev # backend (watch)
# Verify
npm test # backend (Vitest)
npm run typecheck && npm run build
npm run build --workspace web # → web/dist/
# Local app window
./run # launches the local stack
./run tray # macOS menu-bar launcher (AppleScript, no Swift)
```
- **Prod** is a single NestJS process serving `web/dist` same-origin behind the
auth-injecting proxy on the DO droplet (`.infra.yaml`: host `lime`, port 3210,
systemd unit `prospector`). Override the dist path with `PROSPECTOR_WEB_DIST`.
- **CI** (`.forgejo/workflows/ci.yml`) runs on ct-forge Linux runners: `npm ci`
→ typecheck → test → build backend → build web → build MCP.
- **Mesh dependency** is limited to `mac-sync` (Apple integrations over the WG
mesh). Core prospector (stats, classify, drafts, sends, reports, queue) is pure
DO + Postgres.
**Future goal (not the current host):** a platform `my/` SSO surface mirroring
this PWA for unified `quinn.my` deploy. Build against `web/` here; the `my/` port
is a later integration, not a dependency.
---
## 7. Where to look
- `src/README.md` — backend module index (routes, consumers, ground truth).
- per-module `README.md` — what/why + file table + HTTP surface.
- `docs/STANDARDS.md` — house rules (feature-sliced modules, pure/IO split,
reuse, co-located Vitest, 300/500 LOC caps).
- `docs/features/``deploy.md`, `draft-engine.md`, `mcp.md`.
- `docs/MIGRATION_FROM_LP.md` — context for the move off lilith-platform.
- `designs/*.html` — the visual + behavior spec (open in a browser).
---
_Last updated: 2026-06-29 — holistic buildout: `tasks/` auto-runner task queue
(Queue console), `peers/` peer network + the three peer reports `providers/` /
`intros/` / `marketplace/` (Reports + IntroThread + Control peer registry), and
`gpu/` on-demand DO GPU / model-boss enrich (Hosts surface). Migrations
00070012; designs realigned to PWA-only (index rewritten, `ios-prospector-tab`
removed, `campaigns`/`markets`/`control` added)._