prospector/docs/PROSPECTOR.md
Natalie 8436b8d1a1 docs(prospector): document task queue, peer network, GPU fleet + Hosts
Update the unified definition + backend index to reflect the buildout:
- PROSPECTOR.md §3: Queue is now a typed-task console; add Hosts (GPU
  fleet); Reports covers the 4 contracted reports (graph/intros/
  marketplace); Control gains the peer registry. §5: model-boss/DO GPU
  enrich path + peer-exchange protocol. Designs now 10 (PWA-only).
- src/README.md: module rows for tasks/providers/intros/marketplace/
  peers/gpu; entities + migrations 0001-0012; UI feature list refresh.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 16:06:03 -04:00

10 KiB
Raw Blame History

Quinn Prospector — Unified Definition

The single source of truth for what this application is. Read this first; everything else (src/README.md, per-module READMEs, designs/) drills down from here.

One repo, one app. Prospector is a self-contained NestJS backend + React PWA + MCP server, with its own Postgres database. There is no Swift app, no native macOS target, and no platform my/ round-trip in the build. The PWA is the way forward.


1. What it is

Prospector is Quinn's AFK auto-send engine + operator console for inbound prospecting and outbound campaigns. It replaces the old Claude Desktop "coworker/executor" with a focused, installable tool:

  • Backend (src/) — a NestJS service (Node 20) that classifies inbound messages, decides whether to auto-send or hold, drafts replies from canon templates, dispatches through the macsync outbox, and exposes everything under /prospector/*. Owns its Postgres database.
  • Operator PWA (web/) — a Vite + React app served same-origin by the backend. Installs as a standalone Chrome/macOS window (PWA manifest + app-shell service worker). This is the operator's daily workhorse.
  • MCP server (@packages/mcp-prospector/) — a thin adapter over the REST surface so agent coworkers can drive the same flows.

The 10 interactive prototypes in designs/ are the authoritative visual + behavior contract. Open them in a browser; the PWA ports and wires them to real data. Treat them as the spec — they win UI/behavior disputes.


2. Architecture (three pieces, one origin)

                 ┌─────────────────────────────────────────────┐
                 │  Operator (Chrome PWA, installed window)     │
                 │  web/  — React, hash-routed, 9 views         │
                 └───────────────┬─────────────────────────────┘
                                 │  same-origin  /prospector/*
                                 ▼
   agents ──MCP──▶  ┌──────────────────────────────────────────┐
 @packages/         │  Backend  src/  (NestJS, Node 20)         │
 mcp-prospector     │  feature-sliced modules → pure logic      │
                    │  serves web/dist via useStaticAssets      │
                    └───┬───────────────┬──────────────┬────────┘
                        │               │              │
                        ▼               ▼              ▼
                  Postgres        on-demand GPU   HTTP services
                  (own DB)        (LLM classify/  people · mac-sync
                                   draft)          mr-number
  • Same-origin in prod: main.ts serves web/dist (useStaticAssets); the API stays under /prospector. web/src/api.ts targets /prospector in prod and the /api Vite proxy in dev. No SPA-fallback route — the app hash-routes.
  • Auth: a service-token guard protects all /prospector + /internal routes (injected by the fronting proxy in prod; the Vite proxy injects PROSPECTOR_SERVICE_TOKEN in dev). health/ is public.

Layering rule (enforced, see docs/STANDARDS.md)

controller (HTTP only) → service (I/O + orchestration) → pure modules (domain logic, no DB, unit-tested). Cross-module access goes through a feature's index.ts / *Module — never a deep internal import. src/markets/ is the reference module.


3. The operator surfaces (PWA views ↔ backend modules)

Nine nav-rail views, each backed by real /prospector/* endpoints:

View What it does Backend module
Triage (home) Life/Dates/Digital segmented roster + search → detail; toolbar (classify/MR/pastebin refresh) prospects/
Detail thread, Mr. Number panel, draft-from-🌹 + send-via-outbox, teach-loop correction prospects/, corrections/
Queue task console: typed work items (classify→draft→send, backfill) the runner auto-advances; run/cancel/escalate/abort/requeue + bulk + per-task log tasks/, audit/
Campaigns tag + market + msg-age filter → audience preview → confirmed launch + history campaigns/
Reports the 4 contracted reports: auto-qualify funnel + volume + band/market breakdown, provider-graph (cross-instance attestations), warm-intros, marketplace overflow routing reports/, providers/, intros/, marketplace/
Markets tour-stop selector + per-market peak hours/days, conversion, locality markets/
Pastebin live 🌹 canon templates (from macsync Notes) pastebin/
Hosts on-demand DO GPU fleet: droplet status / provision / teardown + model-boss reachability for the rich classify/draft enrich path gpu/
Control kill-switch (GO/PAUSE/AWAY) · digest · activity feed · held · peer registry (register/list/ping/remove sister instances) settings/, audit/, peers/

Supporting backend modules: inbound/ (macsync webhook → classify → decide), classify/ + engine/ (pure classifier/gate/scam/state/booking/send-guard), runner/ + scheduler/ (AFK decision + heartbeat, drains the task queue every 30s), auth/, health/, clients/ (people, mr-number, macsync), entities/ + migrations/. The holistic buildout adds tasks/ (auto-runner task queue — typed classify/draft/send/backfill items), gpu/ (on-demand DO GPU droplet lifecycle + model-boss enrich), peers/ (peer registry + cross-instance exchange protocol, /internal/peers/*), and the three peer-report modules providers/ (attestation graph), intros/ (warm-intro negotiation), and marketplace/ (overflow routing).

Two distinct "market" notions — never conflate

  1. Campaign-targeting market — a coarse E.164 calling-code bucket (src/prospects/segment.ts, e.g. US/CA, MX). Used by Campaigns + Reports.
  2. Tour-stop market — a metro Quinn physically works for a date window, with a timezone (src/markets/registry.ts, currently NYC). Drives Markets stats.

4. Invariants (the safety floor)

  • Every send — manual, AFK runner, or campaign — goes through the macsync outbox, keeps the human_owned hard floor, and writes one shared prospect_drafts audit row.
  • Campaigns never blast Life-band contacts unless life is explicitly selected.
  • The kill-switch (settings/) gates all auto-send. AWAY/PAUSE halts the runner.

5. Dependencies

Internal (npm workspaces): @prospector/app (backend, src/) · @prospector/web (PWA) · @prospector/mcp-prospector (MCP).

Backend stack: NestJS 11 (common/core/platform-express, config, schedule, swagger, typeorm), TypeORM + pg (Postgres), class-validator, class-transformer, rxjs, reflect-metadata. Tests: Vitest.

Web stack: React 18 + Vite 5. MCP stack: @modelcontextprotocol/sdk + zod.

External services (HTTP, declared in .infra.yaml): people (signals) · mac-sync (Notes pastebin, outbox, messages, calendar — the only mesh-dependent path) · mr-number (number reputation). Plus shared managed Postgres (lilith-store-pg, own logical DB/role) and an on-demand GPU droplet for LLM classify/draft.

GPU enrich path (gpu/): manages one short-lived DO GPU droplet via the DO v2 API and routes the rich classify + Quinn-voice draft through the model-boss coordinator (OpenAI-compatible /v1/chat/completions) running on it. It sits behind the fast classifier — additive only, returns null on any failure so callers fall back to fast classify / pastebin render, never blocking a decision; idle droplets self-tear-down. Boots clean with no DO_API_TOKEN/MODEL_BOSS_URL.

Peer network (peers/): a /internal/peers/* cross-instance exchange protocol (per-peer inbound-token guard, separate from the operator service token) that moves provider attestations, warm-intro proposals, and overflow routings between trusted sister prospector instances. Inbound is idempotent on a peer-generated externalId; attestations require the prospect to be known locally, and intros/routings land as drafts behind the human consent gate.


6. Build, run, deploy

# Dev
cd web && npm run dev          # PWA on Vite, proxies /api → backend
npm run start:dev              # backend (watch)

# Verify
npm test                       # backend (Vitest)
npm run typecheck && npm run build
npm run build --workspace web  # → web/dist/

# Local app window
./run                          # launches the local stack
./run tray                     # macOS menu-bar launcher (AppleScript, no Swift)
  • Prod is a single NestJS process serving web/dist same-origin behind the auth-injecting proxy on the DO droplet (.infra.yaml: host lime, port 3210, systemd unit prospector). Override the dist path with PROSPECTOR_WEB_DIST.
  • CI (.forgejo/workflows/ci.yml) runs on ct-forge Linux runners: npm ci → typecheck → test → build backend → build web → build MCP.
  • Mesh dependency is limited to mac-sync (Apple integrations over the WG mesh). Core prospector (stats, classify, drafts, sends, reports, queue) is pure DO + Postgres.

Future goal (not the current host): a platform my/ SSO surface mirroring this PWA for unified quinn.my deploy. Build against web/ here; the my/ port is a later integration, not a dependency.


7. Where to look

  • src/README.md — backend module index (routes, consumers, ground truth).
  • per-module README.md — what/why + file table + HTTP surface.
  • docs/STANDARDS.md — house rules (feature-sliced modules, pure/IO split, reuse, co-located Vitest, 300/500 LOC caps).
  • docs/features/deploy.md, draft-engine.md, mcp.md.
  • docs/MIGRATION_FROM_LP.md — context for the move off lilith-platform.
  • designs/*.html — the visual + behavior spec (open in a browser).

Last updated: 2026-06-29 — holistic buildout: tasks/ auto-runner task queue (Queue console), peers/ peer network + the three peer reports providers/ / intros/ / marketplace/ (Reports + IntroThread + Control peer registry), and gpu/ on-demand DO GPU / model-boss enrich (Hosts surface). Migrations 00070012; designs realigned to PWA-only (index rewritten, ios-prospector-tab removed, campaigns/markets/control added).