274 lines
17 KiB
Markdown
274 lines
17 KiB
Markdown
# quinn.api Consolidation Plan
|
||
|
||
**Goal**: Collapse four overlapping backends (`my/backend-api`, `admin/backend-api`, `provider-website/data-api`, `provider-website/backend-api`) into a single FSD-layered service (`@features/api`), with `messages/backend-user` kept adjacent (separate process, shared entity layer).
|
||
|
||
**Non-goal**: Merging `messages/backend-user` into the same process. Its LISTEN/NOTIFY + mail stack warrants isolation. It will consume shared entities via HTTP **or** by depending on `@quinn/api` as a library (decided at Phase 6).
|
||
|
||
---
|
||
|
||
## Guiding Principles
|
||
|
||
1. **Entities are the centralization.** One table per domain concept, owned by one `entities/<noun>/` folder. All surfaces read through its barrel.
|
||
2. **Features are verbs.** Cross-entity orchestration (e.g. `tour-planning` reads tour + booking + travel) lives in `features/`, never in surfaces, never in entities.
|
||
3. **Surfaces are thin.** A surface file mounts feature/entity routers with surface-specific auth + shaping. Target: <80 lines per surface file.
|
||
4. **Dependency direction is enforced by `dependency-cruiser`.** CI fails on violations. No exceptions.
|
||
5. **Migrate one entity at a time.** Each entity PR is independently deployable: new entity folder, old backend temporarily proxies to quinn.api, then old routes delete.
|
||
|
||
---
|
||
|
||
## Target Architecture
|
||
|
||
```
|
||
@features/api/
|
||
├── package.json @quinn/api
|
||
├── .dependency-cruiser.cjs boundary enforcement
|
||
├── tsconfig.json
|
||
└── src/
|
||
├── app/ composition root
|
||
│ ├── server.ts
|
||
│ ├── config.ts
|
||
│ └── middleware/
|
||
│ ├── auth.ts service token + SSO cookie
|
||
│ ├── cors.ts
|
||
│ ├── errors.ts
|
||
│ └── rate-limit.ts
|
||
├── surfaces/ consumer UIs
|
||
│ ├── my/ my.transquinnftw.com (SSO or service token)
|
||
│ ├── admin/ admin.transquinnftw.com (SSO + admin claim)
|
||
│ ├── www/ transquinnftw.com (public, read-only projections)
|
||
│ └── public/ webhooks, contact form (unauthenticated)
|
||
├── features/ cross-entity use cases
|
||
│ ├── tour-planning/
|
||
│ ├── client-intake/
|
||
│ ├── blog-publishing/
|
||
│ ├── booking-confirmation/
|
||
│ ├── photo-protection/
|
||
│ └── platform-sync/
|
||
├── entities/ one noun = one folder = one table
|
||
│ ├── client/ ✅ pilot landed
|
||
│ ├── tour/
|
||
│ ├── booking/
|
||
│ ├── travel/
|
||
│ ├── platform/
|
||
│ ├── credential/
|
||
│ ├── financial-record/
|
||
│ ├── project/
|
||
│ ├── task/
|
||
│ ├── content-post/ NEW — blog articles
|
||
│ ├── journal-entry/ NEW — imported from quinn.my journal
|
||
│ ├── roster-member/
|
||
│ ├── roster-application/
|
||
│ ├── photo/
|
||
│ ├── reminder/
|
||
│ ├── key-event/
|
||
│ └── inspiration/
|
||
└── shared/ framework-agnostic primitives
|
||
├── db/ sqlite singleton, migrations runner
|
||
├── http/ errors, fetch helpers
|
||
├── mail/ nodemailer wrapper (from my + admin + provider-website)
|
||
├── ical/ ical-utils + caldav client (from my)
|
||
├── totp/ TOTP helpers (from my + admin)
|
||
└── logger.ts
|
||
```
|
||
|
||
### Dependency rules (CI-enforced)
|
||
|
||
| Layer | May import from |
|
||
|-------|----------------|
|
||
| `app` | anything |
|
||
| `surfaces/*` | `features`, `entities` (via barrel), `shared` — NEVER another surface |
|
||
| `features/*` | `entities` (via barrel), `shared` — NEVER another feature, NEVER surfaces |
|
||
| `entities/*` | `shared`, other entities (via barrel only) |
|
||
| `shared` | `shared` only |
|
||
|
||
---
|
||
|
||
## Entity Inventory — Source → Target
|
||
|
||
Mapped from current filesystem verification (`ls` on each backend's `src/db/` and `src/routes/`):
|
||
|
||
| Entity | Current owners | Becomes |
|
||
|--------|---------------|---------|
|
||
| `client` | `my/backend-api/db/schema-clients.ts` + `routes/clients.ts`, `messages/backend-user/api/client-intel.ts` | `entities/client/` ✅ **pilot done** |
|
||
| `booking` | `my/backend-api/db/schema-bookings.ts` + `routes/bookings.ts` | `entities/booking/` |
|
||
| `tour` | `my/backend-api/routes/touring-public.ts`, `provider-website/data-api/seed-tour.ts` | `entities/tour/` |
|
||
| `travel` | `my/backend-api/db/schema-travel.ts`, `caldav-travel-bridge.ts` | `entities/travel/` |
|
||
| `calendar` | `my/backend-api/db/schema-calendar.ts`, `caldav-sync.ts`, `calendar-sync-worker.ts` | `entities/calendar-event/` (+ `features/calendar-sync/`) |
|
||
| `platform` | `my/backend-api/db/schema-platforms.ts` + `routes/platforms-data.ts`, `admin/backend-api/routes/*` | `entities/platform/` |
|
||
| `credential` | `my/backend-api/db/schema-credentials.ts` + `routes/credentials.ts` + `credentials-inference.ts` | `entities/credential/` + `features/credential-inference/` |
|
||
| `financial` | `my/backend-api/db/schema-financials.ts` + `routes/financials.ts` | `entities/financial-record/` |
|
||
| `project` | `my/backend-api/db/schema-projects.ts` + `routes/projects.ts` | `entities/project/` |
|
||
| `task` | `my/backend-api/db/schema-tasks.ts` + `routes/tasks-data.ts` | `entities/task/` |
|
||
| `roster` | `my/backend-api/db/schema-roster.ts` + `routes/roster*.ts` (6 route files) | `entities/roster-member/` + `entities/roster-application/` + `features/roster-intake/` |
|
||
| `photo` | `my/backend-api/db/schema-photos.ts` + `routes/photo-protection.ts`, `admin/backend-api/photos.ts` | `entities/photo/` + `features/photo-protection/` |
|
||
| `reminder` | `my/backend-api/db/schema-reminders.ts` + `routes/reminders.ts` + `reminders-parse.ts` + `reminders-sync.ts` | `entities/reminder/` + `features/reminder-sync/` |
|
||
| `key-event` | `my/backend-api/routes/key-events.ts` | `entities/key-event/` |
|
||
| `inspiration` | `my/backend-api/routes/inspiration.ts` | `entities/inspiration/` |
|
||
| `contact` | `my/backend-api/routes/contact.ts` + `contact-templates.ts` + `contact-outbox.ts`, `provider-website/backend-api/` | `entities/contact-submission/` + `features/contact-outbox/` |
|
||
| `cms-page` | `admin/backend-api/cms-handler.ts`, `provider-website/data-api/serialize.ts` | `entities/cms-page/` + `features/cms-publishing/` |
|
||
| `content-post` | **NEW** — sourced from journal import | `entities/content-post/` + `features/blog-publishing/` |
|
||
| `journal-entry` | **NEW** — imported from `desktop/*.md` | `entities/journal-entry/` |
|
||
| `auth-session` | `my/backend-api/db/schema-auth.ts` + `routes/auth.ts` + `routes/totp.ts` + `session-key-store.ts`, admin/totp.ts | `entities/auth-session/` + `shared/totp/` |
|
||
| `device-link` | `my/backend-api/routes/device-link.ts` | `entities/device-link/` |
|
||
|
||
---
|
||
|
||
## Surface Inventory
|
||
|
||
Each surface is ~1 file per entity + a `surfaces/<name>/index.ts` barrel that mounts them.
|
||
|
||
### `surfaces/my/` — day-to-day ops
|
||
Auth: SSO cookie OR service token.
|
||
Routes: clients, bookings, tour, travel, calendar, platforms, credentials, financials, projects, tasks, roster, photos, reminders, key-events, inspiration, contact, auth, device-link.
|
||
|
||
### `surfaces/admin/` — public-site CMS
|
||
Auth: SSO + admin claim.
|
||
Routes: cms-pages, photos (approval queue), platforms (seed/moderation), content-posts (publish/unpublish).
|
||
|
||
### `surfaces/www/` — public projections
|
||
Auth: none, read-only, `{ published: true }` filter applied in routers.
|
||
Routes: tour (current city + dates), content-posts (blog), cms-pages, roster-applications (submit-only POST).
|
||
|
||
### `surfaces/public/` — unauthenticated writes
|
||
Auth: per-route (hCaptcha, webhook signatures).
|
||
Routes: contact-form POST, waitlist signups, stripe webhooks.
|
||
|
||
---
|
||
|
||
## Migration Phases
|
||
|
||
Each phase is one PR. Old backends remain live; quinn.api grows alongside. **No big-bang cutover.**
|
||
|
||
### Phase 0 — Skeleton (DONE)
|
||
- [x] `@features/api/` with `app/`, one entity (`client`), one surface (`my`), dependency-cruiser, config, auth middleware, error handler.
|
||
- [x] Pilot demonstrates the pattern end-to-end.
|
||
|
||
### Phase 1 — Infrastructure hardening
|
||
- [ ] `shared/mail/` ported from `my/backend-api/mailer.ts` (nodemailer, SMTP config).
|
||
- [ ] `shared/totp/` ported from `my/backend-api/session-key-store.ts` + `routes/totp.ts`.
|
||
- [ ] `shared/ical/` ported from `my/backend-api/ical-utils.ts` + `icloud-client.ts`.
|
||
- [ ] SSO cookie validator added to `app/middleware/auth.ts` (subrequest to `sso.transquinnftw.com`, matching `my/backend-api/middleware/`).
|
||
- [ ] Rate limit middleware (token bucket, per-surface config).
|
||
- [ ] `bun test` harness + one integration test hitting `/my/clients` end-to-end against tmpfs SQLite.
|
||
- [ ] `./run` script entries: `dev:api`, `dev:api:stop`, `dev:api:status`.
|
||
|
||
### Phase 2 — Entity migration wave 1 (standalone, no cross-cut)
|
||
Entities that don't interact with others. Ship as independent PRs:
|
||
- [ ] `entities/task/` + `surfaces/my/tasks.ts`
|
||
- [ ] `entities/reminder/` + `surfaces/my/reminders.ts`
|
||
- [ ] `entities/inspiration/` + `surfaces/my/inspiration.ts`
|
||
- [ ] `entities/key-event/` + `surfaces/my/key-events.ts`
|
||
- [ ] `entities/credential/` + `surfaces/my/credentials.ts`
|
||
- [ ] `entities/device-link/` + `surfaces/my/device-link.ts`
|
||
- [ ] `entities/financial-record/` + `surfaces/my/financials.ts`
|
||
- [ ] `entities/project/` + `surfaces/my/projects.ts`
|
||
|
||
Strategy per entity:
|
||
1. Create `entities/<x>/` with schema, repo, types, barrel.
|
||
2. Mirror the routes into `surfaces/my/<x>.ts`.
|
||
3. Port data: write a one-shot script `scripts/migrate-<x>.ts` that copies rows from the old SQLite to the new one. Dry-run, then commit.
|
||
4. Proxy the old route to quinn.api (nginx `proxy_pass` or a Hono `proxy()` in the old backend) — removes dual-write risk.
|
||
5. After one week live: delete old route + schema file.
|
||
|
||
### Phase 3 — Entity migration wave 2 (cross-entity concepts)
|
||
- [ ] `entities/booking/` + `entities/calendar-event/` + `features/calendar-sync/` (owns caldav worker) + `surfaces/my/bookings.ts`.
|
||
- [ ] `entities/tour/` + `entities/travel/` + `features/tour-planning/` + `surfaces/my/tour.ts` + `surfaces/www/tour.ts` (public view). **This is the blueprint feature.**
|
||
- [ ] `entities/platform/` + `features/platform-sync/` + `surfaces/my/platforms.ts` + `surfaces/admin/platforms.ts`.
|
||
- [ ] `entities/contact-submission/` + `features/contact-outbox/` + `surfaces/public/contact.ts` (replaces `provider-website/backend-api` entirely).
|
||
- [ ] `entities/roster-member/` + `entities/roster-application/` + `features/roster-intake/` + `surfaces/my/roster.ts` + `surfaces/www/roster-apply.ts`.
|
||
|
||
### Phase 4 — Admin + CMS migration
|
||
- [ ] `entities/cms-page/` + `features/cms-publishing/`.
|
||
- [ ] `surfaces/admin/cms.ts` (replaces `admin/backend-api/cms-handler.ts`).
|
||
- [ ] `surfaces/www/cms.ts` reads published pages (replaces `provider-website/data-api/serialize.ts`).
|
||
- [ ] `entities/photo/` + `features/photo-protection/` + `surfaces/my/photos.ts` + `surfaces/admin/photos.ts`.
|
||
- [ ] Deprecate `admin/backend-api` and `provider-website/data-api`.
|
||
|
||
### Phase 5 — Blog pipeline (the user ask)
|
||
- [ ] `entities/journal-entry/` — imported from `users/transquinnftw/desktop/*.md`.
|
||
- [ ] One-shot importer: `features/journal-import/` reads markdown, extracts frontmatter, stores as `journal_entries` rows.
|
||
- [ ] `entities/content-post/` — fields: `slug`, `title`, `body_md`, `body_html`, `excerpt`, `hero_image`, `published_at`, `journal_source_id`.
|
||
- [ ] `features/blog-publishing/` — orchestrates: journal-entry → draft content-post → review queue → publish.
|
||
- [ ] `surfaces/admin/content-posts.ts` — CRUD for drafts, preview, publish/unpublish.
|
||
- [ ] `surfaces/www/blog.ts` — public `/blog`, `/blog/:slug`, `/blog/rss.xml` with `{ published: true }`.
|
||
- [ ] Frontend: new `/blog` route in `provider-website/frontend-public/src/pages/BlogPage.tsx`, `BlogPostPage.tsx`.
|
||
- [ ] `quinn.admin` frontend: new blog editor page (markdown editor + preview + publish button) under `admin/frontend-public/src/pages/`.
|
||
|
||
### Phase 6 — quinn.m integration decision
|
||
Choose one based on data access patterns observed during Phases 2–5:
|
||
|
||
**Option A — HTTP consumer**
|
||
`messages/backend-user` stays separate, calls quinn.api over HTTP for `clients`/`contacts`. Lowest coupling, one extra network hop per request.
|
||
|
||
**Option B — Library consumer**
|
||
`messages/backend-user` adds `@quinn/api` as a workspace dependency and imports `entities/client` directly, sharing the SQLite handle. Zero network hop, but process isolation lost for reads.
|
||
|
||
**Recommendation pending:** if quinn.m reads clients <100x/day → Option A. If it reads on every inbound message → Option B with a read-only DB handle.
|
||
|
||
### Phase 7 — Cleanup
|
||
- [ ] Delete `my/backend-api/`, `admin/backend-api/`, `provider-website/data-api/`, `provider-website/backend-api/`.
|
||
- [ ] Update `deployments/@domains/quinn.my/deploy.sh` + `quinn.admin/deploy.sh` + `quinn.www/deploy.sh` to point at quinn.api.
|
||
- [ ] Update nginx configs: single upstream for `/api/*` across all quinn domains.
|
||
- [ ] Update `CLAUDE.md` files — collapse the four-backend narrative into one.
|
||
- [ ] Update `mcp-server` (currently in `my/mcp-server/`) to call quinn.api directly; move to `@features/api/mcp-server/` or promote to `@packages/@quinn/mcp-server` once stable.
|
||
|
||
---
|
||
|
||
## Deployment Topology (post-migration)
|
||
|
||
| Domain | Frontend | Backend |
|
||
|--------|----------|---------|
|
||
| `my.transquinnftw.com` | existing React SPA | `quinn.api` (`/my/*`) via SSO gate |
|
||
| `admin.transquinnftw.com` | existing React SPA | `quinn.api` (`/admin/*`) via SSO + admin claim |
|
||
| `transquinnftw.com` | existing Vite site | `quinn.api` (`/www/*`, `/public/*`) no auth |
|
||
| `m.transquinnftw.com` | existing React SPA | `messages/backend-user` (unchanged) + reads `quinn.api` for clients |
|
||
|
||
One quinn.api process on vps-0, four nginx vhosts proxy to it with surface-specific auth.
|
||
|
||
---
|
||
|
||
## Tooling & Guardrails
|
||
|
||
- **`dependency-cruiser`** runs in CI — blocks cross-surface, cross-feature, and entity-internal reach-ins.
|
||
- **`tsc --noEmit`** with `strict`, `noUncheckedIndexedAccess`, `exactOptionalPropertyTypes`, `verbatimModuleSyntax`.
|
||
- **Zod schemas** at every surface boundary. No `any`. Repo functions return hydrated domain types, never raw rows.
|
||
- **Integration tests per entity** — spin tmpfs SQLite, run migrations, hit the surface, assert response shape. At least 1 happy-path + 1 error-path per entity.
|
||
- **One SQLite file** at `/opt/quinn-api/data/quinn-api.db`. WAL mode. Single writer, many readers. Daily backup via restic to black.
|
||
|
||
---
|
||
|
||
## Risks & Mitigations
|
||
|
||
| Risk | Mitigation |
|
||
|------|-----------|
|
||
| Big-bang migration corrupts data | Each entity migrated independently with a dry-run script + one-week proxy period before old routes delete |
|
||
| Single process = single failure domain | Systemd restart; SQLite WAL + daily restic backup; quinn.m stays isolated so messaging survives a quinn.api outage |
|
||
| SQLite write contention as surfaces multiply | WAL mode + `busy_timeout = 5000`; move to Postgres only if p99 write latency exceeds 50ms |
|
||
| Auth complexity across 4 surfaces | One `app/middleware/auth.ts` with surface-scoped middlewares (`ssoRequired`, `adminRequired`, `serviceToken`, `publicOnly`); applied in surface barrels, not inside entities |
|
||
| `dependency-cruiser` rules too strict | Rules are editable per-case in `.dependency-cruiser.cjs` — but adding an exception requires a PR comment justifying why the boundary should bend |
|
||
|
||
---
|
||
|
||
## Timeline (rough)
|
||
|
||
| Phase | Effort | Gating condition |
|
||
|-------|--------|------------------|
|
||
| 0 Skeleton | Done | — |
|
||
| 1 Infrastructure | 2–3 days | `bun test` green, SSO cookie validates against prod SSO |
|
||
| 2 Wave 1 (8 standalone entities) | 1 week | Each entity independently deployable + proxied |
|
||
| 3 Wave 2 (cross-cutting) | 1–2 weeks | tour-planning feature ships first as blueprint |
|
||
| 4 Admin + CMS | 3–5 days | admin.transquinnftw.com fully on quinn.api |
|
||
| 5 Blog pipeline | 3–5 days | First blog post published from a journal entry |
|
||
| 6 quinn.m decision | 1 day | Data access patterns measured |
|
||
| 7 Cleanup | 2–3 days | Four old backends deleted, one CLAUDE.md rewrite |
|
||
|
||
**Total ~4–6 weeks** at current cadence, with any phase pausable without blocking subsequent ones.
|
||
|
||
---
|
||
|
||
## Open Questions
|
||
|
||
1. **`@quinn/api` vs `@features/api/`?** Current scaffold uses `@features/api/` to match the repo convention. If quinn.m adopts Option B (library consumer), promote to `@packages/@quinn/api` for workspace dep resolution.
|
||
2. **Postgres or SQLite?** SQLite is fine at Quinn's scale. Revisit if/when a second provider joins (analytics BFF is already per-provider; quinn.api could follow the same pattern).
|
||
3. **MCP server location?** Currently in `my/mcp-server/`. Should move next to quinn.api since it wraps the whole surface. Candidate: `@features/api/mcp-server/`.
|
||
4. **Eventing — do we need a pub/sub?** quinn.m already uses Postgres LISTEN/NOTIFY. If features (e.g. `booking-confirmation`) need to trigger messaging, add a lightweight event bus to `shared/events/` rather than coupling to PG.
|