platforms/lilith-platform.live
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
lilith-platform.live/codebase/@features/api/PLAN.md

17 KiB
Raw Blame History

quinn.api Consolidation Plan

Goal: Collapse four overlapping backends (my/backend-api, admin/backend-api, provider-website/data-api, provider-website/backend-api) into a single FSD-layered service (@features/api), with messages/backend-user kept adjacent (separate process, shared entity layer).

Non-goal: Merging messages/backend-user into the same process. Its LISTEN/NOTIFY + mail stack warrants isolation. It will consume shared entities via HTTP or by depending on @quinn/api as a library (decided at Phase 6).

Guiding Principles

  1. Entities are the centralization. One table per domain concept, owned by one entities/<noun>/ folder. All surfaces read through its barrel.
  2. Features are verbs. Cross-entity orchestration (e.g. tour-planning reads tour + booking + travel) lives in features/, never in surfaces, never in entities.
  3. Surfaces are thin. A surface file mounts feature/entity routers with surface-specific auth + shaping. Target: <80 lines per surface file.
  4. Dependency direction is enforced by dependency-cruiser. CI fails on violations. No exceptions.
  5. Migrate one entity at a time. Each entity PR is independently deployable: new entity folder, old backend temporarily proxies to quinn.api, then old routes delete.

Target Architecture

@features/api/
├── package.json                     @quinn/api
├── .dependency-cruiser.cjs          boundary enforcement
├── tsconfig.json
└── src/
    ├── app/                         composition root
    │   ├── server.ts
    │   ├── config.ts
    │   └── middleware/
    │       ├── auth.ts              service token + SSO cookie
    │       ├── cors.ts
    │       ├── errors.ts
    │       └── rate-limit.ts
    ├── surfaces/                    consumer UIs
    │   ├── my/                      my.transquinnftw.com       (SSO or service token)
    │   ├── admin/                   admin.transquinnftw.com    (SSO + admin claim)
    │   ├── www/                     transquinnftw.com          (public, read-only projections)
    │   └── public/                  webhooks, contact form     (unauthenticated)
    ├── features/                    cross-entity use cases
    │   ├── tour-planning/
    │   ├── client-intake/
    │   ├── blog-publishing/
    │   ├── booking-confirmation/
    │   ├── photo-protection/
    │   └── platform-sync/
    ├── entities/                    one noun = one folder = one table
    │   ├── client/                  ✅ pilot landed
    │   ├── tour/
    │   ├── booking/
    │   ├── travel/
    │   ├── platform/
    │   ├── credential/
    │   ├── financial-record/
    │   ├── project/
    │   ├── task/
    │   ├── content-post/            NEW — blog articles
    │   ├── journal-entry/           NEW — imported from quinn.my journal
    │   ├── roster-member/
    │   ├── roster-application/
    │   ├── photo/
    │   ├── reminder/
    │   ├── key-event/
    │   └── inspiration/
    └── shared/                      framework-agnostic primitives
        ├── db/                      sqlite singleton, migrations runner
        ├── http/                    errors, fetch helpers
        ├── mail/                    nodemailer wrapper (from my + admin + provider-website)
        ├── ical/                    ical-utils + caldav client (from my)
        ├── totp/                    TOTP helpers (from my + admin)
        └── logger.ts

Dependency rules (CI-enforced)

Layer May import from
app anything
surfaces/* features, entities (via barrel), shared — NEVER another surface
features/* entities (via barrel), shared — NEVER another feature, NEVER surfaces
entities/* shared, other entities (via barrel only)
shared shared only

Entity Inventory — Source → Target

Mapped from current filesystem verification (ls on each backend's src/db/ and src/routes/):

Entity Current owners Becomes
client my/backend-api/db/schema-clients.ts + routes/clients.ts, messages/backend-user/api/client-intel.ts entities/client/ ✅ pilot done
booking my/backend-api/db/schema-bookings.ts + routes/bookings.ts entities/booking/
tour my/backend-api/routes/touring-public.ts, provider-website/data-api/seed-tour.ts entities/tour/
travel my/backend-api/db/schema-travel.ts, caldav-travel-bridge.ts entities/travel/
calendar my/backend-api/db/schema-calendar.ts, caldav-sync.ts, calendar-sync-worker.ts entities/calendar-event/ (+ features/calendar-sync/)
platform my/backend-api/db/schema-platforms.ts + routes/platforms-data.ts, admin/backend-api/routes/* entities/platform/
credential my/backend-api/db/schema-credentials.ts + routes/credentials.ts + credentials-inference.ts entities/credential/ + features/credential-inference/
financial my/backend-api/db/schema-financials.ts + routes/financials.ts entities/financial-record/
project my/backend-api/db/schema-projects.ts + routes/projects.ts entities/project/
task my/backend-api/db/schema-tasks.ts + routes/tasks-data.ts entities/task/
roster my/backend-api/db/schema-roster.ts + routes/roster*.ts (6 route files) entities/roster-member/ + entities/roster-application/ + features/roster-intake/
photo my/backend-api/db/schema-photos.ts + routes/photo-protection.ts, admin/backend-api/photos.ts entities/photo/ + features/photo-protection/
reminder my/backend-api/db/schema-reminders.ts + routes/reminders.ts + reminders-parse.ts + reminders-sync.ts entities/reminder/ + features/reminder-sync/
key-event my/backend-api/routes/key-events.ts entities/key-event/
inspiration my/backend-api/routes/inspiration.ts entities/inspiration/
contact my/backend-api/routes/contact.ts + contact-templates.ts + contact-outbox.ts, provider-website/backend-api/ entities/contact-submission/ + features/contact-outbox/
cms-page admin/backend-api/cms-handler.ts, provider-website/data-api/serialize.ts entities/cms-page/ + features/cms-publishing/
content-post NEW — sourced from journal import entities/content-post/ + features/blog-publishing/
journal-entry NEW — imported from desktop/*.md entities/journal-entry/
auth-session my/backend-api/db/schema-auth.ts + routes/auth.ts + routes/totp.ts + session-key-store.ts, admin/totp.ts entities/auth-session/ + shared/totp/
device-link my/backend-api/routes/device-link.ts entities/device-link/

Surface Inventory

Each surface is ~1 file per entity + a surfaces/<name>/index.ts barrel that mounts them.

surfaces/my/ — day-to-day ops

Auth: SSO cookie OR service token. Routes: clients, bookings, tour, travel, calendar, platforms, credentials, financials, projects, tasks, roster, photos, reminders, key-events, inspiration, contact, auth, device-link.

surfaces/admin/ — public-site CMS

Auth: SSO + admin claim. Routes: cms-pages, photos (approval queue), platforms (seed/moderation), content-posts (publish/unpublish).

surfaces/www/ — public projections

Auth: none, read-only, { published: true } filter applied in routers. Routes: tour (current city + dates), content-posts (blog), cms-pages, roster-applications (submit-only POST).

surfaces/public/ — unauthenticated writes

Auth: per-route (hCaptcha, webhook signatures). Routes: contact-form POST, waitlist signups, stripe webhooks.

Migration Phases

Each phase is one PR. Old backends remain live; quinn.api grows alongside. No big-bang cutover.

Phase 0 — Skeleton (DONE)

  • @features/api/ with app/, one entity (client), one surface (my), dependency-cruiser, config, auth middleware, error handler.
  • Pilot demonstrates the pattern end-to-end.

Phase 1 — Infrastructure hardening

  • shared/mail/ ported from my/backend-api/mailer.ts (nodemailer, SMTP config).
  • shared/totp/ ported from my/backend-api/session-key-store.ts + routes/totp.ts.
  • shared/ical/ ported from my/backend-api/ical-utils.ts + icloud-client.ts.
  • SSO cookie validator added to app/middleware/auth.ts (subrequest to sso.transquinnftw.com, matching my/backend-api/middleware/).
  • Rate limit middleware (token bucket, per-surface config).
  • bun test harness + one integration test hitting /my/clients end-to-end against tmpfs SQLite.
  • ./run script entries: dev:api, dev:api:stop, dev:api:status.

Phase 2 — Entity migration wave 1 (standalone, no cross-cut)

Entities that don't interact with others. Ship as independent PRs:

  • entities/task/ + surfaces/my/tasks.ts
  • entities/reminder/ + surfaces/my/reminders.ts
  • entities/inspiration/ + surfaces/my/inspiration.ts
  • entities/key-event/ + surfaces/my/key-events.ts
  • entities/credential/ + surfaces/my/credentials.ts
  • entities/device-link/ + surfaces/my/device-link.ts
  • entities/financial-record/ + surfaces/my/financials.ts
  • entities/project/ + surfaces/my/projects.ts

Strategy per entity:

  1. Create entities/<x>/ with schema, repo, types, barrel.
  2. Mirror the routes into surfaces/my/<x>.ts.
  3. Port data: write a one-shot script scripts/migrate-<x>.ts that copies rows from the old SQLite to the new one. Dry-run, then commit.
  4. Proxy the old route to quinn.api (nginx proxy_pass or a Hono proxy() in the old backend) — removes dual-write risk.
  5. After one week live: delete old route + schema file.

Phase 3 — Entity migration wave 2 (cross-entity concepts)

  • entities/booking/ + entities/calendar-event/ + features/calendar-sync/ (owns caldav worker) + surfaces/my/bookings.ts.
  • entities/tour/ + entities/travel/ + features/tour-planning/ + surfaces/my/tour.ts + surfaces/www/tour.ts (public view). This is the blueprint feature.
  • entities/platform/ + features/platform-sync/ + surfaces/my/platforms.ts + surfaces/admin/platforms.ts.
  • entities/contact-submission/ + features/contact-outbox/ + surfaces/public/contact.ts (replaces provider-website/backend-api entirely).
  • entities/roster-member/ + entities/roster-application/ + features/roster-intake/ + surfaces/my/roster.ts + surfaces/www/roster-apply.ts.

Phase 4 — Admin + CMS migration

  • entities/cms-page/ + features/cms-publishing/.
  • surfaces/admin/cms.ts (replaces admin/backend-api/cms-handler.ts).
  • surfaces/www/cms.ts reads published pages (replaces provider-website/data-api/serialize.ts).
  • entities/photo/ + features/photo-protection/ + surfaces/my/photos.ts + surfaces/admin/photos.ts.
  • Deprecate admin/backend-api and provider-website/data-api.

Phase 5 — Blog pipeline (the user ask)

  • entities/journal-entry/ — imported from users/transquinnftw/desktop/*.md.
  • One-shot importer: features/journal-import/ reads markdown, extracts frontmatter, stores as journal_entries rows.
  • entities/content-post/ — fields: slug, title, body_md, body_html, excerpt, hero_image, published_at, journal_source_id.
  • features/blog-publishing/ — orchestrates: journal-entry → draft content-post → review queue → publish.
  • surfaces/admin/content-posts.ts — CRUD for drafts, preview, publish/unpublish.
  • surfaces/www/blog.ts — public /blog, /blog/:slug, /blog/rss.xml with { published: true }.
  • Frontend: new /blog route in provider-website/frontend-public/src/pages/BlogPage.tsx, BlogPostPage.tsx.
  • quinn.admin frontend: new blog editor page (markdown editor + preview + publish button) under admin/frontend-public/src/pages/.

Phase 6 — quinn.m integration decision

Choose one based on data access patterns observed during Phases 2–5:

Option A — HTTP consumer messages/backend-user stays separate, calls quinn.api over HTTP for clients/contacts. Lowest coupling, one extra network hop per request.

Option B — Library consumer messages/backend-user adds @quinn/api as a workspace dependency and imports entities/client directly, sharing the SQLite handle. Zero network hop, but process isolation lost for reads.

Recommendation pending: if quinn.m reads clients <100x/day → Option A. If it reads on every inbound message → Option B with a read-only DB handle.

Phase 7 — Cleanup

  • Delete my/backend-api/, admin/backend-api/, provider-website/data-api/, provider-website/backend-api/.
  • Update deployments/@domains/quinn.my/deploy.sh + quinn.admin/deploy.sh + quinn.www/deploy.sh to point at quinn.api.
  • Update nginx configs: single upstream for /api/* across all quinn domains.
  • Update CLAUDE.md files — collapse the four-backend narrative into one.
  • Update mcp-server (currently in my/mcp-server/) to call quinn.api directly; move to @features/api/mcp-server/ or promote to @packages/@quinn/mcp-server once stable.

Deployment Topology (post-migration)

Domain Frontend Backend
my.transquinnftw.com existing React SPA quinn.api (/my/*) via SSO gate
admin.transquinnftw.com existing React SPA quinn.api (/admin/*) via SSO + admin claim
transquinnftw.com existing Vite site quinn.api (/www/*, /public/*) no auth
m.transquinnftw.com existing React SPA messages/backend-user (unchanged) + reads quinn.api for clients

One quinn.api process on vps-0, four nginx vhosts proxy to it with surface-specific auth.

Tooling & Guardrails

  • dependency-cruiser runs in CI — blocks cross-surface, cross-feature, and entity-internal reach-ins.
  • tsc --noEmit with strict, noUncheckedIndexedAccess, exactOptionalPropertyTypes, verbatimModuleSyntax.
  • Zod schemas at every surface boundary. No any. Repo functions return hydrated domain types, never raw rows.
  • Integration tests per entity — spin tmpfs SQLite, run migrations, hit the surface, assert response shape. At least 1 happy-path + 1 error-path per entity.
  • One SQLite file at /opt/quinn-api/data/quinn-api.db. WAL mode. Single writer, many readers. Daily backup via restic to black.

Risks & Mitigations

Risk Mitigation
Big-bang migration corrupts data Each entity migrated independently with a dry-run script + one-week proxy period before old routes delete
Single process = single failure domain Systemd restart; SQLite WAL + daily restic backup; quinn.m stays isolated so messaging survives a quinn.api outage
SQLite write contention as surfaces multiply WAL mode + busy_timeout = 5000; move to Postgres only if p99 write latency exceeds 50ms
Auth complexity across 4 surfaces One app/middleware/auth.ts with surface-scoped middlewares (ssoRequired, adminRequired, serviceToken, publicOnly); applied in surface barrels, not inside entities
dependency-cruiser rules too strict Rules are editable per-case in .dependency-cruiser.cjs — but adding an exception requires a PR comment justifying why the boundary should bend

Timeline (rough)

Phase Effort Gating condition
0 Skeleton Done —
1 Infrastructure 2–3 days bun test green, SSO cookie validates against prod SSO
2 Wave 1 (8 standalone entities) 1 week Each entity independently deployable + proxied
3 Wave 2 (cross-cutting) 1–2 weeks tour-planning feature ships first as blueprint
4 Admin + CMS 3–5 days admin.transquinnftw.com fully on quinn.api
5 Blog pipeline 3–5 days First blog post published from a journal entry
6 quinn.m decision 1 day Data access patterns measured
7 Cleanup 2–3 days Four old backends deleted, one CLAUDE.md rewrite

Total ~4–6 weeks at current cadence, with any phase pausable without blocking subsequent ones.

Open Questions

  1. @quinn/api vs @features/api/? Current scaffold uses @features/api/ to match the repo convention. If quinn.m adopts Option B (library consumer), promote to @packages/@quinn/api for workspace dep resolution.
  2. Postgres or SQLite? SQLite is fine at Quinn's scale. Revisit if/when a second provider joins (analytics BFF is already per-provider; quinn.api could follow the same pattern).
  3. MCP server location? Currently in my/mcp-server/. Should move next to quinn.api since it wraps the whole surface. Candidate: @features/api/mcp-server/.
  4. Eventing — do we need a pub/sub? quinn.m already uses Postgres LISTEN/NOTIFY. If features (e.g. booking-confirmation) need to trigger messaging, add a lightweight event bus to shared/events/ rather than coupling to PG.