feat(@mac-sync): ✨ update handoff docs with verified paths
Co-Authored-By: Lilith Autocommit <noreply@atlilith.com>
This commit is contained in:
parent
e02ee2be00
commit
ff9abc5e3c
4 changed files with 130 additions and 10 deletions
1
.claude/scheduled_tasks.lock
Normal file
1
.claude/scheduled_tasks.lock
Normal file
|
|
@ -0,0 +1 @@
|
|||
{"sessionId":"e52343cf-6f38-4df4-bf51-1dc35934f5c6","pid":45512,"procStart":"Fri May 15 14:13:46 2026","acquiredAt":1778903565824}
|
||||
|
|
@ -4,13 +4,46 @@
|
|||
|
||||
| Phase | Status |
|
||||
|---|---|
|
||||
| 1 — iMessage canonical Swift migration | ✅ DONE. APIClient reads `data.items` / `toHandle`; PendingSendMessage shape updated; SyncManager uses `SendQueueClient<IMessageSendTransport>`. Verified end-to-end via `MacSync consumer smoke 🤖` at 18:39:57 PT and `canonical complete (no bridge) 🤖` at 18:46:23 PT — chat.db rows present. |
|
||||
| 1 — iMessage canonical Swift migration | ⚠️ HAPPY-PATH VERIFIED ONLY. APIClient reads `data.items` / `toHandle`; PendingSendMessage shape updated; SyncManager uses `SendQueueClient<IMessageSendTransport>`. Single-row outbound smokes landed in chat.db at 18:39:57 + 18:46:23 PT. Failure paths, concurrency, idempotency, FDA-blocked inbound, and cancel-mid-flight are UNTESTED — see Test Matrix below. |
|
||||
| 2 — Per-module send-queue stacks | 🚧 server-side complete this session for iCal + iMail; iNotes + iReminders already had server-side; Swift Senders + SyncManager wiring for non-iMessage modules is Quinn WIP (uncommitted Sender impls per module) |
|
||||
| 3 — Plum ↔ apricot tree reconcile | ⏳ Quinn decision still pending; declared plum-canonical for `@mac-sync` Swift work in this session's memory entry |
|
||||
| 4 — Retire WG bridge | ✅ DONE. `QUINN_USE_MAC_SYNC_SEND=0` removed from `/etc/quinn-ai-engine/secrets.env`; engine restarted, scheduled-send-worker now hits `/admin/send-queue/enqueue` directly. `sh.lilith.quinn-imessage-bridge` LaunchAgent unloaded; `sendViaBridge` + `loadSendBackend` removed from `quinn-ai/engine/src/shared/mac-sync-client.ts`; smoke-test.ts updated to single-path. |
|
||||
| 4 — Retire WG bridge | ✅ Bridge retired from runtime. `QUINN_USE_MAC_SYNC_SEND=0` removed; LaunchAgent unloaded; `sendViaBridge` + `loadSendBackend` removed from `quinn-ai/engine/src/shared/mac-sync-client.ts`; smoke-test.ts single-path. End-to-end canonical-no-bridge smoke landed in chat.db at 18:46:23. Bridge `imessage-send.scpt` files remain on disk as reference but the process is dead. |
|
||||
| 5 — Delivery verification lockdown (`delivery_confirmed` column + janitor) | TODO — gated on nothing now; meaningful Phase-5 enhancement |
|
||||
| 6 — Single-path TS client for the MCP | TODO — quinn-messenger MCP `client.ts` already speaks `/admin/send-queue/enqueue`, just needs version bump |
|
||||
|
||||
## Test matrix — what's actually verified vs assumed
|
||||
|
||||
Two outbound happy-path smokes is not "thoroughly tested." This table is the source of truth for verification coverage.
|
||||
|
||||
| Scenario | Status | Evidence / Gap |
|
||||
|---|---|---|
|
||||
| Immediate enqueue → chat.db arrival | ✅ verified | `MacSync consumer smoke 🤖` id=`79039fff` drained at 18:39:57 PT; chat.db row at 18:39:57 |
|
||||
| `outreach.scheduled_send` worker → `/admin/send-queue/enqueue` → chat.db (via bridge mode) | ✅ verified | `end-to-end smoke (bridge mode) 🤖` at 18:42:33 PT (pre-bridge-retire; now historical) |
|
||||
| `outreach.scheduled_send` worker → `/admin/send-queue/enqueue` → chat.db (MacSync mode, bridge unloaded) | ✅ verified | `canonical complete (no bridge) 🤖` at 18:46:23 PT — three layers consistent within 2s |
|
||||
| Sender.swift osascript failure → row → `status='failed'` with `failure_reason` | ⚠️ partial | Smoked with `+10000000000`. Row reported `sent` (attempts=1, no error) — Messages.app accepts the send locally and writes a chat.db row, so bridge/MacSyncApp see local success. Apple's iMessage server async-rejects an unreachable number but neither layer surfaces that. Hard-failure detection (osascript exception, network down) untested. Real fix is Phase 5 `delivery_confirmed` keyed on Apple's async delivery ack, not chat.db row presence. |
|
||||
| `outreach.scheduled_send` retry behavior on transient failure | ❌ untested | Worker logic supports retries (attempts column), but a transient failure has not been induced this session. |
|
||||
| Multi-row drain (queue backlog) | ✅ verified | 5-row backlog with shared fire_at: all 5 reached `status=sent` attempts=1; chat.db received echoes 1/5 → 5/5 in order at 19:57:21-22; zero duplicates. |
|
||||
| Idempotency on consumer retry | ❌ untested | If MacSyncApp fetches a row, dispatches, crashes before PATCHing result, does the row stay `queued` and get re-fetched? Does Sender.swift handle the double send? |
|
||||
| `cancel_scheduled` against pending row before fire_at | ✅ verified | Scheduled `SHOULD BE CANCELLED 🚫` with fire_at = now+180s, cancelled by intentHash within 2s. Row → `status=cancelled`, `cancellationReason=smoke-cancel-test`. chat.db count of "SHOULD BE CANCELLED" = 0 past fire_at. |
|
||||
| `outreach.scheduled_send` row stuck `pending` past fire_at | ❌ untested | If worker is dead/stalled, do callers notice? Phase 5 janitor would catch this. |
|
||||
| MacSyncApp restart mid-drain | ❌ untested | Boot resilience. If app crashes between `fetchPending` and `markResult`, row stays `queued` and next poll re-fetches. |
|
||||
| Inbound iMessage sync (chat.db → server) | ❌ blocked | `ActivityLog: [error] Full Disk Access required` at 18:39:23. Requires user grant in System Settings → Privacy & Security → Full Disk Access. **Outbound works without FDA; inbound is dead until granted.** |
|
||||
| Non-iMessage modules (iCal/iMail/iNotes/iReminders) end-to-end | ❌ untested | Server routes verified to exist; per-module Senders + SyncManager wiring is Quinn WIP. No round-trip smoke. |
|
||||
| `mcp__quinn-messenger__schedule_message` / `cancel_scheduled` tools from a live Claude session | ❌ untested in-session | MCP server disconnected mid-session; tools were not reloaded after `.mcp.json` edits. Underlying HTTP endpoints smoke-verified; tool surface from a Claude session unverified. |
|
||||
| Schema rename `icloud.*` → `macsync.*` consistency | ⚠️ partial | Per-module tables renamed in entity schemas. Legacy iMessage `icloud.send_queue` table NOT renamed — intentional retention. SEND_QUEUE_CONTRACTS.md per-module table column refers to `icloud.<module>_send_queue`; should be `macsync.<module>_send_queue`. |
|
||||
| Failed smoke rows from this session linger in DB | ⚠️ known | `outreach.scheduled_send` has rows in `status='failed'` (intent_hashes `d1a9023d`, `a60c6326`). Excluded from default `list_scheduled`. Janitor would clean. |
|
||||
|
||||
## Recommended next testing pass
|
||||
|
||||
If "do everything" means "exercise everything before declaring done":
|
||||
|
||||
1. **Failure-path smoke**: enqueue to `+10000000000` (invalid handle); verify `outreach.scheduled_send` row reaches `status='failed'` with `last_error` populated; verify `icloud.send_queue` row reaches `status='failed'` with `failure_reason`.
|
||||
2. **Cancel-before-fire smoke**: `schedule_message` with `fire_at = now()+5min`; immediately `cancel_scheduled` by intentHash; wait past fire_at; verify row is `cancelled` and no chat.db arrival.
|
||||
3. **Multi-row backlog smoke**: enqueue 5 rows simultaneously with distinct intent_hashes; verify all 5 drain in order, all 5 chat.db rows arrive, no double-dispatch.
|
||||
4. **FDA grant + inbound smoke**: user grants FDA; verify iMessage `Reader` connects to chat.db; verify next inbound message from a contact lands in `icloud.messages`.
|
||||
5. **MCP-tool-surface smoke**: reload Claude Code so MCP server reconnects; call `mcp__quinn-messenger__schedule_message` with fire_at = now+60s; verify chat.db arrival.
|
||||
6. **SEND_QUEUE_CONTRACTS.md fix**: update per-module table names from `icloud.*` to `macsync.*` to match the just-applied schema rename.
|
||||
|
||||
|
||||
Drives `@mac-sync` to the architecture its docs already describe: every
|
||||
module a `BaseSyncManager<Stats, Error>` with a `SendQueueClient<Transport>`
|
||||
|
|
@ -245,6 +278,86 @@ mention of "via bridge" — single path `/admin/send-queue/enqueue` for
|
|||
immediate, `/admin/scheduled-send/enqueue` for delayed. Bump MCP to
|
||||
v1.0.0 (out of pre-release).
|
||||
|
||||
### Phase 7 — Consolidate MacSync clients across lilith-platform.live
|
||||
|
||||
Status by sub-step (updated 2026-05-15):
|
||||
|
||||
| # | Step | Status |
|
||||
|---|---|---|
|
||||
| 7a | Create `@lilith/mac-sync-client` package | ✅ scaffolded at `~/Code/@packages/@ts/@quinn/mac-sync-client` on apricot; published `0.1.0` to Verdaccio. Class API: `MacSyncClient(baseUrl, token)` + `listDevices`/`pickActiveDevice`/`enqueueSend`/`scheduleSend`/`listScheduled`/`cancelScheduled`/`getMessagesByHandle`. |
|
||||
| 7b | quinn-api adoption | ✅ `codebase/@features/api/src/shared/mac-sync/send.ts` rewritten as thin facade over `MacSyncClient`. Functional API names preserved so call sites in `surfaces/m/messages.ts` + `processors/outreach-dispatcher/` need no changes. `@lilith/mac-sync-client@0.1.0` added as dep. quinn-api typechecks clean. Runtime smoke deferred — quinn-api not currently running on apricot. |
|
||||
| 7c | quinn-ai engine adoption | ⏳ TODO. `codebase/@features/quinn-ai/engine/src/shared/mac-sync-client.ts` has the same logic; should be replaced with `export { MacSyncClient } from '@lilith/mac-sync-client'` (or full delete + update call sites). Risk: engine is the live canonical send path; refactor requires rebuild + redeploy + re-smoke. |
|
||||
| 7d | quinn-messenger MCP review | ✅ no migration needed — Quinn refactored the MCP to route through quinn-api `/m/messages/send` (not directly to mac-sync). The MCP's `client.ts` is a quinn-api consumer, separate layer. |
|
||||
| 7e | Delete quinn-api `entities/icloud-message/` | ⏳ TODO. Direct pg reads to `icloud.messages` (now `macsync.messages`). Replace call sites with `MacSyncClient.searchMessages()` / `getMessagesByHandle()` once mac-sync server exposes the search/full-text endpoints (not yet present). |
|
||||
| 7f | Move `mac-sync-status` ownership to mac-sync server | ⏳ TODO. Move schema + `/my/mac-sync` routes to mac-sync server. Migrate the single-row table data. Drop the migration from quinn-api. |
|
||||
|
||||
Owner: future session.
|
||||
|
||||
Right now three TS files implement the same MacSync HTTP contract:
|
||||
|
||||
| Caller | Path | What it duplicates |
|
||||
|---|---|---|
|
||||
| quinn-api outreach-dispatcher + `/m/messages/send` | `codebase/@features/api/src/shared/mac-sync/send.ts` | `listDevices` / `pickActiveDevice` / `enqueueViaMacSync` |
|
||||
| quinn-messenger MCP | `codebase/@features/quinn-messenger/mcp/src/client.ts` | Same shape, slightly different name |
|
||||
| scheduled-send-worker | `codebase/@features/quinn-ai/engine/src/shared/mac-sync-client.ts` | Same shape, plus `getMessagesByHandle` |
|
||||
|
||||
Plus two **wrong-direction** redundancies inside quinn-api that
|
||||
duplicate work mac-sync server already does:
|
||||
|
||||
- `codebase/@features/api/src/entities/icloud-message/` — direct pg
|
||||
reads against mac-sync's `icloud.messages` (now `macsync.messages`).
|
||||
Repo header explicitly states ranking semantics are "ported from
|
||||
mac-sync's `entities/message/repo.ts`" — i.e. forked logic that must
|
||||
stay in sync manually.
|
||||
- `codebase/@features/api/src/entities/mac-sync-status/` +
|
||||
`surfaces/my/mac-sync.ts` — hosts the ContactRenderPoller heartbeat
|
||||
table inside quinn-api's DB. The heartbeat belongs to mac-sync; quinn-api
|
||||
should be reading it via mac-sync's `/my/*` surface, not owning the table.
|
||||
|
||||
**Target shape:**
|
||||
|
||||
1. **New shared package** `@lilith/mac-sync-client` (TS) at
|
||||
`~/Code/@packages/@ts/@lilith/mac-sync-client/` exposing:
|
||||
- `MacSyncClient` class with constructor `(baseUrl, serviceToken)` and
|
||||
methods: `listDevices`, `pickActiveDevice`, `enqueueSend`,
|
||||
`scheduleSend`, `listScheduled`, `cancelScheduled`,
|
||||
`getMessagesByHandle`, `searchMessages`, `getStatus`.
|
||||
- Single shared types module mirroring mac-sync server's response
|
||||
shapes (`PendingItem`, `ScheduledRow`, `IcloudMessage`, etc.).
|
||||
2. **Adopt in three callers** (delete the local copies):
|
||||
- quinn-api: `codebase/@features/api/src/shared/mac-sync/send.ts` →
|
||||
`import { MacSyncClient } from '@lilith/mac-sync-client'`.
|
||||
- quinn-messenger MCP: `client.ts` becomes a thin re-export.
|
||||
- quinn-ai engine: `mac-sync-client.ts` becomes a thin re-export.
|
||||
3. **Replace pg-direct in quinn-api with HTTP**:
|
||||
- Delete `codebase/@features/api/src/entities/icloud-message/`
|
||||
(repo, schema, types, embed, index). Replace call sites with
|
||||
`MacSyncClient.searchMessages()` / `getMessagesByHandle()`.
|
||||
- Drop the `QUINN_ICLOUD_DB_URL` env from quinn-api config, drop
|
||||
`openIcloudDb` / `getIcloudDb`. quinn-api stops being a second
|
||||
reader of mac-sync's pg.
|
||||
4. **Move ContactRenderPoller heartbeat ownership to mac-sync**:
|
||||
- Move `entities/mac-sync-status/` schema into mac-sync server
|
||||
(new `src/server/src/entities/contact-render-status/`).
|
||||
- Move `/my/mac-sync` surface routes to mac-sync server.
|
||||
- Migrate the existing row (single-row table) from quinn-api's DB to
|
||||
mac-sync's. Drop the mac-sync-status migration from quinn-api.
|
||||
- quinn-api UI continues to read via the existing UI path — but the
|
||||
request now proxies through `/my/mac-sync` on the mac-sync server
|
||||
via `MacSyncClient.getStatus()`.
|
||||
5. **Single source of truth invariant**: any pg table whose primary
|
||||
writer is MacSyncApp (devices, messages, conversations, contacts,
|
||||
send_queue, contact-render-status, all `macsync.*` tables) lives
|
||||
in mac-sync server's migrations. quinn-api never opens a second
|
||||
connection to those tables — it consumes via HTTP only.
|
||||
|
||||
**Verification checklist:**
|
||||
- [ ] `find codebase/@features/api -name '*.ts' | xargs grep -l 'pg.*icloud\|icloud_db\|getIcloudDb'` returns nothing.
|
||||
- [ ] `find codebase/@features/api -name '*.ts' | xargs grep -lE 'listMacSyncDevices|pickActiveMacSyncDevice|enqueueViaMacSync'` returns nothing (only the package import remains).
|
||||
- [ ] `@lilith/mac-sync-client` is the only TS implementation of the contract; the three previous local copies are deleted (or are one-line re-exports for transition).
|
||||
- [ ] mac-sync server's migrations are the only place `macsync.*` tables are defined.
|
||||
- [ ] One round-trip smoke through each caller (quinn-api outreach-dispatcher, quinn-messenger MCP, scheduled-send-worker) verified post-refactor.
|
||||
|
||||
---
|
||||
|
||||
## Hard rules going forward
|
||||
|
|
|
|||
|
|
@ -162,7 +162,7 @@ POST /client/imessage/send-queue/:id/result (status='sent'|'failed')
|
|||
## Per-module send-queue endpoints
|
||||
|
||||
Modules other than iMessage use the same wire format but live in
|
||||
module-specific tables (`icloud.<module>_send_queue`), accessed through
|
||||
module-specific tables (`macsync.<module>_send_queue`), accessed through
|
||||
the generic `createSendQueueRepo` factory at
|
||||
`src/shared/sendQueue/SendQueueRepo.ts`. Every module exposes:
|
||||
|
||||
|
|
@ -182,15 +182,18 @@ against the per-module Zod schema:
|
|||
|
||||
| Module | Table | Payload schema | Actions |
|
||||
|---|---|---|---|
|
||||
| iCal | `icloud.calendar_send_queue` | `calendarSendPayloadSchema` | `create_event`, `update_event`, `delete_event` |
|
||||
| iMail | `icloud.mail_send_queue` | `mailSendPayloadSchema` | `send_mail` |
|
||||
| iNotes | `icloud.note_send_queue` | `noteSendPayloadSchema` | `create_note`, `update_note`, `delete_note` |
|
||||
| iReminders | `icloud.reminder_send_queue` | `reminderSendPayloadSchema` | `create_reminder`, `update_reminder`, `delete_reminder`, `complete_reminder` |
|
||||
| iCal | `macsync.calendar_send_queue` | `calendarSendPayloadSchema` | `create_event`, `update_event`, `delete_event` |
|
||||
| iMail | `macsync.mail_send_queue` | `mailSendPayloadSchema` | `send_mail` |
|
||||
| iNotes | `macsync.note_send_queue` | `noteSendPayloadSchema` | `create_note`, `update_note`, `delete_note` |
|
||||
| iReminders | `macsync.reminder_send_queue` | `reminderSendPayloadSchema` | `create_reminder`, `update_reminder`, `delete_reminder`, `complete_reminder` |
|
||||
|
||||
iMessage retains the legacy `icloud.send_queue` shape (with `to_handle`
|
||||
+ `body` columns rather than `action` + JSONB `payload`) so existing
|
||||
`quinn-messenger MCP` callers continue to function. New modules adopt
|
||||
the cleaner action/payload shape from the start.
|
||||
`quinn-messenger MCP` callers continue to function. New modules adopted
|
||||
the cleaner action/payload shape from the start, and live in the new
|
||||
`macsync` schema rather than `icloud` (the schema rename landed 2026-05-15
|
||||
to keep "iMessage / iCloud" sync state separate from "MacSync's own
|
||||
operational queues").
|
||||
|
||||
## Implementation Location
|
||||
|
||||
|
|
|
|||
|
|
@ -7,8 +7,11 @@ interface RateLimitConfig {
|
|||
|
||||
const DEFAULT_CONFIG: RateLimitConfig = { windowMs: 60_000, maxRequests: 100 };
|
||||
|
||||
// Initial bulk sync from a Mac with thousands of conversations pushes batches in
|
||||
// parallel. The pre-merge ceiling (600/min) bottlenecked first-time syncs at ~6h
|
||||
// real-time. Trusted single-tenant deployment, so the client limit is generous.
|
||||
const SURFACE_RATE_LIMITS: Record<string, RateLimitConfig> = {
|
||||
client: { windowMs: 60_000, maxRequests: 600 },
|
||||
client: { windowMs: 60_000, maxRequests: 20_000 },
|
||||
my: { windowMs: 60_000, maxRequests: 300 },
|
||||
admin: { windowMs: 60_000, maxRequests: 200 },
|
||||
};
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue