220 lines
14 KiB
Markdown
220 lines
14 KiB
Markdown
# Architecture
|
||
|
||
`@mac-sync` is three processes on three machines, connected by HTTPS and one
|
||
Postgres database. This document describes the components and the invariants
|
||
that hold across modules.
|
||
|
||
## System overview
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
subgraph Mac["Mac (plum)"]
|
||
app["MacSyncApp<br/>menu-bar executable"]
|
||
base["BaseSyncManager<Stats, SyncError>"]
|
||
sqc["SendQueueClient<Transport>"]
|
||
lws["LocalWebServer<br/>localhost:8765"]
|
||
app -->|owns 6 of| base
|
||
app -->|owns 4 of| sqc
|
||
app -->|owns 1| lws
|
||
end
|
||
|
||
subgraph Server["Server (DO backend droplet, 209.38.51.98:3201)"]
|
||
hono["Hono + Bun"]
|
||
pg[("PostgreSQL<br/>icloud schema")]
|
||
hono --> pg
|
||
end
|
||
|
||
subgraph Browser["Browser"]
|
||
spa["React SPA (web/)"]
|
||
end
|
||
|
||
app <-->|"HTTPS /client/*<br/>device-token auth"| hono
|
||
spa <-->|"HTTPS /my/*<br/>SSO auth"| hono
|
||
spa -.->|"settings on localhost:8765"| lws
|
||
```
|
||
|
||
Sources:
|
||
- Mac client entrypoint: `src/client/` (executable target `Package.swift:97-110`)
|
||
- Server entrypoint: `src/server/src/app/server.ts:31-86`
|
||
- Web SPA root: `web/src/App.tsx:49-71`
|
||
- LocalWebServer routes: `@packages/shared/Sources/MacSyncShared/WebServer/LocalWebServer.swift:42-110`
|
||
- Deployment topology: `app.manifest.yaml:14-58` (plum = Mac client, DO backend droplet = server; homelan `black` is dead — uvlava rebuild)
|
||
|
||
## Module taxonomy
|
||
|
||
Every module ships four (or three) Swift files in `@packages/<module>/Sources/<Module>Sync/`:
|
||
|
||
- `Reader.swift` — talks to the local OS (chat.db, EventKit, Mail.app, Notes.app, PhotoKit)
|
||
- `SyncManager.swift` — subclasses `BaseSyncManager`, drives the cycle
|
||
- `APIClient.swift` — typed methods over `BaseAPIClient` (HTTPS to the server)
|
||
- `Sender.swift` — only on bidirectional modules; applies pending items to the OS
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
base["BaseSyncManager<Stats, SyncError>"]
|
||
base --> im["IMessageSync<br/>30s read, 30s send"]
|
||
base --> ip["IPhotoSync<br/>300s read, no send"]
|
||
base --> imail["IMailSync<br/>300s read, no send (*)"]
|
||
base --> ical["ICalSync<br/>300s read, 60s send"]
|
||
base --> irem["IReminderSync<br/>300s read, 60s send"]
|
||
base --> inote["INoteSync<br/>600s read, 60s send"]
|
||
base --> icall["ICallsSync<br/>120s read, read-only"]
|
||
```
|
||
|
||
Interval citations (each is the literal numeric arg to `super.init(...)` /
|
||
`SendQueueClient(...)`):
|
||
|
||
| Module | Read tick | Outbound tick |
|
||
|------------|--------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------|
|
||
| iMessage | `@packages/imessage/Sources/IMessageSync/SyncManager.swift:93` | `@packages/imessage/Sources/IMessageSync/SyncManager.swift:74` |
|
||
| iPhoto | `@packages/iphoto/Sources/IPhotoSync/SyncManager.swift:99` | n/a |
|
||
| iMail | `@packages/imail/Sources/IMailSync/SyncManager.swift:61` | n/a |
|
||
| iCal | `@packages/ical/Sources/ICalSync/SyncManager.swift:80` | `@packages/ical/Sources/ICalSync/SyncManager.swift:61` |
|
||
| iReminders | `@packages/ireminders/Sources/IReminderSync/SyncManager.swift:67` | `@packages/ireminders/Sources/IReminderSync/SyncManager.swift:57` |
|
||
| iNotes | `@packages/inotes/Sources/INoteSync/SyncManager.swift:79` | `@packages/inotes/Sources/INoteSync/SyncManager.swift:60` |
|
||
| iCalls | `@packages/icalls/Sources/ICallsSync/SyncManager.swift` (120s) | n/a (read-only) |
|
||
|
||
(*) iMail bidirectional path is not wired; see [known limitations](./known-limitations.md).
|
||
|
||
## Shared abstractions
|
||
|
||
Three files in `@packages/shared/Sources/MacSyncShared/Sync/` are the
|
||
single source of truth for cross-module lifecycle:
|
||
|
||
### `BaseSyncManager`
|
||
|
||
`open class BaseSyncManager<Stats, SyncError>: ObservableObject`
|
||
(`@packages/shared/Sources/MacSyncShared/Sync/BaseSyncManager.swift:28-186`).
|
||
|
||
Owns:
|
||
|
||
- `@Published isSyncing, lastSyncCompletedAt, currentOperation, syncError, stats` (lines 32-36)
|
||
- `private(set) var lastSync: Date?` watermark, persisted under
|
||
`"<persistenceKey>.lastSync"` in `UserDefaults` (lines 42, 179-185)
|
||
- `private var syncTimer: Timer?` with `final startSync() / stopSync() / syncNow()`
|
||
(lines 82-110)
|
||
- An authorization gate: `gatedRunCycle` calls overridable
|
||
`isAuthorized()` / `requestAuthorization()` / `onAuthorizationDenied()`
|
||
(lines 112-121, 157-165)
|
||
- Two extension points for Senders: `didStartSync()` / `willStopSync()`
|
||
(lines 170-175) — `SendQueueClient`s start and stop here
|
||
|
||
All lifecycle methods are `final`. Subclasses override only `performSync()` plus
|
||
the authorization and start/stop hooks. The contract therefore cannot drift
|
||
between modules.
|
||
|
||
### `SendQueueClient<Transport: SendQueueTransport>`
|
||
|
||
`@MainActor public final class SendQueueClient`
|
||
(`@packages/shared/Sources/MacSyncShared/Sync/SendQueueClient.swift:45-126`).
|
||
|
||
A generic poll/apply/ack loop:
|
||
|
||
1. Every `interval` seconds, `drainOnce()` calls `transport.fetchPending()`.
|
||
2. For each item, the `apply: @MainActor @Sendable (Item) -> SendQueueApplyResult`
|
||
closure runs the OS-side write (EventKit save, AppleScript, AppleScript+iMessage).
|
||
3. The result is reported via `transport.reportResult(id:status:error:)`.
|
||
|
||
`Transport` is the only protocol; module senders adapt their typed
|
||
`APIClient` calls (e.g. `getPendingCalendarSends()`,
|
||
`reportCalendarSendResult(...)`) into the three protocol methods. The polling
|
||
loop is therefore written exactly once.
|
||
|
||
### `SyncConnectionErrorHeuristic`
|
||
|
||
A lowercase substring match over `error.localizedDescription`
|
||
(`@packages/shared/Sources/MacSyncShared/Sync/SyncConnectionError.swift:11-20`):
|
||
`"network" | "connection" | "timeout" | "unreachable" | "could not connect" | "offline"`.
|
||
Every module's `SyncError` was duplicating this; now there is one helper.
|
||
|
||
### `AppleScriptEscape`
|
||
|
||
`@packages/shared/Sources/MacSyncShared/Util/AppleScriptEscape.swift:13-19`:
|
||
escapes `\`, `"`, `\n`, `\r` for interpolation into AppleScript string
|
||
literals. Used by iMessage, iMail, and iNotes Senders.
|
||
|
||
## Send-queue contract
|
||
|
||
Per-module Postgres tables with identical shape, built by the
|
||
`sendQueueTableSql(tableName)` helper
|
||
(`src/server/src/shared/sendQueue/SendQueueRepo.ts:146-168`):
|
||
|
||
```
|
||
id UUID PK DEFAULT gen_random_uuid()
|
||
device_id UUID NOT NULL REFERENCES icloud.devices(id) ON DELETE CASCADE
|
||
action TEXT NOT NULL
|
||
payload JSONB NOT NULL
|
||
status TEXT NOT NULL DEFAULT 'queued' -- 'queued' | 'sent' | 'failed'
|
||
sent_at TIMESTAMPTZ
|
||
failure_reason TEXT
|
||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||
```
|
||
|
||
`createSendQueueRepo({ tableName, payloadSchema, allowedActions })`
|
||
(`src/server/src/shared/sendQueue/SendQueueRepo.ts:56-139`) returns a typed
|
||
`{ enqueue, listPending, markResult, count }` repo. The factory validates the
|
||
`tableName` against `^[a-z_][a-z0-9_]*\.[a-z_][a-z0-9_]*$` (line 54) because
|
||
table names are concatenated into SQL.
|
||
|
||
| Module | Table | Allowed actions | Admin enqueue route | Client pending route | Client ack route | `/my/*` write routes |
|
||
|------------|--------------------------------|---------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------|--------------------------------------------|---------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||
| iMessage | `icloud.send_queue` (legacy) | bespoke (`entities/send-queue/schema.ts:5-23`) | `POST /admin/send-queue/enqueue` | `GET /client/imessage/send-queue/pending` | `POST /client/imessage/send-queue/:id/result` | none (sends initiated via `/admin/send-queue/enqueue`) |
|
||
| iCal | `icloud.calendar_send_queue` | `create_event`, `update_event`, `delete_event` (`entities/calendarSendItem/types.ts:24`) | `POST /admin/calendar-send-queue/enqueue` | `GET /client/ical/send-queue/pending` | `POST /client/ical/send-queue/:id/result` | `POST /my/calendar/events`, `PUT /my/calendar/events/:id`, `DELETE /my/calendar/events/:id` (`surfaces/my/calendar.ts:80,90,102`) |
|
||
| iReminders | `icloud.reminder_send_queue` | `create_reminder`, `update_reminder`, `delete_reminder` (`entities/reminderSendItem/types.ts:16-18`) | `POST /admin/reminder-send-queue/enqueue` | `GET /client/ireminders/send-queue/pending` | `POST /client/ireminders/send-queue/:id/result` | `POST /my/reminders/`, `PUT /my/reminders/:id`, `DELETE /my/reminders/:id` (`surfaces/my/reminders.ts:73,87,103`) |
|
||
| iNotes | `icloud.note_send_queue` | `create_note`, `update_note`, `delete_note` (`entities/noteSendItem/types.ts:12`) | `POST /admin/note-send-queue/enqueue` | `GET /client/inotes/send-queue/pending` | `POST /client/inotes/send-queue/:id/result` | `POST /my/notes/`, `PUT /my/notes/:id`, `DELETE /my/notes/:id` (`surfaces/my/notes.ts:66,76,88`) |
|
||
|
||
Routes verified against `src/server/src/surfaces/{client,my,admin}/*.ts` and
|
||
wired up in `src/server/src/surfaces/{client,my,admin}/index.ts:11-19`.
|
||
|
||
## Cross-cutting invariants
|
||
|
||
1. **One sync manager per module.** Every module is a `final class` extending
|
||
`BaseSyncManager<ModuleStats, ModuleSyncError>`. Module-specific
|
||
`@Published` state (e.g. `contactSyncInfo` on iMessage, `currentOperation`
|
||
text on iCal) lives on the subclass.
|
||
2. **One send-queue contract.** All non-legacy modules go through
|
||
`createSendQueueRepo`. The iMessage `icloud.send_queue` table predates the
|
||
factory and stays on its own bespoke schema; the Mac client still polls it
|
||
via the same generic `SendQueueClient<IMessageSendTransport>`
|
||
(`@packages/imessage/Sources/IMessageSync/SyncManager.swift:70-86`).
|
||
3. **Single AppleScript escape helper.** `AppleScriptEscape.quote(_)` is the
|
||
one place strings get interpolated into AppleScript source.
|
||
4. **Watermarks are `lastSync` dates per module**, keyed by `persistenceKey` in
|
||
`UserDefaults`. The base updates them at cycle end; modules read them at
|
||
cycle start to scope incremental fetches.
|
||
5. **Authorization is opt-in.** `BaseSyncManager.requestAuthorization()`
|
||
defaults to `true`. Modules that need permission (EventKit for iCal /
|
||
iReminders, Automation for iMail / iNotes, Photos for iPhoto) override it.
|
||
6. **One blob-storage port (server).** Binary blobs go through the
|
||
`ObjectStore` interface (`src/server/src/features/iphoto/storage/`), never a
|
||
vendor path. The `S3Adapter` (`Bun.S3Client`) is **always SigV4-signed** and
|
||
works against any S3-compatible store; the `LocalAdapter` is the dev
|
||
on-disk default. Out-of-band consumers get short-lived **presigned** URLs,
|
||
never bare object URLs. Selection is config (`STORAGE_BACKEND`); deployment
|
||
topology is injected, never compiled in. See
|
||
[iPhoto › Server-side blob storage](modules/iphoto.md#server-side-blob-storage).
|
||
|
||
## Database schema (server)
|
||
|
||
Migrations are applied in order at boot
|
||
(`src/server/src/app/server.ts:36-52`). Tables live in the `icloud` Postgres
|
||
schema.
|
||
|
||
- `devices`, `contacts`, `conversations`, `messages` — iMessage
|
||
- `albums`, `photos` — iPhoto
|
||
- `calendars`, `events` — iCal
|
||
- `reminders` — iReminders
|
||
- `notes` — iNotes
|
||
- `calls` — iCalls (read-only)
|
||
- `send_queue` (legacy iMessage), `calendar_send_queue`, `reminder_send_queue`,
|
||
`note_send_queue` — outbound queues
|
||
- `prospects` — unrelated prospector feature
|
||
|
||
LocalWebServer (on the Mac, port 8765) also exposes `/api/calls` (direct recent
|
||
calls) and includes `callHistoryDb` in `/api/diagnostics` for agent/tool use
|
||
and the Dashboard "Native Subsystems" card.
|
||
|
||
Mail is read-only via Mail.app's own storage (no first-class `mail` entity at
|
||
the time of writing); ingestion goes through `features/imail/`
|
||
(`surfaces/client/imail.ts:35-41`).
|