205 lines
13 KiB
Markdown
205 lines
13 KiB
Markdown
# Architecture
|
|
|
|
`@mac-sync` is three processes on three machines, connected by HTTPS and one
|
|
Postgres database. This document describes the components and the invariants
|
|
that hold across modules.
|
|
|
|
## System overview
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
subgraph Mac["Mac (plum)"]
|
|
app["MacSyncApp<br/>menu-bar executable"]
|
|
base["BaseSyncManager<Stats, SyncError>"]
|
|
sqc["SendQueueClient<Transport>"]
|
|
lws["LocalWebServer<br/>localhost:8765"]
|
|
app -->|owns 6 of| base
|
|
app -->|owns 4 of| sqc
|
|
app -->|owns 1| lws
|
|
end
|
|
|
|
subgraph Server["Server (black, 10.0.0.11:3201)"]
|
|
hono["Hono + Bun"]
|
|
pg[("PostgreSQL<br/>icloud schema")]
|
|
hono --> pg
|
|
end
|
|
|
|
subgraph Browser["Browser"]
|
|
spa["React SPA (web/)"]
|
|
end
|
|
|
|
app <-->|"HTTPS /client/*<br/>device-token auth"| hono
|
|
spa <-->|"HTTPS /my/*<br/>SSO auth"| hono
|
|
spa -.->|"settings on localhost:8765"| lws
|
|
```
|
|
|
|
Sources:
|
|
- Mac client entrypoint: `src/client/` (executable target `Package.swift:97-110`)
|
|
- Server entrypoint: `src/server/src/app/server.ts:31-86`
|
|
- Web SPA root: `web/src/App.tsx:49-71`
|
|
- LocalWebServer routes: `@packages/shared/Sources/MacSyncShared/WebServer/LocalWebServer.swift:42-110`
|
|
- Deployment topology: `app.manifest.yaml:14-58` (plum = Mac client, black = server)
|
|
|
|
## Module taxonomy
|
|
|
|
Every module ships four (or three) Swift files in `@packages/<module>/Sources/<Module>Sync/`:
|
|
|
|
- `Reader.swift` — talks to the local OS (chat.db, EventKit, Mail.app, Notes.app, PhotoKit)
|
|
- `SyncManager.swift` — subclasses `BaseSyncManager`, drives the cycle
|
|
- `APIClient.swift` — typed methods over `BaseAPIClient` (HTTPS to the server)
|
|
- `Sender.swift` — only on bidirectional modules; applies pending items to the OS
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
base["BaseSyncManager<Stats, SyncError>"]
|
|
base --> im["IMessageSync<br/>30s read, 30s send"]
|
|
base --> ip["IPhotoSync<br/>300s read, no send"]
|
|
base --> imail["IMailSync<br/>300s read, no send (*)"]
|
|
base --> ical["ICalSync<br/>300s read, 60s send"]
|
|
base --> irem["IReminderSync<br/>300s read, 60s send"]
|
|
base --> inote["INoteSync<br/>600s read, 60s send"]
|
|
```
|
|
|
|
Interval citations (each is the literal numeric arg to `super.init(...)` /
|
|
`SendQueueClient(...)`):
|
|
|
|
| Module | Read tick | Outbound tick |
|
|
|------------|--------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------|
|
|
| iMessage | `@packages/imessage/Sources/IMessageSync/SyncManager.swift:93` | `@packages/imessage/Sources/IMessageSync/SyncManager.swift:74` |
|
|
| iPhoto | `@packages/iphoto/Sources/IPhotoSync/SyncManager.swift:99` | n/a |
|
|
| iMail | `@packages/imail/Sources/IMailSync/SyncManager.swift:61` | n/a |
|
|
| iCal | `@packages/ical/Sources/ICalSync/SyncManager.swift:80` | `@packages/ical/Sources/ICalSync/SyncManager.swift:61` |
|
|
| iReminders | `@packages/ireminders/Sources/IReminderSync/SyncManager.swift:67` | `@packages/ireminders/Sources/IReminderSync/SyncManager.swift:57` |
|
|
| iNotes | `@packages/inotes/Sources/INoteSync/SyncManager.swift:79` | `@packages/inotes/Sources/INoteSync/SyncManager.swift:60` |
|
|
|
|
(*) iMail bidirectional path is not wired; see [known limitations](./known-limitations.md).
|
|
|
|
## Shared abstractions
|
|
|
|
Three files in `@packages/shared/Sources/MacSyncShared/Sync/` are the
|
|
single source of truth for cross-module lifecycle:
|
|
|
|
### `BaseSyncManager`
|
|
|
|
`open class BaseSyncManager<Stats, SyncError>: ObservableObject`
|
|
(`@packages/shared/Sources/MacSyncShared/Sync/BaseSyncManager.swift:28-186`).
|
|
|
|
Owns:
|
|
|
|
- `@Published isSyncing, lastSyncCompletedAt, currentOperation, syncError, stats` (lines 32-36)
|
|
- `private(set) var lastSync: Date?` watermark, persisted under
|
|
`"<persistenceKey>.lastSync"` in `UserDefaults` (lines 42, 179-185)
|
|
- `private var syncTimer: Timer?` with `final startSync() / stopSync() / syncNow()`
|
|
(lines 82-110)
|
|
- An authorization gate: `gatedRunCycle` calls overridable
|
|
`isAuthorized()` / `requestAuthorization()` / `onAuthorizationDenied()`
|
|
(lines 112-121, 157-165)
|
|
- Two extension points for Senders: `didStartSync()` / `willStopSync()`
|
|
(lines 170-175) — `SendQueueClient`s start and stop here
|
|
|
|
All lifecycle methods are `final`. Subclasses override only `performSync()` plus
|
|
the authorization and start/stop hooks. The contract therefore cannot drift
|
|
between modules.
|
|
|
|
### `SendQueueClient<Transport: SendQueueTransport>`
|
|
|
|
`@MainActor public final class SendQueueClient`
|
|
(`@packages/shared/Sources/MacSyncShared/Sync/SendQueueClient.swift:45-126`).
|
|
|
|
A generic poll/apply/ack loop:
|
|
|
|
1. Every `interval` seconds, `drainOnce()` calls `transport.fetchPending()`.
|
|
2. For each item, the `apply: @MainActor @Sendable (Item) -> SendQueueApplyResult`
|
|
closure runs the OS-side write (EventKit save, AppleScript, AppleScript+iMessage).
|
|
3. The result is reported via `transport.reportResult(id:status:error:)`.
|
|
|
|
`Transport` is the only protocol; module senders adapt their typed
|
|
`APIClient` calls (e.g. `getPendingCalendarSends()`,
|
|
`reportCalendarSendResult(...)`) into the three protocol methods. The polling
|
|
loop is therefore written exactly once.
|
|
|
|
### `SyncConnectionErrorHeuristic`
|
|
|
|
A lowercase substring match over `error.localizedDescription`
|
|
(`@packages/shared/Sources/MacSyncShared/Sync/SyncConnectionError.swift:11-20`):
|
|
`"network" | "connection" | "timeout" | "unreachable" | "could not connect" | "offline"`.
|
|
Every module's `SyncError` was duplicating this; now there is one helper.
|
|
|
|
### `AppleScriptEscape`
|
|
|
|
`@packages/shared/Sources/MacSyncShared/Util/AppleScriptEscape.swift:13-19`:
|
|
escapes `\`, `"`, `\n`, `\r` for interpolation into AppleScript string
|
|
literals. Used by iMessage, iMail, and iNotes Senders.
|
|
|
|
## Send-queue contract
|
|
|
|
Per-module Postgres tables with identical shape, built by the
|
|
`sendQueueTableSql(tableName)` helper
|
|
(`src/server/src/shared/sendQueue/SendQueueRepo.ts:146-168`):
|
|
|
|
```
|
|
id UUID PK DEFAULT gen_random_uuid()
|
|
device_id UUID NOT NULL REFERENCES icloud.devices(id) ON DELETE CASCADE
|
|
action TEXT NOT NULL
|
|
payload JSONB NOT NULL
|
|
status TEXT NOT NULL DEFAULT 'queued' -- 'queued' | 'sent' | 'failed'
|
|
sent_at TIMESTAMPTZ
|
|
failure_reason TEXT
|
|
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
|
```
|
|
|
|
`createSendQueueRepo({ tableName, payloadSchema, allowedActions })`
|
|
(`src/server/src/shared/sendQueue/SendQueueRepo.ts:56-139`) returns a typed
|
|
`{ enqueue, listPending, markResult, count }` repo. The factory validates the
|
|
`tableName` against `^[a-z_][a-z0-9_]*\.[a-z_][a-z0-9_]*$` (line 54) because
|
|
table names are concatenated into SQL.
|
|
|
|
| Module | Table | Allowed actions | Admin enqueue route | Client pending route | Client ack route | `/my/*` write routes |
|
|
|------------|--------------------------------|---------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------|--------------------------------------------|---------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
| iMessage | `icloud.send_queue` (legacy) | bespoke (`entities/send-queue/schema.ts:5-23`) | `POST /admin/send-queue/enqueue` | `GET /client/imessage/send-queue/pending` | `POST /client/imessage/send-queue/:id/result` | none (sends initiated via `/admin/send-queue/enqueue`) |
|
|
| iCal | `icloud.calendar_send_queue` | `create_event`, `update_event`, `delete_event` (`entities/calendarSendItem/types.ts:24`) | `POST /admin/calendar-send-queue/enqueue` | `GET /client/ical/send-queue/pending` | `POST /client/ical/send-queue/:id/result` | `POST /my/calendar/events`, `PUT /my/calendar/events/:id`, `DELETE /my/calendar/events/:id` (`surfaces/my/calendar.ts:80,90,102`) |
|
|
| iReminders | `icloud.reminder_send_queue` | `create_reminder`, `update_reminder`, `delete_reminder` (`entities/reminderSendItem/types.ts:16-18`) | `POST /admin/reminder-send-queue/enqueue` | `GET /client/ireminders/send-queue/pending` | `POST /client/ireminders/send-queue/:id/result` | `POST /my/reminders/`, `PUT /my/reminders/:id`, `DELETE /my/reminders/:id` (`surfaces/my/reminders.ts:73,87,103`) |
|
|
| iNotes | `icloud.note_send_queue` | `create_note`, `update_note`, `delete_note` (`entities/noteSendItem/types.ts:12`) | `POST /admin/note-send-queue/enqueue` | `GET /client/inotes/send-queue/pending` | `POST /client/inotes/send-queue/:id/result` | `POST /my/notes/`, `PUT /my/notes/:id`, `DELETE /my/notes/:id` (`surfaces/my/notes.ts:66,76,88`) |
|
|
|
|
Routes verified against `src/server/src/surfaces/{client,my,admin}/*.ts` and
|
|
wired up in `src/server/src/surfaces/{client,my,admin}/index.ts:11-19`.
|
|
|
|
## Cross-cutting invariants
|
|
|
|
1. **One sync manager per module.** Every module is a `final class` extending
|
|
`BaseSyncManager<ModuleStats, ModuleSyncError>`. Module-specific
|
|
`@Published` state (e.g. `contactSyncInfo` on iMessage, `currentOperation`
|
|
text on iCal) lives on the subclass.
|
|
2. **One send-queue contract.** All non-legacy modules go through
|
|
`createSendQueueRepo`. The iMessage `icloud.send_queue` table predates the
|
|
factory and stays on its own bespoke schema; the Mac client still polls it
|
|
via the same generic `SendQueueClient<IMessageSendTransport>`
|
|
(`@packages/imessage/Sources/IMessageSync/SyncManager.swift:70-86`).
|
|
3. **Single AppleScript escape helper.** `AppleScriptEscape.quote(_)` is the
|
|
one place strings get interpolated into AppleScript source.
|
|
4. **Watermarks are `lastSync` dates per module**, keyed by `persistenceKey` in
|
|
`UserDefaults`. The base updates them at cycle end; modules read them at
|
|
cycle start to scope incremental fetches.
|
|
5. **Authorization is opt-in.** `BaseSyncManager.requestAuthorization()`
|
|
defaults to `true`. Modules that need permission (EventKit for iCal /
|
|
iReminders, Automation for iMail / iNotes, Photos for iPhoto) override it.
|
|
|
|
## Database schema (server)
|
|
|
|
Migrations are applied in order at boot
|
|
(`src/server/src/app/server.ts:36-52`). Tables live in the `icloud` Postgres
|
|
schema.
|
|
|
|
- `devices`, `contacts`, `conversations`, `messages` — iMessage
|
|
- `albums`, `photos` — iPhoto
|
|
- `calendars`, `events` — iCal
|
|
- `reminders` — iReminders
|
|
- `notes` — iNotes
|
|
- `send_queue` (legacy iMessage), `calendar_send_queue`, `reminder_send_queue`,
|
|
`note_send_queue` — outbound queues
|
|
- `prospects` — unrelated prospector feature
|
|
|
|
Mail is read-only via Mail.app's own storage (no first-class `mail` entity at
|
|
the time of writing); ingestion goes through `features/imail/`
|
|
(`surfaces/client/imail.ts:35-41`).
|