macsync/docs/architecture.md

Architecture

@mac-sync is three processes on three machines, connected by HTTPS and one Postgres database. This document describes the components and the invariants that hold across modules.

System overview

flowchart LR
    subgraph Mac["Mac (plum)"]
        app["MacSyncApp<br/>menu-bar executable"]
        base["BaseSyncManager&lt;Stats, SyncError&gt;"]
        sqc["SendQueueClient&lt;Transport&gt;"]
        lws["LocalWebServer<br/>localhost:8765"]
        app -->|owns 6 of| base
        app -->|owns 4 of| sqc
        app -->|owns 1| lws
    end

    subgraph Server["Server (black, 10.0.0.11:3201)"]
        hono["Hono + Bun"]
        pg[("PostgreSQL<br/>icloud schema")]
        hono --> pg
    end

    subgraph Browser["Browser"]
        spa["React SPA (web/)"]
    end

    app <-->|"HTTPS /client/*<br/>device-token auth"| hono
    spa <-->|"HTTPS /my/*<br/>SSO auth"| hono
    spa -.->|"settings on localhost:8765"| lws

Sources:

• Mac client entrypoint: src/client/ (executable target Package.swift:97-110)
• Server entrypoint: src/server/src/app/server.ts:31-86
• Web SPA root: web/src/App.tsx:49-71
• LocalWebServer routes: @packages/shared/Sources/MacSyncShared/WebServer/LocalWebServer.swift:42-110
• Deployment topology: app.manifest.yaml:14-58 (plum = Mac client, black = server)

Module taxonomy

Every module ships four (or three) Swift files in @packages/<module>/Sources/<Module>Sync/:

• Reader.swift — talks to the local OS (chat.db, EventKit, Mail.app, Notes.app, PhotoKit)
• SyncManager.swift — subclasses BaseSyncManager, drives the cycle
• APIClient.swift — typed methods over BaseAPIClient (HTTPS to the server)
• Sender.swift — only on bidirectional modules; applies pending items to the OS

flowchart TD
    base["BaseSyncManager<Stats, SyncError>"]
    base --> im["IMessageSync<br/>30s read, 30s send"]
    base --> ip["IPhotoSync<br/>300s read, no send"]
    base --> imail["IMailSync<br/>300s read, no send (*)"]
    base --> ical["ICalSync<br/>300s read, 60s send"]
    base --> irem["IReminderSync<br/>300s read, 60s send"]
    base --> inote["INoteSync<br/>600s read, 60s send"]
    base --> icall["ICallsSync<br/>120s read, read-only"]

Interval citations (each is the literal numeric arg to super.init(...) / SendQueueClient(...)):

Module	Read tick	Outbound tick
iMessage	@packages/imessage/Sources/IMessageSync/SyncManager.swift:93	@packages/imessage/Sources/IMessageSync/SyncManager.swift:74
iPhoto	@packages/iphoto/Sources/IPhotoSync/SyncManager.swift:99	n/a
iMail	@packages/imail/Sources/IMailSync/SyncManager.swift:61	n/a
iCal	@packages/ical/Sources/ICalSync/SyncManager.swift:80	@packages/ical/Sources/ICalSync/SyncManager.swift:61
iReminders	@packages/ireminders/Sources/IReminderSync/SyncManager.swift:67	@packages/ireminders/Sources/IReminderSync/SyncManager.swift:57
iNotes	@packages/inotes/Sources/INoteSync/SyncManager.swift:79	@packages/inotes/Sources/INoteSync/SyncManager.swift:60
iCalls	@packages/icalls/Sources/ICallsSync/SyncManager.swift (120s)	n/a (read-only)

(*) iMail bidirectional path is not wired; see known limitations.

Shared abstractions

Three files in @packages/shared/Sources/MacSyncShared/Sync/ are the single source of truth for cross-module lifecycle:

BaseSyncManager

open class BaseSyncManager<Stats, SyncError>: ObservableObject (@packages/shared/Sources/MacSyncShared/Sync/BaseSyncManager.swift:28-186).

Owns:

• @Published isSyncing, lastSyncCompletedAt, currentOperation, syncError, stats (lines 32-36)
• private(set) var lastSync: Date? watermark, persisted under "<persistenceKey>.lastSync" in UserDefaults (lines 42, 179-185)
• private var syncTimer: Timer? with final startSync() / stopSync() / syncNow() (lines 82-110)
• An authorization gate: gatedRunCycle calls overridable isAuthorized() / requestAuthorization() / onAuthorizationDenied() (lines 112-121, 157-165)
• Two extension points for Senders: didStartSync() / willStopSync() (lines 170-175) — SendQueueClients start and stop here

All lifecycle methods are final. Subclasses override only performSync() plus the authorization and start/stop hooks. The contract therefore cannot drift between modules.

SendQueueClient<Transport: SendQueueTransport>

@MainActor public final class SendQueueClient (@packages/shared/Sources/MacSyncShared/Sync/SendQueueClient.swift:45-126).

A generic poll/apply/ack loop:

1. Every interval seconds, drainOnce() calls transport.fetchPending().
2. For each item, the apply: @MainActor @Sendable (Item) -> SendQueueApplyResult closure runs the OS-side write (EventKit save, AppleScript, AppleScript+iMessage).
3. The result is reported via transport.reportResult(id:status:error:).

Transport is the only protocol; module senders adapt their typed APIClient calls (e.g. getPendingCalendarSends(), reportCalendarSendResult(...)) into the three protocol methods. The polling loop is therefore written exactly once.

SyncConnectionErrorHeuristic

A lowercase substring match over error.localizedDescription (@packages/shared/Sources/MacSyncShared/Sync/SyncConnectionError.swift:11-20): "network" | "connection" | "timeout" | "unreachable" | "could not connect" | "offline". Every module's SyncError was duplicating this; now there is one helper.

AppleScriptEscape

@packages/shared/Sources/MacSyncShared/Util/AppleScriptEscape.swift:13-19: escapes \, ", \n, \r for interpolation into AppleScript string literals. Used by iMessage, iMail, and iNotes Senders.

Send-queue contract

Per-module Postgres tables with identical shape, built by the sendQueueTableSql(tableName) helper (src/server/src/shared/sendQueue/SendQueueRepo.ts:146-168):

id              UUID PK DEFAULT gen_random_uuid()
device_id       UUID NOT NULL REFERENCES icloud.devices(id) ON DELETE CASCADE
action          TEXT NOT NULL
payload         JSONB NOT NULL
status          TEXT NOT NULL DEFAULT 'queued'    -- 'queued' | 'sent' | 'failed'
sent_at         TIMESTAMPTZ
failure_reason  TEXT
created_at      TIMESTAMPTZ NOT NULL DEFAULT now()
updated_at      TIMESTAMPTZ NOT NULL DEFAULT now()

createSendQueueRepo({ tableName, payloadSchema, allowedActions }) (src/server/src/shared/sendQueue/SendQueueRepo.ts:56-139) returns a typed { enqueue, listPending, markResult, count } repo. The factory validates the tableName against ^[a-z_][a-z0-9_]*\.[a-z_][a-z0-9_]*$ (line 54) because table names are concatenated into SQL.

Module	Table	Allowed actions	Admin enqueue route	Client pending route	Client ack route	/my/* write routes
iMessage	icloud.send_queue (legacy)	bespoke (entities/send-queue/schema.ts:5-23)	POST /admin/send-queue/enqueue	GET /client/imessage/send-queue/pending	POST /client/imessage/send-queue/:id/result	none (sends initiated via /admin/send-queue/enqueue)
iCal	icloud.calendar_send_queue	create_event, update_event, delete_event (entities/calendarSendItem/types.ts:24)	POST /admin/calendar-send-queue/enqueue	GET /client/ical/send-queue/pending	POST /client/ical/send-queue/:id/result	POST /my/calendar/events, PUT /my/calendar/events/:id, DELETE /my/calendar/events/:id (surfaces/my/calendar.ts:80,90,102)
iReminders	icloud.reminder_send_queue	create_reminder, update_reminder, delete_reminder (entities/reminderSendItem/types.ts:16-18)	POST /admin/reminder-send-queue/enqueue	GET /client/ireminders/send-queue/pending	POST /client/ireminders/send-queue/:id/result	POST /my/reminders/, PUT /my/reminders/:id, DELETE /my/reminders/:id (surfaces/my/reminders.ts:73,87,103)
iNotes	icloud.note_send_queue	create_note, update_note, delete_note (entities/noteSendItem/types.ts:12)	POST /admin/note-send-queue/enqueue	GET /client/inotes/send-queue/pending	POST /client/inotes/send-queue/:id/result	POST /my/notes/, PUT /my/notes/:id, DELETE /my/notes/:id (surfaces/my/notes.ts:66,76,88)

Routes verified against src/server/src/surfaces/{client,my,admin}/*.ts and wired up in src/server/src/surfaces/{client,my,admin}/index.ts:11-19.

Cross-cutting invariants

1. One sync manager per module. Every module is a final class extending BaseSyncManager<ModuleStats, ModuleSyncError>. Module-specific @Published state (e.g. contactSyncInfo on iMessage, currentOperation text on iCal) lives on the subclass.
2. One send-queue contract. All non-legacy modules go through createSendQueueRepo. The iMessage icloud.send_queue table predates the factory and stays on its own bespoke schema; the Mac client still polls it via the same generic SendQueueClient<IMessageSendTransport> (@packages/imessage/Sources/IMessageSync/SyncManager.swift:70-86).
3. Single AppleScript escape helper. AppleScriptEscape.quote(_) is the one place strings get interpolated into AppleScript source.
4. Watermarks are lastSync dates per module, keyed by persistenceKey in UserDefaults. The base updates them at cycle end; modules read them at cycle start to scope incremental fetches.
5. Authorization is opt-in. BaseSyncManager.requestAuthorization() defaults to true. Modules that need permission (EventKit for iCal / iReminders, Automation for iMail / iNotes, Photos for iPhoto) override it.
6. One blob-storage port (server). Binary blobs go through the ObjectStore interface (src/server/src/features/iphoto/storage/), never a vendor path. The S3Adapter (Bun.S3Client) is always SigV4-signed and works against any S3-compatible store; the LocalAdapter is the dev on-disk default. Out-of-band consumers get short-lived presigned URLs, never bare object URLs. Selection is config (STORAGE_BACKEND); deployment topology is injected, never compiled in. See iPhoto › Server-side blob storage.

Database schema (server)

Migrations are applied in order at boot (src/server/src/app/server.ts:36-52). Tables live in the icloud Postgres schema.

• devices, contacts, conversations, messages — iMessage
• albums, photos — iPhoto
• calendars, events — iCal
• reminders — iReminders
• notes — iNotes
• send_queue (legacy iMessage), calendar_send_queue, reminder_send_queue, note_send_queue — outbound queues
• prospects — unrelated prospector feature

Mail is read-only via Mail.app's own storage (no first-class mail entity at the time of writing); ingestion goes through features/imail/ (surfaces/client/imail.ts:35-41).