576496ca3e
Generalize the photos-originals rclone-mount pattern to a video-projects prefix so the video studio (and imajin ETL, per storage-portability-plan §2.3) can read/write multi-GB project sources/renders as local files while only hot data stays resident on plum (bounded VFS LRU cache). Lets a small-disk laptop work with large footage without filling APFS. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
116 lines
4.9 KiB
Markdown
116 lines
4.9 KiB
Markdown
# iPhoto (`IPhotoSync`)
|
|
|
|
## Purpose
|
|
|
|
Sync the user's Photos library metadata to the server and upload binaries on
|
|
demand.
|
|
|
|
## Direction
|
|
|
|
Read-only Mac to server. There is no Sender; the web cannot push photos back.
|
|
See the rationale in [known-limitations](../known-limitations.md#iphoto-has-no-sender).
|
|
|
|
## OS surface
|
|
|
|
PhotoKit (`PhotosUI`/`Photos.framework`). Requires Photos library access
|
|
prompted by `IPhotoSync.requestAuthorization()`
|
|
(`@packages/iphoto/Sources/IPhotoSync/SyncManager.swift:118-120, 122-124`).
|
|
|
|
## Files
|
|
|
|
- `Reader.swift` — `PhotosLibraryReader.shared`; enumerates assets and albums
|
|
via `PHAsset` / `PHAssetCollection`.
|
|
- `APIClient.swift` — metadata sync, binary upload, stats
|
|
(`@packages/iphoto/Sources/IPhotoSync/APIClient.swift:129-240`):
|
|
- `syncPhotos`, `syncAlbums`
|
|
- `getPendingUploads` -> `uploadPhoto` (data-in-memory) or `uploadPhotoFromURL` (streamed)
|
|
- `getStats`
|
|
- `SyncManager.swift` — orchestrates metadata sync, then a bounded-concurrency
|
|
upload pass: `metadataBatchSize = 100`, `maxConcurrentUploads = 4`
|
|
(`@packages/iphoto/Sources/IPhotoSync/SyncManager.swift:90-92`).
|
|
- No `Sender.swift`.
|
|
|
|
## Timing
|
|
|
|
- Read interval: **300s**
|
|
(`@packages/iphoto/Sources/IPhotoSync/SyncManager.swift:99`).
|
|
- Metadata batch size: 100.
|
|
- Max concurrent uploads: 4.
|
|
|
|
## Server surface
|
|
|
|
- Entity tables: `icloud.albums`, `icloud.photos`
|
|
(`src/server/src/app/server.ts:42-43`).
|
|
- Client routes (`src/server/src/surfaces/client/iphoto.ts`):
|
|
- `GET /client/iphoto/stats` (line 53)
|
|
- `GET /client/iphoto/upload/pending` (line 58)
|
|
- `POST /client/iphoto/sync` (line 63)
|
|
- `POST /client/iphoto/albums` (line 69)
|
|
- `POST /client/iphoto/upload/:localIdentifier` (line 75)
|
|
- Web routes (`src/server/src/surfaces/my/photos.ts`):
|
|
- `GET /my/photos/` (line 12)
|
|
- `GET /my/photos/albums` (line 18)
|
|
- `GET /my/photos/albums/:albumId` (line 22)
|
|
- No admin send-queue surface.
|
|
|
|
## Server-side blob storage
|
|
|
|
Photo originals + derivatives go through a provider-agnostic `ObjectStore` port
|
|
(`src/server/src/features/iphoto/storage/`), never a vendor-specific path:
|
|
|
|
- `index.ts` — the `ObjectStore` interface (`put`/`get`/`head`/`presignGet`),
|
|
the config factory `createObjectStore()`, an explicit-creds `createS3Store()`
|
|
(used by the imajin ETL source), and a memoized `defaultObjectStore()`.
|
|
- `s3.ts` — `S3Adapter` over `Bun.S3Client`, **always SigV4-signed**; works
|
|
against MinIO, DO Spaces, AWS, R2. Selected by `STORAGE_BACKEND=s3`.
|
|
- `local.ts` — `LocalAdapter` (dev), blobs on disk under `STORAGE_LOCAL_PATH`.
|
|
|
|
Consumers that fetch originals out-of-band (`face-worker`, the imajin ETL)
|
|
receive **short-lived presigned GET URLs** (`presignGet`), never a bare object
|
|
URL — matching a private, TLS-only bucket policy. Upload path: `service.ts`
|
|
`storePhotoBlob` → `store.put`; thumbnail/classify workers read via `store.get`.
|
|
|
|
See env in [dev-setup](../dev-setup.md#boot-the-server) and the design in
|
|
`.project/storage-portability-plan.md`.
|
|
|
|
## Optional: originals offload mount (per-host, e.g. plum)
|
|
|
|
An **opt-in** capability — not part of the default runtime, no code branches on
|
|
it. When a host needs photo originals off local disk, the Photos.app
|
|
`originals/<hex>/<UUID>.ext` subtree is served on demand by `rclone mount` over
|
|
the object store (DO Spaces) with a bounded LRU cache; the SQLite core
|
|
(`database/`) and derivatives (`resources/`) stay local on APFS. Both Photos.app
|
|
and `IPhotoSync` read the same mounted files.
|
|
|
|
- Mount runner: `deploy/photos-originals-mount.sh`.
|
|
- Installer (phased, idempotent; gated symlink repoint): `deploy/install-photos-mount.sh`.
|
|
- Seed (iCloud → Spaces, osxphotos + rclone, keyed by real bundle path):
|
|
`deploy/seed-originals-to-spaces.sh`. Shared Spaces/rclone config:
|
|
`deploy/lib/spaces-env.sh`.
|
|
- The server's blob ingest and the default `deploy/install.sh` have **zero**
|
|
dependency on this. Full rationale + gates (macFUSE approval, iCloud re-seed)
|
|
in `.project/storage-portability-plan.md`.
|
|
|
|
## Web surface
|
|
|
|
- Tab: `/photos` (`web/src/App.tsx:58`).
|
|
- API helpers: `web/src/api/photos.ts`.
|
|
|
|
## Known limitations
|
|
|
|
- No outbound path; web cannot add to the library. See
|
|
[known-limitations](../known-limitations.md#iphoto-has-no-sender).
|
|
- `uploadRate` / `bytesUploaded` statistics reset on every cold start; per
|
|
[known-limitations](../known-limitations.md#iphotostatsuploadrate-is-per-session-only).
|
|
- Server-side enrichment (face extraction via `face-worker`, classification via
|
|
`classify-worker`, thumbnails via `thumbnail-worker`) runs off the uploaded
|
|
blob; the Mac client itself only reads metadata + uploads binaries.
|
|
|
|
## Tests (`@packages/iphoto/Tests/IPhotoSyncTests/`)
|
|
|
|
- `ReaderTests.swift` — exercises `PhotosLibraryReader` helpers that don't
|
|
need a populated library.
|
|
- `SyncStatsTests.swift` — `uploadRate`/`eta` formulas
|
|
(`@packages/iphoto/Sources/IPhotoSync/SyncManager.swift:22-46`).
|
|
|
|
Not covered: actual PhotoKit enumeration, binary upload (both need the OS).
|