# Handoff 03: Outbox — queued, paced, prioritized, time-aware outbound send

_For a coder agent in `@mac-sync`. Created 2026-06-28. Builds on Handoff 02 (calls — done). This is the outbound counterpart: a write-to outbox the triage drops replies into, which macsync sends intelligently._

## Why this exists
Tonight the inbox-triage workflow (Executor) sent ~30 replies by calling an iMessage MCP directly. That exposed real problems:
- **Parallel sends overload** the bridge ("stream closed" after ~8 at once) → had to hand-group into 5s.
- **The MCP reports false success** — `send_imessage` says "sent" even when it bounces (`error=22` = recipient not on iMessage = Android). ~13 of 30 silently failed.
- **Android needs SMS**, recovered manually via AppleScript `send … of (service whose service type = SMS)`.
- **No pacing, no scheduling, no priority, no undo window.**

The fix Quinn wants: **macsync exposes an outbox. The triage just writes rows. macsync processes them — queued, paced, prioritized, channel-smart, time-aware — and records true delivery status.** This subsumes the open "Android/SMS" item from Handoff 02.

## Current state (reuse, don't reinvent)
- **Send-queue infra already exists:** per-module `icloud.<module>_send_queue` tables via the `createSendQueueRepo` factory; the Mac polls them with the generic `SendQueueClient<Transport>` (iMessage 30s; calendar/reminders/notes 60s). iMessage's legacy `icloud.send_queue` is the bespoke precedent. Contract doc: `src/server/SEND_QUEUE_CONTRACTS.md`.
- **Send transport:** the Mac client already sends iMessage. The SMS path (Messages "SMS" service via the iPhone's Text Message Forwarding relay) is proven to work from AppleScript tonight — wire it as a fallback transport.
- Shared: `src/server/src/shared/sendQueue/SendQueueRepo.ts`, `shared/db/index.ts`, `shared/typedstream/decode.ts`.

## Goal
A first-class **outbox**: triage inserts a row `{recipient, body, priority, channel_pref, not_before, dedupe_key}`; macsync sends it at a smart time over the best channel, retries, and writes back true status `{queued→sending→sent→delivered | failed, channel_used, error, attempts}`. No more false success, no manual SMS, no overload.

## Subtasks
1. **Outbox table + repo** via `createSendQueueRepo` (`icloud.outbox_send_queue` or reuse iMessage queue if cleaner). Columns: recipient, body, priority (int), channel_pref (auto|imessage|sms), not_before (ts, for scheduling/undo), attempts, max_attempts, status, channel_used, error, dedupe_key, created_at, sent_at. Index on (status, priority desc, not_before).
2. **Write path for triage.** Smallest surface for the triage client to enqueue (MCP tool or `/my/outbox` POST) — validated, returns row id. Dedupe on `dedupe_key` (e.g. `recipient+hash(body)+day`) so re-runs don't double-send.
3. **Channel-smart sender (Mac client).** Try iMessage; on `error=22`/not-registered, **fall back to SMS** via the relay. Record `channel_used`. **Verify real status** by reading back `chat.db` (`is_sent`, `error`) — never trust the send call's own "ok".
4. **Pacing + priority.** Process highest-priority first; **rate-limit** (jittered, e.g. 1 every 20–90s, cap per minute) so we never trip the overload. Respect `not_before`.
5. **Send-timing scheduler** — set `not_before` from the methodology below (or accept an explicit one from triage). This is the core new logic.
6. **Status write-back + retries.** Surface delivery status for the triage to poll; retry transient failures with backoff up to `max_attempts`; mark hard-fails.
7. **Tests.** Repo SQL (priority/not_before ordering, dedupe), channel fallback, pacing, and the scheduler (table below). `make build`/`make test` green.

## Send-timing methodology (the scheduler) — propose & implement
Goal: **guess a good `not_before` send time** for each reply from (their last-inbound time+day) × (current time+day) × (lead priority). Be timely for hot leads, human-paced otherwise, and avoid dead-of-night cold-texting that reads as desperate.

**Day-parts (recipient-local; infer tz from area code, else default America/New_York where Quinn is):**
- DEAD 03:00–09:00 · MORNING 09:00–12:00 · AFTERNOON 12:00–17:00 · EVENING 17:00–22:00 · LATE 22:00–03:00.
- EVENING + LATE are prime for this work; DEAD is the no-cold-send zone.

**Recency → priority/SLA (drives queue order):**
| recency of their msg | priority | target send |
|---|---|---|
| < 60 min (LIVE) | HOT | now + 60–180s jitter (also covers undo window) — send regardless of day-part, they're looking |
| 1–6 h | HIGH | within current send-friendly window |
| 6–24 h | NORMAL | next send-friendly window, matched to their day-part |
| 1–3 d | LOW | quiet batch, OF-leaning |
| > 3 d | STALE | OF redirect or skip |

**Rules for the `not_before` guess (cold = recency ≥ 60 min):**
1. **Match their rhythm:** people are reachable around when they reached out → bias the send into the **same day-part** they messaged in (late-night texter → send evening/late; lunchtime texter → midday).
2. **Send-friendly window = ~09:00–01:00.** If now is inside it → send now (paced). If now is in DEAD hours → set `not_before` to ~10:00 today (or their matched day-part), unless HOT.
3. **Day-of-week weighting:** Fri/Sat nights + holidays (e.g. Pride) → LATE is prime, send promptly even at 1–2am. Weekday 09:00–17:00 → lower booking intent, deprioritize vs evening. Sun daytime → moderate.
4. **Quiet-hours guardrail:** no cold sends 03:00–09:00 local; HOT/LIVE may override.
5. **Pacing:** even when "send now," the queue spaces sends (jitter) — never a burst.
6. **Cooldown/dedupe:** never two sends to a lead within a cooldown; never resend same body.

**Reference shape:**
```
schedule(recv_at, now, priority):
  if now - recv_at < 60m: return now + rand(60,180)s          # LIVE/HOT
  win = send_friendly(now, dow)                                 # 09:00–01:00, weekend-night extended
  if now in win: return now + paced_jitter()
  return next_open(win, prefer_daypart=daypart(recv_at))        # e.g. 10:00, biased to their rhythm
```
Make the windows/weights config constants, not magic numbers, and log the chosen `not_before` + reason for tuning.

## Constraints (from `@mac-sync/CLAUDE.md`)
NEVER commit · NEVER `pkill node` · no `file:`/`link:` in package.json · complete production code only (STOP→REPORT→WAIT if blocked) · reuse the send-queue factory + `shared/db` singleton · single AppleScript escape helper for any SMS AppleScript.

## Acceptance
- Triage enqueues a reply and macsync sends it over the right channel (iMessage, SMS fallback) with **true** status written back.
- No bursts (paced); priority + `not_before` honored; dedupe prevents double-sends.
- Scheduler sets sane `not_before` per the methodology (tests cover LIVE/cold/dead-hours/weekend-night cases).
- `make build` + `make test` green.

## Hardening notes (from the 2026-06-28 local-stopgap dupe incident)
A manual stopgap (no outbox yet) spammed numbers 4–21× because retries were unbounded. The outbox must guarantee:
- **Per-recipient cooldown:** do not send a NEW message to a recipient who has a successful send within the last ~60 min (separate from `dedupe_key`, which only stops identical bodies).
- **Bounded retry:** a bounce/failure may retry only within a short window (≤~5 min) up to `max_attempts`, then go TERMINAL — never an unbounded/historical re-sweep.
- **Recovery detection caveat:** recovered SMS rows in `chat.db` have `handle_id=0`; detect prior recovery by joining `chat_message_join`→`chat` (per-conversation latest outbound), never by `message.handle_id`, or the "already sent?" check silently fails and re-sends.

## Out of scope
Triage classification/copy (Executor side); commit/deploy.

---

# Implementation log (2026-06-28, @mac-sync coder)

_Appended below the spec; section above left for the originating agent._

## Status: server-side foundation + scheduler DONE & tested; Mac-client sender remains

### Subtask 5 — Send-timing scheduler (DONE, the named centerpiece) ✅ tested green
- `src/server/src/features/outbox/scheduler.ts` — pure, deterministic (`now` + jitter injected). `derivePriority` (recency→HOT/HIGH/NORMAL/LOW/STALE), `dayPartForHour`, tz-aware `schedule()` implementing the methodology: LIVE<60m → prompt+undo window (ignores day-part); cold in send-friendly window (09:00–01:00, **weekend-night extended to 03:00** Fri/Sat) → paced jitter; otherwise **defer to next humane opening biased into the lead's day-part** (lunchtime texter→12:00, evening→17:00, late→22:00), quiet-hours 03:00–09:00 guarded. Windows/weights are config constants (`DEFAULT_SCHEDULER_CONFIG`); chosen `not_before` + reason returned for tuning.
- `src/server/src/features/outbox/timezone.ts` — `timezoneForHandle` (area-code → continental US zone, default Eastern). Separate from prospect's `metroFromHandle` (clock vs marketing question).
- **`bun test scheduler.test.ts` → 12/12 pass** (LIVE/HOT, dead-hours override, in-window weekday, rhythm-biased defer ×2, weekend-night Sat-02:00 in-window vs Wed-02:00 deferred, tz mapping). These RUN here — no DB dependency.

### Subtask 1 — Outbox table + repo (DONE) — dedicated schema, not the generic factory
- `src/server/src/entities/outbox/` — `macsync.outbox` (migration `2026-06-28_outbox_initial`, registered in `app/server.ts`). First-class `priority`/`not_before`/`channel_*`/`attempts`/`dedupe_key` columns (the factory's JSONB payload can't be indexed for drain order). Indexes: `idx_outbox_drain (status, priority DESC, not_before ASC)`, partial-unique `ux_outbox_dedupe (device_id, dedupe_key) WHERE dedupe_key IS NOT NULL`.
- `repo.ts`: `enqueueOutbox` (ON CONFLICT DO NOTHING → returns existing id, `deduped:true`), `listDue` (queued + due, hottest first), `markSending` (attempts++), `markResult` (failed re-queues while attempts remain else hard-fails; sent/delivered terminal), `getOutbox`, `countByStatus`.

### Subtask 2 — Write path for triage (DONE)
- `features/outbox/service.ts` `enqueueReply()` — derives tz, runs the scheduler (unless caller pins `notBefore`), maps priority label→numeric weight, default dedupe key `recipient:fnv1a(body):day`.
- `surfaces/my/outbox.ts` — `POST /my/outbox` (validated enqueue; 201 new / 200 deduped), `GET /my/outbox/stats`, `GET /my/outbox/:id` (poll status).

### Subtask 6 (server half) — Status write-back + retries (DONE)
- `surfaces/client/outbox.ts` — `GET /client/outbox/due`, `POST /client/outbox/:id/sending`, `POST /client/outbox/:id/result`. Retry/terminal logic lives in `markResult`.
- **Security (IDOR fix):** `/client/*` uses per-device token auth, so the mutation endpoints scope to `c.get('deviceId')` — `markSending`/`markResult` add `AND device_id = $N ... RETURNING id` and the surface returns 404 when no owned row matches. A device token cannot claim/complete another device's rows. (The separate flag on `/my/calls` does not apply — `/my/*` is single-tenant service-token auth; `deviceId` there is a filter, not a tenant boundary, identical to every other `/my` surface.)

### Subtask 7 — Tests
- Scheduler test: **runs + green here** (12/12).
- `src/server/src/test/outbox.repo.test.ts` — FakePool repo test (dedupe, drain ordering, retry/terminal). ⚠️ Can't execute here — imports `@/shared/db` → offline `@lilith` registry (`localhost:4874`). Run where registry is up, same as `calls.repo.test.ts`.

### Subtasks 3 & 4 — Mac-client sender (DONE) ✅ builds + tests green
- `@packages/imessage/Sources/IMessageSync/OutboxSendManager.swift` — `@MainActor` paced drain. **Self-rescheduling Timer** (not fixed poll): after each send it waits a jittered 20–90s before the next, so a backlog **can't burst** the bridge (the "stream closed after ~8" failure). Idles at 30s when empty.
- Per item: `POST /client/outbox/:id/sending` (claim → prevents re-drain double-send) → `SendService.shared.send` (existing **iMessage→SMS fallback + chat.db verification** — never trusts osascript) → `POST /client/outbox/:id/result` with **true** status + **`channelUsed`**.
- `Sender.swift`: `send()` now also returns the **channel actually used** (`normalizeChannel` → `imessage`/`sms`), so the server records `channel_used`. Only call site (the legacy `sendQueueClient` closure) is named-access — unaffected.
- APIClient: `fetchOutboxDue` / `claimOutbox` / `reportOutboxResult` + `OutboxAPI` protocol (so the manager is fakeable). Wired into `IMessageSyncManager` lifecycle (`didStartSync`/`willStopSync`) beside the legacy `sendQueueClient`.
- **`swift build` (full MacSyncApp) green; `OutboxSendTests` 3/3 pass** (pacing bounds + channel normalization).

**One follow-up (server-side, noted not built):** if a send's result POST fails after the claim, the row is stuck in `sending` (listDue is queued-only, never refetches it). Add a periodic server sweep `UPDATE macsync.outbox SET status='queued' WHERE status='sending' AND updated_at < now() - interval '10 min'` as the retry backstop.

### Acceptance check
- Scheduler sets sane `not_before` per methodology ✅ (tests cover LIVE/cold/dead-hours/weekend-night)
- Outbox table/repo/write-path/status-writeback ✅ (server)
- Channel-smart send (iMessage→SMS) + **true** verified status + `channel_used` ✅ (`OutboxSendManager` + `Sender.swift`)
- Pacing/priority/`not_before`/dedupe ✅ (server scheduler + client paced drain)
- IDOR guard on client mutations ✅ (device-scoped)
- `make build` green ✅ (full MacSyncApp links); Swift tests green ✅ (scheduler 12/12, outbox-send 3/3); server repo tests written but ⚠️ unrun (offline `@lilith` registry)

**Status: DONE on the @mac-sync side.** Remaining is running the server repo tests where the registry is reachable.

### Follow-up — stale-`sending` sweep (DONE, 2026-06-28)
The optional retry backstop is now built.
- `entities/outbox/repo.ts` `requeueStaleSending(pool, olderThanIso)` — `UPDATE … WHERE status='sending' AND updated_at < $1`, attempt-aware: re-queues while attempts remain else hard-fails (same policy as `markResult`), `RETURNING id, status`.
- `features/outbox/sweep.ts` — `startOutboxSweep()` periodic worker (default every 5 min, reclaim `sending` older than 10 min). Non-overlapping (`sweeping` guard), never throws into the timer, `unref`'d so it doesn't hold the process open. Wired in `app/server.ts` beside `startEmbeddingWorker()`.
- Tests: 2 cases added to `src/test/outbox.repo.test.ts` (reclaim+requeue/hard-fail SQL + params; empty when nothing stale). ⚠️ Unrun here (offline `@lilith` registry), same caveat as the rest.
