life-docs/features/assistant.md

Feature — Assistant

Multi-agent LLM chat system with WebSocket streaming, domain-routed agents, skill-based prompt injection, tool calling, voice synthesis, spaced repetition feedback, companion mode, and memory persistence.

---

Schema

conversations

Column	Type	Constraints	Description
id	uuid	PK
title	varchar(255)	nullable
model	varchar(100)	nullable	Override model
is_voice	boolean	default false	Voice conversation flag
type	varchar(50)	default 'human'	human, companion, imessage
project_key	varchar(100)	nullable	Scoped to project
archived_at	timestamp	nullable	Soft archive
active_agent_key	varchar(100)	nullable	Sticky agent routing
detected_domains	text[]	default '{}'	Accumulated domain tags

Indexes: archived_at, is_voice, type, project_key

messages

Column	Type	Constraints	Description
id	uuid	PK (auto-generated)
conversation_id	uuid	FK → conversations.id, ON DELETE CASCADE
role	enum	NOT NULL	user, assistant, system, tool
content	text	NOT NULL
model	varchar(100)	nullable	Model used for this message
token_count	integer	nullable
duration_ms	integer	nullable	Inference time
metadata	jsonb	default '{}'	Agent info, tool calls, etc.
parent_id	uuid	nullable	For branching conversations
created_at	timestamp	auto

Indexes: conversation_id, role, parent_id, created_at

message_feedback

Column	Type	Constraints	Description
id	uuid	PK
message_id	uuid	FK → messages.id, ON DELETE CASCADE
conversation_id	uuid	NOT NULL
feedback_type	varchar(20)	NOT NULL	positive, negative, correction
dimensions	jsonb	default []	Auto-scored and human-scored dimensions
reason	text	default ''
source	varchar(50)	default 'api'
incorporated	boolean	default false	Used in training export

Indexes: message_id, conversation_id

skills

Column	Type	Constraints	Description
id	uuid	PK
skill_key	varchar(100)	UNIQUE	Loaded from SKILL.md files
name	varchar(255)	NOT NULL
description	text	NOT NULL
category	varchar(50)	NOT NULL
system_prompt_fragment	text	NOT NULL	Injected into system prompt
frontmatter	jsonb	default '{}'	Parsed YAML frontmatter
is_enabled	boolean	default true
always_include	boolean	default false	Include in every prompt
user_invocable	boolean	default true
disable_model_invocation	boolean	default false
domain_module	varchar(100)	nullable
source_path	varchar(500)	nullable	Filesystem path
deleted_at	timestamptz	nullable	Soft delete

Indexes: skill_key (unique), category, is_enabled

agent_definitions

Column	Type	Constraints	Description
id	uuid	PK
agent_key	varchar(100)	UNIQUE	Loaded from AGENT.md files
name	varchar(255)	NOT NULL
description	text	NOT NULL
persona	text	NOT NULL	Full persona prompt
domains	text[]	default '{}'	Domain keywords for routing
model_preference	varchar(100)	nullable	Preferred model ID
persona_style	text[]	default '{}'	Style tags
tools_allowed	text[]	default '{}'	Tool whitelist
skills_filter	text[]	default '{}'	Skill key whitelist
session_modes	text[]	default '{chat}'	chat, voice, companion
always_available	boolean	default false
is_enabled	boolean	default true
frontmatter	jsonb	default '{}'
source_path	varchar(500)	nullable
deleted_at	timestamptz	nullable

Indexes: agent_key (unique), is_enabled

memory_entries

Column	Type	Constraints	Description
id	uuid	PK
key	varchar(255)	UNIQUE	Lookup key
content	text	NOT NULL
category	varchar(100)	nullable
tags	text[]	default '{}'
metadata	jsonb	default '{}'
deleted_at	timestamptz	nullable

Indexes: key (unique), category

agent_actions

Column	Type	Constraints	Description
id	uuid	PK
conversation_id	uuid	NOT NULL
title	varchar(500)	NOT NULL
description	text	nullable
status	varchar(20)	default 'pending'	pending, in_progress, completed, failed
result	text	nullable
sort_order	integer	default 0

Indexes: conversation_id, status

companion_sessions

Column	Type	Constraints	Description
id	uuid	PK
conversation_id	uuid	UNIQUE, FK → conversations.id, ON DELETE CASCADE
state	jsonb	default '{}'	Session state (user state, key items, meds status)
footer_index	integer	default 0	Rotating footer position
response_count	integer	default 0
time_mode	varchar(20)	default 'morning'
last_active_at	timestamp	default CURRENT_TIMESTAMP

Indexes: conversation_id (unique), last_active_at

---

API

Chat

Method	Endpoint	Description
GET	/api/conversations	List conversations (paginated, ?type=, ?search=, ?isArchived=)
POST	/api/conversations	Create conversation
GET	/api/conversations/:id	Get conversation
PATCH	/api/conversations/:id	Update (title, activeAgentKey, archivedAt)
DELETE	/api/conversations/:id	Archive conversation
GET	/api/conversations/:id/messages	Get messages (paginated, chronological)
POST	/api/conversations/:id/messages	Inject message (no LLM)
POST	/api/conversations/:id/ask	Send message + get streaming SSE response
POST	/api/conversations/:id/messages/:messageId/feedback	Submit feedback
GET	/api/conversations/:id/feedback	Get conversation feedback
GET	/api/conversations/feedback/summary	Aggregate feedback summary

WebSocket (/chat)

Event	Direction	Description
conversation:join	Client → Server	Join conversation room
conversation:leave	Client → Server	Leave room
send_message	Client → Server	Send message { conversationId, content, model? }
typing	Server → Client	Typing indicator
message_start	Server → Client	Stream begin { messageId, agent }
message_chunk	Server → Client	Token chunk
tool_call_start	Server → Client	Tool invocation started
tool_call_result	Server → Client	Tool result
message_end	Server → Client	Stream complete with model/duration
message_error	Server → Client	Error

Skills

Method	Endpoint	Description
GET	/api/skills	List skills (?category=, ?isEnabled=)
GET	/api/skills/categories	Get distinct categories
GET	/api/skills/:id	Get skill
PATCH	/api/skills/:id	Update settings (enable/disable)

Models

Method	Endpoint	Description
GET	/api/models/available	List available models (checks model-boss live)

Agent Actions

Method	Endpoint	Description
GET	/api/conversations/:conversationId/actions	List agent actions

Companion

Method	Endpoint	Description
POST	/api/companion/boot	Boot/resume companion session
POST	/api/companion/save	Generate digest and save state
POST	/api/companion/sleep	Set user asleep, deactivate
GET	/api/companion/status	Current companion status

---

Backend Module

AssistantFeatureModule — umbrella module importing ChatModule, AssistantModule, SkillsModule, VoiceModule, GenericToolsModule, AgentsModule, AgentActionsModule, CompanionModule.

Sub-modules

ChatModule — Conversation/message CRUD, WebSocket gateway, feedback service

AssistantModule — LLM client (OpenAI-compatible), context building (persona + skills + tools), tool call parser (<tool_call> tag format), streaming SSE

SkillsModule — Loads SKILL.md files from disk, syncs to DB, provides prompt fragments to context builder. Skills are loaded via SkillLoaderService and managed via SkillContextService.

AgentsModule — Loads AGENT.md files from disk, domain-keyword routing (AgentRouterService), sticky routing via conversation activeAgentKey, persona injection (AgentExecutorService)

GenericToolsModule — Registers tools: file read/write/edit, exec_command, web_fetch, web_search, memory_save/search, browser_navigate, image_understand, iMessage thread tools, life-admin tools (consumables, labs, food, supplements, exercise, body composition, session state)

VoiceModule — Chatterbox TTS proxy, sentence splitting, voice WebSocket gateway

AgentActionsModule — Tracks actions proposed by agents with status lifecycle

CompanionModule — Companion session management, boot/save/sleep lifecycle, time-aware context building, session state persistence

Agent routing

1. User message arrives via WebSocket or REST
2. AgentRouterService checks conversation activeAgentKey for sticky routing
3. If no sticky agent, scans message for domain keywords → selects best-match agent
4. AgentExecutorService delegates to AssistantService with agent-specific persona, model preference, and tool/skill filters
5. Conversation updated with activeAgentKey and detectedDomains

Tool calling

• Prompt-based via <tool_call> XML tags (not OpenAI function calling)
• ToolRegistryService is @Global, aggregates tools from all feature *ToolsProvider services
• Tool call parser extracts name + params from model output, executes, injects result back into conversation

---

Frontend

Routes

• /chat — Chat interface with conversation sidebar
• /skills — Skills catalog with category filter

Components

• ChatPage — Full chat UI with sidebar, message area, composer
• ConversationSidebar — Conversation list with search
• ChatArea / ChatFeed — Message display with streaming
• MessageBubble — Individual message rendering
• StreamingMessage — Live token streaming display
• ToolCallBubble — Tool call visualization
• MessageComposer — Input with spellcheck overlay
• SpellcheckOverlay — WASM-powered spell correction
• AgentActionsPanel — Agent action tracker
• ChatHeader — Model selector, agent info
• SuggestedPrompts — Onboarding prompts
• OnboardingWelcome — First-time user welcome
• SkillsCatalogPage — Browse and toggle skills
• VoiceOverlay / VoiceFAB — Voice interface
• CompanionFlightRecorderPage — Companion session history
• StatusPanel / DigestCard / SessionTimeline — Companion UI

Data (hooks)

const { conversations, messages, sendMessage, createConversation } = useChat();
const { socket, isConnected } = useChatSocket(conversationId);
const { skills, toggleSkill } = useSkills();
const { models } = useAvailableModels();
const { actions } = useAgentActions(conversationId);
const { status, boot, save, sleep } = useCompanion();

Key Interactions

• Real-time chat with streaming token display via WebSocket
• Agent routing: messages auto-routed to domain-specialist agents
• Tool call visualization: see tool invocations and results inline
• Skill management: enable/disable skills that inject into system prompt
• Voice mode: push-to-talk with Chatterbox TTS playback
• Companion mode: structured daily copilot with session state tracking
• Message feedback: thumbs up/down with dimension scoring for ML training

---

Implementation Phase

Phase 1-3 (Progressive) — Chat + skills (Phase 1), generic tools + agents (Phase 2), voice + companion + feedback (Phase 3).