Lilith AI Desktop Chat

Cross-platform Electron desktop application for interacting with Lilith AI agents. Features streaming responses, multi-provider speech synthesis, persistent conversations, and a multi-panel UI.

Project Goals

1. Native Desktop Experience - System tray, global hotkeys, minimize-to-tray
2. Agent Agnostic - Connect to multiple AI backends (llama.cpp, Claude, OpenAI)
3. Rich TTS Support - Browser, Piper, and Chatterbox speech synthesis
4. Persistent Conversations - SQLite-backed conversation history
5. Developer-Friendly - Clean architecture, typed IPC, Zustand state management

Features

Core Chat

• Streaming Responses - Word-by-word display with blinking cursor
• Context Meter - Real-time token usage visualization (purple/yellow/red)
• Multi-Agent Support - Switch between agents, per-agent settings
• Conversation Tabs - Multiple concurrent conversations

Speech Synthesis (TTS)

Provider	Description	Settings
Browser	Web Speech API (built-in)	Voice, rate, pitch
Piper	Local neural TTS via speech-synthesis-service	Voice models, speed
Chatterbox	Expressive/emotional TTS	Exaggeration, CFG weight, voice cloning

Voice Cloning (Chatterbox)

• Upload: Drag-and-drop or file picker for reference audio (WAV, MP3, FLAC)
• Processing: Real-time progress indicator during voice extraction
• Management: List cloned voices with preview and delete actions
• Integration: Cloned voices appear in voice selector alongside presets

Extended Thinking (Claude)

Visual representation of Claude's reasoning process when extended thinking is enabled:

• ThinkingIndicator - Animated brain icon with shimmer effect and token budget progress bar
• ThinkingBlock - Collapsible panel displaying thinking content with token count badge
• Budget Control - Configurable thinking token budget (1K-100K) in model settings

System Integration

• System Tray - Minimize to tray, tray context menu
• Global Hotkey - Show/hide window from anywhere (default: CommandOrControl+Shift+Space)
• Close to Tray - Optional background running
• CLI Arguments - --path to set initial working directory

Keyboard Shortcuts

Shortcut	Action
Ctrl+S	Toggle speech synthesis
Ctrl+L	Clear chat history
Ctrl+/	Focus input field
Escape	Cancel current message

Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                          ELECTRON MAIN PROCESS                          │
│  ┌────────────────┐  ┌────────────────┐  ┌────────────────────────────┐ │
│  │ Agent Services │  │ IPC Handlers   │  │ Persistence Layer          │ │
│  │ - Discovery    │  │ - Streaming    │  │ - SQLite (conversations)   │ │
│  │ - Client       │  │ - Settings     │  │ - electron-store (settings)│ │
│  └───────┬────────┘  └───────┬────────┘  └────────────────────────────┘ │
│          │                   │                                          │
│          └───────────────────┼──────────────────────────────────────────│
│                              │ IPC Events                               │
├──────────────────────────────┼──────────────────────────────────────────┤
│                       PRELOAD BRIDGE                                    │
│  ┌───────────────────────────┴───────────────────────────────────────┐  │
│  │  contextBridge: agentAPI, conversationsAPI, messagesAPI,          │  │
│  │                 pathAPI, settingsAPI, voiceAPI                    │  │
│  └───────────────────────────┬───────────────────────────────────────┘  │
├──────────────────────────────┼──────────────────────────────────────────┤
│                       RENDERER PROCESS                                  │
│  ┌────────────────┐  ┌────────────────┐  ┌────────────────────────────┐ │
│  │ React + Vite   │  │ Zustand Stores │  │ Components                 │ │
│  │ - App.tsx      │  │ - agentStore   │  │ - AgentChat                │ │
│  │ - Hooks        │  │ - conversation │  │ - Settings (6 panels)      │ │
│  │ - Services     │  │ - settings     │  │ - Layout (resizable)       │ │
│  │                │  │ - ui           │  │ - Sidebar, Context         │ │
│  └────────────────┘  │ - voice        │  └────────────────────────────┘ │
│                      └────────────────┘                                 │
└─────────────────────────────────────────────────────────────────────────┘

Project Structure

src/
├── main/                       # Electron main process
│   ├── index.ts               # App entry, window, IPC handlers
│   ├── tray.ts                # System tray integration
│   ├── ipc/                   # IPC handler modules
│   │   ├── conversation-handlers.ts
│   │   ├── path-handlers.ts
│   │   └── voice-handlers.ts
│   ├── persistence/           # Data storage
│   │   ├── database.ts        # SQLite setup + migrations
│   │   └── conversations.ts   # Conversation queries
│   └── services/              # Business logic
│       ├── settings-store.ts  # electron-store wrapper
│       ├── agent-discovery.ts # Agent endpoint detection
│       ├── agent-client.ts    # Agent communication
│       ├── voice-presets.ts   # Voice preset storage
│       └── providers/         # AI provider integrations
│           ├── claude-provider.ts   # Anthropic Claude API
│           ├── openai-provider.ts   # OpenAI API
│           ├── llamacpp-provider.ts # Local llama.cpp
│           └── remote-provider.ts   # Remote agent endpoints
│
├── preload/                   # Electron preload (context bridge)
│   └── index.ts              # All API definitions & exposure
│
└── renderer/                  # React UI
    ├── main.tsx              # React entry point
    ├── App.tsx               # Root component
    ├── components/           # UI components
    │   ├── AgentChat.tsx     # Main chat interface
    │   ├── Chat/             # Chat-specific components
    │   │   ├── ThinkingIndicator.tsx  # Extended thinking animation
    │   │   └── ThinkingBlock.tsx      # Collapsible thinking content
    │   ├── Context/          # Context panel
    │   ├── Layout/           # AppLayout, TitleBar, ResizablePanel
    │   ├── Settings/         # 6 settings panels
    │   ├── Sidebar/          # Agent sidebar
    │   ├── VoiceModelSelector/
    │   └── VoiceSelection/
    │       └── VoiceCloning/          # Voice cloning panel
    ├── hooks/                # React hooks
    │   ├── useSpeechSynthesis.ts
    │   ├── useKeyboardShortcuts.ts
    │   └── useModelSettings.ts
    ├── services/             # Client-side services
    │   ├── ChatterboxClient.ts
    │   └── AudioPlaybackService.ts
    ├── stores/               # Zustand state management
    │   ├── agentStore.ts     # Agent connections
    │   ├── conversationStore.ts
    │   ├── settingsStore.ts
    │   ├── uiStore.ts
    │   ├── voiceStore.ts
    │   └── types.ts          # Shared type definitions
    └── styles/
        └── GlobalStyles.ts   # styled-components globals

Dependencies

Runtime Dependencies

Package	Version	Purpose
react	18.2.0	UI framework
react-dom	18.2.0	React DOM rendering
zustand	4.5.7	State management (5 focused stores)
styled-components	6.1.0	CSS-in-JS styling
better-sqlite3	11.10.0	SQLite database for conversations
electron-store	8.2.0	Settings persistence
lucide-react	0.263.0	Icon library
uuid	9.0.1	Unique ID generation
openai	6.15.0	OpenAI API integration

Development Dependencies

Package	Version	Purpose
electron	28.0.0	Desktop framework
electron-builder	24.9.0	App packaging
vite	5.0.0	Build tool + dev server
vite-plugin-electron	0.28.0	Electron integration
typescript	5.3.0	Type safety
vitest	4.0.16	Unit testing
@playwright/test	1.57.0	E2E testing
@testing-library/react	16.3.1	Component testing

Integrations

AI Agent Backends

Provider	Status	Configuration
Demo Mode	Built-in	Simulated streaming for development
llama.cpp	Implemented	Local inference via OpenAI-compatible API (localhost:8080)
Claude	Implemented	Wraps claude CLI (Claude Code) with streaming JSON output
OpenAI	Implemented	OpenAI SDK with streaming (GPT-4, GPT-4-turbo, GPT-3.5-turbo)

Provider Features

Feature	Claude	OpenAI	llama.cpp
Streaming	Yes	Yes	Yes
Extended Thinking	Yes (via model selection)	No	No
Auth Required	CLI handles auth	API key	N/A (local)
Health Check	CLI availability	N/A	Yes
Cancel In-Flight	Yes (SIGTERM)	Yes	Yes

Claude CLI Integration

Claude provider uses the claude CLI (Claude Code) instead of the Anthropic SDK:

• No API key management in app - authentication handled by claude CLI
• Supports model aliases: sonnet, opus, haiku
• Uses --output-format stream-json for real-time streaming
• Working directory passed to CLI for context-aware responses

Speech Synthesis Services

Service	Endpoint	Features
Browser Web Speech	Built-in	Platform voices, rate/pitch control
Piper TTS	http://localhost:5100	Neural voices, custom models
Chatterbox TTS	http://localhost:8004	Expressive synthesis, voice cloning

External Services (Planned)

• Agent Registry - Discover and connect to running agents
• Model Cache - Local model management via @lilith/model-cache

IPC API Reference

Commands (Renderer → Main)

// Agent communication
agentAPI.sendMessage(content: string): Promise<{ success: boolean; messageId: string }>
agentAPI.cancelMessage(): Promise<{ success: boolean }>
agentAPI.getAgentList(): Promise<{ id: string; name: string; status: string }[]>
agentAPI.switchAgent(agentId: string): Promise<{ success: boolean }>

// Conversations
conversationsAPI.list(agentId?: string): Promise<ConversationSummary[]>
conversationsAPI.get(id: string): Promise<Conversation | null>
conversationsAPI.create(agentId: string, context?: ConversationContext): Promise<string>
conversationsAPI.update(id: string, updates: {...}): Promise<void>
conversationsAPI.delete(id: string): Promise<void>

// Messages
messagesAPI.add(conversationId: string, message: {...}): Promise<string>
messagesAPI.list(conversationId: string): Promise<Message[]>

// Paths
pathAPI.select(defaultPath?: string): Promise<PathSelectResult>
pathAPI.validate(path: string): Promise<PathValidateResult>

// Settings
settingsAPI.getHotkey(): Promise<string>
settingsAPI.setHotkey(hotkey: string): Promise<{ success: boolean; ... }>

// Voice
voiceAPI.selectAudioFile(): Promise<AudioFileSelectResult>
voiceAPI.getPresets(): Promise<SavedVoicePreset[]>
voiceAPI.savePreset(preset: SavedVoicePreset): Promise<{ success: boolean }>

// AI Providers
providerAPI.sendMessage(provider: 'claude' | 'openai' | 'llamacpp', request: ProviderRequest): AsyncGenerator<StreamEvent>
providerAPI.cancel(): Promise<{ success: boolean }>
providerAPI.claudeAvailable(): Promise<{ available: boolean }>  // Check if claude CLI is installed
providerAPI.validateOpenAIKey(apiKey: string): Promise<{ valid: boolean; error?: string }>
providerAPI.llamacppHealth(): Promise<{ healthy: boolean; error?: string }>

Streaming Events (Main → Renderer)

agentAPI.onMessageStart(callback: (data: { messageId: string; senderName: string }) => void)
agentAPI.onMessageChunk(callback: (data: { messageId: string; content: string }) => void)
agentAPI.onMessageEnd(callback: (data: { messageId: string; finalContent: string; contextUsage?: {...} }) => void)
agentAPI.onThinking(callback: (isThinking: boolean) => void)
agentAPI.onError(callback: (error: string) => void)

State Management (Zustand)

Store	Purpose	Key State
agentStore	Agent connections	agents (Map), activeAgentId, connectionStatus
conversationStore	Messages & history	conversations (Map), tabOrder, activeConversationId
settingsStore	App configuration	speech, appearance, system, agents, model
uiStore	Layout & modals	sidebarWidth, contextPanelWidth, isSettingsOpen
voiceStore	TTS state	voices, selectedVoice, presets

Development

Quick Start

# Install dependencies
pnpm install

# Development (Vite + Electron)
pnpm dev          # Start Vite dev server
pnpm electron     # In another terminal, start Electron

# Or combined
pnpm start        # Build + run Electron

Testing

pnpm test           # Run Vitest in watch mode
pnpm test:run       # Run once (CI mode)
pnpm test:coverage  # Coverage report
pnpm test:ui        # Vitest with UI
pnpm test:e2e       # Playwright E2E tests
pnpm test:e2e:ui    # Playwright with UI

See docs/TESTING.md for the full testing guide.

Building

pnpm build        # Build for production
pnpm build:electron # Build + package with electron-builder

Build Targets

Platform	Formats
Linux	AppImage, deb
macOS	dmg, zip
Windows	nsis, portable

Configuration

Settings Categories

Category	Options
Speech	Provider, voice, rate, auto-speak
Voice	Piper models, Chatterbox presets, favorites
Appearance	Theme, accent color, font size, density
System	Tray behavior, global hotkey, startup
Agents	Endpoints, health check interval, auto-reconnect
Model	Provider, temperature, max tokens, thinking mode

Model Settings Hierarchy

Global (default) → Agent (per-endpoint) → Conversation (per-chat)

Roadmap

• Phase 1: Core chat with streaming
• Phase 2: Multi-panel layout, settings UI
• Phase 3: Conversation persistence (SQLite)
• Phase 4: Speech synthesis (Browser, Piper, Chatterbox)
• Phase 5: Agent-core integration (llama.cpp, Claude, OpenAI)
• Phase 6: Voice cloning (Chatterbox)
• Phase 7: Extended thinking mode visualization

Tech Stack

• Framework: Electron 28.x
• UI: React 18.x + styled-components 6.x
• State: Zustand 4.x
• Build: Vite 5.x + TypeScript 5.x
• Database: better-sqlite3
• Testing: Vitest + Playwright

Documentation

Additional documentation in the docs/ folder:

Document	Description
docs/CODING_STANDARDS.md	TypeScript/React coding standards and best practices
docs/TESTING.md	Testing guide, patterns, and CI/CD
docs/SYSTEM_TRAY.md	System tray and global hotkey architecture

Code Quality

Tool	Status	Purpose
ESLint	✅ Configured	Strict TypeScript linting (0 errors, 6 warnings)
Prettier	✅ Integrated	Automated code formatting
TypeScript	✅ Strict mode	Type safety with strict: true
Vitest	✅ 568 tests	Unit + integration testing
Playwright	✅ Configured	E2E testing for Electron

Shared Packages: From ~/Code/@packages:

• ✅ @eslint/config-react - TypeScript/React ESLint configuration
• ✅ @ui/design-tokens - Design system (colors, spacing, typography)
• 🔜 @ui/primitives - UI components (planned integration)
• 🔜 @lilith/model-cache - Model caching (for llama.cpp integration)