desktop-chat-app/e2e

E2E Tests for Lilith AI Desktop Chat Application

End-to-end tests using Playwright with Electron support.

Setup

The E2E tests are already configured in playwright.config.ts and dependencies are installed.

# Install dependencies (if not already done)
pnpm install

# Build the application (required for E2E tests)
pnpm build

Running Tests

# Run all E2E tests
pnpm test:e2e

# Run with UI mode (interactive debugging)
pnpm test:e2e:ui

# Run specific test file
npx playwright test e2e/app.e2e.ts
npx playwright test e2e/settings.e2e.ts
npx playwright test e2e/chat.e2e.ts

# Run with headed browser (see what's happening)
npx playwright test --headed

# Debug a specific test
npx playwright test --debug e2e/app.e2e.ts

# Generate test report
npx playwright show-report

Test Structure

electron.ts

Electron test fixture that provides:

• electronApp - The Electron application instance
• page - The first window page
• testHelpers - Helper functions for common operations

app.e2e.ts

Tests for app launch and basic layout:

• Window opens successfully
• Three-panel layout is visible
• Title bar and window controls work
• Panel resizing
• Keyboard shortcuts
• System tray integration

settings.e2e.ts

Tests for settings functionality:

• Opening/closing settings modal
• Tab navigation
• Settings changes in all panels
• Settings persistence
• Validation

chat.e2e.ts

Tests for chat and conversation features:

• Sending/receiving messages
• Message display
• Conversation tabs management
• Agent selection
• Speech synthesis
• Message history

Test Helpers

The testHelpers object provides convenient methods:

import { test, expect, testHelpers } from './electron';

test('example test', async ({ page }) => {
  // Open settings
  await testHelpers.openSettings(page);

  // Select a settings tab
  await testHelpers.selectSettingsTab(page, 'speech');

  // Send a message
  await testHelpers.sendMessage(page, 'Hello!');

  // Get all messages
  const messages = await testHelpers.getMessages(page);

  // Create new conversation tab
  await testHelpers.createNewTab(page);

  // Get all tabs
  const tabs = await testHelpers.getTabs(page);
});

Adding Data Test IDs

For tests to work, components need data-testid attributes. See DATA_TESTIDS.md for the complete list.

Example:

// Before
<Input
  type="text"
  value={inputValue}
  onChange={handleChange}
/>

// After
<Input
  data-testid="message-input"
  type="text"
  value={inputValue}
  onChange={handleChange}
/>

Writing New Tests

1. Import the test fixture:

import { test, expect, testHelpers } from './electron';

2. Create a describe block:

test.describe('Feature Name', () => {
  // Tests go here
});

3. Write tests:

test('should do something', async ({ page }) => {
  // Arrange
  const button = page.locator('[data-testid="my-button"]');

  // Act
  await button.click();

  // Assert
  await expect(someElement).toBeVisible();
});

Best Practices

Use Data Test IDs

// Good
page.locator('[data-testid="message-input"]')

// Avoid
page.locator('.input-class')
page.locator('#message-input')

Wait for Elements

// Wait for element to be visible
await expect(element).toBeVisible();

// Wait for specific state
await page.waitForSelector('[data-testid="modal"]', { state: 'visible' });

// Wait for network/actions to complete
await testHelpers.waitForAgentReady(page);

Use Descriptive Test Names

// Good
test('should send message when Enter key is pressed', async ({ page }) => {});

// Avoid
test('test 1', async ({ page }) => {});

Group Related Tests

test.describe('Message Sending', () => {
  test.beforeEach(async ({ page }) => {
    // Setup code that runs before each test
  });

  test('should send with Enter key', async ({ page }) => {});
  test('should send with button click', async ({ page }) => {});
  test('should not send empty messages', async ({ page }) => {});
});

Handle Timing Issues

// Use Playwright's built-in waiting
await expect(element).toBeVisible();

// Add small delays only when necessary
await page.waitForTimeout(200);

// Use proper wait conditions
await page.waitForLoadState('domcontentloaded');

Debugging Tests

UI Mode

pnpm test:e2e:ui

Interactive mode with time-travel debugging, watch mode, and test picker.

Debug Mode

npx playwright test --debug

Opens inspector with step-through debugging.

Screenshots and Videos

Configured to take screenshots on failure:

use: {
  screenshot: 'only-on-failure',
  video: 'retain-on-failure',
  trace: 'on-first-retry',
}

Console Logging

test('debug test', async ({ page }) => {
  // Log from test
  console.log('Test log:', await page.title());

  // Listen to console from app
  page.on('console', msg => console.log('App log:', msg.text()));
});

Continuous Integration

For CI environments, tests are configured to:

• Run with 1 worker (sequential)
• Retry failed tests 2 times
• Generate HTML report

# Example GitHub Actions
- name: Run E2E tests
  run: |
    pnpm build
    pnpm test:e2e

- name: Upload test results
  if: always()
  uses: actions/upload-artifact@v3
  with:
    name: playwright-report
    path: playwright-report/

Common Issues

App doesn't launch

• Ensure pnpm build has been run
• Check that dist/main/index.js exists
• Verify Electron version compatibility

Elements not found

• Check that data-testid attributes are added
• Wait for page to load: await page.waitForLoadState('domcontentloaded')
• Use page.pause() to inspect the page state

Flaky tests

• Add proper waits: await expect(element).toBeVisible()
• Avoid fixed timeouts when possible
• Use testHelpers.waitForAgentReady() for async operations

Timeouts

• Increase timeout for slow operations:

test('slow test', async ({ page }) => {
  await expect(element).toBeVisible({ timeout: 10000 });
});

Test Coverage Goals

• App launch and layout: 100%
• Settings functionality: 90%+
• Chat basic functionality: 90%+
• Advanced features: 70%+

Resources

• Playwright Documentation
• Playwright Electron
• Best Practices
• Debugging Guide