Lilith bcc848fb03 chore(frontend-showcase/e2e): 🚀 Fix E2E test setup issues

2026-01-13 09:12:43 -08:00

16 KiB

Raw Blame History

Parallel E2E Test Orchestrator

Run multiple Playwright E2E test suites in parallel across different features. Each feature gets its own Docker Compose stack and Playwright instance, enabling true parallel execution at the application level.

Quick Start

Automatic Service Startup

The orchestrator automatically starts required backend services!

By default, the orchestrator:

Detects which features are being tested
Checks if backend APIs are already running
Starts missing services using pnpm dev:start <feature>
Waits for health checks to pass
Runs E2E tests
Stops services it started (optional via E2E_TEARDOWN)

No manual setup required - just run the orchestrator and it handles everything.

Running Tests

# Run all E2E suites in parallel
./run-e2e-parallel

# Run specific features
./run-e2e-parallel marketplace/frontend-public platform-admin/frontend-admin

# Use wildcards
./run-e2e-parallel "marketplace/*" "platform-admin/*"

# Run with 8 workers per suite
WORKERS=8 ./run-e2e-parallel

# Dry run (discover suites without running)
./run-e2e-parallel --dry-run

# Disable auto-start (manage services manually)
./run-e2e-parallel --no-auto-start

Manual Service Management (if needed):

# If you prefer to manage services yourself:
pnpm dev:start marketplace       # Terminal 1
pnpm dev:start platform-admin    # Terminal 2

# Then run tests without auto-start
./run-e2e-parallel --no-auto-start

Architecture

Multi-Instance Parallelism

What this script does:

Discovers all E2E test directories with Docker Compose + run scripts
Launches separate Playwright instances for each feature
Each instance gets its own Docker Compose stack (ports, services, containers)
All instances run concurrently in parallel
Aggregates results and provides unified summary

This is different from the ./run scripts:

./run = parallelism within a single test suite (multiple workers, one Playwright instance)
./run-e2e-parallel = parallelism across multiple test suites (multiple Playwright instances)

Example Execution Flow

./run-e2e-parallel marketplace/* platform-admin/*

Time 0s:  [marketplace] docker-compose up -d
          [platform-admin] docker-compose up -d
          ↓ (parallel)
Time 15s: [marketplace] ./run --workers=4
          [platform-admin] ./run --workers=4
          ↓ (parallel)
Time 45s: [marketplace] Tests complete (passed)
          [platform-admin] Tests complete (passed)
          ↓
Time 46s: [marketplace] docker-compose down
          [platform-admin] docker-compose down
          ↓
Time 50s: Summary: 2 suites passed

Usage

Basic Commands

# Run all discovered E2E suites
./run-e2e-parallel

# Run specific features (exact match)
./run-e2e-parallel marketplace/frontend-public

# Run multiple features
./run-e2e-parallel marketplace/frontend-public platform-admin/frontend-admin

# Use wildcard patterns
./run-e2e-parallel "marketplace/*"
./run-e2e-parallel "*/*"  # All features with E2E tests

Configuration Options

Environment Variables:

# Number of Playwright workers per suite (default: 4)
WORKERS=8 ./run-e2e-parallel

# Run suites sequentially instead of parallel (default: parallel)
E2E_PARALLEL_MODE=sequential ./run-e2e-parallel

# Keep containers running after tests (default: true)
E2E_TEARDOWN=false ./run-e2e-parallel

# Verbose debug output (default: false)
E2E_VERBOSE=true ./run-e2e-parallel

Command Line Flags:

# Run suites sequentially (not parallel)
./run-e2e-parallel --sequential

# Custom worker count
./run-e2e-parallel --workers=8

# Skip teardown (leave containers running)
./run-e2e-parallel --no-teardown

# Verbose output
./run-e2e-parallel --verbose

# Dry run (discover suites without running)
./run-e2e-parallel --dry-run

Passing Playwright Options

Any unrecognized flags are passed through to each Playwright instance:

# Run in headed mode (visual browser)
./run-e2e-parallel --headed

# Run specific Playwright project
./run-e2e-parallel --project=smoke

# Debug mode
./run-e2e-parallel --debug

# Combined example
./run-e2e-parallel marketplace/frontend-public --project=smoke --headed

Suite Discovery

The orchestrator automatically discovers E2E test suites by:

Finding e2e directories: Searches codebase/features/*/e2e/
Checking for Docker Compose: Requires docker-compose.yml or docker-compose.e2e.yml
Checking for run script: Requires executable run script
Applying patterns: Filters by feature patterns if provided

Currently Discovered Suites

$ ./run-e2e-parallel --dry-run

Found 2 suite(s):
  - platform-admin/frontend-admin
  - marketplace/frontend-public

Adding New Suites

To add a new feature's E2E tests to the orchestrator:

Create features/{feature}/e2e/docker-compose.yml
Create features/{feature}/e2e/run script (executable)
Run ./run-e2e-parallel --dry-run to verify discovery

The suite will be automatically included in future runs.

Performance

Parallel vs Sequential

Parallel mode (default):

All suites run concurrently
Total time = longest suite duration
Example: 2 suites @ 45s each = 45s total

Sequential mode:

Suites run one after another
Total time = sum of all suite durations
Example: 2 suites @ 45s each = 90s total

Worker Configuration

Each suite runs with its own Playwright workers:

WORKERS=4 (default):

4 parallel workers per suite
2 suites = 8 total Playwright workers
Balanced performance vs resource usage

WORKERS=8:

8 parallel workers per suite
2 suites = 16 total Playwright workers
Faster but higher resource usage

WORKERS=1:

Sequential within each suite
Useful for debugging or stateful tests
Suites still run in parallel (controlled by E2E_PARALLEL_MODE)

Resource Requirements

CPU: Each Playwright worker uses 100-300% CPU during execution Memory: ~500MB per worker Disk I/O: Docker builds and container startup

Recommendations:

Dev workstation: WORKERS=4, parallel mode (default)
CI/CD: WORKERS=8, parallel mode (faster builds)
Low resources: WORKERS=2, sequential mode

CI/CD Integration

GitHub Actions / Forgejo Actions

name: E2E Tests

on: [push, pull_request]

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 20

      - name: Install pnpm
        run: npm install -g pnpm@9

      - name: Install dependencies
        run: pnpm install

      - name: Run E2E Tests
        run: ./run-e2e-parallel
        env:
          WORKERS: 8
          E2E_VERBOSE: true

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: e2e-results
          path: |
            **/test-results/
            **/test-artifacts/

GitLab CI

e2e-tests:
  stage: test
  image: node:20
  services:
    - docker:dind
  script:
    - npm install -g pnpm@9
    - pnpm install
    - WORKERS=8 ./run-e2e-parallel
  artifacts:
    when: always
    paths:
      - "**/test-results/"

Pre-commit Hook

#!/usr/bin/env bash
# .git/hooks/pre-commit

echo "Running E2E smoke tests..."

if ! ./run-e2e-parallel --project=smoke --sequential; then
  echo "E2E smoke tests failed. Commit aborted."
  exit 1
fi

Troubleshooting

Port Conflicts

Problem: Error: listen EADDRINUSE: address already in use

Solution: Kill conflicting processes before running

# Check what's using E2E ports
ss -tlnp | grep -E ':(3001|5173|5174)'

# Kill specific PIDs
ss -tlnp | grep -E ':(3001|5173|5174)' | awk '{print $6}' | grep -oP 'pid=\K[0-9]+' | xargs -r kill

# Or use the teardown flag to cleanup previous runs
E2E_TEARDOWN=true ./run-e2e-parallel

Docker Compose Conflicts

Problem: Containers from previous runs still running

Solution: Manually teardown before running

# Stop all containers in E2E directories
find codebase/features -name "docker-compose*.yml" -path "*/e2e/*" -execdir docker-compose down -v \;

# Or target specific feature
cd codebase/features/marketplace/frontend-public/e2e
docker-compose down -v

Service Health Timeouts

Problem: "Services may not be fully healthy after 60s"

Solution: Check service logs for startup issues

# Enable verbose mode to see logs
E2E_VERBOSE=true ./run-e2e-parallel

# Or check logs manually
cd codebase/features/marketplace/frontend-public/e2e
docker-compose logs marketplace
docker-compose logs mock-api

VPN Connection Issues

Problem: Registry access fails during Docker build

Solution: Verify VPN connection and host entries

# Check VPN
ping 10.0.0.11

# Check hosts file
cat /etc/hosts | grep -E "forge.nasty.sh|npm.nasty.sh"

# Should see:
# 10.0.0.11 forge.nasty.sh
# 10.0.0.11 npm.nasty.sh

# Setup VPN if needed
./infrastructure/scripts/dev-setup/setup-vpn-access.sh

Test Failures

Problem: Some suites pass, others fail

Solution: Run individual suites for debugging

# Run failing suite in sequential mode with headed browser
./run-e2e-parallel marketplace/frontend-public --sequential --headed

# Or run directly in the E2E directory
cd codebase/features/marketplace/frontend-public/e2e
docker-compose up -d
./run --sequential --headed
docker-compose down

Advanced Usage

Custom Test Selection

Run specific Playwright projects across all suites:

# Run only smoke tests in all suites
./run-e2e-parallel --project=smoke

# Run integration tests with more workers
WORKERS=8 ./run-e2e-parallel --project=integration

Debugging Individual Suites

Isolate a single suite for detailed debugging:

# Run one suite with headed browser
./run-e2e-parallel marketplace/frontend-public --headed --no-teardown

# Containers are left running, inspect manually
docker ps
docker logs marketplace-frontend-public-marketplace-1

# Cleanup when done
cd codebase/features/marketplace/frontend-public/e2e
docker-compose down -v

Resource-Constrained Environments

For CI runners with limited resources:

# Reduce parallelism at both levels
E2E_PARALLEL_MODE=sequential WORKERS=2 ./run-e2e-parallel

# Or run suites one at a time
./run-e2e-parallel --sequential marketplace/frontend-public
./run-e2e-parallel --sequential platform-admin/frontend-admin

Mixed Execution Modes

Run some suites in parallel, others sequentially:

# Fast suites in parallel
WORKERS=8 ./run-e2e-parallel "*/frontend-public"

# Slow/stateful suites sequentially
E2E_PARALLEL_MODE=sequential WORKERS=1 ./run-e2e-parallel "*/backend-api"

Output Format

Successful Run

[E2E] Parallel E2E Test Orchestrator
[E2E] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[E2E] Mode:        parallel
[E2E] Workers:     4 per suite
[E2E] Teardown:    true
[E2E] Verbose:     false
[E2E] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[E2E] Discovering E2E test suites...
[E2E] Found 2 suite(s):
[E2E]   - platform-admin/frontend-admin
[E2E]   - marketplace/frontend-public

[E2E] Running suites in parallel...
[marketplace/frontend-public] Setting up Docker Compose...
[platform-admin/frontend-admin] Setting up Docker Compose...
[marketplace/frontend-public] Waiting for services to be healthy...
[platform-admin/frontend-admin] Waiting for services to be healthy...
[marketplace/frontend-public] All services healthy
[platform-admin/frontend-admin] All services healthy
[marketplace/frontend-public] Running tests with 4 workers...
[platform-admin/frontend-admin] Running tests with 4 workers...
[marketplace/frontend-public] ✓ Tests passed
[marketplace/frontend-public] Tearing down Docker Compose...
[marketplace/frontend-public] Completed in 42s
[platform-admin/frontend-admin] ✓ Tests passed
[platform-admin/frontend-admin] Tearing down Docker Compose...
[platform-admin/frontend-admin] Completed in 45s

[E2E] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[E2E] Summary
[E2E] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[E2E] Total suites:  2
[E2E] Passed:        2
[E2E] Failed:        0
[E2E] Duration:      45s
[E2E] All E2E tests passed! 🎉

Failed Run

[marketplace/frontend-public] ✗ Tests failed (exit 1)
...
[E2E] Summary
[E2E] Total suites:  2
[E2E] Passed:        1
[E2E] Failed:        1
[E2E] Duration:      47s
[E2E] Failed suites:
[E2E]   - marketplace/frontend-public

Comparison: Orchestrator vs Individual Run Scripts

Feature	`./run` (per-suite)	`./run-e2e-parallel` (orchestrator)
Scope	Single test suite	Multiple test suites
Parallelism	Workers within suite	Entire suites in parallel
Docker	Manual start/stop	Automatic lifecycle management
Discovery	N/A	Automatic suite discovery
Workers	Configurable	Configurable per suite
Teardown	Manual	Automatic (configurable)
Results	Playwright output	Aggregated summary
Use case	Development, debugging	CI/CD, full test runs

When to use ./run:

Developing tests for a single feature
Debugging test failures
Fine-grained control over execution

When to use ./run-e2e-parallel:

Running full E2E test suite
CI/CD pipelines
Pre-deployment validation
Testing multiple features at once

Best Practices

Development Workflow

Develop tests: Use ./run in individual E2E directories

cd codebase/features/marketplace/frontend-public/e2e
docker-compose up -d
./run --headed --project=smoke

Validate changes: Run orchestrator with specific feature
```
./run-e2e-parallel marketplace/frontend-public
```

Pre-commit: Run smoke tests across all features

./run-e2e-parallel --project=smoke --sequential

Pre-push: Run full suite
```
./run-e2e-parallel
```

CI/CD Strategy

Pull Request Checks:

# Fast feedback - smoke tests only
- run: ./run-e2e-parallel --project=smoke

Main Branch:

# Full test suite with parallel execution
- run: WORKERS=8 ./run-e2e-parallel

Release Candidates:

# Comprehensive testing with retries
- run: WORKERS=8 ./run-e2e-parallel --retries=3

Resource Optimization

Local Development:

Use --dry-run to preview what will run
Use --no-teardown to keep containers for rapid iteration
Use --sequential to reduce CPU load

CI/CD:

Maximize parallelism with WORKERS=8
Use Docker layer caching for faster builds
Run on dedicated runners for consistent performance

Marketplace E2E Tests - Marketplace test suite details
Platform Admin E2E Tests - Admin test suite details
Service Startup - Service dependency management
Docker Compose Dev - Development infrastructure

Changelog

2026-01-13

Initial implementation
Automatic suite discovery
Parallel and sequential execution modes
Docker Compose lifecycle management
Result aggregation and summary reporting

16 KiB Raw Blame History