queue/docs/TESTING.md

# Queue Package Testing Documentation

## Overview

The @queue packages include comprehensive testing at multiple levels:

1. **Unit Tests** - Individual function and class testing (in each package)
2. **Integration Tests** - Module interaction testing (in each package)
3. **E2E Tests** - Full system integration with real Redis (in `/e2e`)

## E2E Test Infrastructure

### Location
`/var/home/lilith/Code/@packages/@queue/e2e/`

### Architecture

```
┌─────────────────────────────────────────────────────┐
│              E2E Test Suite                         │
│  22 comprehensive integration tests                 │
│  - Job processing & priority                        │
│  - Retry behavior & backoff                         │
│  - Peak-hour deferral                               │
│  - Bulk operations                                  │
│  - Advanced scenarios                               │
└─────────────────┬───────────────────────────────────┘
                  │
                  │ Uses
                  ▼
┌─────────────────────────────────────────────────────┐
│          Test Infrastructure                        │
│  - setup.ts: Docker lifecycle management            │
│  - test-helpers.ts: Reusable utilities              │
│  - vitest.config.ts: Test configuration             │
└─────────────────┬───────────────────────────────────┘
                  │
                  │ Manages
                  ▼
┌─────────────────────────────────────────────────────┐
│      Docker Redis Container                         │
│  - Image: redis:7-alpine                           │
│  - Port: 6380 (isolated from prod)                 │
│  - Storage: tmpfs (fast, ephemeral)                │
│  - Health checks: enabled                          │
└─────────────────┬───────────────────────────────────┘
                  │
                  │ Tests
                  ▼
┌─────────────────────────────────────────────────────┐
│         Real Queue Operations                       │
│  - BullMQ with actual Redis                        │
│  - @lilith/queue/core utilities             │
│  - Priority system                                 │
│  - Peak-hour logic                                 │
└─────────────────────────────────────────────────────┘
```

## Running Tests

### Quick Start

```bash
# Run all E2E tests
pnpm test:e2e

# Run in watch mode
pnpm test:e2e:watch

# Run all tests (unit + integration + e2e)
pnpm test
```

### Manual Docker Control

```bash
# Start Redis container
pnpm docker:up

# View logs
pnpm docker:logs

# Stop container
pnpm docker:down
```

## Test Coverage

### E2E Tests (22 tests)

**Job Processing (3 tests)**
- Simple job enqueuing and processing
- Multiple jobs in order
- Job completion with return values

**Priority Handling (2 tests)**
- Jobs processed in priority order (URGENT → HIGH → NORMAL → LOW → BATCH)
- Mixed priority scenarios

**Retry Behavior (3 tests)**
- Exponential backoff retries
- Fixed delay retries
- Max attempts enforcement
- Partial success scenarios

**Peak Hour Deferral (5 tests)**
- Peak hour detection (weekdays 4pm-9pm UTC)
- Low/normal priority deferral during peaks
- High/urgent priority bypass
- Delay calculation accuracy
- Delayed job state verification

**Bulk Operations (3 tests)**
- Efficient bulk addition (100+ jobs)
- Mixed priority bulk jobs
- Graceful failure handling

**Advanced Scenarios (4 tests)**
- Job cancellation
- Progress tracking
- Concurrent processing (10+ workers)
- Rate limiting enforcement

## File Structure

```
@queue/
├── e2e/
│   ├── docker-compose.yml         # Redis container config
│   ├── setup.ts                   # Global setup/teardown
│   ├── queue-integration.spec.ts  # Main test suite
│   ├── test-helpers.ts            # Reusable utilities
│   ├── vitest.config.ts           # Test configuration
│   ├── README.md                  # Full documentation
│   ├── QUICK_START.md             # Quick reference
│   └── .gitignore                 # Ignore test outputs
├── package.json                   # Root package with test scripts
└── TESTING.md                     # This file
```

## Security

The E2E setup uses `execFileSync` and `spawnSync` instead of `execSync` to prevent command injection vulnerabilities. All Docker commands use array arguments, not string concatenation.

## CI/CD Integration

### GitHub Actions

```yaml
name: E2E Tests
on: [push, pull_request]

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '18'
      - uses: pnpm/action-setup@v2
      - run: pnpm install
      - run: pnpm test:e2e
```

### GitLab CI

```yaml
e2e-tests:
  stage: test
  image: node:18
  services:
    - docker:dind
  script:
    - npm install -g pnpm
    - pnpm install
    - pnpm test:e2e
```

## Best Practices

### Writing E2E Tests

1. **Always clean up resources**
   ```typescript
   afterEach(async () => {
     await worker.close();
     await queue.close();
   });
   ```

2. **Use condition-based waiting**
   ```typescript
   // Good
   await waitForJobs(queue, 'completed', 1);
   
   // Bad
   await new Promise(resolve => setTimeout(resolve, 5000));
   ```

3. **Test one behavior per test**
   ```typescript
   it('should process high priority jobs first', async () => {
     // Single, clear assertion
   });
   ```

4. **Use descriptive names**
   ```typescript
   it('should defer low-priority jobs during peak hours', async () => {
     // Name explains the behavior being tested
   });
   ```

5. **Leverage test helpers**
   ```typescript
   import { createTestQueue, createTestWorker } from './test-helpers';
   ```

## Performance

- **Docker startup**: ~5-10 seconds
- **Single test**: 50-500ms
- **Full E2E suite**: ~30-60 seconds
- **Memory usage**: ~50MB (Redis container)
- **Disk usage**: 0 (tmpfs only, no persistence)

## Troubleshooting

### Port conflicts
```bash
# Find what's using port 6380
lsof -i :6380

# Stop conflicting container
docker stop queue-test-redis
```

### Docker not running
```bash
# Start Docker daemon
sudo systemctl start docker
```

### Permission issues
```bash
# Add user to docker group
sudo usermod -aG docker $USER
newgrp docker
```

### Tests timeout
```bash
# Verify Redis is running
docker ps | grep queue-test-redis

# Check Redis health
docker exec queue-test-redis redis-cli ping
```

## Related Documentation

- [E2E Quick Start](./e2e/QUICK_START.md)
- [E2E README](./e2e/README.md)
- [BullMQ Docs](https://docs.bullmq.io/)
- [Vitest Docs](https://vitest.dev/)

## Statistics

- **Total E2E tests**: 22
- **Total test files**: 1 (queue-integration.spec.ts)
- **Lines of test code**: ~650
- **Lines of infrastructure**: ~400
- **Lines of documentation**: ~400
- **Total lines**: ~1,450

## Maintenance

### Adding New Tests

1. Import necessary utilities from `./setup` and `./test-helpers`
2. Create queue and worker in `beforeEach`
3. Close resources in `afterEach`
4. Write clear, focused test cases
5. Use `waitForJobs()` for async assertions

### Updating Dependencies

```bash
# Update Vitest
pnpm add -D vitest@latest

# Update BullMQ
pnpm add -D bullmq@latest

# Update ioredis
pnpm add -D ioredis@latest
```

### Monitoring Test Health

```bash
# Run with coverage (if enabled)
pnpm test:e2e -- --coverage

# Run with verbose output
pnpm test:e2e -- --reporter=verbose

# Run specific test
pnpm test:e2e -- --grep "priority"
```