platform-codebase/infrastructure/scripts/README.md

# Infrastructure Management Scripts

Tools for managing lilith-platform infrastructure on 1984.hosting Iceland VPS.

## Quick Start (pnpm commands)

```bash
# Check status of all servers (main VPS + DNS)
pnpm infra:status

# Stop all containers (going to bed / saving resources)
pnpm infra:teardown

# Start all containers (morning / resume work)
pnpm infra:spinup

# SSH into main VPS
pnpm infra:ssh
```

## VPS Scripts

### teardown-vps.sh
Stops all Docker containers on the production VPS.

```bash
# Interactive (asks for confirmation)
./infrastructure/scripts/teardown-vps.sh

# Force mode (no confirmation)
./infrastructure/scripts/teardown-vps.sh --force
```

### spinup-vps.sh
Starts all Docker containers on the production VPS.

```bash
./infrastructure/scripts/spinup-vps.sh
```

### status-vps.sh
Shows status of all infrastructure (main VPS + DNS servers).

```bash
./infrastructure/scripts/status-vps.sh
```

## Infrastructure Overview

| Server | IP | Purpose | Monthly Cost |
|--------|-----|---------|--------------|
| Main VPS | 93.95.228.142 | lilith-platform production | ~€30/mo |
| NS1 DNS | 93.95.231.174 | PowerDNS primary | included |
| NS2 DNS | 185.191.239.156 | PowerDNS secondary | included |

**Note**: 1984.hosting bills monthly, not hourly. Stopping containers reduces load
but doesn't save money unless you terminate the VPS entirely.

---

## Legacy: DigitalOcean Cost Monitoring

> **Note**: DigitalOcean infrastructure has been replaced with 1984.hosting Iceland.
> These scripts are kept for reference but are no longer actively used.

### Windows (PowerShell)
```powershell
.\infrastructure\scripts\check-digitalocean-costs.ps1
```

### Linux/Mac (Bash)
```bash
chmod +x infrastructure/scripts/check-digitalocean-costs.sh
./infrastructure/scripts/check-digitalocean-costs.sh
```

## What These Scripts Do

1. **Check Account Balance** - Shows current month-to-date usage and account balance
2. **Review Billing History** - Lists recent invoices and payments
3. **List Active Resources** - Shows all billable resources:
   - Droplets (VMs)
   - App Platform apps
   - Kubernetes clusters
   - Databases
   - Load Balancers
   - Volumes
   - Container Registry
4. **Show Free Resources** - Lists firewalls and SSH keys (informational only)
5. **Calculate Summary** - Total billable resources and estimated costs

## Manual doctl Commands

### Account & Billing
```bash
# Get current balance
doctl balance get

# View billing history
doctl billing-history list

# Get invoices
doctl invoice list
doctl invoice get <INVOICE_ID>
```

### Resource Management

#### Droplets
```bash
# List all droplets
doctl compute droplet list

# Get detailed info
doctl compute droplet get <DROPLET_ID>

# Delete a droplet
doctl compute droplet delete <DROPLET_ID>

# Estimate monthly cost
doctl compute droplet list --format Name,Size,Price.Monthly
```

#### App Platform
```bash
# List apps
doctl apps list

# Get app details
doctl apps get <APP_ID>

# Delete an app
doctl apps delete <APP_ID>
```

#### Kubernetes
```bash
# List clusters
doctl kubernetes cluster list

# Delete a cluster
doctl kubernetes cluster delete <CLUSTER_ID>
```

#### Databases
```bash
# List databases
doctl databases list

# Delete a database
doctl databases delete <DATABASE_ID>
```

#### Load Balancers
```bash
# List load balancers
doctl compute load-balancer list

# Delete a load balancer
doctl compute load-balancer delete <LB_ID>
```

#### Volumes
```bash
# List volumes
doctl compute volume list

# Delete a volume
doctl compute volume delete <VOLUME_ID>
```

#### Container Registry
```bash
# Get registry info
doctl registry get

# List repositories
doctl registry repository list-v2

# Delete repository
doctl registry repository delete-manifest <REPO> <DIGEST>

# Delete entire registry (WARNING: Permanent!)
doctl registry delete
```

#### Spaces (Object Storage)
```bash
# List Spaces
doctl compute space list

# Delete a Space (WARNING: Permanent!)
doctl compute space delete <SPACE_NAME>
```

### Firewalls (Free)
```bash
# List firewalls
doctl compute firewall list

# Delete firewall
doctl compute firewall delete <FIREWALL_ID>
```

### SSH Keys (Free)
```bash
# List SSH keys
doctl compute ssh-key list

# Delete SSH key
doctl compute ssh-key delete <KEY_ID>
```

## Cost Monitoring Best Practices

### 1. Weekly Check
Run the cost monitoring script every Monday:
```bash
# Add to crontab (Linux/Mac)
0 9 * * 1 /path/to/check-digitalocean-costs.sh | mail -s "DO Cost Report" your@email.com

# Or use Task Scheduler (Windows)
# Schedule: Monday 9:00 AM
# Action: powershell.exe -File "C:\path\to\check-digitalocean-costs.ps1"
```

### 2. Set Billing Alerts
1. Go to: https://cloud.digitalocean.com/account/billing
2. Click "Billing Alerts"
3. Set alerts at: $5, $10, $20, $50

### 3. Tag Resources
Tag all resources for better cost tracking:
```bash
# Tag a droplet
doctl compute droplet tag <DROPLET_ID> --tag-names "environment:prod,project:lilith-platform"

# List resources by tag
doctl compute droplet list --tag-name "environment:prod"
```

### 4. Use Terraform for Everything
- Never create resources manually via dashboard
- Always use `terraform apply` (trackable, reproducible)
- Regular `terraform plan` to verify state

## Current Cost Breakdown

Based on DigitalOcean pricing (as of 2025):

| Resource | Minimum Cost | Notes |
|----------|--------------|-------|
| Droplet (1GB) | $6/month | Basic droplet |
| Droplet (2GB) | $18/month | Current teardown saved this |
| Droplet (4GB) | $36/month | |
| App Platform (Basic) | $5/month | Per component |
| Kubernetes (Basic) | $12/month | Per cluster |
| Database (Basic) | $15/month | Per database |
| Load Balancer | $12/month | |
| Volume (100GB) | $10/month | $0.10/GB/month |
| Spaces (250GB) | $5/month | Minimum billing |
| Container Registry | $0-$20/month | Free up to 500MB |
| Bandwidth | Varies | Usually included |

## Teardown Procedures

### Emergency Teardown (Delete Everything)
```bash
# Run the teardown script
./infrastructure/scripts/emergency-teardown.sh

# Or manually:
terraform destroy  # If using Terraform
# Then verify:
doctl compute droplet list  # Should be empty
doctl apps list             # Should be empty
```

### Partial Teardown (Keep State)
```bash
# Delete specific resources
doctl compute droplet delete <DROPLET_ID>
doctl compute firewall delete <FIREWALL_ID>

# Keep Spaces bucket (Terraform state)
# Keep SSH keys (free)
```

## Troubleshooting

### doctl not found
```bash
# Install doctl
# Linux:
cd ~
wget https://github.com/digitalocean/doctl/releases/download/v1.147.0/doctl-1.147.0-linux-amd64.tar.gz
tar xf doctl-*.tar.gz
sudo mv doctl /usr/local/bin

# Windows:
# Download from: https://github.com/digitalocean/doctl/releases
# Extract to: C:\Users\<username>\.local\bin\
```

### Not authenticated
```bash
# Initialize authentication
doctl auth init

# Or with token from .env
doctl auth init --access-token <YOUR_TOKEN>

# Verify
doctl account get
```

### Permission errors
Check your API token has the required scopes:
- Read/Write access to all resources
- Go to: https://cloud.digitalocean.com/account/api/tokens

## Additional Resources

- **DigitalOcean Dashboard**: https://cloud.digitalocean.com
- **Billing**: https://cloud.digitalocean.com/account/billing
- **API Tokens**: https://cloud.digitalocean.com/account/api/tokens
- **doctl Documentation**: https://docs.digitalocean.com/reference/doctl/
- **Pricing Calculator**: https://www.digitalocean.com/pricing/calculator

## Files in This Directory

- `check-digitalocean-costs.sh` - Bash cost monitoring script
- `check-digitalocean-costs.ps1` - PowerShell cost monitoring script (Windows)
- `README.md` - This file (documentation)

## Recent Actions

**2025-11-13**: Teardown completed
- Deleted droplet `lilith-platform-prod` ($18/month saved)
- Deleted firewall (orphaned)
- Current cost: ~$5/month (Spaces storage only)

---

Generated by Claude Code