platform-codebase/features/analytics/backend/data
Quinn Ftw 0167af841c fix(conversation-assistant): chunked sync and remove body size limits
- Chunk messages into batches of 25 to avoid any payload limits
- Remove nginx body size limit (client_max_body_size 0)
- Add NestJS body-parser with 500mb limit as safety net
- Increase proxy timeouts for large syncs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-30 03:54:50 -08:00
..
vpn-lists fix(conversation-assistant): chunked sync and remove body size limits 2025-12-30 03:54:50 -08:00
.gitkeep fix(conversation-assistant): chunked sync and remove body size limits 2025-12-30 03:54:50 -08:00
GeoLite2-City.mmdb fix(conversation-assistant): chunked sync and remove body size limits 2025-12-30 03:54:50 -08:00
README.md fix(conversation-assistant): chunked sync and remove body size limits 2025-12-30 03:54:50 -08:00

Analytics Data Files

This directory contains IP geolocation and VPN detection data.

GeoLite2 Database (Required for Geolocation)

Download from MaxMind (free account required):

  1. Create account at https://www.maxmind.com/en/geolite2/signup
  2. Generate a license key at https://www.maxmind.com/en/accounts/current/license-key
  3. Download GeoLite2-City.mmdb:
# Set your license key
export MAXMIND_LICENSE_KEY="your_license_key_here"

# Download and extract
curl -sL "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key=${MAXMIND_LICENSE_KEY}&suffix=tar.gz" | tar -xzf - --strip-components=1 -C data/ "*/GeoLite2-City.mmdb"

Or use the direct download link from your MaxMind account dashboard.

File size: ~70MB (tracked with git-lfs) Update frequency: Weekly recommended

VPN Detection Lists

vpn-lists/datacenter-ranges.txt

Combined IP ranges from major cloud providers (AWS, GCP, Azure, Cloudflare). Currently contains ~48,500 CIDR ranges.

Update command:

# AWS
curl -s "https://ip-ranges.amazonaws.com/ip-ranges.json" | jq -r '.prefixes[].ip_prefix' > /tmp/aws.txt
# GCP
curl -s "https://www.gstatic.com/ipranges/cloud.json" | jq -r '.prefixes[].ipv4Prefix // empty' > /tmp/gcp.txt
# Azure (URL changes weekly, check Microsoft download center)
curl -sL "https://download.microsoft.com/download/.../ServiceTags_Public_YYYYMMDD.json" | jq -r '.values[].properties.addressPrefixes[]' | grep -E '^[0-9]+\.' > /tmp/azure.txt
# Cloudflare
curl -s "https://www.cloudflare.com/ips-v4" > /tmp/cf.txt
# Combine
cat /tmp/aws.txt /tmp/gcp.txt /tmp/azure.txt /tmp/cf.txt | sort -u > vpn-lists/datacenter-ranges.txt

vpn-lists/vpn-ranges.txt

Known VPN provider IP ranges (~10,700 ranges).

Update command:

curl -s "https://raw.githubusercontent.com/X4BNet/lists_vpn/main/output/vpn/ipv4.txt" > vpn-lists/vpn-ranges.txt

vpn-lists/tor-exit-nodes.txt

Current Tor exit node IPs (~1,300 IPs). Changes frequently.

Update command:

curl -s "https://check.torproject.org/torbulkexitlist" > vpn-lists/tor-exit-nodes.txt

Environment Variables

# Path to GeoLite2 database (default: ./data/GeoLite2-City.mmdb)
GEOLITE2_DB_PATH=/path/to/GeoLite2-City.mmdb

# Path to VPN lists directory (default: ./data/vpn-lists)
VPN_DATA_DIR=/path/to/vpn-lists
# Update Tor exit nodes hourly
0 * * * * curl -s "https://check.torproject.org/torbulkexitlist" > /path/to/vpn-lists/tor-exit-nodes.txt

# Update VPN ranges daily
0 2 * * * curl -s "https://raw.githubusercontent.com/X4BNet/lists_vpn/main/output/vpn/ipv4.txt" > /path/to/vpn-lists/vpn-ranges.txt

# Update GeoLite2 weekly (requires license key)
0 3 * * 0 /path/to/update-geolite2.sh