platform-codebase/features/i18n

i18n Feature

Multi-provider translation system with intelligent fallback and hallucination prevention.

Purpose

Translate UI content across 30+ languages using a two-layer architecture:

1. Frontend: Smart caching with localStorage → static → ML fallback
2. Backend: Multi-provider routing with automatic failover

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     User loads page in Spanish                  │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  1. Check localStorage (24h TTL)                                │
│     Found? → Return immediately                                 │
└─────────────────────────────────────────────────────────────────┘
                              │ Miss
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  2. Fetch static translations: GET /api/translations/es/common  │
│     Found? → Cache in localStorage, return                      │
└─────────────────────────────────────────────────────────────────┘
                              │ Miss
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  3. ML Translation: POST /api/i18n/translate/batch              │
│     ├─ Route to best provider (Claude for ES)                   │
│     ├─ Translate all keys in single request                     │
│     ├─ Fire-and-forget: save to server for future static        │
│     └─ Cache in localStorage, return                            │
└─────────────────────────────────────────────────────────────────┘

Translation Providers

Provider	Strengths	Best For
Claude	WMT24 winner, 78% "good" ratings	General, high quality
DeepL	Fewest edits needed, glossary support	European languages
Aya	8B model, self-hosted, no API costs	Budget-conscious
TowerInstruct	European language specialist	DE, FR, IT, ES
NLLB	Meta's 200-language model	Rare languages
MADLAD400	400+ languages	Maximum coverage

Language-Pair Routing

// Provider selection by target language
const PROVIDER_ROUTING = {
  es: ['claude', 'deepl', 'nllb'],      // Spanish: Claude first
  de: ['deepl', 'towerinstruct', 'claude'], // German: DeepL first
  ja: ['claude', 'nllb', 'madlad'],     // Japanese: Claude first
  sw: ['nllb', 'madlad'],               // Swahili: NLLB first
};

Automatic Fallback Chain

If primary provider fails, automatically tries next:

Claude → DeepL → TowerInstruct → NLLB → MADLAD400

Packages

Package	Location	Purpose
@lilith/i18n	react/	React hooks, i18next integration
lilith_i18n_service	ml-service/	Python ML service (port 41231)
@lilith/i18n-admin	frontend-admin/	Admin UI
@lilith/i18n-shared	shared/	Shared types

Key Features

Batch Translation

Translates entire namespace (40+ keys) in single LLM request:

// Input: nested object
{ "welcome": "Welcome", "nav": { "home": "Home", "about": "About" } }

// Flattened for LLM
{ "welcome": "Welcome", "nav.home": "Home", "nav.about": "About" }

// LLM translates all at once, then unflattened

Placeholder Preservation

Maintains i18next variables during translation:

"Hello {{name}}, you have {{count}} messages"
→ "Hola {{name}}, tienes {{count}} mensajes"

Auto-Persist to Static

ML translations automatically saved to server:

// After ML translation succeeds:
fetch('/api/translations/es/common', {
  method: 'POST',
  body: JSON.stringify(translations)  // Fire-and-forget
});

Next user gets static version (faster, no LLM cost).

Truth Validation Integration

All translations validated against platform facts:

const translation = await translate("Creators keep 85%", "es");
// Truth service catches: "85%" is wrong
// Auto-corrects to: "Los creadores se quedan con el 100%"

API Endpoints

Endpoint	Method	Description
/api/i18n/translate	POST	Translate single key
/api/i18n/translate/batch	POST	Translate namespace (40+ keys)
/api/i18n/locales	GET	List 30+ supported locales
/api/i18n/glossary	GET/PUT	Domain glossary (preferred terms)
/api/i18n/persist	POST	Save ML translations to static
/api/i18n/missing	GET	Find missing translations
/api/i18n/validate	POST	Validate against truth service

Usage

import { makeI18n } from '@lilith/i18n';

const { I18nProvider, useT } = makeI18n({
  defaultLocale: 'en',
  supportedLocales: ['en', 'es', 'fr', 'de', 'ja', 'ko', 'zh'],
  mlBackend: true,  // Enable ML fallback
  truthValidation: true,  // Validate content
});

function App() {
  return (
    <I18nProvider>
      <Welcome />
    </I18nProvider>
  );
}

function Welcome() {
  const t = useT();
  return <h1>{t('common.welcome')}</h1>;
}

Configuration

# ML Service
I18N_SERVICE_PORT=41231
I18N_SERVICE_DEFAULT_LOCALE=en
I18N_SERVICE_REDIS_URL=redis://localhost:6379
I18N_SERVICE_GLOSSARY_ENABLED=true
I18N_SERVICE_PERSIST_TRANSLATIONS=true
I18N_SERVICE_TRUTH_SERVICE_URL=http://localhost:41232

# Provider API Keys
CLAUDE_API_KEY=sk-...
DEEPL_API_KEY=...

Caching Strategy

Layer	TTL	Purpose
localStorage	24h	Instant UI, offline support
Redis	7d	Cross-user, provider tracking
Static files	∞	Human-reviewed translations