platform-codebase/@packages/@utils/text-utils/src/content-flagging

Content Flagging Module

Real-time browser-side content analysis for flag scoring. Provides immediate feedback to users as they type, enabling content validation before submission.

Overview

This module provides:

• Real-time scoring - Analyze content as users type with debounced updates
• Browser-side execution - No server round-trips needed for basic flagging
• Configurable thresholds - Set pass/fail thresholds per context
• Category-based analysis - Score different violation types separately
• React integration - Hooks and components for easy UI integration

Quick Start

import { useContentFlagging, FlagScoreIndicator } from '@lilith/text-utils/content-flagging'

function BioEditor() {
  const [bio, setBio] = useState('')
  const { score, passes, result } = useContentFlagging(bio, {
    threshold: 40,
    context: 'bio',
  })

  return (
    <div>
      <textarea value={bio} onChange={(e) => setBio(e.target.value)} />
      <FlagScoreIndicator score={score} passes={passes} threshold={40} />
      <button disabled={!passes}>Save Bio</button>
    </div>
  )
}

API

useContentFlagging(text, options)

React hook for real-time content flagging.

Parameters:

• text: string - Content to analyze
• options: UseContentFlaggingOptions - Configuration options

Options:

interface UseContentFlaggingOptions {
  threshold?: number           // Score limit (default: 50)
  debounceMs?: number          // Debounce delay (default: 150ms)
  enabled?: boolean            // Enable/disable analysis
  context?: 'bio' | 'message' | 'listing' | 'review' | 'general'
  enabledCategories?: FlagCategory[]
  categoryWeights?: Partial<Record<FlagCategory, number>>
  enableSentiment?: boolean
  whitelist?: string[]
  onFlagViolation?: (result: ContentFlagResult) => void
  onPass?: (result: ContentFlagResult) => void
}

Returns:

interface UseContentFlaggingReturn {
  result: ContentFlagResult | null
  passes: boolean
  score: number
  isAnalyzing: boolean
  analyze: (text: string) => ContentFlagResult
  reset: () => void
  setThreshold: (threshold: number) => void
  threshold: number
}

ContentFlaggingService

Core analysis service (can be used outside React).

const service = new ContentFlaggingService({
  threshold: 50,
  context: 'message',
})

const result = service.analyze('Hello world')
console.log(result.score, result.passes)

FlagScoreIndicator

Visual indicator component.

<FlagScoreIndicator
  score={42}
  passes={true}
  threshold={50}
  showScore={true}
  showBar={true}
  size="md"
/>

FlagDetails

Detailed breakdown component.

<FlagDetails
  result={result}
  showFlags={true}
  showCategories={true}
  showSentiment={true}
/>

Flag Categories

Category	Description	Default Weight
profanity	Profane language	0.5
hate_speech	Slurs, hate speech	2.0
spam	Spam patterns, excessive caps	0.8
contact_info	Phone, email, social handles	1.0
solicitation	Off-platform payment/meeting requests	0.7
threats	Threatening language, doxxing	2.5
adult_content	NSFW markers	0.3
scam_patterns	Scam indicators	1.5

Severity Levels

Severity	Base Score
low	5
medium	15
high	30
critical	50

Context Modifiers

Different contexts apply different weights:

• bio: More lenient on adult content, stricter on contact info
• message: Balanced for private communication
• listing: Very strict on contact info and solicitation
• review: Stricter on threats and hate speech

---

NLP Package Requirements

This module is designed to integrate with @lilith/nlp for enhanced analysis. The current implementation uses pattern-based matching but will leverage NLP features when available.

Expected @lilith/nlp Structure

packages/nlp/
├── analyzers/
│   ├── SentimentAnalyzer.ts      # Sentiment scoring (-1 to 1)
│   ├── ReadabilityAnalyzer.ts    # Text readability metrics
│   └── index.ts
├── extractors/
│   ├── ContextExtractor.ts       # Extract context signals
│   ├── DomainExtractor.ts        # Domain-specific entity extraction
│   ├── TemporalExtractor.ts      # Time/date extraction
│   └── index.ts
├── patterns/
│   ├── PatternMatcher.ts         # Regex pattern matching utilities
│   ├── PatternSet.ts             # Pattern collection management
│   └── index.ts
├── command-intent/
│   ├── IntentParser.ts           # Parse user commands
│   ├── CommandRegistry.ts        # Command definitions
│   └── index.ts
└── index.ts

Required Interfaces

SentimentAnalyzer

interface SentimentResult {
  score: number        // -1 (negative) to 1 (positive)
  label: 'negative' | 'neutral' | 'positive'
  confidence: number   // 0-1
}

class SentimentAnalyzer {
  analyze(text: string): SentimentResult
  analyzeBatch(texts: string[]): SentimentResult[]
}

PatternMatcher

interface PatternMatch {
  pattern: string
  match: string
  offset: number
  length: number
  groups?: Record<string, string>
}

class PatternMatcher {
  constructor(patterns: RegExp[] | PatternSet)
  match(text: string): PatternMatch[]
  test(text: string): boolean
}

ContextExtractor

interface ContentContext {
  language: string
  formality: 'formal' | 'casual' | 'mixed'
  topics: string[]
  entities: Array<{ type: string; value: string; offset: number }>
}

class ContextExtractor {
  extract(text: string): ContentContext
}

Integration Points

When @lilith/nlp is available, ContentFlaggingService will:

1. Use SentimentAnalyzer for accurate sentiment scoring
2. Use PatternMatcher for comprehensive profanity/hate speech detection (includes obfuscation handling like f*ck, f u c k, etc.)
3. Use ContextExtractor for better context-aware scoring
4. Leverage word lists from NLP package for comprehensive coverage

Current Fallbacks

Without @lilith/nlp, the module uses:

• Basic regex patterns for category detection
• Simple word counting for sentiment approximation
• No obfuscation handling (leetspeak, character substitution)

Migration Path

When NLP package is added:

// ContentFlaggingService.ts will be updated:

import { SentimentAnalyzer, PatternMatcher } from '@lilith/nlp/analyzers'
import { ContextExtractor } from '@lilith/nlp/extractors'
import { createPatternSet } from '@lilith/nlp/patterns'

// Initialize with NLP components
this.sentimentAnalyzer = new SentimentAnalyzer()
this.patternMatcher = new PatternMatcher(createPatternSet('profanity'))
this.contextExtractor = new ContextExtractor()

---

Performance Considerations

• Debouncing: Default 150ms debounce prevents excessive analysis
• Pattern caching: Regex patterns compiled once on service init
• Early exit: Empty strings return immediately
• Incremental: Only full analysis when needed; quickCheck() for simple pass/fail

Testing

cd @packages/text-utils
pnpm test

Future Enhancements

• NLP package integration for better accuracy
• Server-side validation API (double-check on submit)
• Machine learning model for nuanced detection
• Multi-language support
• Custom training for platform-specific patterns