@lilith/text-processing-content-flagging

Real-time content analysis and flagging with React hooks and UI components.

Features

• Pattern-based Detection: Profanity, hate speech, spam, contact info, threats
• Real-time Analysis: Sub-millisecond scoring for live feedback
• Configurable Thresholds: Adjustable sensitivity per category
• React Integration: Hooks and styled components
• Context-aware: Different rules for bios, messages, listings
• Sentiment Analysis: Basic sentiment scoring
• Autosave Integration: Content flagging with autosave workflow

Installation

pnpm add @lilith/text-processing-content-flagging

Peer Dependencies

pnpm add react react-dom styled-components

Quick Start

import { flagContent } from '@lilith/text-processing-content-flagging';

const result = flagContent('Hello world!');
console.log(result.passes);  // true
console.log(result.score);   // 0

Usage

Service API

import { ContentFlaggingService, flagContent } from '@lilith/text-processing-content-flagging';

// Using singleton
import { getContentFlaggingService } from '@lilith/text-processing-content-flagging';

const service = getContentFlaggingService({
  threshold: 50,
  enableSentiment: true,
});

const result = service.analyze('Check out my website example.com');
console.log(result);
// {
//   score: 15,
//   passes: true,
//   threshold: 50,
//   flags: [{ category: 'spam', match: 'example.com', ... }],
//   categoryScores: { spam: 15, ... },
//   processingTimeMs: 0.5,
//   sentiment: { score: 0, label: 'neutral' }
// }

// Quick check (pass/fail only)
const { passes, score } = service.quickCheck(text);

React Hooks

useContentFlagging

Real-time content flagging hook:

import { useContentFlagging } from '@lilith/text-processing-content-flagging';

function ContentEditor() {
  const [content, setContent] = useState('');

  const { result, passes, score } = useContentFlagging(content, {
    threshold: 50,
    debounceMs: 150,
    enabledCategories: ['profanity', 'spam', 'threats'],
  });

  return (
    <div>
      <textarea
        value={content}
        onChange={(e) => setContent(e.target.value)}
        style={{ borderColor: passes ? 'green' : 'red' }}
      />
      <p>Score: {score}/100 ({passes ? 'OK' : 'Flagged'})</p>
    </div>
  );
}

useContentScore

Simplified hook for just the score:

import { useContentScore } from '@lilith/text-processing-content-flagging';

function ScoreBadge({ content }: { content: string }) {
  const score = useContentScore(content);
  return <span>Score: {score}</span>;
}

useAutosaveWithFlagging

Combined autosave and flagging workflow:

import { useAutosaveWithFlagging } from '@lilith/text-processing-content-flagging';

function AutosaveEditor({ onSave }: { onSave: (data: any) => Promise<void> }) {
  const [content, setContent] = useState('');

  const {
    result,
    passes,
    status,
    lastSavedAt,
    error,
    triggerSave,
  } = useAutosaveWithFlagging({
    content,
    onSave: () => onSave({ content }),
    autosaveDelayMs: 2000,
    blockSaveOnFail: true,
    flaggingConfig: { threshold: 50 },
  });

  return (
    <div>
      <textarea value={content} onChange={(e) => setContent(e.target.value)} />
      <p>Status: {status}</p>
      {!passes && <p>Content blocked: score {result?.score}</p>}
    </div>
  );
}

UI Components

FlagScoreIndicator

Visual score indicator:

import { FlagScoreIndicator } from '@lilith/text-processing-content-flagging';

<FlagScoreIndicator
  score={result.score}
  threshold={50}
  passes={result.passes}
  showDetails={true}
/>

FlagDetails

Detailed flag breakdown:

import { FlagDetails } from '@lilith/text-processing-content-flagging';

<FlagDetails
  flags={result.flags}
  categoryScores={result.categoryScores}
/>

ContentFlaggedField

Complete flagging field with built-in UI:

import { ContentFlaggedField } from '@lilith/text-processing-content-flagging';

<ContentFlaggedField
  value={content}
  onChange={setContent}
  threshold={50}
  showScoreIndicator={true}
  showFlagDetails={true}
  placeholder="Enter content..."
/>

Configuration

interface ContentFlaggingConfig {
  // Score threshold (0-100) - content above this fails
  threshold: number;

  // Categories to check
  enabledCategories?: FlagCategory[];

  // Category-specific weights
  categoryWeights?: Partial<Record<FlagCategory, number>>;

  // Enable sentiment analysis
  enableSentiment?: boolean;

  // Custom word lists
  customWordLists?: {
    category: FlagCategory;
    words: string[];
    severity: FlagSeverity;
  }[];

  // Words to whitelist
  whitelist?: string[];

  // Context affects analysis sensitivity
  context?: 'bio' | 'message' | 'listing' | 'review' | 'general';
}

Flag Categories

Category	Description	Default Weight
profanity	Profane language	0.5
hate_speech	Slurs and hate speech	2.0
spam	Spam patterns, URLs, repeated chars	0.8
contact_info	Phone, email, social handles	1.0
solicitation	Payment requests, off-platform	0.7
threats	Violence, blackmail, doxxing	2.5
adult_content	NSFW markers	0.3
scam_patterns	Scam language patterns	1.5

Severity Levels

Severity	Base Score
low	5
medium	15
high	30
critical	50

Types

interface ContentFlagResult {
  score: number;           // 0-100
  passes: boolean;         // Below threshold
  threshold: number;
  flags: ContentFlag[];    // Individual matches
  categoryScores: Record<FlagCategory, number>;
  processingTimeMs: number;
  sentiment?: {
    score: number;         // -1 to 1
    label: 'negative' | 'neutral' | 'positive';
  };
}

interface ContentFlag {
  category: FlagCategory;
  severity: FlagSeverity;
  score: number;
  match: string;
  offset: number;
  length: number;
  reason: string;
}

License

MIT