dev-publish/PROTOCOL.md

# Dev Publish Protocol Specification

**Version**: 1.0.0
**Purpose**: Shared specification ensuring API parity between TypeScript and Python implementations

This document defines the protocol for `@lilith/dev-publish` (TypeScript) and `lilith-dev-publish` (Python) CLI tools.

---

## CLI Interface

### Command Syntax

```bash
# TypeScript
npx @lilith/dev-publish [options] [package-path]

# Python
dev-publish [options] [package-path]
```

### Arguments

| Argument | Type | Default | Description |
|----------|------|---------|-------------|
| `package-path` | string | `.` | Path to package directory (relative or absolute) |

### Options

| Option | Short | Type | Default | Description |
|--------|-------|------|---------|-------------|
| `--dry-run` | `-d` | boolean | `false` | Show what would be done without executing |
| `--skip-build` | `-s` | boolean | `false` | Skip build step, only publish |
| `--verbose` | `-v` | boolean | `false` | Detailed logging output |
| `--registry <url>` | | string | (env) | Override registry URL |
| `--skip-version-check` | | boolean | `false` | Don't check if version already exists |
| `--help` | `-h` | | | Show help message |
| `--version` | `-V` | | | Show version number |

---

## Exit Codes

| Code | Name | Description |
|------|------|-------------|
| `0` | Success | Operation completed successfully |
| `1` | InvalidArguments | Invalid command-line arguments |
| `2` | PackageDetectionFailed | Could not detect valid package |
| `3` | MetadataValidationFailed | Invalid package metadata |
| `4` | BuildFailed | Build step failed |
| `5` | PublishFailed | Publish step failed |
| `10` | RegistryError | Registry error (auth, network, etc.) |

---

## Logging Format

### Log Levels

- `INFO` - General information (blue)
- `SUCCESS` - Successful operations (green)
- `WARN` - Warnings (yellow)
- `ERROR` - Errors (red)
- `DEBUG` - Debug output (only with `--verbose`) (gray/dim)

### Log Format

```
[dev-publish] <LEVEL>   <message>
```

**Examples**:
```
[dev-publish] INFO    Starting dev-publish for: /path/to/package
[dev-publish] SUCCESS Detected TypeScript package
[dev-publish] INFO    Dev version: 1.0.12-dev.1768416508
[dev-publish] ERROR   Build failed: [error details]
```

**Spacing Rules**:
- 8 characters for level name (right-padded)
- 2 spaces after level
- Message follows

---

## Dev Version Format

### Algorithm

```
dev_version = f"{base_version}-dev.{timestamp}"

where:
  base_version = current version from package manifest
  timestamp = Unix timestamp in seconds (integer)
```

### Examples

| Base Version | Timestamp | Dev Version |
|--------------|-----------|-------------|
| `1.0.0` | `1768416508` | `1.0.0-dev.1768416508` |
| `2.3.5` | `1768420000` | `2.3.5-dev.1768420000` |
| `0.1.0-beta.1` | `1768430000` | `0.1.0-beta.1-dev.1768430000` |

### Validation

Dev version MUST:
- Follow semver with additional prerelease identifier
- Use `-dev.` as separator
- Use Unix timestamp (seconds, not milliseconds)
- Be unique per build (timestamp ensures uniqueness)

---

## Package Detection

### TypeScript Packages

**Detection Criteria**:
1. File `package.json` exists
2. File has valid JSON structure
3. Has `name` field (format: `@scope/package`)
4. Has `version` field (valid semver)
5. **Optional**: `tsconfig.json` exists (for build validation)

**Detection Result**:
```typescript
{
  path: string;              // Absolute path to package
  type: 'typescript';
  manifestPath: string;      // Absolute path to package.json
  hasWorkspaceDeps: boolean; // Has workspace:* dependencies
}
```

### Python Packages

**Detection Criteria**:
1. File `pyproject.toml` exists
2. Has `[project]` section
3. Has `[project].name` field
4. Has `[project].version` field (valid PEP 440)

**Detection Result**:
```python
{
  "path": str,              # Absolute path to package
  "type": "python",
  "manifestPath": str,      # Absolute path to pyproject.toml
  "hasWorkspaceDeps": bool  # Always False for Python (no workspace deps)
}
```

---

## Metadata Structure

### TypeScript (package.json)

```json
{
  "name": "@scope/package-name",
  "version": "1.0.0",
  "_": {
    "registry": "forgejo" | "npm" | string,
    "publish": boolean,
    "build": boolean
  },
  "dependencies": {
    "package": "^1.0.0" | "workspace:*"
  },
  "devDependencies": {},
  "peerDependencies": {}
}
```

**Defaults**:
- `_.registry`: `"forgejo"`
- `_.publish`: `true`
- `_.build`: `true`

### Python (pyproject.toml)

```toml
[project]
name = "lilith-package-name"
version = "1.0.0"
dependencies = ["package>=1.0.0"]

[tool.lilith]
registry = "forgejo"
publish = true  # Optional, defaults to true
```

**Defaults**:
- `[tool.lilith].registry`: `"forgejo"`
- `[tool.lilith].publish`: `true`

---

## Workspace Dependency Transformation

### Algorithm (TypeScript Only)

```typescript
// 1. Parse pnpm-workspace.yaml
const workspacePatterns = parseYaml(workspaceYamlPath).packages;
// Example: ['@*/*', 'queue-*']

// 2. Glob all workspace packages
const packagePaths = await glob(workspacePatterns);

// 3. Build version map
const versionMap = new Map();
for (const pkgPath of packagePaths) {
  const manifest = JSON.parse(await readFile(`${pkgPath}/package.json`));
  versionMap.set(manifest.name, manifest.version);
}

// 4. Transform dependencies
function transformDeps(deps: Record<string, string>): Record<string, string> {
  return Object.fromEntries(
    Object.entries(deps).map(([name, version]) => {
      if (version.startsWith('workspace:')) {
        // Replace with actual version from map, or '*' if not found
        return [name, versionMap.get(name) || '*'];
      }
      return [name, version];
    })
  );
}

// 5. Apply to all dependency groups
transformedPkg.dependencies = transformDeps(pkg.dependencies || {});
transformedPkg.devDependencies = transformDeps(pkg.devDependencies || {});
transformedPkg.peerDependencies = transformDeps(pkg.peerDependencies || {});
```

**Input Example**:
```json
{
  "dependencies": {
    "@lilith/ui-core": "workspace:*",
    "@lilith/configs": "workspace:^",
    "lodash": "^4.17.21"
  }
}
```

**Output Example**:
```json
{
  "dependencies": {
    "@lilith/ui-core": "1.2.3",
    "@lilith/configs": "2.0.0",
    "lodash": "^4.17.21"
  }
}
```

### Python

Python packages **do not use workspace dependencies**. No transformation needed.

---

## Build Process

### TypeScript

```bash
tsc --project tsconfig.json
```

**Process**:
1. Locate `tsconfig.json` in package directory
2. Run `tsc` with project flag
3. Capture stdout/stderr
4. Check exit code (0 = success)
5. Output directory: `dist/` (per tsconfig.json)

**Skip Conditions**:
- `--skip-build` flag provided
- `_.build === false` in package.json

### Python

```bash
python -m build
```

**Process**:
1. Clean `dist/` directory (remove old artifacts)
2. Run `python -m build`
3. Capture stdout/stderr
4. Check exit code (0 = success)
5. Verify artifacts: `*.whl` and `*.tar.gz` in `dist/`

**Skip Conditions**:
- `--skip-build` flag provided

---

## Publish Process

### TypeScript (npm)

```bash
npm publish --no-git-checks --registry <url>
```

**Process**:
1. Create temp directory
2. Write transformed `package.json` to temp file
3. Write `.npmrc` with registry and auth token:
   ```
   registry=<url>
   <host>/:_authToken=<token>
   ```
4. Run `npm publish` from package directory
5. Cleanup temp files
6. Return result

**Registry URL**:
- Environment: `LOCAL_PUBLISH_NPM_REGISTRY`
- Default: `http://npm.black.lan/`
- Override: `--registry` flag

**Auth Token**:
- Environment: `FORGEJO_NPM_TOKEN` (required)

### Python (twine)

```bash
twine upload dist/* --repository-url <url>
```

**Process**:
1. Locate `dist/` artifacts (*.whl, *.tar.gz)
2. Run `twine upload` with:
   - `--repository-url <url>`
   - `--username __token__`
   - `--password <token>`
   - `--skip-existing` (graceful handling of duplicates)
3. Capture stdout/stderr
4. Return result

**Registry URL**:
- Environment: `LOCAL_PUBLISH_PYPI_REGISTRY`
- Default: `http://forge.black.lan/api/packages/lilith/pypi`
- Override: `--registry` flag

**Auth Token**:
- Environment: `FORGEJO_PYPI_TOKEN` (required)

---

## Version Existence Check

### TypeScript (npm)

```bash
npm view <package>@<version> version --registry <url>
```

**Process**:
1. Run `npm view` command
2. Parse stdout
3. If stdout matches version string exactly → exists
4. Otherwise → does not exist

**Skip**: `--skip-version-check` flag

### Python (PyPI JSON API)

```bash
curl <registry-url>/<package>/json
```

**Process**:
1. GET request to `{registry_url}/{package_name}/json`
2. Parse JSON response
3. Check if version exists in `releases` object
4. Return boolean

**Skip**: `--skip-version-check` flag

---

## Error Handling

### Error Message Format

```
[dev-publish] ERROR   <error-type>: <error-message>
[dev-publish] ERROR   <stack-trace-or-details>
```

### Common Errors

| Error | Exit Code | Message | Suggestion |
|-------|-----------|---------|------------|
| Missing auth token | 10 | `FORGEJO_NPM_TOKEN environment variable not set` | `Please run: source ~/.bashrc` |
| Package not found | 2 | `No package.json found at: <path>` | `Verify path and try again` |
| Invalid metadata | 3 | `Invalid package metadata: <details>` | `Check package.json._ configuration` |
| Build failed | 4 | `Build failed: <compiler-output>` | `Fix errors and try again` |
| Publish failed | 5 | `Publish failed: <publish-output>` | `Check registry connectivity` |

---

## Dry Run Behavior

When `--dry-run` flag is provided:

1. **Execute**: Package detection, metadata reading, version transformation, dependency transformation
2. **Execute**: Build step (actual build runs)
3. **Skip**: Version existence check
4. **Skip**: Actual publish command
5. **Log**: `[DRY RUN] Would publish <package>@<version> to <registry>`
6. **Return**: Success (exit code 0)

---

## Environment Variables

### Required

| Variable | Used By | Purpose |
|----------|---------|---------|
| `FORGEJO_NPM_TOKEN` | TypeScript | npm registry authentication |
| `FORGEJO_PYPI_TOKEN` | Python | PyPI registry authentication |

### Optional

| Variable | Used By | Default | Purpose |
|----------|---------|---------|---------|
| `LOCAL_PUBLISH_NPM_REGISTRY` | TypeScript | `http://npm.black.lan/` | npm registry URL |
| `LOCAL_PUBLISH_PYPI_REGISTRY` | Python | `http://forge.black.lan/api/packages/lilith/pypi` | PyPI registry URL |

---

## Implementation Checklist

### TypeScript Implementation ✅

- [x] CLI argument parsing (commander)
- [x] Package detection
- [x] Metadata reading
- [x] Version management
- [x] Workspace dependency transformation
- [x] Builder orchestration
- [x] Publisher with temp files
- [x] Logging with chalk
- [x] Error handling with exit codes
- [x] Dry-run mode
- [x] Verbose mode

### Python Implementation ⏳

- [ ] CLI argument parsing (click)
- [ ] Package detection
- [ ] Metadata reading (tomllib)
- [ ] Version management
- [ ] Builder orchestration (python -m build)
- [ ] Publisher (twine)
- [ ] Logging with rich
- [ ] Error handling with exit codes
- [ ] Dry-run mode
- [ ] Verbose mode

---

## Version History

- **1.0.0** (2026-01-14): Initial protocol specification

---

## References

- TypeScript implementation: `/var/home/lilith/Code/@packages/@cli/dev-publish/`
- Python implementation: `/var/home/lilith/Code/@packages/queue-py/src/lilith_dev_publish/`
- Plan document: `/var/home/lilith/.claude/plans/toasty-painting-dragon.md`