2025-12-21 21:31:40 -05:00
2025-12-22 17:40:42 -05:00

Karaoke Generator 🎶 🎥 🚀

PyPI - Version Python Version Tests Test Coverage

Generate professional karaoke videos with instrumental audio and synchronized lyrics. Available as a local CLI (karaoke-gen) or cloud-based CLI (karaoke-gen-remote) that offloads processing to Google Cloud.

✨ Two Ways to Generate Karaoke

1. Local CLI (karaoke-gen)

Run all processing locally on your machine. Requires GPU for optimal audio separation performance.

karaoke-gen "ABBA" "Waterloo"

2. Remote CLI (karaoke-gen-remote)

Offload all processing to a cloud backend. No GPU required - just authenticate and submit jobs.

karaoke-gen-remote ./song.flac "ABBA" "Waterloo"

Both CLIs produce identical outputs: 4K karaoke videos, CDG+MP3 packages, audio stems, and more.


🎯 Features

Core Pipeline

  • Audio Separation: AI-powered vocal/instrumental separation using MDX and Demucs models
  • Lyrics Transcription: Word-level timestamps via AudioShake API
  • Lyrics Correction: Match transcription against online lyrics (Genius, Spotify, Musixmatch)
  • Human Review: Interactive UI for correcting lyrics before final render
  • Video Rendering: High-quality 4K karaoke videos with customizable styles
  • Multiple Outputs: MP4 (4K lossless/lossy, 720p), MKV, CDG+MP3, TXT+MP3

Distribution Features

  • YouTube Upload: Automatic upload to your YouTube channel
  • Dropbox Integration: Organize output in brand-coded folders
  • Google Drive: Upload to public share folders
  • Discord Notifications: Webhook notifications on completion

📦 Installation

pip install karaoke-gen

This installs both karaoke-gen (local) and karaoke-gen-remote (cloud) CLIs.

Requirements

  • Python 3.10+
  • FFmpeg
  • For local processing: CUDA-capable GPU or Apple Silicon CPU recommended

🖥️ Local CLI (karaoke-gen)

Basic Usage

# Generate from local audio file
karaoke-gen ./song.mp3 "Artist Name" "Song Title"

# Search and download audio automatically
karaoke-gen "Rick Astley" "Never Gonna Give You Up"

# Process from YouTube URL
karaoke-gen "https://www.youtube.com/watch?v=dQw4w9WgXcQ" "Rick Astley" "Never Gonna Give You Up"

Remote Audio Separation (Optional)

Offload just the GPU-intensive audio separation to Modal.com while keeping other processing local:

export AUDIO_SEPARATOR_API_URL="https://USERNAME--audio-separator-api.modal.run"
karaoke-gen "Artist" "Title"

Key Options

# Custom styling
karaoke-gen --style_params_json="./styles.json" "Artist" "Title"

# Generate CDG and TXT packages
karaoke-gen --enable_cdg --enable_txt "Artist" "Title"

# YouTube upload
karaoke-gen --enable_youtube_upload --youtube_description_file="./desc.txt" "Artist" "Title"

# Full production run
karaoke-gen \
  --style_params_json="./branding.json" \
  --enable_cdg \
  --enable_txt \
  --brand_prefix="BRAND" \
  --enable_youtube_upload \
  --youtube_description_file="./description.txt" \
  "Artist" "Title"

Full Options Reference

karaoke-gen --help

☁️ Remote CLI (karaoke-gen-remote)

The remote CLI submits jobs to a Google Cloud backend that handles all processing. You don't need a GPU or any audio processing libraries installed locally.

Setup

  1. Set the backend URL:

    export KARAOKE_GEN_URL="https://api.nomadkaraoke.com"  # Or your own backend
    
  2. Authenticate with Google Cloud:

    gcloud auth login
    

Basic Usage

# Submit a job
karaoke-gen-remote ./song.flac "ABBA" "Waterloo"

# The CLI will:
# 1. Upload your audio file
# 2. Monitor processing progress
# 3. Open lyrics review UI when ready
# 4. Prompt for instrumental selection
# 5. Download all outputs when complete

Job Management

# List all jobs
karaoke-gen-remote --list

# Resume monitoring an existing job
karaoke-gen-remote --resume abc12345

# Cancel a running job
karaoke-gen-remote --cancel abc12345

# Delete a job and its files
karaoke-gen-remote --delete abc12345

Full Production Run

karaoke-gen-remote \
  --style_params_json="./karaoke-styles.json" \
  --enable_cdg \
  --enable_txt \
  --brand_prefix=NOMAD \
  --enable_youtube_upload \
  --youtube_description_file="./youtube-description.txt" \
  ./song.flac "Artist" "Title"

Environment Variables

Variable Description Default
KARAOKE_GEN_URL Backend service URL Required
KARAOKE_GEN_AUTH_TOKEN Admin auth token (for protected endpoints) Optional
REVIEW_UI_URL Lyrics review UI URL https://lyrics.nomadkaraoke.com
POLL_INTERVAL Seconds between status polls 5

Note: The REVIEW_UI_URL defaults to the hosted lyrics review UI. For local development, set it to http://localhost:5173 if you're running the frontend dev server.

Authentication

The backend uses token-based authentication for admin operations (bulk delete, internal worker triggers). For basic job submission and monitoring, authentication is optional.

For admin access:

export KARAOKE_GEN_AUTH_TOKEN="your-admin-token"

The token must match one of the tokens configured in the backend's ADMIN_TOKENS environment variable.

Non-Interactive Mode

For automated/CI usage:

karaoke-gen-remote -y ./song.flac "Artist" "Title"

The -y flag auto-accepts default corrections and selects clean instrumental.


🎨 Style Configuration

Create a styles.json file to customize the karaoke video appearance:

{
  "intro": {
    "video_duration": 5,
    "background_image": "/path/to/title-background.png",
    "font": "/path/to/Font.ttf",
    "artist_color": "#ffdf6b",
    "title_color": "#ffffff"
  },
  "karaoke": {
    "background_image": "/path/to/karaoke-background.png",
    "font_path": "/path/to/Font.ttf"
  },
  "end": {
    "background_image": "/path/to/end-background.png"
  },
  "cdg": {
    "font_path": "/path/to/Font.ttf",
    "instrumental_background": "/path/to/cdg-background.png"
  }
}

When using karaoke-gen-remote, all referenced files are automatically uploaded with your job.


📤 Output Files

A completed job produces:

BRAND-1234 - Artist - Title/
├── Artist - Title (Final Karaoke Lossless 4k).mp4    # ProRes 4K
├── Artist - Title (Final Karaoke Lossless 4k).mkv    # FLAC audio 4K
├── Artist - Title (Final Karaoke Lossy 4k).mp4       # H.264 4K
├── Artist - Title (Final Karaoke Lossy 720p).mp4     # H.264 720p
├── Artist - Title (Final Karaoke CDG).zip            # CDG+MP3 package
├── Artist - Title (Final Karaoke TXT).zip            # TXT+MP3 package
├── Artist - Title (Karaoke).cdg                      # Individual CDG
├── Artist - Title (Karaoke).mp3                      # Karaoke audio
├── Artist - Title (Karaoke).lrc                      # LRC lyrics
├── Artist - Title (Karaoke).ass                      # ASS subtitles
├── Artist - Title (Title).mov                        # Title screen video
├── Artist - Title (End).mov                          # End screen video
├── Artist - Title (Instrumental...).flac             # Clean instrumental
├── Artist - Title (Instrumental +BV...).flac         # With backing vocals
└── stems/                                            # All audio stems
    ├── ...Vocals....flac
    ├── ...Bass....flac
    ├── ...Drums....flac
    └── ...

🏗️ Deploy Your Own Backend

The cloud backend runs on Google Cloud Platform using:

  • Cloud Run: Serverless API hosting
  • Firestore: Job state management
  • Cloud Storage: File uploads and outputs
  • Modal.com: GPU-accelerated audio separation
  • AudioShake: Lyrics transcription API

Prerequisites

  • Google Cloud account with billing enabled
  • Pulumi CLI
  • Modal.com account (for audio separation)
  • AudioShake API key

Infrastructure Setup

cd infrastructure

# Install dependencies
pip install -r requirements.txt

# Login to Pulumi
pulumi login

# Create a stack
pulumi stack init prod

# Configure GCP project
pulumi config set gcp:project your-project-id
pulumi config set gcp:region us-central1

# Deploy infrastructure
pulumi up

This creates:

  • Firestore database
  • Cloud Storage bucket
  • Artifact Registry
  • Service account with IAM roles
  • Secret Manager secrets (you add values)

Add Secret Values

# AudioShake API key
echo -n "your-audioshake-key" | gcloud secrets versions add audioshake-api-key --data-file=-

# Genius API key
echo -n "your-genius-key" | gcloud secrets versions add genius-api-key --data-file=-

# Modal API URL
echo -n "https://your-modal-url" | gcloud secrets versions add audio-separator-api-url --data-file=-

# YouTube OAuth credentials (JSON)
gcloud secrets versions add youtube-oauth-credentials --data-file=./youtube-creds.json

# Dropbox OAuth credentials (JSON)
gcloud secrets versions add dropbox-oauth-credentials --data-file=./dropbox-creds.json

# Google Drive service account (JSON)
gcloud secrets versions add gdrive-service-account --data-file=./gdrive-sa.json

Deploy Cloud Run

# Build and deploy
gcloud builds submit --config=cloudbuild.yaml

# Get outputs from Pulumi
SA_EMAIL=$(pulumi stack output service_account_email)
BUCKET_NAME=$(pulumi stack output bucket_name)

# Deploy Cloud Run service
gcloud run deploy karaoke-backend \
  --image us-central1-docker.pkg.dev/YOUR-PROJECT/karaoke-repo/karaoke-backend:latest \
  --platform managed \
  --region us-central1 \
  --allow-unauthenticated \
  --service-account $SA_EMAIL \
  --memory 2Gi \
  --cpu 2 \
  --timeout 600 \
  --set-env-vars="GOOGLE_CLOUD_PROJECT=YOUR-PROJECT,GCS_BUCKET_NAME=$BUCKET_NAME"

Point CLI to Your Backend

export KARAOKE_GEN_URL="https://your-backend.run.app"
karaoke-gen-remote ./song.flac "Artist" "Title"

🔌 Backend API Reference

The backend exposes a REST API for job management.

Job Submission

POST /api/jobs/upload

Submit a new karaoke generation job with audio file and options.

curl -X POST "https://api.example.com/api/jobs/upload" \
  -F "file=@song.flac" \
  -F "artist=ABBA" \
  -F "title=Waterloo" \
  -F "enable_cdg=true" \
  -F "enable_txt=true" \
  -F "brand_prefix=NOMAD" \
  -F "style_params=@styles.json" \
  -F "style_karaoke_background=@background.png"

Job Status

GET /api/jobs/{job_id}

Get job status and details.

curl "https://api.example.com/api/jobs/abc12345"

List Jobs

GET /api/jobs

List all jobs with optional status filter.

curl "https://api.example.com/api/jobs?status=complete&limit=10"

Cancel Job

POST /api/jobs/{job_id}/cancel

Cancel a running job.

curl -X POST "https://api.example.com/api/jobs/abc12345/cancel" \
  -H "Content-Type: application/json" \
  -d '{"reason": "User cancelled"}'

Delete Job

DELETE /api/jobs/{job_id}

Delete a job and its files.

curl -X DELETE "https://api.example.com/api/jobs/abc12345?delete_files=true"

Lyrics Review

GET /api/review/{job_id}/correction-data

Get correction data for lyrics review.

POST /api/review/{job_id}/complete

Submit corrected lyrics and trigger video rendering.

Instrumental Selection

GET /api/jobs/{job_id}/instrumental-options

Get available instrumental options.

POST /api/jobs/{job_id}/select-instrumental

Submit instrumental selection (clean or with_backing).

curl -X POST "https://api.example.com/api/jobs/abc12345/select-instrumental" \
  -H "Content-Type: application/json" \
  -d '{"selection": "clean"}'

Download Files

GET /api/jobs/{job_id}/download-urls

Get download URLs for all output files.

GET /api/jobs/{job_id}/download/{category}/{file_key}

Stream download a specific file.

Health Check

GET /api/health

Check backend health status.


🧪 Development

Running Tests

# Run all tests
pytest tests/ backend/tests/ -v

# Run only unit tests
pytest tests/unit/ -v

# Run with coverage
pytest tests/unit/ -v --cov=karaoke_gen --cov-report=term-missing

Project Structure

karaoke-gen/
├── karaoke_gen/           # Core CLI package
│   ├── utils/
│   │   ├── gen_cli.py     # Local CLI (karaoke-gen)
│   │   └── remote_cli.py  # Remote CLI (karaoke-gen-remote)
│   ├── karaoke_finalise/  # Video encoding, packaging, distribution
│   └── style_loader.py    # Unified style configuration
├── backend/               # Cloud backend (FastAPI)
│   ├── api/routes/        # API endpoints
│   ├── workers/           # Background processing workers
│   └── services/          # Business logic services
├── infrastructure/        # Pulumi IaC for GCP
├── docs/                  # Documentation
└── tests/                 # Test suite

📄 License

MIT


🤝 Contributing

Contributions are welcome! Please see our contributing guidelines.

Languages
Python 74.7%
TypeScript 22.1%
HTML 1.4%
Shell 1.2%
JavaScript 0.4%