Compare commits

..
Author SHA1 Message Date
Andrew BeveridgeandClaude Opus 4.5 7b5a150e81 docs: add self-hosted runner documentation
- Update README.md with CI/CD runner status
- Add lessons learned for self-hosted runner setup
- Create archive doc with full implementation details

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 12:05:27 -05:00
Andrew Beveridge ☄️ 0b57b8f610 Merge pull request #95 from nomadkaraoke/feature/self-hosted-runner
feat: add self-hosted GitHub Actions runner on GCP
2025-12-28 11:51:59 -05:00
Andrew BeveridgeandClaude Opus 4.5 050ca6a441 fix: add setup-python and install-poetry to self-hosted runner jobs
Addresses CodeRabbit review feedback - the self-hosted runner jobs
need proper Python and Poetry setup via actions rather than relying
on pre-installed versions. This ensures Python 3.13 compatibility.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 11:49:09 -05:00
Andrew BeveridgeandClaude Opus 4.5 2725352d89 feat: add self-hosted GitHub Actions runner on GCP
Adds a dedicated GCP VM (e2-standard-4, 200GB SSD) to run GitHub Actions
jobs that were hitting disk space limits on GitHub-hosted runners (14GB).

Infrastructure changes:
- New service account `github-runner` with minimal permissions
- Secret Manager secret for GitHub PAT (runner registration)
- Compute instance with startup script that installs Docker, Python 3.13,
  Node.js 20, Java 21, Poetry, FFmpeg, and registers as a GitHub runner

Workflow changes:
- `package-build-test` now uses self-hosted runner
- `backend-unit-tests` now uses self-hosted runner
- `backend-emulator-tests` now uses self-hosted runner
- `deploy-backend` now uses self-hosted runner
- Removed disk cleanup steps (no longer needed with 200GB)

Setup requires adding GitHub PAT to Secret Manager after `pulumi up`:
  echo -n "ghp_xxx" | gcloud secrets versions add github-runner-pat --data-file=-

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 11:18:50 -05:00
Andrew Beveridge ☄️ 762cdf03c9 Merge pull request #94 from nomadkaraoke/fix/pip-install-quotes-91
docs: Add quotes to pip install commands for cross-shell compatibility
2025-12-28 11:11:47 -05:00
Andrew BeveridgeandClaude Opus 4.5 7df3a21525 docs: Add quotes to pip install commands for cross-shell compatibility
Wrap package names with extras notation in double quotes to prevent
zsh and other shells from interpreting square brackets as glob patterns.

Fixes #91

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 11:10:09 -05:00
Andrew Beveridge ☄️ f72f772f96 Merge pull request #88 from nomadkaraoke/feature/mobile-ux-improvements
feat: improve mobile responsiveness across all UIs
2025-12-28 03:13:58 -05:00
Andrew BeveridgeandClaude Opus 4.5 7a30407fdb fix: Address CodeRabbit review feedback
- Add language identifier to markdown code block in docs
- Increase Undo/Redo button touch targets from 32px to 44px

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 03:08:53 -05:00
Andrew BeveridgeandClaude Opus 4.5 e0e35f5335 feat: improve mobile responsiveness across all UIs
- Main Frontend:
  - Stack Artist/Title inputs vertically on mobile (< 640px)
  - Improve tab button touch targets with min-height 44px
  - Make header responsive with icon-only buttons on small screens
  - Hide button text labels on mobile, show icons only

- Lyrics Review UI (Material UI):
  - Make handler panel width responsive (100% on mobile)
  - Fix button toolbar to wrap on mobile
  - Adjust heights for mobile viewports

- Instrumental Selection UI (Static HTML):
  - Add comprehensive mobile media queries
  - Stack bottom section vertically on mobile
  - Make selection panel full-width on mobile
  - Increase touch targets to 44px minimum
  - Allow scrolling on mobile viewports

- Testing:
  - Add mobile device projects to Playwright config (Pixel 7, iPhone 14)
  - Create comprehensive mobile responsiveness test suite
  - Test no horizontal overflow, touch targets, form usability

- Documentation:
  - Add MOBILE-UX-BEST-PRACTICES.md with patterns and guidelines

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 03:08:05 -05:00
Andrew Beveridge ☄️ 79be51c95d Merge pull request #89 from nomadkaraoke/refactor/documentation-restructure
docs: Restructure documentation for LLM agent clarity
2025-12-28 03:00:33 -05:00
Andrew BeveridgeandClaude Opus 4.5 a3f0c08f99 fix: Address markdown linting issues from CodeRabbit review
- Wrap bare URLs in angle brackets in CLAUDE.md (MD034)
- Add language specifiers to fenced code blocks (MD040):
  - `text` for directory structures and ASCII diagrams
  - `http` for HTTP request examples

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 02:58:22 -05:00
Andrew BeveridgeandClaude Opus 4.5 986a5e499f docs: Restructure documentation for LLM agent clarity
## Summary
Complete documentation reorganization to make it more effective for AI agents working on this codebase.

## Changes Made
- Consolidated 88 docs into 5 active files + archive
- New structure: README.md, ARCHITECTURE.md, DEVELOPMENT.md, API.md, LESSONS-LEARNED.md
- Added Claude Code slash commands: /docs-review (pre-merge), /docs-maintain (periodic)
- Updated CLAUDE.md with documentation maintenance rules
- Moved all historical docs to docs/archive/ with YYYY-MM-DD prefixes

## New Documentation Structure
```
docs/
├── README.md          # Current status + navigation
├── ARCHITECTURE.md    # System design
├── DEVELOPMENT.md     # Dev setup, testing, deployment
├── API.md             # Backend API reference
├── LESSONS-LEARNED.md # Key insights for future agents
└── archive/           # 93 historical docs with date prefixes
```

## Testing
- All 57 backend tests pass
- No code changes, docs-only reorganization

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 02:43:02 -05:00
Andrew Beveridge ☄️ b16f57c267 Merge pull request #86 from nomadkaraoke/fix/consolidate-instrumental-ui
Consolidate instrumental review UI to single source
2025-12-28 01:44:06 -05:00
Andrew Beveridge ☄️ 22a5b119ec Merge pull request #87 from nomadkaraoke/fix/ci-skip-disk-cleanup
ci: skip disk cleanup when backend deployment not needed
2025-12-28 01:43:15 -05:00
Andrew BeveridgeandClaude Opus 4.5 5600df6783 ci: skip disk cleanup when backend deployment not needed
Move version check before the "Free disk space" step and add a
conditional so it only runs when deployment is actually needed.
This saves ~3+ minutes on pushes to main when the version tag
already exists.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 01:42:11 -05:00
Andrew BeveridgeandClaude Opus 4.5 9bed9839b8 Fix seekTo undefined audio variable and use absolute logo URL
- Add missing 'const audio = document.getElementById' in seekTo()
- Use absolute URL for logo with onerror fallback for CLI mode

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 01:35:32 -05:00
Andrew Beveridge ede3c5eca0 Merge main into fix/consolidate-instrumental-ui 2025-12-28 01:29:30 -05:00
Andrew BeveridgeandClaude Opus 4.5 5ecc4bb1ae Consolidate instrumental review UI to single source
Remove duplicate instrumental UI from frontend/public/ and apply PR #85
fixes to the canonical source in karaoke_gen/instrumental_review/static/.

This ensures both CLI and cloud versions use the same code:
- CLI: Served by FastAPI InstrumentalReviewServer
- Cloud: CI copies from karaoke_gen/ to GitHub Pages

Fixes applied to canonical source:
- Use duration_seconds instead of duration from waveform API
- Add NaN guards to formatTime(), seekTo(), and click/drag handlers
- Replace microphone emoji with Nomad Karaoke logo

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 01:26:38 -05:00
Andrew Beveridge ☄️ 10cb868481 Merge pull request #80 from nomadkaraoke/feature/local-whisper-transcriber
Add local Whisper transcription support
2025-12-28 01:24:18 -05:00
Andrew Beveridge 02a64faecc Updated deps 2025-12-28 01:07:25 -05:00
Andrew BeveridgeandClaude Opus 4.5 ba4ec72040 Merge main into feature/local-whisper-transcriber
Resolved version conflict by keeping 0.76.24 from feature branch.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 01:07:06 -05:00
Andrew Beveridge ☄️ ae2846b90b Merge pull request #85 from nomadkaraoke/fix/instrumental-ui-duration-nan
Fix instrumental UI NaN duration causing seek to fail
2025-12-28 01:05:27 -05:00
Andrew BeveridgeandClaude Opus 4.5 8c0eab7d7b Increase logo height to 40px
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 01:04:13 -05:00
Andrew BeveridgeandClaude Opus 4.5 0f51f595a3 Replace microphone emoji with Nomad Karaoke logo in instrumental UI
- Added logo-img CSS class for proper image sizing
- Updated header to use the SVG logo instead of emoji

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 01:02:18 -05:00
Andrew BeveridgeandClaude Opus 4.5 98bb30114e Fix instrumental UI NaN duration causing seek to fail
The waveform API returns `duration_seconds` but the frontend was reading
`waveformData.duration` which is undefined, causing duration to be NaN.

Fixes:
- Read `duration_seconds` from API response instead of `duration`
- Add guard in formatTime() to return '0:00' for NaN/Infinity
- Add guard in seekTo() to prevent setting audio.currentTime to NaN
- Add guards in waveform click handlers to prevent NaN calculations

This fixes the 'NaN:NaN' display and the TypeError when clicking to seek.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 01:01:26 -05:00
Andrew Beveridge ☄️ e88110a25c Merge pull request #84 from nomadkaraoke/fix/instrumental-ui-auth
Fix instrumental UI authentication token not being passed in URL
2025-12-28 00:48:34 -05:00
Andrew Beveridge 12a62db7f8 Remove test files and add to gitignore 2025-12-28 00:48:13 -05:00
Andrew Beveridge ☄️ c4a0cb2a63 Merge pull request #83 from nomadkaraoke/fix/docker-cache-ignore-error
Fix Docker registry cache error on first build
2025-12-28 00:47:45 -05:00
Andrew BeveridgeandClaude Opus 4.5 b0a16a90a4 Fix OutputPaths attribute access (dataclass not dict)
OutputPaths is a dataclass, not a dictionary. Changed .get() calls
to proper attribute access (.video, .ass).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 00:46:54 -05:00
Andrew BeveridgeandClaude Opus 4.5 2f6cd7a745 Fix Docker registry cache error and optimize CI workflow
Changes:
1. Add ignore-error=true to cache-from/cache-to so builds continue
   gracefully when cache doesn't exist (first build)

2. Add aggressive disk space cleanup at start of deploy-backend using
   jlumbroso/free-disk-space action (fixes "No space left on device")

3. Skip tests on main branch - tests only run on PRs now
   - PRs run full test suite with path filtering
   - Main branch only runs build/deploy jobs
   - CI Gate only required for PRs, not main

This significantly speeds up deployments on main since tests
already passed on the PR before merge.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 00:46:02 -05:00
Andrew BeveridgeandClaude Opus 4.5 1da1a17235 Fix instrumental UI authentication token not being passed in URL
The instrumental review UI was getting 401 errors because the frontend
JobCard component was looking for the token in job.state_data.instrumental_token
when it's actually stored at job.instrumental_token (same pattern as review_token).

Changes:
- Add instrumental_token field to Job interface in api.ts
- Fix JobCard.tsx getInstrumentalUrl() to use job.instrumental_token
- Update test to use correct token location

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 00:40:44 -05:00
Andrew Beveridge daa6807b39 Remove accidentally committed test audio files 2025-12-28 00:36:34 -05:00
Andrew BeveridgeandClaude Opus 4.5 06c8587e99 Fix missing lyrics_results argument in generate_outputs call
The OutputGenerator.generate_outputs() method requires a lyrics_results
argument. Since we're at the video rendering stage after loading
corrections from JSON, we pass an empty dict as the lyrics were already
written during the transcription phase.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 00:35:24 -05:00
Andrew Beveridge 81c7d1cc7f Merge main into feature/local-whisper-transcriber 2025-12-28 00:31:56 -05:00
Andrew BeveridgeandClaude Opus 4.5 b2c8608e0c Use default styles when no output_styles_json provided
When rendering video or CDG without a styles file, the OutputGenerator
now uses sensible default karaoke styles instead of raising an error.
This allows end-to-end karaoke generation to work without requiring a
styles JSON file.

Changes:
- Add _get_default_styles() method with embedded default karaoke styles
- Modify init to use defaults when styles file is missing/None
- Update test to verify default styles are loaded correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 00:29:18 -05:00
Andrew Beveridge ☄️ d6f22f5494 Merge pull request #82 from nomadkaraoke/feature/github-actions-docker-builds
Replace Cloud Build with GitHub Actions for Docker builds
2025-12-28 00:25:11 -05:00
Andrew BeveridgeandClaude Opus 4.5 ea1c659828 Replace Cloud Build with GitHub Actions for Docker builds
Cost optimization: Migrate from expensive Cloud Build ($100+/month) to
free GitHub Actions with registry caching.

Changes:
- Add .dockerignore to reduce build context (~1GB → ~10MB)
- Rewrite deploy-backend job to use docker/build-push-action
- Use Artifact Registry as persistent layer cache (mode=max)
- Keep cloudbuild*.yaml as fallback for manual builds
- Add documentation for new build process

Benefits:
- $0 monthly cost (public repo, free GitHub Actions)
- Faster builds with registry-cached layers
- Simpler deployment pipeline (all in one workflow)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 00:22:39 -05:00
Andrew BeveridgeandClaude Opus 4.5 8bd8f3343f Fix pip extras for local-whisper installation
Poetry extras require the optional dependency to be in the main
[tool.poetry.dependencies] section with `optional = true`, not just
in a separate group. The group syntax is only for Poetry's own
`poetry install --with` command.

Now `pip install karaoke-gen[local-whisper]` correctly installs
whisper-timestamped.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 00:19:19 -05:00
Andrew BeveridgeandClaude Opus 4.5 ce07080fc7 Add disk space cleanup to backend-unit-tests CI job
Prevents transient "No space left on device" failures on GitHub Actions
runners by freeing up disk space before running tests.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 00:13:13 -05:00
Andrew BeveridgeandClaude Opus 4.5 eceab9f422 Fix LocalWhisper integration in lyrics_processor.py
- Add LocalWhisper check to _check_transcription_providers()
- Update error message to include LocalWhisper as option 3
- Pass runpod_api_key, whisper_runpod_id, and enable_local_whisper
  to TranscriberConfig

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 00:04:31 -05:00
Andrew Beveridge ☄️ e4223634d1 Merge pull request #81 from nomadkaraoke/fix/remove-instrumental-selector-export
Fix stale exports in job/index.ts and add frontend build CI check
2025-12-27 23:52:42 -05:00
Andrew BeveridgeandClaude Opus 4.5 6586ea3584 Update poetry.lock for local-whisper extras
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 23:52:23 -05:00
Andrew BeveridgeandClaude Opus 4.5 b7c1850a3b Fix stale exports in job/index.ts and add frontend build CI check
The previous PR removed InstrumentalSelector.tsx but didn't update the
barrel export file, breaking the production build.

Changes:
- Remove stale InstrumentalSelector and StatusBadge exports from index.ts
- Add frontend-build job to CI workflow to catch build failures on PRs
- Add frontend-build to ci-gate and deploy-frontend dependencies

This ensures future PRs will fail CI if they break the frontend build.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 23:51:42 -05:00
Andrew BeveridgeandClaude Opus 4.5 e4dbd7393e Add local Whisper transcription support
Implements LocalWhisperTranscriber for running Whisper inference locally
using the whisper-timestamped library, addressing issue #68.

Changes:
- Add LocalWhisperTranscriber class with lazy model loading
- Support auto-detection of device (CPU/CUDA/MPS) with fallback
- Add configuration via environment variables (WHISPER_MODEL_SIZE,
  WHISPER_DEVICE, WHISPER_CACHE_DIR, WHISPER_LANGUAGE)
- Integrate into controller with priority 3 (fallback after cloud services)
- Add whisper-timestamped as optional dependency [local-whisper]
- Add comprehensive unit tests
- Update README with setup instructions and troubleshooting guide

Installation:
  pip install karaoke-gen[local-whisper]

For CPU-only systems:
  pip install torch torchaudio --index-url https://download.pytorch.org/whl/cpu
  pip install karaoke-gen[local-whisper]

Fixes #68

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 23:40:34 -05:00
Andrew Beveridge ☄️ 7736a4a8f7 Merge pull request #79 from nomadkaraoke/feature/instrumental-review-ui
Replace primitive instrumental selector with full-featured review UI
2025-12-27 23:36:45 -05:00
Andrew BeveridgeandClaude Opus 4.5 156b83ace1 Replace primitive instrumental selector with full-featured review UI
The web frontend now opens the same rich instrumental review UI that the
CLI uses. This provides waveform visualization, audio playback for
different stems, mute region selection, and custom instrumental creation.

Changes:
- Add instrumental review static HTML to frontend/public/instrumental/
- Update JobCard to open /instrumental/ page instead of showing dialog
- Remove InstrumentalSelector.tsx primitive dialog component
- Remove orphaned instrumental-review components (never used)
- Update tests to reflect link-based navigation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 23:34:20 -05:00
Andrew Beveridge ☄️ 8978eb6d96 Merge pull request #77 from nomadkaraoke/fix/flacfetch-youtube-url
Update flacfetch to 0.15.1 (fixes YouTube URL download)
2025-12-27 23:10:12 -05:00
Andrew Beveridge ☄️ c4143947ca Merge pull request #78 from nomadkaraoke/feature/mandatory-worktrees
Make git worktrees mandatory for all changes
2025-12-27 23:09:07 -05:00
Andrew BeveridgeandClaude Opus 4.5 8a28c95786 Simplify flacfetch workflow: no PR required
flacfetch changes can be committed directly to main once tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 23:07:36 -05:00
Andrew BeveridgeandClaude Opus 4.5 48acb6c4e1 Add flacfetch as a related project with workflow docs
Documents that flacfetch is a dependency we own, located at
/Users/andrew/Projects/flacfetch, and provides the workflow for
making changes to it and updating karaoke-gen's dependency.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 23:07:05 -05:00
Andrew BeveridgeandClaude Opus 4.5 da60495144 Clarify version bump rule applies only to code changes
Documentation-only changes (CLAUDE.md, README.md, etc.) don't require
version bumps since they don't affect deployed code.

Also reverts the unnecessary version bump from the previous commit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 23:04:39 -05:00
Andrew BeveridgeandClaude Opus 4.5 e282f40758 Make git worktrees mandatory for all changes
Since all changes must go through a PR, worktrees are now required for
all changes (not just non-trivial ones). This keeps the main directory
clean on main and enables parallel work.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 23:03:54 -05:00
Andrew BeveridgeandClaude Opus 4.5 aaeda8679a Update flacfetch to 0.15.1 (fixes YouTube URL download)
Fixes the "Release.__init__() missing 1 required positional argument: 'quality'"
error that occurred when downloading from YouTube URLs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 22:59:31 -05:00
Andrew Beveridge ☄️ 6da6656c15 Merge pull request #76 from nomadkaraoke/fix/remove-deployment-tracking
Remove deployment tracking to fix PR banner
2025-12-27 22:50:44 -05:00
Andrew BeveridgeandClaude Opus 4.5 d81591ca99 Remove deployment tracking to fix PR banner
Remove the `environment` block from the deploy-frontend job to prevent
GitHub from creating deployment entries. This fixes the "This branch
has not been deployed" banner that appears on all PRs.

The deployment to GitHub Pages still works via actions/deploy-pages@v4,
but without tracked environment deployments.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 22:49:44 -05:00
Andrew Beveridge ☄️ 0e66ac84ec Merge pull request #74 from nomadkaraoke/fix/youtube-url-support
Fix YouTube URL support in karaoke-gen CLI
2025-12-27 22:48:12 -05:00
Andrew BeveridgeandClaude Opus 4.5 d8e3542850 Address CodeRabbit lint feedback
- Remove f-string prefix from strings without placeholders (F541)
- Add 'from e' to ImportError handler to preserve stack trace (B904)
- Use raw string for regex pattern in pytest match (RUF043)
- Remove unused result variables (F841)
- Remove redundant KaraokePrep import in fixture (F811)
- Rename unused mock parameters with underscore prefix (ARG002)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 22:36:22 -05:00
Andrew Beveridge ☄️ 8b66e97459 Merge pull request #75 from nomadkaraoke/fix/progress-display-during-parallel-processing
Fix progress display to show step 4 when workers are active during download
2025-12-27 22:34:53 -05:00
Andrew Beveridge 8dc0a96734 Added CI gate check 2025-12-27 22:24:52 -05:00
138 changed files with 3752 additions and 1490 deletions

No files matched your search

+74
View File
@@ -0,0 +1,74 @@
---
description: Periodic documentation maintenance and organization check
allowed-tools: Read, Glob, Grep, Edit, Write, Bash
---
# Documentation Maintenance Check
You are performing periodic maintenance on the project documentation.
## Checks to Perform
### 1. Structure Compliance
Verify docs/ follows the correct structure:
```
docs/
├── README.md # Current status + navigation
├── ARCHITECTURE.md # System design
├── DEVELOPMENT.md # Dev setup, testing, deployment
├── API.md # Backend API reference
├── LESSONS-LEARNED.md # Accumulated wisdom
└── archive/ # Historical docs (YYYY-MM-DD-topic.md)
```
Report any files that don't belong or are misplaced.
### 2. Freshness Check
- Read `docs/README.md` - is the status section current?
- Check recent git commits - do they suggest docs should be updated?
- Look for stale information that contradicts current code
### 3. Archive Organization
- Verify all archive files have YYYY-MM-DD prefix
- Check for any docs that should be archived (completed work, old plans)
- Remove truly obsolete content (empty files, duplicate info)
### 4. Cross-Reference Check
- Verify CLAUDE.md points to current doc locations
- Check that docs reference each other correctly
- Ensure no broken internal links
### 5. Content Quality
- Flag any docs over 500 lines (may need splitting)
- Identify duplicate information across docs
- Note sections that seem outdated
## Actions to Take
1. **Fix structural issues** - Move/rename misplaced files
2. **Update stale content** - Refresh outdated information
3. **Archive completed work** - Move old docs to archive/
4. **Report findings** - Summarize what was found and fixed
## Output Format
Provide a maintenance report:
```
## Documentation Maintenance Report - YYYY-MM-DD
### Issues Found
- [list of issues]
### Actions Taken
- [list of fixes made]
### Recommendations
- [suggestions for future attention]
### Status
- [ ] Structure: OK/Fixed
- [ ] Freshness: OK/Updated
- [ ] Archive: OK/Organized
- [ ] Cross-refs: OK/Fixed
- [ ] Quality: OK/Flagged
```
+43
View File
@@ -0,0 +1,43 @@
---
description: Review current session for documentation updates before merging PR
allowed-tools: Read, Glob, Grep, Edit, Write
---
# Pre-Merge Documentation Review
You are reviewing this chat session to identify documentation updates needed before merging the current PR.
## Your Task
1. **Analyze the current session** - What work was done? What was learned? What changed?
2. **Check if docs need updates:**
- Does `docs/README.md` status section need updating?
- Should anything be added to `docs/LESSONS-LEARNED.md`?
- Does `docs/ARCHITECTURE.md` need changes (if architecture changed)?
- Does `docs/DEVELOPMENT.md` need changes (if dev workflow changed)?
- Does `docs/API.md` need changes (if API changed)?
3. **For significant completed work:**
- Create `docs/archive/YYYY-MM-DD-topic.md` summarizing the work
- Use today's date and a descriptive topic name
4. **Update docs/README.md** with:
- Current status if it changed
- Any new known issues discovered
- Link to new archive doc if created
## Guidelines
- Keep updates concise and factual
- Focus on information useful for future AI agents
- Don't duplicate information that's in the code
- Archive docs should capture "what was done and why" not implementation details
## Output
After reviewing, either:
- Make the necessary documentation updates
- Report "No documentation updates needed" with brief explanation why
Remember: Good documentation helps the next agent continue where you left off.
+83
View File
@@ -0,0 +1,83 @@
# .dockerignore - Reduce Docker build context size
# This is critical for fast builds as it reduces upload time and cache invalidation
# Git
.git/
.gitignore
.gitmodules
# Node.js / Frontend (not needed for backend image)
node_modules/
frontend/
frontend-react/
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# Python virtual environments (use Docker's Python instead)
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
.venv/
venv/
backend/venv/
ENV/
env/
*.egg-info/
.eggs/
dist/
build/
*.egg
# IDE
.idea/
.vscode/
*.swp
*.swo
*~
.DS_Store
# Testing (not needed in production image)
.pytest_cache/
.coverage
htmlcov/
.tox/
.nox/
coverage.xml
*.cover
tests/
backend/tests/
# Documentation (not needed in Docker image)
docs/
*.md
!README.md
# Infrastructure (not needed for Docker build)
infrastructure/
# CI/CD configs
.github/
# Cloud Build configs (keep for reference but not in image)
cloudbuild*.yaml
# Local development files
lyrics_transcriber_local/
*.local
.env
.env.*
*.bak
*.tmp
Thumbs.db
# Large test/sample files
tests/data/
*.mp3
*.wav
*.mp4
*.mkv
*.flac
+221 -168
View File
@@ -18,11 +18,13 @@ concurrency:
jobs:
# ============================================
# PATH FILTERING - Determine what changed
# Only needed for PRs (main branch skips tests)
# ============================================
changes:
name: "Detect Changes"
runs-on: ubuntu-latest
if: github.ref != 'refs/heads/main'
outputs:
frontend: ${{ steps.filter.outputs.frontend }}
backend: ${{ steps.filter.outputs.backend }}
@@ -61,14 +63,14 @@ jobs:
echo "| Dependencies | ${{ steps.filter.outputs.deps }} |" >> $GITHUB_STEP_SUMMARY
# ============================================
# FRONTEND TESTS
# FRONTEND TESTS (PRs only - main skips tests)
# ============================================
frontend-unit-tests:
name: "Frontend - Unit Tests"
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.frontend == 'true' || github.ref == 'refs/heads/main' }}
if: github.ref != 'refs/heads/main' && needs.changes.outputs.frontend == 'true'
steps:
- uses: actions/checkout@v4
@@ -95,15 +97,40 @@ jobs:
path: frontend/coverage/
retention-days: 7
frontend-build:
name: "Frontend - Build"
runs-on: ubuntu-latest
needs: changes
if: github.ref != 'refs/heads/main' && needs.changes.outputs.frontend == 'true'
steps:
- uses: actions/checkout@v4
- name: Setup Node.js with caching
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: frontend/package-lock.json
- name: Install dependencies
working-directory: frontend
run: npm ci --legacy-peer-deps
- name: Build frontend
working-directory: frontend
env:
NEXT_PUBLIC_API_URL: https://api.nomadkaraoke.com
run: npm run build
# ============================================
# PACKAGE TESTS (karaoke-gen Python CLI)
# PACKAGE TESTS (PRs only - main skips tests)
# ============================================
package-unit-tests:
name: "Package - Unit Tests (70% coverage)"
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main' }}
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
steps:
- uses: actions/checkout@v4
with:
@@ -161,7 +188,7 @@ jobs:
name: "Package - Integration Tests"
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main' }}
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
steps:
- uses: actions/checkout@v4
with:
@@ -206,11 +233,15 @@ jobs:
- name: Run integration tests
run: poetry run pytest tests/integration/ -v
package-build:
name: "Package - Build & Install"
runs-on: ubuntu-latest
package-build-test:
name: "Package - Build & Install (Test)"
runs-on: [self-hosted, linux, gcp]
needs: [changes, package-unit-tests, package-integration-tests]
if: ${{ always() && (needs.package-unit-tests.result == 'success' || needs.package-unit-tests.result == 'skipped') && (needs.package-integration-tests.result == 'success' || needs.package-integration-tests.result == 'skipped') && (needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main') }}
if: >-
github.ref != 'refs/heads/main' && always() &&
(needs.package-unit-tests.result == 'success' || needs.package-unit-tests.result == 'skipped') &&
(needs.package-integration-tests.result == 'success' || needs.package-integration-tests.result == 'skipped') &&
(needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
steps:
- uses: actions/checkout@v4
with:
@@ -227,49 +258,23 @@ jobs:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Cache Poetry dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-package-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
restore-keys: |
venv-package-${{ runner.os }}-py3.13-
- name: Build package
run: poetry build
- name: Free disk space
uses: jlumbroso/free-disk-space@main
with:
tool-cache: false
android: true
haskell: true
dotnet: false
large-packages: false
docker-images: false
swap-storage: false
- name: Test package installation
run: |
pip install dist/*.whl
python -m karaoke_gen.utils.gen_cli --help
- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
name: package-dist
path: dist/
retention-days: 1
# ============================================
# BACKEND TESTS (Cloud Run API service)
# BACKEND TESTS (PRs only - main skips tests)
# ============================================
backend-unit-tests:
name: "Backend - Unit Tests"
runs-on: ubuntu-latest
runs-on: [self-hosted, linux, gcp]
needs: changes
if: ${{ needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main' }}
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -287,14 +292,6 @@ jobs:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Cache Poetry dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-backend-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
restore-keys: |
venv-backend-${{ runner.os }}-py3.13-
- name: Install dependencies
run: poetry install
@@ -312,17 +309,10 @@ jobs:
backend-emulator-tests:
name: "Backend - Emulator Integration Tests"
runs-on: ubuntu-latest
runs-on: [self-hosted, linux, gcp]
needs: changes
if: ${{ needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main' }}
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
steps:
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf /usr/local/share/boost
df -h
- name: Checkout code
uses: actions/checkout@v4
with:
@@ -339,28 +329,12 @@ jobs:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Cache Poetry dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-backend-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
restore-keys: |
venv-backend-${{ runner.os }}-py3.13-
- name: Set up Java 21 (required by Firestore emulator)
uses: actions/setup-java@v4
with:
distribution: "temurin"
java-version: "21"
- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@v2
with:
version: "latest"
- name: Install Firestore emulator
run: gcloud components install beta cloud-firestore-emulator --quiet
- name: Install dependencies
run: poetry install
@@ -384,7 +358,7 @@ jobs:
name: "Backend - Code Quality"
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || github.ref == 'refs/heads/main' }}
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true')
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -419,9 +393,78 @@ jobs:
echo "✅ Basic syntax validation passed"
# ============================================
# DEPLOYMENTS (main branch only)
# CI GATE - Required status check for PRs
# Only runs on PRs, not on main
# ============================================
ci-gate:
name: "CI Gate"
runs-on: ubuntu-latest
needs: [frontend-unit-tests, frontend-build, package-unit-tests, package-integration-tests, backend-unit-tests, backend-emulator-tests, backend-lint]
if: github.ref != 'refs/heads/main' && always()
steps:
- name: Check results
run: |
results=(
"${{ needs.frontend-unit-tests.result }}"
"${{ needs.frontend-build.result }}"
"${{ needs.package-unit-tests.result }}"
"${{ needs.package-integration-tests.result }}"
"${{ needs.backend-unit-tests.result }}"
"${{ needs.backend-emulator-tests.result }}"
"${{ needs.backend-lint.result }}"
)
for result in "${results[@]}"; do
if [[ "$result" == "failure" || "$result" == "cancelled" ]]; then
echo "❌ CI failed"
exit 1
fi
done
echo "✅ CI passed"
# ============================================
# DEPLOYMENTS (main branch only)
# Tests already passed on PR, so we skip them here
# ============================================
package-build:
name: "Package - Build"
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
steps:
- uses: actions/checkout@v4
with:
submodules: true
- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Cache Poetry dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-package-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
restore-keys: |
venv-package-${{ runner.os }}-py3.13-
- name: Build package
run: poetry build
- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
name: package-dist
path: dist/
retention-days: 1
package-publish:
name: "Deploy - Publish to PyPI"
runs-on: ubuntu-latest
@@ -512,11 +555,7 @@ jobs:
deploy-frontend:
name: "Deploy - Frontend (GitHub Pages)"
runs-on: ubuntu-latest
needs: [frontend-unit-tests]
if: ${{ always() && github.ref == 'refs/heads/main' && github.event_name == 'push' && (needs.frontend-unit-tests.result == 'success' || needs.frontend-unit-tests.result == 'skipped') }}
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -587,20 +626,28 @@ jobs:
deploy-backend:
name: "Deploy - Backend (Cloud Run)"
runs-on: ubuntu-latest
needs: [backend-unit-tests, backend-emulator-tests, backend-lint]
if: ${{ always() && github.ref == 'refs/heads/main' && github.event_name == 'push' && (needs.backend-unit-tests.result == 'success' || needs.backend-unit-tests.result == 'skipped') && (needs.backend-emulator-tests.result == 'success' || needs.backend-emulator-tests.result == 'skipped') && (needs.backend-lint.result == 'success' || needs.backend-lint.result == 'skipped') }}
runs-on: [self-hosted, linux, gcp]
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
permissions:
contents: read
id-token: write
env:
REGISTRY: us-central1-docker.pkg.dev
PROJECT_ID: nomadkaraoke
REPOSITORY: karaoke-repo
BASE_IMAGE_NAME: karaoke-backend-base
APP_IMAGE_NAME: karaoke-backend
steps:
# ============================================
# Check if deployment is needed FIRST
# ============================================
- name: Checkout code
uses: actions/checkout@v4
with:
submodules: false
fetch-depth: 0 # Need full history for tag check
- name: Set up Python (for version extraction)
- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: "3.13"
@@ -624,6 +671,8 @@ jobs:
echo "✨ Tag v$VERSION does not exist - proceeding with deployment"
fi
# No disk cleanup needed - self-hosted runner has 200GB SSD
- name: Authenticate to Google Cloud (Workload Identity)
if: steps.version_check.outputs.needs_deploy == 'true'
id: auth
@@ -638,21 +687,37 @@ jobs:
with:
version: "latest"
- name: Check if base image needs rebuild
- name: Configure Docker for Artifact Registry
if: steps.version_check.outputs.needs_deploy == 'true'
run: gcloud auth configure-docker ${{ env.REGISTRY }} --quiet
# ============================================
# Docker Build with BuildKit + Registry Cache
# This replaces expensive Cloud Build with free GitHub Actions
# Registry cache persists between builds for fast rebuilds
# ============================================
- name: Set up Docker Buildx
if: steps.version_check.outputs.needs_deploy == 'true'
uses: docker/setup-buildx-action@v3
- name: Calculate base image hash
if: steps.version_check.outputs.needs_deploy == 'true'
id: base-hash
run: |
# Hash both poetry.lock AND Dockerfile.base - rebuild if either changes
CURRENT_HASH=$(cat poetry.lock backend/Dockerfile.base | sha256sum | cut -d' ' -f1)
echo "hash=$CURRENT_HASH" >> $GITHUB_OUTPUT
echo "📋 Base content hash: $CURRENT_HASH"
- name: Check if base image exists and is current
if: steps.version_check.outputs.needs_deploy == 'true'
id: check-base
run: |
echo "🔍 Checking if base image needs rebuild..."
BASE_IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest"
# Hash both poetry.lock AND Dockerfile.base - rebuild if either changes
CURRENT_HASH=$(cat poetry.lock backend/Dockerfile.base | sha256sum | cut -d' ' -f1)
echo "Current combined hash (poetry.lock + Dockerfile.base): $CURRENT_HASH"
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
BASE_IMAGE="us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-base:latest"
# Try to get the existing base image's hash label
EXISTING_HASH=""
if docker pull $BASE_IMAGE 2>/dev/null; then
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "base.content.hash"}}' 2>/dev/null || echo "")
# Fallback to old label name for backwards compatibility
@@ -661,10 +726,10 @@ jobs:
fi
echo "Existing base image hash: $EXISTING_HASH"
else
echo "⚠️ No existing base image found"
echo "⚠️ No existing base image found - will build"
fi
if [ "$CURRENT_HASH" != "$EXISTING_HASH" ]; then
if [ "${{ steps.base-hash.outputs.hash }}" != "$EXISTING_HASH" ]; then
echo "🔄 Base image needs rebuild (poetry.lock or Dockerfile.base changed)"
echo "needs_rebuild=true" >> $GITHUB_OUTPUT
else
@@ -672,88 +737,73 @@ jobs:
echo "needs_rebuild=false" >> $GITHUB_OUTPUT
fi
- name: Rebuild base image (if needed)
- name: Build and push base image (if needed)
if: steps.version_check.outputs.needs_deploy == 'true' && steps.check-base.outputs.needs_rebuild == 'true'
run: |
echo "🏗️ Rebuilding base image with updated dependencies..."
uses: docker/build-push-action@v6
with:
context: .
file: backend/Dockerfile.base
push: true
tags: |
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:${{ github.sha }}
# Registry cache for ALL layers (mode=max) - persists between builds
# ignore-error=true allows build to continue if cache doesn't exist yet
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:cache,ignore-error=true
cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:cache,mode=max,ignore-error=true
build-args: |
BUILD_DATE=${{ github.run_id }}
BUILD_ID=${{ github.sha }}
BASE_CONTENT_HASH=${{ steps.base-hash.outputs.hash }}
# Provenance disabled for smaller image and faster push
provenance: false
BASE_BUILD_ID=$(gcloud builds submit \
--config=cloudbuild-base.yaml \
--project=nomadkaraoke \
--format='value(id)' \
--async \
.)
- name: Build and push app image
if: steps.version_check.outputs.needs_deploy == 'true'
uses: docker/build-push-action@v6
with:
context: .
file: backend/Dockerfile
push: true
tags: |
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:latest
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:${{ github.sha }}
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:v${{ steps.version_check.outputs.version }}
# Registry cache for app layers
# ignore-error=true allows build to continue if cache doesn't exist yet
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:cache,ignore-error=true
cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:cache,mode=max,ignore-error=true
provenance: false
echo "Base Build ID: $BASE_BUILD_ID"
echo "Build URL: https://console.cloud.google.com/cloud-build/builds/$BASE_BUILD_ID?project=nomadkaraoke"
# ============================================
# Deploy to Cloud Run
# ============================================
echo "Waiting for base image build to complete..."
while true; do
BUILD_STATUS=$(gcloud builds describe $BASE_BUILD_ID --project=nomadkaraoke --format='value(status)')
echo "Base build status: $BUILD_STATUS"
if [ "$BUILD_STATUS" = "SUCCESS" ]; then
echo "✅ Base image build completed successfully"
break
elif [ "$BUILD_STATUS" = "FAILURE" ] || [ "$BUILD_STATUS" = "CANCELLED" ] || [ "$BUILD_STATUS" = "TIMEOUT" ]; then
echo "❌ Base image build failed with status: $BUILD_STATUS"
echo ""
echo "==================== BUILD LOGS ===================="
gcloud logging read "resource.type=build AND resource.labels.build_id=$BASE_BUILD_ID" \
--project=nomadkaraoke \
--format="value(textPayload)" \
--limit=200 \
--order=asc || echo "Failed to retrieve logs from Cloud Logging"
echo "===================================================="
gcloud builds log $BASE_BUILD_ID --project=nomadkaraoke 2>&1 || true
exit 1
fi
sleep 30
done
- name: Trigger Cloud Build
- name: Deploy to Cloud Run
if: steps.version_check.outputs.needs_deploy == 'true'
run: |
echo "🚀 Triggering Cloud Build for deployment..."
echo "Commit: ${{ github.sha }}"
echo "Version: ${{ steps.version_check.outputs.version }}"
VERSION="${{ steps.version_check.outputs.version }}"
IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:${{ github.sha }}"
BUILD_ID=$(gcloud builds submit \
--config=cloudbuild.yaml \
--project=nomadkaraoke \
--format='value(id)' \
--async \
.)
echo "🚀 Deploying to Cloud Run..."
echo "Image: $IMAGE"
echo "Version: $VERSION"
echo "Build ID: $BUILD_ID"
echo "Build URL: https://console.cloud.google.com/cloud-build/builds/$BUILD_ID?project=nomadkaraoke"
echo "Waiting for build to complete..."
while true; do
BUILD_STATUS=$(gcloud builds describe $BUILD_ID --project=nomadkaraoke --format='value(status)')
echo "Build status: $BUILD_STATUS"
if [ "$BUILD_STATUS" = "SUCCESS" ]; then
echo "✅ Build completed successfully"
break
elif [ "$BUILD_STATUS" = "FAILURE" ] || [ "$BUILD_STATUS" = "CANCELLED" ] || [ "$BUILD_STATUS" = "TIMEOUT" ]; then
echo "❌ Build failed with status: $BUILD_STATUS"
echo ""
echo "==================== BUILD LOGS ===================="
gcloud logging read "resource.type=build AND resource.labels.build_id=$BUILD_ID" \
--project=nomadkaraoke \
--format="value(textPayload)" \
--limit=200 \
--order=asc || echo "Failed to retrieve logs from Cloud Logging"
echo "===================================================="
echo ""
gcloud builds log $BUILD_ID --project=nomadkaraoke 2>&1 || true
exit 1
fi
sleep 30
done
gcloud run deploy karaoke-backend \
--image $IMAGE \
--region us-central1 \
--platform managed \
--allow-unauthenticated \
--memory 16Gi \
--cpu 8 \
--cpu-boost \
--execution-environment gen2 \
--timeout 1800 \
--concurrency 1 \
--max-instances 50 \
--min-instances 0 \
--set-env-vars "GOOGLE_CLOUD_PROJECT=${{ env.PROJECT_ID }},GCS_BUCKET_NAME=karaoke-gen-storage-${{ env.PROJECT_ID }},FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1,DEFAULT_ENABLE_YOUTUBE_UPLOAD=true,DEFAULT_BRAND_PREFIX=NOMAD" \
--set-secrets "AUDIOSHAKE_API_TOKEN=audioshake-api-key:latest,GENIUS_API_TOKEN=genius-api-key:latest,SPOTIFY_COOKIE_SP_DC=spotify-cookie:latest,RAPIDAPI_KEY=rapidapi-key:latest,DEFAULT_DISCORD_WEBHOOK_URL=discord-releases-webhook:latest,ADMIN_TOKENS=admin-tokens:latest,RED_API_KEY=red-api-key:latest,RED_API_URL=red-api-url:latest,OPS_API_KEY=ops-api-key:latest,OPS_API_URL=ops-api-url:latest,FLACFETCH_API_URL=flacfetch-api-url:latest,FLACFETCH_API_KEY=flacfetch-api-key:latest"
- name: Verify deployment and version
if: steps.version_check.outputs.needs_deploy == 'true'
@@ -804,6 +854,9 @@ jobs:
echo "Commit: ${{ github.sha }}"
echo "Branch: ${{ github.ref_name }}"
echo "Run: ${{ github.run_number }}"
echo ""
echo "📊 Built on self-hosted GCP runner (200GB SSD, no disk space issues)"
echo " Runner: gcp-runner with 4 vCPU, 16GB RAM"
- name: Skipped deployment summary
if: steps.version_check.outputs.needs_deploy == 'false'
+3
View File
@@ -227,3 +227,6 @@ cython_debug/
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
flacfetch
# Test output directories
Rick Astley*/
+73 -107
View File
@@ -1,131 +1,97 @@
# Project Guidelines for AI Assistants
# Karaoke-Gen AI Assistant Guidelines
This document contains guidelines and rules for AI assistants working on this codebase.
## Project Overview
## Core Principles
**Karaoke video generation platform** - CLI tool and web service that creates professional karaoke videos with synchronized lyrics.
**Always be testing.** Proactively evaluate, read, run, fix, and improve tests whenever you touch any part of the system—frontend, backend, infrastructure. Every code change should be thoroughly tested. Leave everything better tested than you found it.
- **Production**: <https://gen.nomadkaraoke.com> (frontend), <https://api.nomadkaraoke.com> (backend)
- **CLI**: `karaoke-gen` (local), `karaoke-gen-remote` (cloud)
- **Repo**: <https://github.com/nomadkaraoke/karaoke-gen>
**Always be documenting.** Continuously read, write, improve, update, and organize markdown documentation. Reference existing docs and keep them accurate as the codebase evolves.
## Quick Reference
**Build for maintainability.** Keep code SOLID, modular, and clean. Favor simplicity and clarity. Future developers (and AI agents) will thank you.
| What | Where |
|------|-------|
| Current status | `docs/README.md` |
| Architecture | `docs/ARCHITECTURE.md` |
| Dev setup & testing | `docs/DEVELOPMENT.md` |
| API reference | `docs/API.md` |
| Past learnings | `docs/LESSONS-LEARNED.md` |
## Branching Workflow
## Tech Stack
**Never commit directly to `main`.** Always work on a feature branch.
- **Backend**: FastAPI on Cloud Run, Firestore, GCS, Secret Manager
- **Frontend**: Next.js on Cloudflare Pages
- **Processing**: Modal (GPU audio separation), AudioShake/Whisper (transcription)
- **Infra**: Pulumi IaC, GitHub Actions CI/CD
Create descriptive branch names like `feature/add-discord-notifications`, `fix/audio-sync-issue`, or `refactor/cleanup-api-routes`.
## Essential Rules
### Using Git Worktrees
**For non-trivial changes** (new features, multi-file edits, refactors):
- Create a git worktree in the parent folder before making any file changes:
### Git Workflow
- **Never commit directly to `main`** - always use feature branches
- **Use git worktrees** for all changes:
```bash
git worktree add ../karaoke-gen-feat-xyz feature/xyz
git worktree add -b feature/xyz ../karaoke-gen-feat-xyz main
```
- This keeps the main directory clean on `main` and enables parallel work on multiple features
- Name the worktree folder descriptively, e.g., `../karaoke-gen-fix-audio-sync`
**For trivial changes** (typos, single-line fixes):
- Creating a branch in the current directory is acceptable
- Use `git checkout -b fix/typo-xyz` before making changes
**Cleanup:** When done with a worktree, remove it with:
```bash
git worktree remove ../karaoke-gen-feat-xyz
```
### Creating a Pull Request
Once all changes are complete, tests pass, and commits are pushed, create a PR with:
1. **Summary** - Brief overview of the changes
2. **Changes Made** - List of specific modifications
3. **Testing** - How changes were tested (unit tests, manual testing)
4. **Related Issues** - Reference any GitHub issues (e.g., "Fixes #123")
Only merge after review and CI checks pass.
## Infrastructure Changes
All Google Cloud infrastructure (IAM, Cloud Storage, Cloud Run, Secret Manager, Artifact Registry, etc.) must be managed via **Pulumi IaC** in the `infrastructure/` folder.
### Rules
- **Reading/querying** with `gcloud` CLI is fine for investigation and debugging
- **Temporary changes** via `gcloud` CLI or Console are acceptable for urgent fixes or experimentation, but **must be cleaned up** and properly codified in the IaC afterward
- **Permanent changes** must go through Pulumi — never leave `gcloud` or Console changes as the final state
### Workflow
1. Make changes in `infrastructure/`
2. Run `pulumi preview` and verify the output
3. Run `pulumi up` to apply
### Authentication Issues
If you encounter any authentication issues with `gcloud` or `pulumi` (e.g., expired tokens, permission errors), **stop and notify the user** so they can run the appropriate auth commands in their external terminal:
```bash
gcloud auth login
gcloud auth application-default login
```
Do not attempt to work around auth issues or make assumptions about credentials.
## Version Management
When committing and pushing changes, always bump the patch version in `pyproject.toml`.
The version is located at `tool.poetry.version` in `pyproject.toml`. For example, if the current version is `0.70.3`, bump it to `0.70.4`.
This ensures every commit pushed to the repository has a unique version number for tracking and deployment purposes.
## Testing Requirements
**Before committing and pushing ANY changes, you MUST run the full test suite.**
### Backend Test Command
- Create PR with summary, changes, testing info
- Merge only after CI passes
### Testing (Required Before Commit)
```bash
# Backend (10 min timeout, limit output)
make test 2>&1 | tail -n 500
```
**Important for AI agents:**
- Set a **10 minute timeout** when running tests (they can take a while with emulators)
- Always pipe output through `tail -n 500` to limit output and preserve context window
- The tail output will show test results and any failures at the end
This runs all backend tests in order and fails fast if any test fails:
1. Unit tests (karaoke_gen package) with coverage check
2. Backend unit tests
3. E2E integration tests with emulators (auto-starts/stops emulators)
### Frontend Test Command
When working in the `frontend/` directory, run:
```bash
# Frontend
cd frontend && npm run test:all 2>&1 | tail -n 200
```
This runs:
1. **Unit tests** (Jest) - Component and utility tests
2. **E2E tests** (Playwright) - Browser-based integration tests
### Version Bumping
- Bump `tool.poetry.version` in `pyproject.toml` for code changes
- Skip for docs-only changes
**Note:** E2E tests require `KARAOKE_ACCESS_TOKEN` environment variable to be set in `frontend/.env.local`.
### Infrastructure
- All GCP changes via Pulumi in `infrastructure/`
- `gcloud` CLI for reading/debugging only
- Stop and notify user on auth issues
Individual test commands:
- `npm run test:unit` - Run unit tests only
- `npm run test:e2e` - Run E2E tests only
- `npm run test:ci` - Run unit tests with coverage
## Documentation Maintenance
### Pre-commit Checklist
### Structure
```text
docs/
├── README.md # Current status + navigation (UPDATE THIS)
├── ARCHITECTURE.md # System design
├── DEVELOPMENT.md # Dev setup, testing, deployment
├── API.md # Backend API reference
├── LESSONS-LEARNED.md # Key insights for future agents
└── archive/ # Historical docs (YYYY-MM-DD-topic.md)
```
- [ ] `make test` passes (for backend changes)
- [ ] `cd frontend && npm run test:all` passes (for frontend changes)
- [ ] Version in `pyproject.toml` has been bumped
- [ ] `coderabbit review --plain` has been run to get comprehensive analysis and suggestions for cleaner, more maintainable code. Apply the feedback to improve accessibility, structure, and best practices.
### Rules
1. **Before merging PRs**: Run `/docs-review` to check if docs need updating
2. **Periodically**: Run `/docs-maintain` to verify docs are organized
3. **For significant work**: Create `docs/archive/YYYY-MM-DD-topic.md`
4. **Add learnings**: Update `docs/LESSONS-LEARNED.md` with insights
5. **Keep status current**: Update `docs/README.md` when project state changes
**Do not commit or push code that fails tests. Fix any test failures before proceeding.**
### What Goes Where
- **README.md**: Current status, known issues, quick links
- **ARCHITECTURE.md**: System design, data flow, tech decisions
- **DEVELOPMENT.md**: Setup, testing, deployment, CI/CD
- **API.md**: Endpoints, request/response formats
- **LESSONS-LEARNED.md**: What worked, what didn't, gotchas
- **archive/**: Completed work, session summaries, old plans
## Related Projects
### flacfetch
Location: `/Users/andrew/Projects/flacfetch`
Workflow: Make changes on `main`, bump version, push, wait for CI, then `poetry update flacfetch` in karaoke-gen.
## Pre-Commit Checklist
- [ ] Tests pass (`make test` / `npm run test:all`)
- [ ] Version bumped (if code changed)
- [ ] `/docs-review` run (check for doc updates)
- [ ] PR created with clear description
+72 -2
View File
@@ -71,8 +71,40 @@ export AUDIOSHAKE_API_TOKEN="your_audioshake_token"
Get an API key at [https://www.audioshake.ai/](https://www.audioshake.ai/) - business only, at time of writing this.
#### Option 2: Whisper via RunPod
Open-source alternative using OpenAI's Whisper model on RunPod infrastructure.
#### Option 2: Local Whisper (No Cloud Required)
Run Whisper directly on your local machine using whisper-timestamped. Works on CPU, NVIDIA GPU (CUDA), or Apple Silicon.
```bash
# Install with local Whisper support
pip install "karaoke-gen[local-whisper]"
# Optional: Configure model size (tiny, base, small, medium, large)
export WHISPER_MODEL_SIZE="medium"
# Optional: Force specific device (cpu, cuda, mps)
export WHISPER_DEVICE="cpu"
```
**Model Size Guide:**
| Model | VRAM | Speed | Quality |
|-------|------|-------|---------|
| tiny | ~1GB | Fast | Lower |
| base | ~1GB | Fast | Basic |
| small | ~2GB | Medium | Good |
| medium | ~5GB | Slower | Better |
| large | ~10GB | Slowest | Best |
**CPU-Only Installation** (no GPU required):
```bash
# Pre-install CPU-only PyTorch first
pip install torch torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install "karaoke-gen[local-whisper]"
```
Local Whisper runs automatically as a fallback when no cloud transcription services are configured.
#### Option 3: Whisper via RunPod
Cloud-based alternative using OpenAI's Whisper model on RunPod infrastructure.
```bash
export RUNPOD_API_KEY="your_runpod_key"
@@ -574,6 +606,44 @@ If the output video has quality problems:
- Check available codecs: `ffmpeg -codecs`
- For 4K output, ensure sufficient disk space (10GB+ per track)
### Local Whisper Issues
#### GPU Out of Memory
If you get CUDA out of memory errors:
```bash
# Use a smaller model
export WHISPER_MODEL_SIZE="small" # or "tiny"
# Or force CPU mode
export WHISPER_DEVICE="cpu"
```
#### Slow Transcription on CPU
CPU transcription is significantly slower than GPU. For faster processing:
- Use a smaller model (`tiny` or `base`)
- Consider using cloud transcription (AudioShake or RunPod)
- On Apple Silicon, the `small` model offers good speed/quality balance
#### Model Download Issues
Whisper models are downloaded on first use (~1-3GB depending on size). If downloads fail:
- Check your internet connection
- Set a custom cache directory: `export WHISPER_CACHE_DIR="/path/with/space"`
- Models are cached in `~/.cache/whisper/` by default
#### whisper-timestamped Not Found
If you get "whisper-timestamped is not installed":
```bash
pip install "karaoke-gen[local-whisper]"
# Or install directly:
pip install whisper-timestamped
```
#### Disabling Local Whisper
If you want to disable local Whisper (e.g., to force cloud transcription):
```bash
export ENABLE_LOCAL_WHISPER="false"
```
---
## 🧪 Development
+183
View File
@@ -0,0 +1,183 @@
# API Reference
Base URL: `https://api.nomadkaraoke.com` (production) or `http://localhost:8000` (local)
## Authentication
All endpoints except `/health` and `/api/themes` require authentication.
```bash
curl -H "Authorization: Bearer YOUR_TOKEN" ...
```
## Endpoints
### Health
```http
GET /health
```
No auth required. Returns service status.
### Jobs
#### Create Job with Upload
```http
POST /api/jobs/upload
Content-Type: multipart/form-data
file: <audio file>
artist: "Artist Name"
title: "Song Title"
```
Returns job ID. Triggers async processing.
#### Get Job Status
```http
GET /api/jobs/{job_id}
```
Response includes:
- `status` - Current job state
- `file_urls` - Signed URLs for outputs (when ready)
- `state_data` - Worker progress details
- `timeline` - Processing history
#### List Jobs
```http
GET /api/jobs
GET /api/jobs?status=complete
GET /api/jobs?limit=10&offset=0
```
#### Delete Job
```http
DELETE /api/jobs/{job_id}
```
### Review
#### Get Correction Data
```http
GET /api/review/{job_id}/correction-data
```
Returns lyrics correction data for the review UI.
#### Complete Review
```http
POST /api/review/{job_id}/complete
Content-Type: application/json
{
"corrections": [...],
"corrected_segments": [...]
}
```
Saves corrections and triggers video rendering.
#### Generate Preview
```http
POST /api/review/{job_id}/preview-video
```
Generates preview video during review.
#### Stream Audio
```http
GET /api/review/{job_id}/audio/{stem_type}
```
Streams audio for review playback.
### Instrumental Selection
```http
POST /api/jobs/{job_id}/select-instrumental
Content-Type: application/json
{
"selection": "clean" // or "with_backing"
}
```
### Audio Search
```http
GET /api/audio-search?q=song+name
```
Search for songs (flacfetch integration).
### Themes
```http
GET /api/themes
```
No auth required. Returns available video themes.
### Internal (Admin Only)
These endpoints are used by workers and require admin tokens.
```http
POST /api/internal/jobs/{job_id}/trigger-audio
POST /api/internal/jobs/{job_id}/trigger-lyrics
POST /api/internal/jobs/{job_id}/trigger-screens
POST /api/internal/jobs/{job_id}/trigger-render-video
POST /api/internal/jobs/{job_id}/trigger-video
```
## Job States
| State | Description |
|-------|-------------|
| `pending` | Created, not started |
| `downloading` | Processing input |
| `separating_stage1` | Audio separation (1/2) |
| `separating_stage2` | Audio separation (2/2) |
| `transcribing` | Lyrics transcription |
| `generating_screens` | Title/end screens |
| `awaiting_review` | Waiting for human review |
| `in_review` | Human reviewing |
| `review_complete` | Review submitted |
| `rendering_video` | Generating karaoke video |
| `awaiting_instrumental_selection` | Waiting for selection |
| `instrumental_selected` | Selection made |
| `generating_video` | Final encoding |
| `complete` | Done |
| `failed` | Error |
## Error Responses
```json
{
"detail": "Error message"
}
```
Common status codes:
- `400` - Bad request
- `401` - Unauthorized
- `404` - Job not found
- `500` - Server error
## Rate Limits
No rate limits currently implemented.
## Webhooks
Not yet implemented. Jobs must be polled for status.
+162
View File
@@ -0,0 +1,162 @@
# Architecture
## System Overview
```text
┌─────────────────────────────────────────────────────────────────┐
│ Frontend (Cloudflare Pages) │
│ Next.js + TypeScript │
│ https://gen.nomadkaraoke.com │
└───────────────────────────┬─────────────────────────────────────┘
│ HTTPS
▼
┌─────────────────────────────────────────────────────────────────┐
│ Backend API (Cloud Run) │
│ FastAPI + Python 3.12 │
│ https://api.nomadkaraoke.com │
│ │
│ Workers: │
│ • Audio Worker - Stem separation via Modal API │
│ • Lyrics Worker - Transcription via AudioShake │
│ • Screens Worker - Title/end screen generation │
│ • Render Worker - Karaoke video with lyrics │
│ • Video Worker - Final assembly and encoding │
└───────────────────────────┬─────────────────────────────────────┘
│
┌─────────────┼─────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────────┐
│ Firestore│ │ GCS │ │Secret Manager│
│ (jobs) │ │ (files) │ │ (API keys) │
└──────────┘ └──────────┘ └──────────────┘
```
## Processing Pipeline
```text
1. UPLOAD
POST /api/jobs/upload
• Save audio to GCS
• Create job in Firestore
• Trigger parallel workers
2. PARALLEL PROCESSING
┌─────────────────┐ ┌─────────────────┐
│ Audio Worker │ │ Lyrics Worker │
│ Modal API │ │ AudioShake │
│ 2-stage sep │ │ Auto-correct │
└────────┬────────┘ └────────┬────────┘
└──────────┬───────────┘
▼
3. SCREENS GENERATION
• Title screen (5s video)
• End screen (5s video)
▼
4. HUMAN REVIEW (AWAITING_REVIEW)
• React UI for lyrics correction
• Preview video generation
• Submit corrections
▼
5. VIDEO RENDERING
• Merge corrections with original data
• Generate karaoke video with lyrics
▼
6. INSTRUMENTAL SELECTION
• User chooses: clean or with_backing
▼
7. FINAL VIDEO
• Remux with selected instrumental
• Concatenate: title + karaoke + end
• Encode to 4 formats
▼
8. COMPLETE
• User downloads via signed URLs
```
## Job States
| State | Description | Human Action |
|-------|-------------|--------------|
| `pending` | Job created | - |
| `downloading` | Processing audio | - |
| `separating_stage1` | Audio separation stage 1 | - |
| `separating_stage2` | Audio separation stage 2 | - |
| `transcribing` | Lyrics transcription | - |
| `generating_screens` | Creating title/end screens | - |
| `awaiting_review` | **Waiting for human** | Review lyrics |
| `in_review` | Human actively reviewing | Edit lyrics |
| `review_complete` | Review submitted | - |
| `rendering_video` | Generating karaoke video | - |
| `awaiting_instrumental_selection` | **Waiting for human** | Choose audio |
| `instrumental_selected` | Selection made | - |
| `generating_video` | Final encoding | - |
| `complete` | All done | Download files |
| `failed` | Error occurred | - |
## GCS File Structure
```text
jobs/{job_id}/
├── input/
│ └── original.flac # Uploaded audio
├── stems/
│ ├── vocals.flac
│ ├── instrumental_clean.flac
│ ├── instrumental_with_backing.flac
│ └── ...
├── lyrics/
│ ├── corrections.json # Auto-corrected
│ ├── corrections_updated.json # After human review
│ └── karaoke.ass
├── screens/
│ ├── title.mov
│ └── end.mov
├── videos/
│ └── with_vocals.mkv # Karaoke video
└── finals/
├── lossless_4k.mp4
├── lossless_4k.mkv
├── lossy_4k.mp4
└── lossy_720p.mp4
```
## LyricsTranscriber Integration
**Key Design Decision**: We use LyricsTranscriber as a **library**, not a server.
```text
CLI Mode (Original): Cloud Mode (Our Approach):
──────────────────── ──────────────────────────
LyricsTranscriber LyricsTranscriber
├── transcribe() ├── transcribe() ✅
├── correct() ├── correct() ✅
├── ReviewServer.start() │
│ └── BLOCKS (waits) │ (Skip server, save JSON)
│ │ ... ASYNC GAP ...
│ │ (Human reviews via React)
└── OutputGenerator │
└── OutputGenerator ✅
(called after review)
```
**Use**: `LyricsTranscriber`, `CorrectionResult`, `OutputGenerator`, `OutputConfig`
**Don't use**: `ReviewServer`, blocking waits
## External Services
| Service | Purpose | Required |
|---------|---------|----------|
| Modal | GPU audio separation | Yes |
| AudioShake | Lyrics transcription | Yes |
| Genius | Reference lyrics | Yes |
| YouTube API | Video upload | Optional |
## Tech Stack
- **Backend**: FastAPI, Python 3.12, Cloud Run
- **Frontend**: Next.js, TypeScript, Cloudflare Pages
- **Database**: Firestore
- **Storage**: Google Cloud Storage
- **Secrets**: Google Secret Manager
- **IaC**: Pulumi
- **CI/CD**: GitHub Actions
+216
View File
@@ -0,0 +1,216 @@
# Development Guide
## Quick Start
### Prerequisites
- Python 3.12
- Node.js 20+
- Docker (for GCS emulator)
- Google Cloud SDK with Firestore emulator
```bash
# Install Firestore emulator
gcloud components install cloud-firestore-emulator
```
### Backend Setup
```bash
# Create and activate venv
python3.12 -m venv venv
source venv/bin/activate
# Install dependencies
poetry install
# Copy environment template
cp .env.example .env
# Edit .env with required API keys
```
### Frontend Setup
```bash
cd frontend
npm install
cp .env.example .env.local
# Edit .env.local with KARAOKE_ACCESS_TOKEN
```
## Running Locally
### Backend with Emulators
```bash
# Start emulators and backend
./scripts/run-backend-local.sh --with-emulators
# Backend runs at http://localhost:8000
# API docs at http://localhost:8000/docs
```
### Frontend
```bash
cd frontend
npm run dev
# Runs at http://localhost:3000
```
## Testing
### Backend Tests
```bash
# Run all tests (required before commit)
make test 2>&1 | tail -n 500
# This runs in order:
# 1. Unit tests with coverage
# 2. Backend unit tests
# 3. E2E integration tests with emulators
```
**Important**: Set 10 minute timeout. Tests can take a while with emulators.
### Frontend Tests
```bash
cd frontend
npm run test:all 2>&1 | tail -n 200
# Individual commands:
npm run test:unit # Jest unit tests
npm run test:e2e # Playwright E2E tests
```
### Test Architecture
```
┌─────────────────────────────────┐
│ Unit Tests (Mocked) │ Fast, isolated
│ ~62 tests, <1s │
├─────────────────────────────────┤
│ Emulator Integration Tests │ Local Firestore/GCS
│ ~11 tests, ~2s │
├─────────────────────────────────┤
│ E2E Tests │ Full API + Frontend
│ Requires deployed backend │
└─────────────────────────────────┘
```
## Deployment
### Automatic (CI/CD)
Push to `main` triggers GitHub Actions:
1. Run tests
2. Build Docker image
3. Deploy to Cloud Run
4. Deploy frontend to Cloudflare Pages
### Manual Backend Deploy
```bash
# Preview infrastructure changes
cd infrastructure
pulumi preview
# Apply changes
pulumi up
```
### Manual Frontend Deploy
```bash
cd frontend
npm run build
npx wrangler pages deploy out
```
## Environment Variables
### Backend (.env)
```bash
# Required
AUDIOSHAKE_API_TOKEN=...
GENIUS_ACCESS_TOKEN=...
AUDIO_SEPARATOR_API_URL=... # Modal API endpoint
# GCP (auto-configured in Cloud Run)
GOOGLE_CLOUD_PROJECT=...
GCS_BUCKET_NAME=...
# Auth
ADMIN_TOKENS=token1,token2
```
### Frontend (.env.local)
```bash
NEXT_PUBLIC_API_URL=http://localhost:8000
KARAOKE_ACCESS_TOKEN=... # For E2E tests
```
## Key Scripts
| Script | Purpose |
|--------|---------|
| `make test` | Run all backend tests |
| `./scripts/run-backend-local.sh` | Start backend with emulators |
| `./scripts/start-emulators.sh` | Start emulators only |
| `./scripts/stop-emulators.sh` | Stop emulators |
## Project Structure
```
karaoke-gen/
├── backend/
│ ├── api/routes/ # FastAPI endpoints
│ ├── models/ # Pydantic models
│ ├── services/ # Business logic
│ ├── workers/ # Background workers
│ └── tests/ # Backend tests
├── frontend/
│ ├── src/
│ │ ├── app/ # Next.js pages
│ │ ├── components/ # React components
│ │ └── lib/ # Utilities
│ └── tests/ # Frontend tests
├── infrastructure/ # Pulumi IaC
├── karaoke_gen/ # CLI package
└── scripts/ # Dev scripts
```
## Troubleshooting
### Emulators Won't Start
```bash
# Check if already running
lsof -i :8080 # Firestore
lsof -i :4443 # GCS
# Stop them
./scripts/stop-emulators.sh
```
### Tests Fail with Auth Errors
Ensure `ADMIN_TOKENS` is set in environment or `.env` file.
### Docker Issues
```bash
# Ensure Docker is running
docker info
# For GCS emulator
docker pull fsouza/fake-gcs-server
```
### Coverage Below Target
Run `./scripts/run-tests.sh --coverage` and check `htmlcov/index.html` to see uncovered lines.
+164
View File
@@ -0,0 +1,164 @@
# Lessons Learned
Key insights for future AI agents working on this codebase.
## Architecture Decisions
### LyricsTranscriber: Library, Not Server
**Problem**: LyricsTranscriber's `ReviewServer` blocks waiting for human input - incompatible with async cloud architecture.
**Solution**: Use LyricsTranscriber as a library:
- Call `transcribe()` and `correct()` for processing
- Save `corrections.json` to GCS
- Skip `ReviewServer` entirely
- Call `OutputGenerator` only after human review completes
**What to use**: `LyricsTranscriber`, `CorrectionResult`, `OutputGenerator`, `OutputConfig`, `CorrectionOperations`
**What NOT to use**: `ReviewServer`, `server.start()`, blocking waits
### Video Generation Timing
**Wrong**: Generate video during lyrics worker, then review
**Right**: Lyrics worker saves JSON → Human reviews → Render worker generates video
Video must include human-corrected lyrics, so generation happens AFTER review.
### Parallel Processing
Audio separation and lyrics transcription run in parallel:
- Both can fail independently
- Progress tracked separately in `state_data`
- Screens generation waits for both to complete
## Common Gotchas
### Pydantic Model Fields
**Problem**: Setting a field that doesn't exist in the Pydantic model silently does nothing.
```python
# BAD - fails silently if field doesn't exist
job.input_media_gcs_path = "path/to/file"
# GOOD - explicit field in model
class Job(BaseModel):
input_media_gcs_path: Optional[str] = None
```
**Lesson**: Always verify Pydantic model has the field before setting it.
### Styles Configuration
ASS subtitle generator requires ALL style fields present and properly typed:
```python
required_fields = [
"font", "font_path", "ass_name", # ass_name often missed
"primary_color", "secondary_color", "outline_color", "back_color",
"bold", "italic", "underline", "strike_out",
"scale_x", "scale_y", "spacing", "angle",
"border_style", "outline", "shadow",
"margin_l", "margin_r", "margin_v", "encoding"
]
# Critical: font_path must be string, not None
"font_path": "" # Correct
"font_path": None # Causes ASS writer to fail
```
### Frontend Data Format
The React review UI sends partial correction data:
```python
# Frontend sends:
{"corrections": [...], "corrected_segments": [...]}
# Backend must merge with original corrections.json
```
### Firestore Consistency
**Problem**: Update job, trigger worker immediately, worker reads stale data.
**Solution**: Verify updates propagated before triggering workers:
```python
# Update job
await job_manager.update_job(job_id, updates)
# Verify update visible
job = await job_manager.get_job(job_id)
if job.input_media_gcs_path:
await trigger_worker()
```
## Testing Insights
### Emulator Tests Catch Real Bugs
Unit tests with mocks didn't catch the `input_media_gcs_path` bug. Emulator integration tests did because they use real Firestore behavior.
**Lesson**: Run `make test` which includes emulator tests before every commit.
### Test Isolation
Emulator tests need their own `conftest.py` that does NOT mock `google.cloud.*`. Mocks from unit tests can leak into integration tests.
## Deployment Notes
### Cloud Run Timeouts
Default 5-minute timeout may not be enough for video encoding. Consider:
- Cloud Run Jobs for long-running tasks
- Breaking encoding into smaller chunks
- Increasing timeout for specific operations
### Secret Manager Access
Cloud Run service account needs `Secret Manager Secret Accessor` role. This is managed via Pulumi in `infrastructure/`.
### CI/CD: GitHub Actions
Migrated from Cloud Build to GitHub Actions for:
- Simpler configuration
- Better visibility
- Faster feedback
### Self-Hosted Runner on GCP
GitHub-hosted runners have ~14GB disk, which caused failures during Docker builds. Solution: self-hosted runner on GCP with 200GB SSD.
**Key learnings:**
1. **GPG in startup scripts**: Always use `gpg --batch` for non-interactive execution
2. **setup-python on self-hosted**: Needs Python in tool cache at `$RUNNER/_work/_tool/Python/<version>/x64/` with `x64.complete` marker
3. **Debian vs Ubuntu**: PPAs don't work on Debian; use upstream repos (Temurin, NodeSource)
4. **pyenv for Python**: More reliable than source builds or distro packages for specific versions
See `docs/archive/2025-12-28-self-hosted-github-runner.md` for full details.
## Performance Observations
| Operation | Duration |
|-----------|----------|
| Audio separation | 5-8 min |
| Lyrics transcription | 2-3 min |
| Auto-correction | 30 sec |
| Screens generation | 30 sec |
| Video rendering | 10-15 min |
| Final encoding | 5-10 min |
| **Total** | 30-50 min |
Human review time varies (5-15 min typical).
## What We'd Do Differently
1. **Add Pydantic model fields test first** - Would have caught the silent field issue immediately
2. **Use emulator tests from day one** - Faster feedback than deploying to test
3. **Design for async human review upfront** - Avoided rearchitecting after discovering ReviewServer blocks
4. **Keep docs minimal and current** - Less documentation, but always accurate
+318
View File
@@ -0,0 +1,318 @@
# Mobile UX Best Practices
This document outlines the mobile-first UX principles and implementation patterns used in this project to ensure a great experience across all screen sizes.
## Core Principles
### 1. Mobile-First Responsive Design
Design for the smallest screen first, then progressively enhance for larger screens.
```css
/* Base styles for mobile */
.element {
width: 100%;
padding: 12px;
}
/* Enhance for larger screens */
@media (min-width: 640px) {
.element {
width: 280px;
padding: 16px;
}
}
```
### 2. No Horizontal Overflow
The most critical mobile UX issue is horizontal scrolling. Always ensure content fits within the viewport width.
**Testing for overflow:**
```typescript
async function hasNoHorizontalOverflow(page: Page): Promise<boolean> {
return await page.evaluate(() => {
const docWidth = document.documentElement.scrollWidth;
const viewportWidth = window.innerWidth;
return docWidth <= viewportWidth + 2; // Small tolerance for rounding
});
}
```
**Common causes of horizontal overflow:**
- Fixed widths (e.g., `width: 340px`)
- Horizontal flex layouts without wrapping
- Images without `max-width: 100%`
- Tables without horizontal scroll containers
### 3. Touch Target Sizing
Interactive elements must be easily tappable. Follow these minimum sizes:
| Platform | Minimum Size | Recommended |
|----------|--------------|-------------|
| Apple iOS | 44x44px | 48x48px |
| Material Design | 48x48px | 48x48px |
| Web (general) | 32x32px | 44x44px |
**Implementation:**
```tsx
<Button className="min-h-[44px] px-3">
<Icon className="w-4 h-4" />
<span className="hidden sm:inline">Label</span>
</Button>
```
### 4. Progressive Disclosure
Show less content on mobile, with clear paths to access more:
- **Icon-only buttons on mobile, labels on desktop:**
```tsx
<Icon className="w-4 h-4 sm:mr-2" />
<span className="hidden sm:inline">Action Label</span>
```
- **Collapsed navigation on mobile**
- **Truncated text with expansion options**
## Layout Patterns
### Responsive Grid
Use responsive grid columns that stack on mobile:
```tsx
// Bad - always 2 columns
<div className="grid grid-cols-2 gap-4">
// Good - stack on mobile, 2 columns on larger screens
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3 sm:gap-4">
```
### Flexible Containers
Avoid fixed widths. Use percentage-based or responsive widths:
```css
/* Bad */
.sidebar {
width: 340px;
}
/* Good */
.sidebar {
width: 100%;
}
@media (min-width: 768px) {
.sidebar {
width: 340px;
}
}
```
### Flex Wrap for Toolbars
Allow toolbar items to wrap on smaller screens:
```tsx
<div className="flex flex-wrap gap-2 items-center">
{/* Items will wrap naturally */}
</div>
```
## Typography
### Readable Font Sizes
Minimum font sizes for readability without zooming:
| Element | Minimum Size | Recommended |
|---------|--------------|-------------|
| Body text | 14px | 16px |
| Labels | 12px | 14px |
| Small text/badges | 11px | 12px |
### Responsive Headings
```tsx
<h1 className="text-base sm:text-xl font-bold truncate">
Title Text
</h1>
```
## Form Design
### Input Fields on Mobile
- Stack form fields vertically on mobile
- Use full-width inputs for easier tapping
- Ensure proper input type for mobile keyboard optimization
```tsx
<input
type="email" // Shows email keyboard
inputMode="numeric" // Shows number pad
className="w-full min-h-[44px]"
/>
```
### Labels and Placeholders
- Labels should be visible (not just placeholders)
- Adequate spacing between label and input
- Clear validation feedback
## Testing Strategy
### Automated Testing with Playwright
Test across multiple viewport sizes representing real devices:
```typescript
const MOBILE_VIEWPORTS = {
'iPhone SE': { width: 375, height: 667 },
'iPhone 14': { width: 390, height: 844 },
'Pixel 7': { width: 412, height: 915 },
'iPad Mini': { width: 768, height: 1024 },
};
for (const [name, viewport] of Object.entries(MOBILE_VIEWPORTS)) {
test(`page works on ${name}`, async ({ page }) => {
await page.setViewportSize(viewport);
await page.goto('/');
// Test for horizontal overflow
const hasOverflow = await page.evaluate(() =>
document.documentElement.scrollWidth > window.innerWidth + 2
);
expect(hasOverflow).toBe(false);
});
}
```
### Visual Regression Testing
Capture screenshots at different viewports to catch visual regressions:
```typescript
await expect(page).toHaveScreenshot(`homepage-${deviceName}.png`, {
fullPage: true,
maxDiffPixelRatio: 0.1,
});
```
### Touch Target Validation
```typescript
const buttons = page.locator('button, a, [role="button"]');
for (const button of await buttons.all()) {
const box = await button.boundingBox();
if (box) {
expect(box.width).toBeGreaterThanOrEqual(32);
expect(box.height).toBeGreaterThanOrEqual(32);
}
}
```
## Framework-Specific Guidelines
### Tailwind CSS (Next.js Frontend)
Use responsive prefixes consistently:
```plaintext
sm: 640px - Small tablets and up
md: 768px - Tablets and up
lg: 1024px - Laptops and up
xl: 1280px - Desktops and up
```
**Common patterns:**
```tsx
// Stack on mobile, side-by-side on larger
className="flex flex-col sm:flex-row"
// Hide on mobile, show on larger
className="hidden sm:block"
// Reduce padding on mobile
className="p-3 sm:p-4"
// Smaller font on mobile
className="text-xs sm:text-sm"
```
### Material UI (Lyrics Review UI)
Use the `useMediaQuery` hook:
```tsx
const theme = useTheme();
const isMobile = useMediaQuery(theme.breakpoints.down('md'));
return (
<Box sx={{
flexDirection: isMobile ? 'column' : 'row',
width: isMobile ? '100%' : '280px',
}}>
```
### Vanilla CSS (Instrumental Review)
Use CSS media queries:
```css
.element {
width: 100%;
}
@media (min-width: 768px) {
.element {
width: 340px;
}
}
```
## Checklist for New Features
Before merging any UI changes, verify:
- [ ] No horizontal overflow on any mobile viewport (320px - 480px)
- [ ] All interactive elements are at least 44x44px
- [ ] Text is readable without zooming (minimum 11px)
- [ ] Form inputs stack vertically on mobile
- [ ] Navigation is accessible on mobile
- [ ] Images scale properly with `max-width: 100%`
- [ ] Modals/dialogs fit within mobile viewport
- [ ] Buttons show icons-only on very small screens
- [ ] Playwright mobile tests pass
## Common Fixes
### Fix: Horizontal Overflow
1. **Check for fixed widths** - Replace with responsive values
2. **Add flex-wrap** - Allow content to wrap
3. **Add overflow-x: hidden** - On containers (use sparingly)
4. **Check images** - Add `max-width: 100%`
### Fix: Touch Target Too Small
1. **Add minimum height** - `min-h-[44px]`
2. **Add padding** - `p-3` or `px-4 py-3`
3. **Use icon-only on mobile** - Hide text labels
### Fix: Text Too Small
1. **Use responsive font sizes** - `text-xs sm:text-sm`
2. **Set minimum body font size** - At least 14px
3. **Increase line height** - For better readability
## Resources
- [Apple Human Interface Guidelines - Touch Targets](https://developer.apple.com/design/human-interface-guidelines/accessibility)
- [Material Design - Touch Targets](https://m3.material.io/foundations/accessible-design/accessibility-basics)
- [Tailwind CSS Responsive Design](https://tailwindcss.com/docs/responsive-design)
- [Playwright Testing](https://playwright.dev/docs/test-mobile)
+34 -177
View File
@@ -1,193 +1,50 @@
# Karaoke-Gen Documentation
This directory contains all documentation for the karaoke-gen project, organized by purpose.
> **Status**: Production-ready. First full E2E flow completed 2025-12-28.
## 📋 Current Status (Updated: 2025-12-08)
## What Works
**Project:** Cloud-hosted karaoke generation with async human review
**Backend:** 🔄 Phase 1.3 (85%) - Implementing review architecture
**Frontend:** ⏸️ Not started (waiting for backend review flow)
**Deployment:** ✅ Cloud Run with custom domain (api.nomadkaraoke.com)
**Testing:** 🔄 Local emulator testing in progress
- **Web App**: https://gen.nomadkaraoke.com - Upload audio, review lyrics, download videos
- **Backend API**: https://api.nomadkaraoke.com - FastAPI on Cloud Run
- **CLI Tools**: `karaoke-gen` (local), `karaoke-gen-remote` (cloud)
### Key Discovery
## Current State
We identified that the `LyricsTranscriber` library's `ReviewServer` **blocks** waiting for human input, which doesn't work in our async cloud architecture. The solution is to:
| Component | Status |
|-----------|--------|
| Audio upload & separation | Working |
| Lyrics transcription | Working |
| Human lyrics review | Working |
| Preview video generation | Working |
| Instrumental selection | Working |
| Multi-format encoding | Working |
| Token-based auth | Working |
| Payment flow | Not started |
| CI/CD self-hosted runner | Working (GCP) |
1. Use LyricsTranscriber for transcription + auto-correction
2. Save corrections to GCS for async human review
3. Use `OutputGenerator` to render video **after** review completes
## Known Issues
See `00-current-plan/CURRENT-STATUS.md` for full details.
- CDG format generation requires additional style configuration
- Long audio files (>10 min) may timeout on some workers
---
## Quick Links
## 📂 Documentation Structure
| Doc | Purpose |
|-----|---------|
| [ARCHITECTURE.md](ARCHITECTURE.md) | System design, data flow, integrations |
| [DEVELOPMENT.md](DEVELOPMENT.md) | Local setup, testing, deployment |
| [API.md](API.md) | Backend endpoint reference |
| [LESSONS-LEARNED.md](LESSONS-LEARNED.md) | Key insights for future agents |
| [archive/](archive/) | Historical documentation |
### `00-current-plan/` - **START HERE**
Active migration plan and current status.
| File | Purpose |
|------|---------|
| **`CURRENT-STATUS.md`** | Where we are now, what's working, what's next |
| **`WHATS-NEXT.md`** | Detailed plan for review architecture implementation |
| **`WORKER-IMPLEMENTATION-PLAN.md`** | Worker responsibilities and implementation details |
| `MASTER-PLAN.md` | Overall migration strategy |
| `CLOUD-MIGRATION-REQUIREMENTS.md` | Architectural analysis from CLI workflow |
### `01-reference/` - Reference Documentation
Core system documentation.
| File | Purpose |
|------|---------|
| **`ARCHITECTURE.md`** | Cloud + CLI architecture with diagrams |
| `KARAOKE-GEN-CLI-WORKFLOW.md` | CLI workflow breakdown (8 stages) |
| `API-MANUAL-TESTING.md` | How to test backend API with curl |
| `AUTHENTICATION.md` | Token-based auth system |
| `CLOUD-RUN-ARCHITECTURE.md` | Cloud Run concepts |
### `02-implementation-history/` - Session Logs
Historical implementation records.
### `03-deployment/` - Deployment Guides
How to deploy and operate the system.
| File | Purpose |
|------|---------|
| **`EMULATOR-TESTING.md`** | Local testing with Firestore/GCS emulators |
| **`SETUP-VENV.md`** | Python venv setup |
| **`LOCAL-VALIDATION.md`** | Validate before deploying |
| **`OBSERVABILITY-GUIDE.md`** | Debug production issues |
### `04-testing/` - Testing Guides
| File | Purpose |
|------|---------|
| **`TESTING-GUIDE.md`** | Complete testing guide |
---
## 🚀 Quick Start
### I want to understand the current state:
1. Read `00-current-plan/CURRENT-STATUS.md` - where we are now
2. Read `00-current-plan/WHATS-NEXT.md` - what needs to be done
3. Read `01-reference/ARCHITECTURE.md` - system architecture
### I want to work on the backend:
1. Read `03-deployment/EMULATOR-TESTING.md` - local testing setup
2. Read `00-current-plan/WORKER-IMPLEMENTATION-PLAN.md` - worker details
3. Run `./scripts/run-backend-local.sh --with-emulators`
### I want to test:
```bash
# Start backend with emulators
./scripts/run-backend-local.sh --with-emulators
# Upload test file
curl -X POST http://localhost:8000/api/jobs/upload \
-F "file=@tests/data/waterloo10sec.flac" \
-F "artist=ABBA" \
-F "title=Waterloo"
# Check status
curl http://localhost:8000/api/jobs/{job_id}
```
---
## 🎯 Current Phase: Review Architecture
**Goal:** Implement async human review workflow
**What Works:**
- ✅ File upload → GCS
- ✅ Audio separation via Modal API
- ✅ Lyrics transcription via AudioShake
- ✅ Auto-correction via LyricsTranscriber
- ✅ Title/end screen generation
- ✅ Instrumental selection endpoint
**What's Being Implemented:**
- 🔄 Review API endpoints (`GET/POST /api/jobs/{id}/review`)
- 🔄 Render Video Worker (uses `OutputGenerator` after review)
- 🔄 Updated state machine with `RENDERING_VIDEO` state
**The Key Insight:**
```
WRONG: Lyrics Worker → Generate Video → Review → ???
RIGHT: Lyrics Worker → Save JSON → Human Review → Render Video → Final Assembly
```
Video generation happens AFTER human review, not during the lyrics worker.
---
## 📝 Key Technical Decisions
### Why Not Use LyricsTranscriber's ReviewServer?
The `ReviewServer` class (in `lyrics_transcriber/review/server.py`):
- Starts a local web server on port 8000
- Opens a browser window
- **BLOCKS** the thread until human submits corrections
- Designed for local CLI operation
This doesn't work in Cloud Run where:
- We can't open browsers
- Workers must be non-blocking
- Human review happens asynchronously via React frontend
**Solution:** Use the data structures and `OutputGenerator` directly, skip the server.
### Why Separate Video Rendering?
## Workflow Summary
```
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Lyrics Worker │ ──▶ │ Human Reviews │ ──▶ │ Render Worker │
│ │ │ (React UI) │ │ │
│ • Transcribe │ │ • Corrects │ │ • OutputGenerator│
│ • Auto-correct │ │ • Saves JSON │ │ • with_vocals.mkv│
│ • Save JSON │ │ │ │ │
│ • NO VIDEO │ │ │ │ │
└─────────────────┘ └─────────────────┘ └─────────────────┘
Upload Audio → Parallel Processing → Human Review → Video Generation → Download
│ │
(Modal + AudioShake) (LyricsTranscriber UI)
```
The video must include the human-corrected lyrics, so it can only be generated AFTER review.
## For AI Agents
---
## 🔄 Job State Flow
```
PENDING → DOWNLOADING → [parallel audio+lyrics] → GENERATING_SCREENS
↓
AWAITING_REVIEW → IN_REVIEW → REVIEW_COMPLETE
↓
RENDERING_VIDEO (NEW)
↓
AWAITING_INSTRUMENTAL_SELECTION → INSTRUMENTAL_SELECTED
↓
GENERATING_VIDEO → COMPLETE
```
Two human interaction points:
1. **AWAITING_REVIEW** - User corrects lyrics
2. **AWAITING_INSTRUMENTAL_SELECTION** - User chooses audio track
---
## 📞 Getting Help
1. Check `00-current-plan/CURRENT-STATUS.md` first
2. Look for specific guides in `03-deployment/`
3. Review session history in `02-implementation-history/`
### Common Issues
| Issue | Solution |
|-------|----------|
| Emulator not starting | Check ports 8080, 4443 are free |
| "Missing lyrics video" | Review hasn't happened yet - expected behavior |
| Workers not progressing | Check `state_data` in Firestore |
| FFmpeg errors | Ensure FFmpeg is in PATH |
Start with `CLAUDE.md` in the repo root for essential rules, then reference these docs as needed. Keep this README current when project state changes.
File renamed without changes.
File renamed without changes.
@@ -0,0 +1,80 @@
# Self-Hosted GitHub Actions Runner on GCP
**Date**: 2025-12-28
**Status**: Deployed and operational
## Problem
GitHub-hosted runners have ~14GB disk space. CI jobs were failing with "No space left on device" during:
- Docker builds (multi-stage, large dependencies)
- Emulator tests (Docker + Java + Firestore/GCS emulators)
- Package builds with poetry
Workarounds like `jlumbroso/free-disk-space` action helped but weren't reliable.
## Solution
Deployed a self-hosted runner on GCP with 200GB SSD via Pulumi IaC.
### Infrastructure Added
| Resource | Purpose |
|----------|---------|
| `github-runner` VM | e2-standard-4, 200GB SSD, Debian 12 |
| `github-runner` service account | Minimal permissions for runner |
| `github-runner-pat` secret | PAT for runner registration |
| IAM bindings | Artifact Registry reader, Secret Manager accessor, logging |
### Jobs Moved to Self-Hosted
- `package-build-test`
- `backend-unit-tests`
- `backend-emulator-tests`
- `deploy-backend`
These use `runs-on: [self-hosted, linux, gcp]` with labels.
## Issues Encountered
### 1. GPG tty errors in startup script
**Problem**: `gpg --dearmor` failed with "cannot open /dev/tty" in non-interactive startup.
**Fix**: Add `--batch` flag to all GPG commands.
### 2. setup-python can't find Python 3.13 on Debian 12
**Problem**: `actions/setup-python@v5` couldn't find pre-built Python 3.13 for Debian 12 on self-hosted runners.
**Fix**:
1. Install Python 3.13 via pyenv on the runner
2. Copy to tool cache: `_work/_tool/Python/3.13.0/x64/`
3. Create marker file: `x64.complete`
### 3. Java 21 not in Debian repos
**Problem**: `openjdk-21-jdk` not available in Debian 12 stable.
**Fix**: Use Temurin (Adoptium) repository directly.
## Key Files
- `infrastructure/__main__.py` - Runner VM and IAM setup
- `infrastructure/README.md` - Setup instructions
- `.github/workflows/ci.yml` - Jobs using self-hosted runner
## Lessons for Future
1. **Self-hosted runners need explicit tool setup** - Unlike GitHub-hosted, tools aren't pre-installed in expected locations
2. **setup-python tool cache location**: `$RUNNER_DIR/_work/_tool/Python/<version>/x64/`
3. **GPG in scripts**: Always use `--batch` for non-interactive execution
4. **Debian vs Ubuntu**: Many GitHub Actions assume Ubuntu; PPAs don't work on Debian
## Cost
~$50/month for e2-standard-4 running 24/7 (covered by GCP free credits).
## PRs
- #95: Initial self-hosted runner setup
- #97: Make startup script reproducible (pyenv, tool cache, GPG fixes)
File renamed without changes.
File renamed without changes.
@@ -0,0 +1,197 @@
# GitHub Actions Docker Builds - Cost Optimization
**Date:** 2025-12-28
**Problem:** Cloud Build costing $100+/month with slow builds
**Solution:** Move Docker builds to GitHub Actions with registry caching
---
## Overview
This document describes the migration from Google Cloud Build to GitHub Actions for Docker image builds. This change eliminates Cloud Build costs while improving build speeds through better caching.
## Cost Comparison
| Approach | Monthly Cost | Build Speed | Caching |
|----------|-------------|-------------|---------|
| **Cloud Build (N1_HIGHCPU_32)** | ~$100+ | 5-15 min | Limited |
| **GitHub Actions + Registry Cache** | **$0** | 2-5 min | Excellent |
### Why Cloud Build Was Expensive
1. **N1_HIGHCPU_32 pricing**: ~$0.064/build-minute (21x more than default)
2. **No persistent layer cache**: Each build re-downloaded dependencies
3. **Cold start overhead**: Spinning up build VMs added latency
4. **Large dependency tree**: torch, transformers, etc. = ~500 packages
### Why GitHub Actions Is Free
- **Public repositories**: Unlimited build minutes on GitHub-hosted runners
- **Registry cache**: Layers cached in Artifact Registry, persist between builds
- **BuildKit mode=max**: Caches ALL intermediate layers, not just final
---
## Architecture
```
┌─────────────────────────────────────────────────────────────────────────┐
│ NEW BUILD PIPELINE │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ GitHub Actions (Free) │
│ ├── docker/setup-buildx-action (BuildKit) │
│ ├── google-github-actions/auth (Workload Identity) │
│ └── docker/build-push-action │
│ ├── cache-from: type=registry,ref=...karaoke-backend:cache │
│ └── cache-to: type=registry,ref=...karaoke-backend:cache,mode=max│
│ │
│ Artifact Registry │
│ ├── karaoke-backend:latest (production image) │
│ ├── karaoke-backend:cache (build cache - ALL layers) │
│ ├── karaoke-backend-base:latest (base image with deps) │
│ └── karaoke-backend-base:cache (base cache) │
│ │
│ Build Times (Expected): │
│ ├── First build (cold): ~8-10 min │
│ ├── Code-only changes: ~2-3 min (cache hit) │
│ └── Dependency changes: ~6-8 min (partial cache hit) │
│ │
└─────────────────────────────────────────────────────────────────────────┘
```
---
## How It Works
### Two-Stage Build Pattern
We maintain the existing two-stage build pattern for efficiency:
1. **Base Image** (`karaoke-backend-base`)
- Contains Python 3.11, ffmpeg, system dependencies
- All pip dependencies pre-installed
- Only rebuilt when `poetry.lock` or `Dockerfile.base` changes
- Hash-based change detection via image labels
2. **App Image** (`karaoke-backend`)
- Built FROM the base image
- Contains only application code
- Fast rebuilds (~2 min) for code-only changes
### Registry Cache
The key optimization is using Artifact Registry as a persistent layer cache:
```yaml
- uses: docker/build-push-action@v6
with:
cache-from: type=registry,ref=.../karaoke-backend:cache
cache-to: type=registry,ref=.../karaoke-backend:cache,mode=max
```
**Benefits:**
- `mode=max` caches ALL layers, not just the final image
- Cache persists between CI runs (unlike GitHub Actions cache which has 10GB limit)
- Layers only re-built when inputs change
---
## Files Changed
### New Files
- **`.dockerignore`** - Reduces build context from ~1GB to ~10MB
### Modified Files
- **`.github/workflows/ci.yml`** - `deploy-backend` job rewritten to use:
- `docker/setup-buildx-action@v3`
- `docker/build-push-action@v6`
- Registry caching to Artifact Registry
### Kept (as fallback)
- **`cloudbuild.yaml`** - Still works for manual triggers
- **`cloudbuild-base.yaml`** - Still works for manual base image builds
---
## Manual Builds (Fallback)
If GitHub Actions fails, you can still use Cloud Build:
```bash
# Build base image (only if dependencies changed)
gcloud builds submit --config=cloudbuild-base.yaml --project=nomadkaraoke
# Build and deploy app image
gcloud builds submit --config=cloudbuild.yaml --project=nomadkaraoke
```
---
## Monitoring
### Check Build Cache Usage
View cache images in Artifact Registry:
```bash
gcloud artifacts docker images list \
us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo \
--include-tags
```
You should see `*:cache` tags for both base and app images.
### Compare Build Times
- **GitHub Actions**: Check workflow run times in Actions tab
- **Previous Cloud Build**: ~5-15 min per build
- **Expected with cache**: ~2-3 min for code changes
---
## Troubleshooting
### Cache Miss (Slow Build)
If builds are slow despite no dependency changes:
1. Check if cache image exists:
```bash
docker pull us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend:cache
```
2. Force cache rebuild by pushing to `main` with a small change
### Disk Space Issues
GitHub Actions runners have limited disk. If base image builds fail:
1. The workflow includes `jlumbroso/free-disk-space` action
2. It removes Android SDK, Haskell, .NET, etc. to free ~20GB
### Authentication Issues
If `docker push` fails with permission errors:
1. Check Workload Identity is configured correctly
2. Verify service account has `roles/artifactregistry.writer`
---
## Future Improvements
1. **Parallel base/app builds**: If base is unchanged, start app build immediately
2. **GitHub Actions cache fallback**: Use GHA cache when registry is slow
3. **Build metrics**: Track build times and cache hit rates
---
## References
- [Docker Build with GitHub Actions](https://docs.docker.com/build/ci/github-actions/)
- [Registry Cache Backend](https://docs.docker.com/build/cache/backends/registry/)
- [Cache Management](https://docs.docker.com/build/ci/github-actions/cache/)
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Loaded 100 of 138 files, more files were not shown because too many files have changed in this diff. Show more