mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
main
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
f8d0febcf5 |
Translated lyrics beneath the karaoke lyrics for language learners (v0.260.0) (#1101)
## Summary
New job-submission option: **Add translated lyrics** → pick a language
(33 UI locales) → the karaoke videos show an AI translation of each
lyric line in smaller text beneath the synced lyrics, so language
learners can sing songs in another language while learning what the
words mean.
- **Job creation**: toggle + language picker (default = UI language) on
the Customize step, with a **live preview** of a real rendered frame
(`GET /api/themes/translation-preview?language=xx`; pre-generated sample
translations, no LLM call at preview time). Any job, public or private,
free for now. `translation_language` accepted by every creation route;
admin can edit it.
- **Post-review translation**: right after the job enters
RENDERING_VIDEO, the render worker translates the *final reviewed*
lyrics line by line with Gemini
(`backend/services/lyrics_translation.py`, `LYRICS_TRANSLATION_MODEL`,
GCS-cached so theme/admin re-renders are free) into
`jobs/{id}/lyrics/translations.json`. Not editable. A failure never
blocks the render (renders without translations, recorded in
`state_data.lyrics_translation`); a song already in the target language
gets no row.
- **Rendering** (landscape + portrait, GCE worker + local fallback):
`LyricsSegment.translation`; when present, slots get taller and screens
show 3 lines; a `Karaoke.Translation` style (unsung colour, ~½ size)
draws one static event beneath each line, sanitized, wrapped/scaled to
fit. Jobs without translations render identically.
- **RTL + LTR together**: the translation is its own ASS event with auto
paragraph direction, so e.g. a Hebrew song keeps its right-to-left
highlight with an English translation beneath (and vice versa). Long
lines split by the resizer share their translation across the pieces
(grapheme-safe for Thai/CJK).
Design: `docs/archive/2026-10-01-translated-lyrics-plan.md`.
## Testing
- `make test` all green (root 2014, backend 5416, emulator 150, frontend
1583).
- New: `tests/unit/lyrics_transcriber/output/test_translated_lyrics.py`
(layout/positions/style/sanitizing/fit/mixed RTL-LTR/Thai/portrait),
`test_translated_lyrics_render.py` (libass pixel tests incl.
Hebrew/Arabic lyrics with English/Arabic translations and CJK),
`backend/tests/test_lyrics_translation.py`,
`backend/tests/test_render_video_worker_translations.py`, GCE endpoint
test, frontend CustomizeStep/GuidedJobFlow/api tests, prod E2E
`e2e/production/translated-lyrics.spec.ts`.
- `scripts/run-render-tests-linux.sh`: 135 passed on Debian ffmpeg AND
prod's static ffmpeg (no skips).
- Real-data check: job 5710831e (Hebrew) translated to English by Gemini
and rendered with the nomad theme — RTL highlight intact, English
beneath, split line shares translation.
Local CodeRabbit CLI unavailable in this workspace (no seat), so leaving
this PR for the CodeRabbit bot; an independent local review was done and
its findings fixed in the second commit.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an opt-in translated-lyrics option to job setup, with language
selection and a preview.
* Translations are generated after lyric review and appear beneath
lyrics in landscape and portrait videos, with text layout for multiple
languages and writing directions.
* Translation settings can be changed or cleared by admins.
* **Bug Fixes**
* If translation fails or is unavailable, video rendering continues
without translated lyrics.
* **Documentation**
* Updated API and product documentation with translated-lyrics setup and
behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
||
|
|
5a9d0024a5 |
fix(quick-version): ffmpeg 4.4 never-ending render + fast ONNX model (v0.246.1) (#1074)
## Summary First production make-it quick version (job `405eed71`, Rihanna "Push Up On Me") failed with `render timed out`: - **Render never ended**: the GPU image ships ffmpeg **4.4** (Ubuntu 22.04). With the guide-vocals `amix` in the filtergraph plus the infinite lavfi colour + looped PNG inputs, it ignores `-shortest` — reproduced in `ubuntu:22.04`: a 20 s song produced 540 s+ of video until killed. ffmpeg 8 (local dev) stops fine, which is why local tests passed. Fix: every input and the output are bounded with `-t <duration>` (colour source `d=`). Verified in `ubuntu:22.04`: 3.7 s render, exactly 20.0 s audio+video. - **Quick separation too slow**: the baked roformer took 14 s load + 124 s for a 3:15 song on the L4. Switched to fastgen's `UVR-MDX-NET-Inst_HQ_4.onnx`. It's downloaded on load until the next GPU base rebuild bakes it (`download_models.py` `EXTRA_MODELS`), with a fallback to the baked roformer if the load fails. Prod timeline observed for that job (for the <4 min goal): job created 05:22:12 → **audio-download-job container took 3m46s to start** (existing issue, 1m40–4m across recent executions; separate follow-up) → audio in GCS 05:26:46 → GPU worker started 05:27:10 → quick separation done 05:29:49 → render hung. ## Testing - `backend/tests/test_quick_version.py`: 40 pass (new: every stream bounded to the duration; model fallback; both-fail raises). - Docker `ubuntu:22.04` ffmpeg 4.4.2 reproduction before/after (see above). 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Quick audio separation now uses a faster model by default, with an alternate model available if the primary model cannot be loaded. * **Bug Fixes** * Quick video rendering is now limited to the song’s duration, helping prevent renders from continuing beyond the audio or failing to finish. * Model loading failures are handled more reliably: if both the primary and alternate models fail to load, the error is reported. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
23ebee4657 |
fix(backfill): correct date filter, private-track path, and folder guard (#843)
## Summary Follow-up to #842. Operational correctness fixes to `backend/scripts/backfill_original_audio.py`, found while running the original-audio backfill against production (**344 jobs restored, 0 errors**). As merged, the script silently did nothing and would have mis-routed private tracks. ## Changes - **Date filter:** `created_at` is stored as an ISO **string** in Firestore, so the server-side `created_after`/`created_before` datetime filters matched nothing ("Considering 0 jobs"). Now fetches completed jobs (ordered desc) and filters the date window in Python; default fetch limit raised to 5000 to bound the lookback. - **Private-track path:** uses `get_effective_distribution_for_job()` for the Dropbox path, so private (`NOMADNP`) tracks resolve to the **NonPublished** path instead of the job's stored public `dropbox_path` (was skipping 130+ private tracks as "folder missing"). - **Folder guard:** only uploads into a track folder that **already exists** (new `skip-folder-missing`), so drifted brand/artist/title naming can't create an orphan folder with a lone FLAC. - **Env filter:** filters environment in Python (avoids needing a 3-field composite Firestore index); `--env` now defaults to none (test jobs skip naturally — no brand/Dropbox folder). - **Resilience:** per-job planning errors are counted, not fatal. ## Testing - [x] 8 planner unit tests pass (`test_backfill_original_audio.py`), incl. new `skip-folder-missing` case - [x] Proven end-to-end in production: 344 original-audio files restored to the correct public/private Dropbox folders, idempotent re-run confirmed (already-present skipped, transient errors retried clean) ## Review - [x] Validated by actually executing against prod; script-only change, no runtime/service impact @coderabbitai ignore --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
113197293b |
fix: restore missing (Title/End).mov + original audio in Dropbox folders (#842)
## Summary - Organised Dropbox track folders stopped containing the 5-second screen videos (`(Title).mov`/`(End).mov`) and the original input audio (`(flacfetch).flac` / `(uploaded).mp3`). Root cause: `_upload_to_dropbox` uploads the whole `output_dir`, and the monolith→distributed-GCE migration quietly narrowed what lands there. - Restores both file types for new jobs, adds an audio-only backfill for past jobs, and adds a daily-E2E guard so this regression class is caught automatically. - Bundles a CLI fix for an ffmpeg interactive-prompt hang in `--finalise-only` (found while regenerating the missing files locally). ## Changes **Screen `.mov`s (forward):** GCE encoder copies its generated `title/end.mov` into `outputs/` with canonical names → plumbed via `EncodingOutput.title_mov_path`/`end_mov_path`, the `GCEEncodingBackend` filename mapping, and `_download_gce_encoded_files` back into `output_dir`. **Original audio (forward):** new `backend/services/original_audio.py` (source tag `(uploaded)`/`(flacfetch)`/`(YouTube)` + GCS path + filename); orchestrator `_stage_original_audio_for_upload()` fetches it into `output_dir` before folder uploads (non-fatal, only when a folder upload will occur). **Backfill (audio only):** `backend/scripts/backfill_original_audio.py` — dry-run by default, `--live` to upload, skips when audio is gone from GCS or already present. `.mov` can't be backfilled (needs re-encode). Adds `DropboxService.file_exists` (re-raises non-not-found `ApiError`). **Daily E2E guard:** `audio_download_worker` records original audio in `file_urls['input']['audio']`; `_upload_results` registers `title_mov`/`end_mov` under `file_urls['finals']`; `happy-path-real-user.spec.ts` STEP 10.5 asserts the admin files manifest contains `finals/title_mov`, `finals/end_mov`, `input/audio`, `finals/lossless_4k_mp4`. **CLI ffmpeg-hang fix:** `karaoke_finalise` only added `-y` in non-interactive mode, so a pre-existing output made ffmpeg block on an invisible `Overwrite? [y/N]` prompt until the 600s timeout (silently aborting at step 2/6). `-y` is now unconditional + `stdin=subprocess.DEVNULL` on all ffmpeg `subprocess.run` calls. ## Testing - [x] 38 new/updated unit tests; all affected suites green (`encoding_interface`, `video_worker_orchestrator`, `gce_encoding_worker`, `original_audio`, `backfill_original_audio`, `dropbox_service`, `upload_results_screen_movs`, `audio_download_worker`, full `karaoke_finalise` suite) - [x] Full backend suite: +38 passing vs main, 0 new failures (12 pre-existing `asyncio.get_event_loop()` ordering failures on main, unrelated) - [x] Output-file completeness now verified by the daily GHA E2E (STEP 10.5) ## Review - [x] Local CodeRabbit review completed (scoped to this branch) — 1 major finding fixed (`file_exists` masking non-not-found errors), re-review clean (0 findings) @coderabbitai ignore --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
dfcd79e5c7 |
fix: GPU app image — preserve pinned torch cu126 with --no-deps (#614)
## Summary - Use `--no-deps` in app Dockerfile when building on GPU base image (detected via `MODEL_DIR` env var) - Without this, `pip install -e .` upgrades torch from pinned cu126 back to cu130 (pyproject.toml has `torch>=2.7`) and reinstalls audio-separator[cpu], breaking both CUDA and GPU ONNX Runtime This is a follow-up to #613 — the GPU base image had the right packages, but the app layer's `pip install -e .` undid the pinning. ## Test plan - [ ] CI passes - [ ] Deploy, trigger audio separation job - [ ] Verify Job logs show CUDA available + successful separation @coderabbitai ignore --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
92ae3fb28f |
feat: run audio separation directly on GPU in audio worker (#608)
## Summary - **Eliminates HTTP hop** between the audio worker Cloud Run Job and the audio-separator Cloud Run Service by running the `Separator` class directly with an L4 GPU attached - **Adds GPU Docker image** (`karaoke-backend-gpu`) with CUDA 12.6, PyTorch, and pre-baked ensemble preset models - **Moves audio worker to us-east4** with L4 GPU, 4 CPU, 16Gi memory — old separator service kept for rollback ## Changes ### Code - `audio_processor.py` — Ensemble presets wired into local Separator path (2-stage: `instrumental_clean` + `karaoke`) - `audio_worker.py` — Dual-mode support: `MODEL_DIR` env var → local GPU, `AUDIO_SEPARATOR_API_URL` → remote API - `create_audio_processor()` — New `model_file_dir` parameter to enable local GPU mode ### Infrastructure (Pulumi) - `gpu_artifact_registry.py` — New Artifact Registry in us-east4 for GPU images - `cloud_run.py` — Audio worker job: GPU (nvidia-l4), us-east4, 16Gi, `MODEL_DIR=/models` - `__main__.py` — Wire in GPU registry + Cloud Build IAM ### Docker - `Dockerfile.gpu-base` — CUDA 12.6 + Python 3.11 + audio-separator[gpu] + baked models - `cloudbuild-gpu-base.yaml` — Cloud Build config (E2_HIGHCPU_32, 100GB disk, 1h timeout) - `download_models.py` — Bakes `instrumental_clean` + `karaoke` preset models at build time ### Tests - 8 new unit tests covering factory config, mode selection, and preset wiring - Updated existing test patch targets for module-level Separator import ## Testing - [x] 22/22 unit tests pass (8 new + 14 existing) - [x] Code review completed (subagent-driven + final review) - [x] CodeRabbit CLI review completed locally - [ ] GPU image build (requires `gcloud builds submit`) - [ ] Production verification with `under_pressure_harmonies.flac` ## Rollback Plan Set `AUDIO_SEPARATOR_API_URL` back on the audio worker job → falls through to remote API path. Old separator service is still running (scale-to-zero). ## Design & Plan - Design spec: `docs/archive/2026-03-26-direct-gpu-audio-separation-design.md` - Implementation plan: `docs/archive/2026-03-26-direct-gpu-audio-separation-plan.md` @coderabbitai ignore --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
7b7c0f7ff4 |
fix: allow PENDING → DOWNLOADING_AUDIO state transition (#445)
## Summary - Adds `DOWNLOADING_AUDIO` as a valid transition from `PENDING` in the state machine - Fixes 500 error on the `create-from-search` endpoint deployed in #444 - The `create-from-search` path creates a job (PENDING) then transitions directly to DOWNLOADING_AUDIO since audio is pre-selected from the search session ## Changes - `backend/models/job.py` — Add DOWNLOADING_AUDIO to PENDING transitions - `backend/scripts/generate_state_diagram.py` — Keep diagram in sync - `backend/tests/test_standalone_search.py` — Add state transition test ## Testing - All 31 standalone search tests pass - All 158 related tests pass (standalone + models + audio search) @coderabbitai ignore --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
5901137724 |
feat: make state machine transitions robust with explicit failure handling (#370)
## Summary Jobs 06cfea29 and 984da08b got stuck at `pending` status because state transitions failed silently. This comprehensive fix makes the state machine more robust through several improvements: - **Explicit failure handling**: `transition_to_state()` now raises `InvalidStateTransitionError` by default instead of returning False silently - **Centralized processing pattern**: New `start_job_processing()` helper ensures atomic transition + worker trigger across all handlers - **Worker validation**: All workers now validate job status before processing via `validate_worker_can_run()` - **Runtime consistency checks**: New `job_health_service.py` detects stuck jobs and state inconsistencies - **Health endpoint**: `/health/job-consistency` endpoint for admin dashboard integration - **Admin button fixes**: Reset to pending now clears parallel processing flags; removed deprecated states ## Changes ### Core Services - `services/job_manager.py`: Added `InvalidStateTransitionError`, `start_job_processing()`, and updated `transition_to_state()` to raise by default - `services/job_health_service.py`: New service for consistency checks and worker validation - `exceptions.py`: Added `InvalidStateTransitionError` exception class ### API Routes - `routes/health.py`: Added `/health/job-consistency` endpoint - `routes/admin.py`: Updated STATE_DATA_CLEAR_KEYS and ALLOWED_RESET_STATES - `routes/audio_search.py`: Fixed to properly transition from DOWNLOADING_AUDIO to DOWNLOADING - `routes/users.py`: Added error handling for state transition failures - `routes/file_upload.py`: Uses centralized `start_job_processing()` ### Workers - All workers (audio, lyrics, screens, video, render_video): Added status validation via `validate_worker_can_run()` ### Frontend - `admin/jobs/page.tsx`: Replaced "Lyrics" + "Inst." buttons with combined "Review" button ### Tests - 26 tests for job_health_service - 11 integration tests for state machine flow - 4 tests for job-consistency endpoint - Updated existing tests for new exception behavior ### Documentation - `docs/LESSONS-LEARNED.md`: Added "Make State Transitions Loud, Not Silent" entry - `docs/ARCHITECTURE.md`: Added state machine robustness note - `docs/archive/2026-02-02-state-machine-robustness-plan.md`: Complete implementation plan ## Test plan - [x] All 1850+ unit tests pass - [x] CodeRabbit review issues addressed - [ ] Manual verification of admin reset buttons in staging - [ ] Verify stuck jobs are properly detected by `/health/job-consistency` 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- @coderabbitai ignore --> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |