mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
YouTube downloads depended on one logged-in cookie session on one VM. This tries TornadoAPI first and keeps flacfetch as the fallback. Plan: workspace `docs/archive/2026-11-03-tornado-youtube-download-plan.md`. Alert: https://discord.com/channels/1198691101943541770/1265517528608669707/1567241547739570298 Alert: https://discord.com/channels/1198691101943541770/1265517528608669707/1567252495535841386 Alert: https://discord.com/channels/1198691101943541770/1265517528608669707/1567252497620537415 ## Inactive until the secret exists Nothing changes in prod when this merges. The deploy adds `TORNADO_API_KEY` only once secret `tornado-api-key` has a version, and creating it needs `/gcp-admin`. `YOUTUBE_DOWNLOAD_PROVIDER=flacfetch` switches it off. ## What changed - **`tornado_client.py`:** create job, poll with a 180s deadline (cancels on timeout), stream the delivered file, delete it from Tornado. Retries with one idempotency key per download; honours `Retry-After`. - **`YouTubeDownloadService`:** Tornado first for YouTube, then flacfetch. WARNING per fallback, ERROR on 401/402/403, so an outage or lapsed billing is visible. Non-YouTube URLs stay on flacfetch. - **Availability check:** Tornado metadata first, because flacfetch's check rejects valid videos while its cookies are bot-blocked. If the lookup fails, flacfetch decides as before. ## Spike results (real API, 11 jobs) - `audio_only` + `format: opus` gives **packet-identical audio** to flacfetch's real job input (`Tk3ty5ansqs`, stream MD5 match). `audio_codec: copy` or no `format` gives 128k AAC instead, so the request omits it. - `mBJivIsN0n0`, which bot-blocked job 526bf46c, completed in about 5s of processing. - An age-restricted video completed without cookies; a missing video ends in `Warning`. - Billed about 25.6 MB for a 4-minute song. ## Testing - 79 mocked tests pass (`test_tornado_client.py`, `test_youtube_download_service.py`). - Live run through `YouTubeDownloadService` with the GCS upload stubbed, using a URL with a playlist parameter: only the single video was sent. - Local CodeRabbit: all findings fixed; the last small fix wasn't re-reviewed. ## Not verified - **A full pipeline job with this input.** It can only run once the secret exists. Files are Ogg Opus named `.ogg` (flacfetch's were WebM Opus), so the codec is the same but the container is new to this path. Run the daily E2E or one admin job first. - **Deviation from the plan:** the Tornado job id is logged but not saved on the gen job, because a restart loses the in-request download on the flacfetch path too. A restart can orphan one job, which costs a fraction of a cent. - **CI's secret check:** it depends on the deploy service account being able to read the secret; the log line `tornado-api-key wired:` shows the result. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_012RYLVTvqXyL4QyA2EjzQMG Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>