mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
main
2052
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
4e43449559 |
fix(backend): stop API event-loop freezes + identify users in degradation telemetry (v0.264.0) (#1120)
## Summary The "Reconnecting" pill and the "servers unavailable" banner (164 / 49 events, 2026-09-19 → 10-02) were mostly caused by the single API instance's **event loop freezing for 20–45s**. Every request on the instance, `/api/health` included, completed at the same moment. The full evidence and plan are in `docs/archive/2026-10-03-backend-loop-freezes-telemetry-plan.md`. ### What froze the loop, and the fix for each - **Screens worker:** it ran as a post-response `async def` BackgroundTask doing 4K PIL renders, GCS transfers and ffmpeg transcodes on the loop. Under `--cpu-throttling` it was also CPU-starved. **Fix:** it now runs inline in its Cloud Tasks request (CPU stays allocated, failures still return 200 so Cloud Tasks doesn't double-run it), with the heavy steps in `asyncio.to_thread`. Also offloaded: style load, timing signals, pre-apply. - **Matplotlib font cache:** it was rebuilt on every new instance (17/28 episodes). **Fix:** it's pre-built in `Dockerfile.base` (`MPLCONFIGDIR=/opt/mplconfig`), and pyplot is imported lazily. - **Hourly stale-review cron:** an `async def` with zero awaits doing ≤1500 reads and SMTP sends. **Fix:** it runs in a thread. - **166 `async def` routes that never awaited** (sync Firestore/GCS/email/Gemini on the loop, e.g. magic-link verify). **Fix:** they're now plain `def`, so they run in FastAPI's threadpool. The new AST guard `test_no_blocking_async_routes.py` fails CI on new ones. It also flags sync routes that call `asyncio.create_task`/`get_running_loop`; that caught `confirm_duration` and `trigger_youtube_queue_processing`, which stay async. ### Safety net - `backend/services/loop_watchdog.py`: a heartbeat plus a daemon thread. Any stall over 1s logs `EVENT_LOOP_STALL` with the loop thread's stack and a `culprit=file:line`. Stalls of 5s or more are written to `client_events` as `server_loop_stall`. - Pulumi: the log metric `backend/event_loop_stalls` plus an alert on ≥2 stalls of ≥10s in 30 min. **This still needs a local `pulumi up`.** Pulumi is never applied by CI, and the admin gcloud login has expired. ### Telemetry gaps fixed - **Identity:** `user_email` used to be hard-coded to null. The frontend now sends its session token, and the server resolves `user_email`, `is_admin`, `is_internal` and `is_test`. A client-sent email is ignored. - **New fields:** `device_fingerprint` (the existing FingerprintJS visitorId), `tab_id`, `episode_id`, and `tenant`. - **New event:** `banner_recovered`, carrying `duration_ms` and `peak_status`. - **Report:** `scripts/client_events_report.py` summarises the data, excluding admin/internal/test accounts by default. ## Testing - `make test` passes: backend 5529, plus the emulator suite, karaoke_gen unit tests, and frontend Jest 1623. - New tests: - Watchdog: a deterministic state machine plus detection of a real blocking call. - Event-loop responsiveness: a ticker keeps ticking while the heavy steps run. - Screens inline and failure semantics. - `client_events` identity and fields. - The AST guard. - The report summary. - Frontend reporter and banner episode tests. - Local CodeRabbit CLI review: 0 findings. 🤖 Generated with [Claude Code](https://claude.com/claude-code) @coderabbitai ignore --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
4015b24857 |
feat(audio-search): title-matched best pick; Spotify track match is Recommended, not 'Limited sources' (v0.264.0) (#1122)
## Summary
Live kjbox night 2026-10-01: a singer's make-it for **Braxton Keith -
The Chair** got 10 Spotify results (3 were the right track) and no
torrents. Because Spotify counts as "not lossless", the search was
**tier 3 "Limited sources found"** with a wrong-track best pick ("I Own
This Bar"), so the singer pasted a YouTube link. That then hit the
YouTube bot-check.
- `getBestResult(results, searchTitle)`: when any torrent/Spotify result
is a **confirmed** track-name match, only those compete. YouTube is
never promoted this way. Ties go to seeders, then popularity
(`view_count`). If nothing in the pool ranks, the fallback stays within
the matches.
- `isConfirmedTitleMatch`: the track filename must actually be compared
and equal to the title, ignoring only trailing version suffixes (`(feat.
…)`, `[…]`, ` - 2013 Remaster`). "The Chairman" ≠ "The Chair"; "7
Rings", "Mr. Brightside" and "(I Can't Get No) Satisfaction" work.
- `checkFilenameMismatch` now strips only real audio extensions. Before,
"Mr. Brightside" lost ". Brightside" as an "extension".
- `getSearchConfidence`: a title-matched Spotify best pick is **tier 2**
(Recommended), with no "No lossless sources" warning and a "Spotify
audio" reason.
kjbox ports this 1:1 in `static-sing/audio_rank.js` (separate PR).
## Test plan
- [x] `audio-search-utils.test.ts`: 10 new tests from the real Braxton
Keith results plus edge cases (86 total)
- [x] Full frontend jest: 1617 passed
- [x] Local CodeRabbit: 3 cycles, all findings fixed
@coderabbitai ignore
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
v0.264.0
|
||
|
|
7ea86a8dda |
test(e2e): self-heal Stripe Checkout (fillPostalCode) (#1119)
## 🩹 Stripe Checkout self-heal Stripe changed its hosted checkout page. In [run #197](https://github.com/nomadkaraoke/karaoke-gen/actions/runs/37140934446) every deterministic variant of **fillPostalCode** failed; the Gemini fallback completed the step(s), and the purchase was then **verified server-side** (credit granted). This PR only adds the recipe(s) to `learned-variants.json`, so future runs replay them deterministically (no LLM). It auto-merges once CI passes. ### Learned recipes ```json [ { "stepId": "fillPostalCode", "model": "gemini-3.5-flash-lite", "llmCalls": 1, "actions": [ { "type": "fill", "target": { "by": "css", "css": "#billingPostalCode", "frameUrlIncludes": "checkout.stripe.com/c/pay" }, "value": "{{POSTAL_CODE}}" } ] } ] ``` @coderabbitai ignore 🤖 Generated by the E2E Daily self-healing workflow Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
c8461e56d0 |
fix(e2e): self-heal CI approval — correct pending detection, re-approve open PRs (#1121)
## Summary This fixes the approval step from #1118, found by the live test on run 37140934446. That run reported `approved=true`, but bot PR #1119's CI run was still `action_required`: the old check treated *any* existing `pull_request` run as already running. - `approve_ci` now succeeds only once a `pull_request` CI run exists that is past `action_required`. It approves pending runs as it finds them. - When a self-heal PR is already open, the job now runs `approve_ci` for that PR instead of exiting. So a stuck bot PR gets unstuck on the next run. - Polling is tolerant of transient `gh`/`jq` failures (from the local CodeRabbit review). An empty branch name fails fast. ## Testing - `actionlint` is clean for the changed lines; local CodeRabbit review: 2 findings, both fixed. - After merge: a forced-LLM run on `main` will hit the "PR already open" path for #1119. That approves its CI with the workflow's own token and should let it auto-merge with no manual input. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
1c9104a79d |
fix(e2e): self-heal job approves its PR's CI run (no manual step) (#1118)
## Summary This follows #1116. The live test of the self-heal loop (bot PR #1117) merged, but only after one manual step: approving its CI run. - **Approve the bot PR's CI run.** A bot-authored PR's `pull_request` CI run is created as **action_required** (waiting for approval), and until it runs it outranks the passing CI Gate from the dispatched run. The `self-heal-pr` job now approves that run via `POST /actions/runs/{id}/approve`. If approval fails, it dispatches `ci.yml` and the Discord alert says the PR needs a one-click approval, with a link. - **Ruleset dependency, now documented.** `require_extra_approval_for_unattributed_changes` is now set to `false` on the `protect main` ruleset (11423976). It's a newer GitHub setting that defaults to true and blocks bot-authored PRs until a human approves. This is recorded in `docs/TESTING.md` and `docs/LESSONS-LEARNED.md`. ## Testing - `actionlint` is clean for the changed lines. - After merge: a forced-LLM run on `main` on a step with no learned recipe yet (`fillPostalCode`) will open a fresh bot PR and exercise approval → CI → auto-merge without manual input. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
84f709ea0e |
test(e2e): self-heal Stripe Checkout (selectCard) (#1117)
## 🩹 Stripe Checkout self-heal Stripe changed its hosted checkout page. In [run #196](https://github.com/nomadkaraoke/karaoke-gen/actions/runs/37139075045) every deterministic variant of **selectCard** failed; the Gemini fallback completed the step(s), and the purchase was then **verified server-side** (credit granted). This PR only adds the recipe(s) to `learned-variants.json`, so future runs replay them deterministically (no LLM). It auto-merges once CI passes. ### Learned recipes ```json [ { "stepId": "selectCard", "model": "gemini-3.1-flash-lite", "llmCalls": 3, "actions": [ { "type": "click", "target": { "by": "css", "css": "[data-testid='card-accordion-item']", "frameUrlIncludes": "checkout.stripe.com/c/pay" } } ] } ] ``` @coderabbitai ignore 🤖 Generated by the E2E Daily self-healing workflow Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
4768164a94 |
feat(e2e): self-healing Stripe Checkout (LLM fallback + auto-merging fix PRs) (#1116)
## Summary Stripe's hosted checkout page has broken the daily E2E **5 times since April** (#916, #919, #945, #1115…). Each fix added another selector, and each new Stripe change still needed a human. This PR makes the Stripe stage repair itself, with no manual steps. **How it works.** `stripe-checkout.ts` now runs the checkout as verified steps (`selectCard`, `fillCardNumber`, `fillCardExpiry`, `fillCardCvc`, `fillCardholderName`, `fillPostalCode`, `uncheckSaveInfo`, `submitPayment`). Each step tries, in order: 1. **learned variants** (`learned-variants.json`, newest first), 2. the existing hand-written **built-in** selectors, 3. a **Gemini fallback** that drives the page one action at a time until the step's check passes. Model list: `gemini-3.5-flash-lite` → `gemini-3.1-flash-lite` → `gemini-3.8-flash`; IDs that aren't available are skipped. A step only counts as done when its **check** passes, never just because a click didn't throw (the root cause of run #192's misleading failure). **Fully self-healing loop.** When the LLM repairs a step *and* the purchase is confirmed server-side (credit granted), the new `self-heal-pr` job (on `main` only): - opens a **data-only** PR that changes only `learned-variants.json` (label `e2e-self-heal`, `@coderabbitai ignore`), - enables auto-merge, - dispatches `ci.yml` on the branch, because PRs opened with the workflow token don't trigger CI. The next run replays the recipe without the LLM. Discord and the daily email report 🩹 when this happens, and only one self-heal PR is open at a time. **Safety** - Card data never reaches the LLM. It may only use per-step `{{CARD_NUMBER}}`-style placeholders, which are substituted locally. Field values and card-number-like digit runs are redacted from what it sees, and inputs are masked in screenshots. Tests assert no secret appears in the prompt. - Guardrails refuse clicks on Link/Klarna/Cash App/Bank/"Back to…", and on the final Pay button outside `submitPayment`. They check the **resolved element** (text, aria-label, test ID, submit type), not just the model's label. - The Pay step's check is "redirected to the success page OR credit granted server-side" (throttled), so the fallback never submits twice. The cancel URL doesn't count as success. **Gemini key:** the AI Studio `gemini-api-key`, billed outside the nomadkaraoke project. A `GEMINI_API_KEY` repo secret is used if set; otherwise it's read from Secret Manager via keyless login, and the credential file is deleted before the tests run. With no key, the test runs deterministic-only, as before. Also passes the existing `E2E_STRIPE_ZIP` secret, which was never wired up. ## Testing - **Offline:** `playwright.selfheal.config.ts`, 21 tests with fixture Stripe pages served on fake checkout.stripe.com/js.stripe.com and a stub LLM. Covers: known layout uses no LLM; unknown layout → LLM repairs it → recipe learned → replayed without the LLM; no secrets reach the LLM; guardrails, including CSS targets; deterministic-only failure mode; unit tests for redaction, merging and frame keys. Added to CI's smoke job. - **Live (real Stripe, real Gemini) on this branch,** with `self_heal_force_llm=selectCard`: - [run 37137960148](https://github.com/nomadkaraoke/karaoke-gen/actions/runs/37137960148): `gemini-3.5-flash-lite` completed "choose Card" in one call, credit granted 1→2. This exposed the session ID in Stripe's URL path, now fixed. - [run 37138459027](https://github.com/nomadkaraoke/karaoke-gen/actions/runs/37138459027): repeated after the review fixes. Recipe `[data-testid="card-accordion-item"]` @ `checkout.stripe.com/c/pay`, credit granted. - The PR-opening job is skipped on branches by design. - After merge, I'll dispatch once on `main` with `self_heal_force_llm=selectCard` to exercise the PR → CI → auto-merge path. - Local CodeRabbit review: 2 findings, both fixed. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
2b68cca183 |
fix(e2e): expand Stripe Checkout card accordion (daily E2E run #192) (#1115)
## Summary Daily E2E Stage 1 failed (run #192): `Could not find card field: cardNumber in any frame`. **Root cause:** Stripe changed its Checkout payment-method accordion. The Card row is now a plain `Card` text label with a visually-hidden `button "Pay with card"`. There is no longer a radio/button named `Card`, so the helper never expanded the accordion. This only affects the test. Customers can still pay. (Runs #190/#191 on Oct 1-2 failed for a different, already-fixed reason: the Postmark 403 outage, so the magic-link email was never sent.) ## Changes - `frontend/e2e/helpers/stripe-checkout.ts`: the chooser now matches `Card` / `Pay with card` (radio, button, or exact label text). If the fields still don't appear, it dispatches a click on the hidden `Pay with card` button. On failure it writes `test-results/stripe-checkout-dom.html` so the next markup change can be diagnosed from the artifacts. ## Testing - Type-checks; `playwright --list` loads the spec. - Not run end to end yet: that needs a live Stripe charge ($0.50). Verify with a `workflow_dispatch` of *E2E Daily Test* on this branch, or the next scheduled run. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Automated checkout checks now recognize additional card-selection options, including card choices presented as buttons or radio options, and retry when card fields are slow to appear. If the fields remain unavailable, the checks save diagnostic details from the checkout frames to help investigate the issue. These updates affect the checkout test process; no changes to the customer-facing payment flow are described. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
d6972f10aa |
infra(monitoring): alert when Postmark SMTP-fallback sends fail + document static-IP options (#1114)
## Summary This is a follow-up to #1112/#1113 and decide#147. Postmark confirmed (ticket #11562402) that the nginx 403s are an IP-reputation block on shared Google us-central1 egress IPs, and they won't lift it. The SMTP fallback is working in prod: on 2026-10-02 between 19:10 and 20:17 UTC, 12 of 12 blocked sends were delivered with 0 failures. To save money, we're **deferring** a static egress IP and adding a free tripwire instead. ## Changes - `infrastructure/modules/monitoring.py`: `create_email_delivery_observability()` adds: - A log-based metric `email/postmark_smtp_fallback_failures`, plus an alert policy ("Email - Postmark SMTP fallback failing (emails being lost)") that fires on any failed fallback send in karaoke-backend **or** karaoke-decide. - The filter matches **both** `jsonPayload.message` (gen logs structured JSON) and `textPayload` (decide logs plain text). - `docs/archive/2026-10-02-postmark-ip-block-options.md`: full write-up covering the evidence, Postmark's answer, how the fallback works, the alert, and the static-IP options. It includes **measured costs**: reusing the existing `github-runners-nat` with a reserved IP is ~$4–7/mo. It also has Pulumi steps, NAT sizing, and how to verify the reserved IP is used. - `docs/TROUBLESHOOTING.md` and `infrastructure/monitoring/README.md` link to it. ## Testing / rollout - Filter validated against real logs: - 0 failures, which is correct. - Positive control on the gen JSON branch: 12/12 real fallback sends matched. - Positive control on the decide text branch: 5/5 real lines matched. - `pulumi preview` showed: +2 to create, 274 unchanged. - **Already applied locally with `pulumi up`** (repo rule). The first apply hit the log-metric propagation race, and re-running succeeded; that gotcha is now documented in the code. The alert is live, enabled, and has the email channel attached. - Cost: $0 now. Alert policies cost $0.35/mo from 2027-09-01. Local CodeRabbit review: 2 findings (Cloud NAT sizing, reserved-IP verification in the options doc), both addressed. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
5181e42b46 |
fix(email): failed SMTP QUIT after an accepted send doesn't fail the send (v0.263.2) (#1113)
## Summary This is a follow-up to #1112. CodeRabbit found the issue while reviewing the same fix in karaoke-decide. If Postmark accepts the message but the SMTP `QUIT` fails during the `with` block's exit, `_send_via_smtp` reported a failure, and a caller retry could then send the email twice. Cleanup errors are now swallowed, falling back to `close()`, and only errors from the send itself fail the send. ## Testing - Added `test_quit_failure_after_send_still_reports_success`. Email tests: 72 passed. The same change was reviewed by local CodeRabbit in karaoke-decide#147. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>v0.263.2 |
||
|
|
eb1c914e4a |
fix(email): fall back to Postmark SMTP when the API edge returns HTML 403 (v0.263.1) (#1112)
## Summary Since 2026-10-02 11:25 UTC **every** backend email has failed: magic links, made-for-you order confirmations and admin alerts (e.g. job `18bee12c`). Postmark's API edge answers our us-central1 Cloud Run egress IPs with a bare nginx HTML 403. The retries added in v0.261.1 never get past it, because a block lasts hours. - Postmark IP Allowlisting is **off** (checked in the UI at account and server level). karaoke-decide (us-central1) gets the same 403s, but us-east4 jobs and residential IPs don't. That points to a network-level block on Postmark's side. Support ticket **#11562402** is filed. - Postmark SMTP is served by separate infrastructure and is already enabled on the server. ## Changes - `PostmarkEmailProvider`: on an HTML 403 (edge block), go straight to SMTP (`smtp.postmarkapp.com:587`, STARTTLS, server token as username and password, `X-PM-Message-Stream: outbound`). The API retry was dropped for this case; it only added about 4s of latency. - Also fall back to SMTP when every API attempt hits a connect timeout (the message never reached Postmark, so no duplicate risk). - Unchanged: 429/5xx are still retried on the API. JSON Postmark errors and read timeouts never fall back. - SMTP sends return no MessageID and clear `last_message_id`. Header-injection `ValueError`s are caught. - `docs/TROUBLESHOOTING.md` runbook updated. Version bumped to 0.263.1. ## Testing - 71 email unit tests, plus `make test-backend` (unit + emulator) passing. - **A real send through `_send_via_smtp` from a local machine** to andrew@nomadkaraoke.com shows as `Sent` on the outbound stream in Postmark's message API, and the Unicode subject came through intact. - After deploy: trigger a magic-link email and confirm `via Postmark SMTP fallback` in the backend logs. Local CodeRabbit review: 1 finding (header-injection ValueError), fixed. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>v0.263.1 |
||
|
|
b90ad82d90 |
docs: record Vertex and Gemini API shutdown in nomadkaraoke (#1111)
Records the manual (non-Pulumi) infra changes from the Gemini migration (#1108): Vertex IAM removal, the cross-project Aquarius grant removal, disabling aiplatform + generativelanguage in nomadkaraoke, deleting the unused restricted API key, and how to rotate `gemini-api-key`. Docs only, so no version bump. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated deployment guidance to record recent API access and credential changes, including disabled services and removal of an unused restricted key. The guidance also describes the associated project and billing setup, with steps for re-enabling services or rotating credentials. No end-user product behavior changes are included. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
0fa10e4330 |
feat(frontend): friendly cold-start waking state and backend pre-warm (v0.263.0) (#1110)
## Summary `karaoke-backend` now runs with min-instances 0, so a request that hits a cold backend waits ~15-20s (measured 18.4s) while Cloud Run boots an instance. Cloud Run queues requests during that window, so the health probe times out exactly as it does during a recycle, and users saw the amber "Reconnecting…" / "having trouble reaching our servers" banner. This replaces that with a calm cold-start experience and pre-warms the backend. ### Behaviour (`frontend/lib/backend-status.ts`) - New `waking` status. **Cold-start rule:** if no backend response succeeded in the `REACHABLE_FRESH_MS` (10 min) before the oldest stalled read began (fresh page load, or a tab idle long enough for scale-to-zero), a stall is treated as a cold start: - probe `/api/health` after `COLD_PROBE_AFTER_MS` (1s) instead of 10s, so a normal warm load (reads finish in <1s) never sends an extra probe; - once the read is ≥3s old **and** the probe failed (4s timeout), show the waking card (~5s into a cold start); - it clears the instant reads complete. - **Escalation:** past `WAKING_ESCALATE_MS` (60s) the normal reconnecting/unavailable logic takes over. A stall that starts within 10 min of a successful response is a recycle/outage, so the existing banner applies unchanged. - The probe still gates everything: a warm backend with a merely slow endpoint shows nothing. - `apiFetch` marks the backend reachable on every non-transient response, for any method. GET hard timeout (45s) and retry logic are unchanged and already ride out a ~20-30s cold start (the queued request simply completes). ### Pre-warm - `prewarmBackend()`: an untracked, fire-and-forget `GET /api/health` (`keepalive`, `cache: no-store`, no custom headers, so no CORS preflight). It runs on app load via the global `BackendStatusBanner` (all `[locale]` pages) and the admin layout, and again when a tab becomes visible after >10 min hidden. It's skipped if the backend answered in the last 10 min. Success marks the backend reachable. - Companion PR nomadkaraoke/public-website#31 pre-warms from the marketing homepage Hero (no-cors, idle-time). ### UI - Waking card: neutral card colours plus a tenant-primary spinner. "Starting up our karaoke servers…" / "This usually takes about 20 seconds. Thanks for your patience!". No warning colours and no dismiss button. - New `banner_waking` degradation event, added to the frontend type and the backend `client-events` allow-list. It includes `since_reachable_ms`. - Strings translated for all 33 locales (Gemini Developer API path from #1108). ## Testing - Store unit tests with fake timers. Cold start → waking (~5s) → clears. Fast cold read shows nothing. Warm slow endpoint shows nothing. Prolonged stall → `unavailable` after 60s. Previously reachable then stall → `reconnecting`. Reachable >10 min ago → waking. Pre-warm de-dupe, failure and visibility re-warm are covered. - apiFetch tests: a fresh session hitting a cold backend shows `waking` (never the warning) and the GET completes at 18s. The pre-warm request shape (no headers, keepalive) is checked. - Backend: `banner_waking` accepted by `/api/client-events`. - `make test` passes. - Real browser check: `next dev` against a local stub that holds every request for 18s from the first hit. The waking card appears by ~7s, the dashboard shows "Loading jobs…" (no error or empty state) and everything resolves at 18.3s. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a startup status banner for cold starts, with an estimate that startup usually takes about 20 seconds. The banner is available in multiple languages. * The app now checks backend availability when needed and after a tab has been hidden for an extended period, helping requests recover from cold starts. * **Documentation** * Updated API telemetry documentation to include the new startup event. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.263.0 |
||
|
|
8490ab75eb |
chore(gemini): move every Gemini call from Vertex AI to the Gemini Developer API (#1108)
## Summary Moves **every** Gemini call in gen off **Vertex AI** (billed to the `nomadkaraoke` GCP project, ~$85/mo) onto the **Gemini Developer API** using the `gemini-api-key` Secret Manager secret (an AI Studio key billed outside this project). Once deployed and verified, `aiplatform.googleapis.com` / `generativelanguage.googleapis.com` get disabled in nomadkaraoke. ### Key injection - `backend/services/gemini_client.get_genai_client(timeout_ms)` — key from `GEMINI_API_KEY`, else Secret Manager `gemini-api-key` via `settings.get_secret` (runtime SA already has project-level `secretAccessor`). No `project`/`location`. - Service: CI `--set-secrets ... GEMINI_API_KEY=gemini-api-key:latest`. - Jobs: Pulumi secret env on `lyrics-transcription-job`, `video-encoding-job` (renders + translations), `nomad-error-monitor` — **already applied** (targeted `pulumi up`, only that env added). Other jobs fall back to Secret Manager. - Local scripts (`frontend/scripts/translate.py`, `scripts/measure_autocorrect_cost.py`, `scripts/test_agentic_correction.py`): env, else `gcloud secrets versions access` (in-process only). - **Rotation:** add a new secret version; new instances / job executions pick it up. No code change. ### Degradation (quota / billing / invalid key; per-minute 429s stay retryable) | Caller | Behaviour | |---|---| | auto-correct | non-retryable 503 → compare mode drops the Gemini leg, keeps Opus | | match-judge / free-text / parse-titles / tenant bulk | existing fallbacks (no suggestion / blanks / regex) | | credit eval | `pending_review` (fail-closed, unchanged) | | lyrics translation | no retry; render proceeds without translation, status recorded | | custom lyrics | clean 503 "temporarily unavailable…" (+ 5-min timeout so it can't hang) | | error-monitor LLM | no retry; alerts sent without AI summary | | translate.py | exit 2 "Gemini quota/credit exhausted — top up AI Studio or rotate gemini-api-key" | `note_gemini_failure()` also sends one Discord ops alert per 6h (Firestore-throttled across instances via new `ops_alerts.send_throttled_ops_alert`). ### Infra - Pulumi: removes `roles/aiplatform.user` from `karaoke-backend` — **will be applied after this deploys and Vertex traffic is confirmed at zero**. - Agentic correction default model `gemini/gemini-3.8-flash` (`vertexai/` prefix still accepted, routed to the Developer API). - Version 0.261.1 → 0.262.0. Docs: CLAUDE.md, LESSONS-LEARNED, README, ARCHITECTURE, API, etc. ## Testing - New: `backend/tests/services/test_backend_gemini_client.py` (classifier, key resolution, no-Vertex client kwargs, alert throttle, per-caller degradation), `tests/unit/lyrics_transcriber/test_agentic_gemini_provider.py`, `frontend/scripts/test_gemini_client.py`, lyrics-translation quota test. - `pytest backend/tests --ignore=backend/tests/emulator` — all pass; `pytest tests/unit` — 2021 passed. - Verified `gemini-3.8-flash` is listed on the Developer API and answered a 5-token test call with the key. - `translate.py --dry-run` on `frontend/messages` and `backend/translations` OK. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Updates** * AI-powered lyric translation, custom lyrics, auto-correction, title parsing, match judging, and related features now use the Gemini Developer API. * AI features use the configured Gemini API key. Quota, billing, or key issues may show an unavailable message or use an existing fallback instead of repeated attempts. * **Documentation** * Updated product, API, setup, and recovery guidance to reflect Gemini API key configuration and usage. * Updated guidance for handling AI service availability and quota issues. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.262.0 |
||
|
|
49464e8fcc |
chore(cost): GCP cost cuts round 2 (v0.261.4) (#1107)
Round 2 of the GCP cost cuts, approved by Andrew. Target: the whole project under $300/mo; the current run-rate is about $400. ## Changes - **karaoke-backend min-instances 1 → 0** (`ci.yml`), about $24/mo. The backend Cloud Run service isn't defined in Pulumi, so the CI flag is the only place this is set. Andrew accepts a ~15s cold start. The 60s `/api/health` uptime check and the `*/10` schedulers keep an instance warm most of the time. With `--cpu-throttling` (request-based billing), a warm instance above the minimum costs nothing while idle. - **audio-separator Cloud Run GPU service shut down** (us-east4), about $22/mo. It is now behind the Pulumi flag `audioSeparatorServiceEnabled`, off by default. The service, SA, IAM bindings, output bucket and AR repo were already removed with a targeted `pulumi up`; the bucket needed the break-glass SA. `audio-separation-job` is untouched; it uses `karaoke-backend-gpu/karaoke-backend`. The separator health check no longer depends on `AUDIO_SEPARATOR_API_URL`, which also fixes a status card that always showed "offline". The CLI now logs a clear hint when the remote API fails. Companion PR: nomadkaraoke/python-audio-separator#309 makes its Cloud Run deploy manual-only. - **backup-to-aws no longer copies job files to S3**, about $15/mo egress. Job files now rely on GCS versioning plus soft-delete. Firestore, secrets, BigQuery, kn-data and git-bundle backups are unchanged. The freshness monitor no longer checks `gcs/job-files/`, and DISASTER-RECOVERY.md is updated. - **Artifact Registry cleanup policies**, about $8–10/mo. Per package: keep the 10 most recent versions plus anything tagged `latest`, `content-` or `cache`; delete all other versions older than 7d. Until now, tagged per-commit images were never deleted. Dropped the CI fallback that copied the GPU base once from us-east4, and deleted the legacy 13 GB us-east4 GPU base package. - Docs: GCP-COST-OPTIMIZATION.md (new section, including how to redeploy the separator on demand), ARCHITECTURE.md, DISASTER-RECOVERY.md. Changes made outside Pulumi (gcf-artifacts cleanup policy, destroying superseded secret versions) are recorded in GCP-COST-OPTIMIZATION.md. ## Tests - `backend/tests/test_system_status_endpoint.py` (new separator status tests) - `infrastructure/functions/backup_to_aws` (49 tests, including a new one that checks job files aren't synced) - `tests/unit/test_audio_remote.py`, `test_audio_processor_local_gpu.py` 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Service Availability** * The hosted audio-separation service is no longer available. Audio separation can run locally; users relying on the hosted service will need another deployment. * **Backup and Recovery** * New job files are no longer copied to off-site S3 backups. S3 recovery is limited to files backed up through September 30, 2026; newer outputs may need to be regenerated. * **System Status** * Audio-separator health status now reports the locally installed package version without checking a remote service. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.261.4 |
||
|
|
e4dc50d9e3 |
fix(ci): sanitize PR title in deploy env vars (v0.261.3) (#1109)
The v0.261.2 main deploy (run 36945703908) failed at **Deploy to Cloud Run (candidate revision)** with: `argument --set-env-vars: Bad syntax for dict arg` The #1106 PR title contained a comma, and `PR_TITLE` is spliced into the comma-delimited `--set-env-vars`. The bug was already there and that title was the first to trigger it. Its Cloud Build step succeeded. Fix: in both build-meta steps, commas become `;`, and newlines and shell-active characters (`\" \` $ \\`) are removed. Version bumped to 0.261.3. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.261.3 |
||
|
|
dd795c66a0 |
ci(cost): build backend images in Cloud Build, run deploy-backend on ubuntu-latest (#1106)
## Summary Retires the self-hosted `gha-build-*` runner VMs (e2-standard-8, `docker-build` label, ~$21/mo). `deploy-backend` now runs on free `ubuntu-latest`; the Docker images are built by **Cloud Build in us-central1**. ## Why not build on the hosted runner? (egress math) The app images are built `FROM` bases in us-central1 Artifact Registry, and the app layer has a `RUN pip install`, so the full base must be pulled each deploy: - CPU base `karaoke-backend-base`: **4.22 GB** compressed - GPU base `karaoke-backend-gpu-base`: **13.1 GB** compressed - ~120 main deploys/month (119 in Sept, 11 so far today) On a GitHub-hosted runner that is GCP internet egress: 17.3 GB × 120 × $0.12/GB ≈ **$250/mo** (CPU-only would still be ≈ $61/mo). Both are worse than the $21 VM. Cloud Build in us-central1 pulls same-region (free). Build-minutes on the default-pool e2-standard-2 machine are in the **2,500 min/month free tier** ($0.006/min after that). A test build of the real images (CPU and GPU in parallel) took **6m55s**, about the same as today's ~7.3 min of sequential builds on the VM. That's ~840 min/mo, so the expected cost is ~$0 (worst case ~$5/mo). ## Changes - `infrastructure/cloudbuild/backend-images.yaml` + `build-backend-images.sh`: buildx (docker-container) builds with the **same tags and registry caches** as before. CPU and GPU build as parallel steps, and each build gets a fresh metadata-server token. - `ci.yml` `deploy-backend`: `runs-on: ubuntu-latest`. It keeps the hash checks (manifest + ~10 KB config blob only) and submits `gcloud builds submit --async`, then streams the logs and checks the status. - Base rebuilds (≈ monthly) use e2-highcpu-8 with 300 GB disk and a 2 h timeout. - A Cloud Build still running when the job is cancelled or fails is cancelled. CI concurrency cancels superseded deploys, and an orphaned build would otherwise still push `:latest`. - Removed the disk-space step, the local base pull, and the one-time us-east4→us-central1 GPU-base copy. The copy finished 2026-09-26, and on a hosted runner it would stream 13 GB through egress. - `build-runner-images.yml`: `variants=all` no longer bakes the unused `build` family. It can still be built with an explicit `variants=build`. - Docs: EPHEMERAL-GHA-RUNNERS, SELF-HOSTED-RUNNERS, GCP-COST-OPTIMIZATION. No Pulumi or IAM changes were needed. `github-actions-deployer` already has `cloudbuild.builds.editor`, `storage.admin` and `iam.serviceAccountUser`, and the default Cloud Build SA (compute default) already has `artifactregistry.writer` on both repos. ## Rollback (one commit) `git revert <squash-merge sha of this PR>` and push to main. That restores `runs-on: [self-hosted, linux, gcp, docker-build]` and the old in-job buildx steps. Nothing outside the repo was changed: the runner-manager `build` family is untouched, and the `gha-runner-build` GCE images (20260812, 20260901) are kept so the dispatcher can still launch `gha-build-*` VMs. If the build image has been pruned by then, re-bake it with Actions → Build GHA Runner Images → `variants=build` (only `variants=all` no longer includes it). ## Testing - I ran the real CPU and GPU builds from this branch's config in Cloud Build us-central1, with `cbtest-*` tags and no cache writes. Both succeeded in 6m55s, and the us-east4 push uploaded only the app layers. - `actionlint` and `shellcheck` report no new findings. - Full verification is the first main deploy after merge. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * Backend image builds now run through a managed build service in the central US region, with CPU and GPU images published as part of the deployment workflow. * The default “all variants” selection now excludes the build variant; explicitly requested variants remain available. * **Documentation** * Updated runner and cost documentation to reflect the current build and deployment setup. * **Maintenance** * Updated the package version to 0.261.2. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.261.2 |
||
|
|
17d60c58a0 |
fix(cost): low Gemini thinking for simple callers + dedupe proactive auto-correct (#1105)
## Summary
GCP cost cut: Vertex Gemini 3.8 Flash is now the largest variable cost
(~$85-90/mo), mostly thinking tokens.
### 1. `thinking_level="low"` for simple extraction/classification
callers
| Caller | Model setting | Before | After |
|---|---|---|---|
| `match_judge/ai.py`, `match_judge/free_text.py` | `match_judge_model`
| default (high/dynamic) | low |
| `tenant_bulk/analyze.py` (filename pairing) | `match_judge_model` |
default | low |
| `parse_titles/ai.py` | `parse_titles_model` | default | low |
| `credit_evaluation_service.py` (grant/deny) | `credit_eval_model` |
default | low |
| `frontend/scripts/translate.py` | n/a | medium | low |
**Unchanged on purpose:** auto-correct (quality eval), custom lyrics,
lyrics translation, and agentic correction (`agentic_ai_model` is
dormant because `SKIP_CORRECTION=true`; its router makes no LLM call).
Timeouts: low thinking only makes these calls faster, so the existing
deadlines (`match_judge_timeout_ms` 12s, `parse_titles_timeout_ms` 20s)
need no change.
**Live check (Vertex, ADC):** 8 match-judge cases plus a 4-file
parse-titles batch, default vs low. Same canonical artist/title on every
case and identical parses. One verdict changed: `beatles/hey jude` went
from content to cosmetic, and cosmetic is the correct label for a
casing/"The" fix. Thinking tokens dropped from 1519 to 167, and max
latency from 5.5s to 2.7s.
**translate.py A/B:** 5 ICU-plural/placeholder strings into es/de/ja/ar.
Low used 0 thinking tokens and medium used ~33k. Both preserved all
placeholders and produced equivalent quality.
### 2. Duplicate proactive auto-correct (job 96cf1100 and about 55% of
jobs)
Root cause: the lyrics worker calls `advance_to_screens_if_ready`
*before* `_trigger_proactive_auto_correct`. The screens worker's
`ensure_and_pre_apply` checks the suggestion cache while the proactive
run is still in flight, gets a miss, and starts a second identical
multi-model run (Claude Opus plus Gemini). In job 96cf1100 both runs
were on the same instance and finished 4s apart. From prod logs for
2026-09-17 to 10-01: **92 of 167 jobs generated twice** (9 of them
across instances), at about $0.07 per duplicate.
Fix (`backend/workers/auto_correct_worker.py`):
- In-process single-flight per job. A concurrent caller awaits the
in-flight task, shielded so it can't cancel it.
- Cross-instance GCS lease
(`jobs/{id}/lyrics/auto_correct_inflight.json`) with generation fencing
(create-only; stale takeover and release only against the observed/own
generation; owner token; 240s TTL; timed-out run keeps its lease). A
non-holder polls off the event loop for the holder's cache for up to
150s instead of regenerating. If the holder fails, the UI's existing
on-demand fallback takes over.
## Tests
- `backend/tests/test_proactive_auto_correct.py`: concurrent calls share
one run (fails on old code), waiting on a peer's lease, taking over a
stale lease, and not regenerating after a peer fails.
- `backend/tests/test_gemini_thinking_level.py`: each changed caller
sends `thinking_level=low`.
- Related suites pass locally. `test_internal_api.py` has 7 failures
locally that also fail on `origin/main` (local env).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* AI-powered features now use a lower thinking level for tasks such as
title parsing, match evaluation, credit evaluation, bulk analysis, and
translation.
* Proactive auto-correct now coordinates concurrent suggestion requests,
reuses completed suggestions, and avoids starting a duplicate run when
another request is already in progress.
* If another request ends without producing suggestions, auto-correct
skips the request rather than immediately generating them again.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
||
|
|
7e51ba7cc1 |
fix(email): retry transient Postmark failures; diagnose missing E2E emails (v0.261.1) (#1104)
## Summary E2E Daily #190 (2026-10-01) failed in Stage 1 because Postmark rejected all 3 magic-link sends with a bare **nginx HTML 403**. That's an edge/IP-level block, not Postmark's JSON error format. Since 2026-09-30 05:26 UTC there have been 11 such failures from `karaoke-backend` (us-central1), in bursts. They also dropped real users' magic links and a lyrics reminder. `video-encoding-job` (us-east4) was never affected. ## Changes - **`PostmarkEmailProvider` retry**: 429, 5xx, non-JSON 403 and connect timeouts are retried with backoff (1s, 3s; 3 attempts total). - JSON Postmark errors (bad token, invalid request, suppressed recipient) are not retried: they're deterministic. - Read timeouts are not retried: Postmark may already have accepted the email and has no idempotency key, so a retry could send it twice. - **E2E `waitForEmail` diagnosis**: on timeout, it queries `GET /api/admin/users/{email}/emails` and reports either "backend has NO record of sending" (a server-side send failure) or "backend sent N email(s) but none reached testmail.app". This replaces the misleading testmail `AbortError`. Applies to every spec that waits for an email. - `docs/TROUBLESHOOTING.md`: new section on Postmark HTML 403s. - Version 0.261.0 → 0.261.1. ## Testing - New `TestPostmarkRetry` unit tests: HTML 403 then success, giving up after 3 attempts, 429/5xx, no retry on JSON 403/422/406, connect timeout retried, read timeout not retried. - `make test` passes: backend 2017 + 5436 + 150; frontend Jest 1591. - Diagnosis helper: every branch checked with a stubbed `fetch`. Against the real prod endpoint, today's failed E2E address shows 0 emails and an address from a passing run shows 2. ## Not in this PR The underlying block is probably IP-based. If retries don't absorb it, the follow-ups are Postmark support and/or a static outgoing IP (Direct VPC egress + Cloud NAT). 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Email delivery now automatically retries certain temporary service and connection failures, with brief delays between attempts. * Email wait timeouts in end-to-end testing now include additional diagnostic details, and temporary polling errors no longer stop checks early. * **Documentation** * Added troubleshooting guidance for missing emails, including how to distinguish delivery blocks from authentication errors and what to check when retries do not resolve the issue. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>v0.261.1 |
||
|
|
d4fcab521c |
Translated lyrics follow-ups + review Waveforms 'separating' spinner (v0.261.0) (#1103)
## Summary Follow-ups from Andrew's test of translated lyrics (job 948b6760), plus a review-page UX gap found while investigating that job. - **Translation row is now secondary:** it renders at 70% of the unsung colour's opacity, with outline and shadow faded to match. A theme's `karaoke.translation_color` still wins. (`subtitles.py:TRANSLATION_OPACITY`) - **YouTube label:** published videos with translations get "(With Translation into X)" in the title, before "(Karaoke)". Only the artist/title get truncated, never the label. The description gets an up-front notice. This covers the live upload and the quota queue, and applies only when `state_data.lyrics_translation.status == "translated"`. Turning translation off and re-rendering clears the outcome. - **Review preview modal:** an always-visible note explains the preview is a quick black-background sync check, not the final look. Jobs with a translation language also get a note that translations are added after review. - **Waveforms toggle:** shows a spinner and tooltip while audio separation is still producing the vocal stem (peaks endpoint returns 202). Previously the rows just stayed empty. ## Testing - Jest: lyrics-review suite (245+) incl. new TranscriptionView/Preview/loader tests - pytest: youtube description/queue/orchestrator, lyrics_translation, translated-lyrics unit tests - Linux libass render tests (Debian + prod static ffmpeg) pass - Independent review agent: 2 findings fixed (Tooltip data-state clobbering toggle state; stale translation outcome) No local CodeRabbit run (CLI has no seat on this org), so leaving this open to the CodeRabbit bot. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * YouTube titles and descriptions now identify the language of translated lyrics. Titles are kept within the platform’s length limit. * Translated lyrics appear with softer opacity by default. * The lyric-review screen explains how the quick preview differs from the final video, including that translated lyrics are added after review. * A spinner and notice indicate when vocal waveforms are still being prepared. * **Bug Fixes** * Translation status now reflects when translation is turned off, preventing stale translated-language labels. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>v0.261.0 |
||
|
|
bc1fc774eb |
Translated lyrics: don't split Japanese/Chinese translations mid-word (v0.260.1) (#1102)
## Summary Follow-up to #1101, found while verifying it on prod (job ca2e881a, English song → Japanese): when a long lyric line is split across rows, its translation is shared between the pieces, and unspaced scripts were cut at an arbitrary character — e.g. 無|駄 ("waste") across two rows, worse in portrait. `split_text_proportionally` now scores each split point for unspaced scripts: free after punctuation (、。!?…), cheap after a Japanese particle (を は が に…) or before a kanji/katakana word start following hiragana, a full "part" of cost for a mid-word cut. It also never leaves a piece empty when there are enough characters. Before → after (real prod line): `ねえ、おかしいでしょ、無` / `駄な時間を省いてあげる` → `ねえ、おかしいでしょ、` / `無駄な時間を省いてあげる`. Thai can still split inside a word (no word-boundary data without a dictionary). ## Testing - New unit tests for the prod case and a particle-boundary case; all 201 lyrics output tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved how translated lyrics are split across rows for unspaced text. Splits now favor punctuation and natural language boundaries, helping avoid cutting through words where possible. * Thai text may still be split within words. * **Chores** * Updated the project version to 0.260.1. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.260.1 |
||
|
|
f8d0febcf5 |
Translated lyrics beneath the karaoke lyrics for language learners (v0.260.0) (#1101)
## Summary
New job-submission option: **Add translated lyrics** → pick a language
(33 UI locales) → the karaoke videos show an AI translation of each
lyric line in smaller text beneath the synced lyrics, so language
learners can sing songs in another language while learning what the
words mean.
- **Job creation**: toggle + language picker (default = UI language) on
the Customize step, with a **live preview** of a real rendered frame
(`GET /api/themes/translation-preview?language=xx`; pre-generated sample
translations, no LLM call at preview time). Any job, public or private,
free for now. `translation_language` accepted by every creation route;
admin can edit it.
- **Post-review translation**: right after the job enters
RENDERING_VIDEO, the render worker translates the *final reviewed*
lyrics line by line with Gemini
(`backend/services/lyrics_translation.py`, `LYRICS_TRANSLATION_MODEL`,
GCS-cached so theme/admin re-renders are free) into
`jobs/{id}/lyrics/translations.json`. Not editable. A failure never
blocks the render (renders without translations, recorded in
`state_data.lyrics_translation`); a song already in the target language
gets no row.
- **Rendering** (landscape + portrait, GCE worker + local fallback):
`LyricsSegment.translation`; when present, slots get taller and screens
show 3 lines; a `Karaoke.Translation` style (unsung colour, ~½ size)
draws one static event beneath each line, sanitized, wrapped/scaled to
fit. Jobs without translations render identically.
- **RTL + LTR together**: the translation is its own ASS event with auto
paragraph direction, so e.g. a Hebrew song keeps its right-to-left
highlight with an English translation beneath (and vice versa). Long
lines split by the resizer share their translation across the pieces
(grapheme-safe for Thai/CJK).
Design: `docs/archive/2026-10-01-translated-lyrics-plan.md`.
## Testing
- `make test` all green (root 2014, backend 5416, emulator 150, frontend
1583).
- New: `tests/unit/lyrics_transcriber/output/test_translated_lyrics.py`
(layout/positions/style/sanitizing/fit/mixed RTL-LTR/Thai/portrait),
`test_translated_lyrics_render.py` (libass pixel tests incl.
Hebrew/Arabic lyrics with English/Arabic translations and CJK),
`backend/tests/test_lyrics_translation.py`,
`backend/tests/test_render_video_worker_translations.py`, GCE endpoint
test, frontend CustomizeStep/GuidedJobFlow/api tests, prod E2E
`e2e/production/translated-lyrics.spec.ts`.
- `scripts/run-render-tests-linux.sh`: 135 passed on Debian ffmpeg AND
prod's static ffmpeg (no skips).
- Real-data check: job 5710831e (Hebrew) translated to English by Gemini
and rendered with the nomad theme — RTL highlight intact, English
beneath, split line shares translation.
Local CodeRabbit CLI unavailable in this workspace (no seat), so leaving
this PR for the CodeRabbit bot; an independent local review was done and
its findings fixed in the second commit.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an opt-in translated-lyrics option to job setup, with language
selection and a preview.
* Translations are generated after lyric review and appear beneath
lyrics in landscape and portrait videos, with text layout for multiple
languages and writing directions.
* Translation settings can be changed or cleared by admins.
* **Bug Fixes**
* If translation fails or is unavailable, video rendering continues
without translated lyrics.
* **Documentation**
* Updated API and product documentation with translated-lyrics setup and
behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
v0.260.0
|
||
|
|
b7a3864a37 |
Missing-lyrics insert: keep synced-reference timings inside the gap (v0.259.1) (#1099)
## Summary Follow-up to #1098, found while fixing job 5710831e. LRCLIB references are line-synced, and the last reference line in a gap can end **after** the next transcribed line starts. On 5710831e it ends at 33.07s while the next line starts at 32.40s. "Insert these lines" copied those timings verbatim, so the inserted words overlapped the following line. `fitTimedLinesToWindow` clamps each inserted timed line to the window between the neighbouring lines' words, with a 0.05s pad: - A line that fits keeps its exact reference timing. - An overflowing line is compressed affinely, keeping its word proportions. - A line with no room left falls back to **untimed**, so the existing submit guard forces a Tap To Sync. ## Testing - 4 new unit tests, including a 5710831e regression using the real numbers: inserted words never overlap the next line, lines that fit keep exact timing, push-after-previous, the untimed fallback, and untimed lines passing through unchanged. - Jest: 1561/1561. - Playwright missing-lyrics scenarios: 4/4. No local CodeRabbit review (the CLI has no seat for this org), so this is left open for the CodeRabbit bot. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Timed lyrics inserted into gaps now stay within the available interval and avoid overlapping neighboring lyrics. Timings are preserved when they already fit; overflowing lines are adjusted to fit when there is enough room. * If there isn’t enough room to assign valid timings, or the original timing has no duration, inserted lyrics are left untimed instead. Lyrics that are already untimed continue to pass through unchanged. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.259.1 |
||
|
|
08b574a86c |
Review UI: flag possible missing lyrics and insert the reference lines; mount sonner toasts (v0.259.0) (#1098)
## Summary Phase 2b of the missing-lyrics work (#1095 detector, #1097 gate). When the job's `vocal_gaps` analysis has an **evidenced** gap (the lead vocal is singing, there are no lyrics, and the reference lyrics place lines there), the lyrics review UI now shows: - **An amber callout:** "Possible missing lyrics at 0:20–0:32 — the lead vocal is singing here but there are no lyrics." It lists the expected lines and their source (synced reference lines first), with **Play** and **Insert these lines** buttons. - **A marker row** at the gap's position in Synced Lyrics (Simple, Advanced and Waveforms views). **Insert these lines:** - Adds one segment per reference line, at the right point in time. - If the gap falls inside a segment, that segment is split. - In duets, the singer follows Add Segment's rule. - It's one undo step, logged in the edit log with `origin: missing_lyrics_gap`. **Timing of inserted words:** - If a **synced reference** (e.g. LRCLIB) has the same line starting inside the gap, its word timings are used. - Otherwise the words go in **untimed**, so the existing "words have no timing" submit guard forces a Tap To Sync. Timings are never invented. - A re-sync hint lists inserted lines with **Sync timing** buttons that open the existing Edit modal / Tap To Sync. It accumulates across gaps. **When the marker hides:** - When words exist inside the gap, with a 0.05s edge tolerance for the backend's rounding. - When the analysis doesn't match the lyrics as loaded (stale guard). - For audio-only gaps, which are too noisy to show. - In read-only/replay mode, the marker shows but Insert is disabled, with an accessible explanation. - The displayed range and Play use the review's timing offset. **Also fixed (pre-existing bug):** sonner's `<Toaster>` was never mounted, so **every sonner toast was silently dropped**. That covers lyrics and instrumental review and the admin pages, and includes the untimed-words submit guard this feature relies on. It's now mounted in the root layout, and the guard message is translated (it was hardcoded English). ## Testing - Jest: 51 tests for the feature (gap detection with tolerance, ordering/split, untimed vs synced timing, submit guard counting, duet singer, offset, stale guard, read-only tooltip, memoisation, re-sync hint). - Playwright (`e2e/regression/lyrics-review.spec.ts`), 4 scenarios: untimed insert → pending sync → undo; Sync timing opens Tap To Sync; LRCLIB-timed insert closes the gap → undo; **blocked submit shows the toast** (verified to fail without the Toaster mount). - `make test`: frontend 1557/1557 plus all backend stages. - i18n: all new strings in 33 locales. - Two local code-review rounds; findings addressed. No local CodeRabbit review (the CLI has no seat for this org), so this is left open for the CodeRabbit bot. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Lyrics review now highlights vocal gaps that may be missing lyrics, with time ranges and available reference lines. * Insert suggested lines into the transcript, undo the change, and open inserted lyrics in the timing editor. Playback and timing-sync options help review each gap. * **Bug Fixes** * Video submission now warns when lyrics contain untimed words that need synchronization. * Improved accessibility labels for undo and redo controls. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.259.0 |
||
|
|
777daaf25f |
Missing-lyrics gate: block auto-approval when singing has no lyrics and the reference has lines there (v0.258.0) (#1097)
## Summary Phase 2a of the missing-lyrics work (phase 1: #1095, shadow detector + backfill). **Audit** (262 completed jobs, 60 days, server-side `scripts/audit_vocal_gaps.py`): - Longest unlyricked vocal run per job: p50 0.7s, p90 5.1s, p99 22.4s. 47 jobs (18%) have a run of 3s or more, far too many to gate on audio alone. - Cross-checked against **synced** reference lyrics (LRCLIB line timestamps): of 57 suspect gaps that have a synced reference, only 15 contain a reference line. Audio-only is ~25% precise. The rest are vocal chops, samples, ad-libs and synth bleed, mostly in drum & bass tracks. - **Evidence-based rule:** a gap is reliable when the reference lyrics place lines in it. **Changes** - `vocal_gaps` 0.2.0: records the synced reference lines inside each gap. `evidenced` = suspect vocal run AND reference lines, either synced or between the neighbouring anchors. - Scorer 0.5.0: new never-auto `missing-lyrics-gate` with a readable reason, e.g. *"possible missing lyrics: ~10s of singing at 0:20-0:32 has no transcribed words, and the reference lyrics have 3 line(s) there"*. Audio-only gaps don't gate. - The executor analyzes vocal gaps *before* scoring (off the event loop, fingerprinted cache) and passes them to both scoring passes. It fails open on pending audio or analysis errors. - `GET /api/review/{id}/correction-data` now includes `vocal_gaps`, for the upcoming review-UI marker (phase 2b). On job 5710831e, the synced LRCLIB lines inside the gap are exactly the three dropped lines. The genuine instrumentals have none. ## Testing - `test_vocal_gaps.py`: synced lines with edge tolerance; section headers ignored; evidence requires both singing and reference (instrumental with misplaced reference lines doesn't gate; singing without reference doesn't gate). - `test_scorer.py`: reasons only for evidenced gaps; the gate overrides confident lyrics; non-evidenced gaps don't gate. - `test_executor.py`: evidenced gaps block auto-approval with the reason and skip the timing check; cached evidenced gaps still gate. - `test_combined_review_flow.py`: `vocal_gaps` is included in correction-data. - Full backend suite: 5211 passed. Versioned 0.258.0 to follow #1096 (0.257.0). No local CodeRabbit review (the CLI has no seat for this org), so this is left open for the CodeRabbit bot. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.258.0 |
||
|
|
f89de165cc |
Admin: re-render any completed job without review (v0.257.0) (#1096)
## Summary
An admin-only **Re-render** button on completed jobs in the `/app` jobs
list. It regenerates title/end cards, the karaoke video, all encoded
formats, packaging and distribution from the job's existing reviewed
lyrics, instrumental selection and style snapshot, **with no
lyrics/instrumental review**. Motivating case: re-rendering
already-delivered videos after a renderer fix (RTL/Hebrew, v0.255.0).
`POST /api/admin/jobs/{job_id}/rerender` with body `{"notify_customer":
false}`.
**Behaviour (decided with Andrew)**
- Works for consumer and tenant jobs, public or private. Uses the job's
existing style; no theme re-snapshot.
- **Published outputs are deleted up front, like Edit (YouTube, GDrive,
Dropbox), but the brand code is kept, not recycled.** The pipeline
re-publishes under the same NOMAD-xxxx. YouTube gets a new upload and
therefore a new URL.
- Outputs are only deleted for destinations the re-render will actually
re-publish (`plan_republish` uses the same conditions as the video
worker). Anything else is kept, with a warning shown to the admin.
- **No customer email/push/Discord by default.** The confirm dialog has
an "Email the customer when done" checkbox. Already-notified
requests-board voters aren't re-emailed; voter emails still owed from
the original publish do go out.
- A failed admin re-render is set to FAILED with an in-progress marker
(scoped to `review_token`, so it can't leak into later runs). It can be
retried by **admins only**; `/retry` returns 403 to customers while the
marker is active.
- **Mutual exclusion with the YouTube upload queue:** the claim and the
queue's `mark_processing` are Firestore transactions that each read both
documents. A re-render is refused with 409 while an upload is in flight.
A queued or failed deferred upload is cancelled atomically when YouTube
will be re-published. The queue never starts an upload for a job that is
mid re-render.
- After a successful upload, the queue never re-queues an entry, so a
later error can't cause a duplicate upload.
- Admin re-render completions don't increment the customer's
completed-jobs count.
**Refactors**
- New `backend/services/published_outputs_cleanup.py`, shared by Edit,
admin delete-outputs and the re-render.
- The Dropbox folder delete now uses the uploader's sanitised name and
also tries the legacy raw name. This is a deliberate fix for titles with
special characters.
- Remaining duplicate copies (`jobs.py` ~2440,
`visibility_change_service`) are listed as follow-ups in the design doc.
## Testing
- 173 tests in `test_admin_rerender.py` +
`test_published_outputs_cleanup.py`, built through three local
code-review rounds. They cover:
- claim and validation
- retry gating
- marker scoping
- publish planning (each mismatch case)
- queue exclusion in both directions, no starvation, and that the
re-render's own later upload is never cancelled
- the no-re-queue-after-upload rule
- notification gating on both pipelines
- community voter cases plus reconcile
- restart atomicity
- legacy Dropbox naming
- tenant re-render and Edit unchanged
- Frontend Jest: button visibility (admin vs non-admin, complete vs
not), dialog, payload, and warnings toast.
- `make test`: backend 1972 + 5372 + 150, frontend 1506.
- i18n: new strings in all 33 locales (frontend) plus backend
translations.
Docs: `docs/API.md`,
`docs/archive/2026-10-01-admin-rerender-any-job.md`.
No local CodeRabbit review (the CLI has no seat for this org), so this
is left open for the CodeRabbit bot. Three local code-review rounds ran;
their findings are addressed in the fix commits.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Administrators can re-render eligible completed tracks without another
review, retaining existing track settings and brand code.
* Choose whether to notify the customer; notifications are off by
default. Published outputs are replaced when applicable, and warnings
identify outputs that remain unchanged.
* Administrators can resume eligible failed or cancelled re-renders.
Re-render controls and status messages are available in multiple
languages.
* **Bug Fixes**
* Improved cleanup and handling of published files and queued YouTube
uploads during re-renders, with safer retries and clearer failure
reporting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
v0.257.0
|
||
|
|
e2f7dbd13c |
Detect sung stretches with no transcribed lyrics (shadow) + backfill endpoint (v0.256.0) (#1095)
## Summary
Transcription can silently **drop whole sung lines**. On job 5710831e
(Hebrew), the words stop at 20.66s and resume at 32.40s while the lead
vocal is ~96% active, and both reference sources have three pre-chorus
lines there. `SectionDetector` (gap ≥ 10s, no audio check) then rendered
"♪ INSTRUMENTAL ♪" over the singing. Nothing caught it:
- the scorer's anchor coverage measures how much of the *transcribed*
text matches the reference, so omissions don't lower it (98.8% here);
- `timing_check` G3 (`max_unclaimed_run_s`) does fire on this job but is
shadow-only and only computed for AUTO-bound jobs;
- the review UI shows nothing at the gap.
This PR is **phase 1: a shadow signal plus server-side backfill for
calibration**. It changes no verdicts or rendering.
- `backend/services/auto_approval/vocal_gaps.py`: for every
transcription gap ≥ 3s, records the lead-vocal active fraction, the
longest breath-bridged vocal run (with a 1.5s allowance for a held note
carried over from the previous word), and the reference lines between
the neighbouring words' reference positions. Section headers are dropped
and ±1-word anchor drift at line boundaries is snapped. `suspect` = run
≥ 3s.
- The executor analyzes every job once stems exist (any verdict), off
the event loop, and stores `state_data.vocal_gaps` keyed by an
`input_key` fingerprint so resets/edits recompute. A summary goes in
`processing_metadata.auto_approval.vocal_gaps`.
- `POST /api/internal/jobs/{id}/vocal-gaps[?dry_run=true]` (admin) plus
`scripts/audit_vocal_gaps.py` for backfill/calibration. Cloud Run does
the stem work.
**On the real job:** one suspect gap, 20.66–32.40s (10.26s run after the
allowance), whose reference lines are exactly the three missing Genius
lines. The three genuine instrumentals have ≤ 0.16s runs.
## Testing
- `test_vocal_gaps.py` (25 tests, synthesized audio + synthetic
correction data):
- dropped lines vs instrumental vs ad-lib vs held note
- a short dropped line diluted by a long gap
- breath bridging
- silent stem = error
- malformed reference data
- reference snapping (off-by-one, single-word line, headers, reversed
chorus mapping)
- Executor tests: stored without changing the outcome; analyzed for
review-bound jobs; cache hit only for the same `input_key`, stale
recompute; pending audio / no stem / error / exception all non-fatal;
backfill uses the reviewed lyrics + original references; dry run.
- Internal endpoint tests (store / dry_run / admin).
- Full backend suite: 5199 passed.
- A local code review's findings were addressed in the second commit:
event loop, stale cache, silent stem, held notes, reference robustness,
audit ordering.
## Next (after deploy)
Run the audit → hand-check suspects → choose thresholds → phase 2:
scorer REVIEW gate with an explicit reason, no fake INSTRUMENTAL over
vocals, review-UI marker with one-click insert of the reference lines.
Also noted: anchor reference ids look shifted by one word at `[Section]`
header boundaries. Separate issue, documented in the plan doc.
No local CodeRabbit review (the CLI has no seat for this org), so this
is left open for the CodeRabbit bot.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added vocal-gap analysis to identify stretches of missing lyrics that
overlap with lead vocals. Results are saved with job data and can be
requested through an admin-only internal API.
* Analysis also runs automatically during auto-approval when vocal stems
are available. It does not change job status or approval decisions.
* **Documentation**
* Documented the internal API, its dry-run option, and the analysis
result statuses.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
v0.256.0
|
||
|
|
354e43bab2 |
Render Hebrew/Arabic karaoke right-to-left; fix title-card tofu (v0.255.0) (#1094)
## Summary
A Hebrew KaraokeHunt job (5710831e, Omer Adam – "שני משוגעים") shipped
with its lyrics **laid out and highlighted left-to-right** and a **title
card of "?" boxes**. Diagnosed by measuring rendered pixels (not by
eye), reproduced locally and with prod's static ffmpeg 7.0.2 in Docker.
**Root causes**
- ASS style `Encoding=0` makes libass force LTR paragraphs; the `{\kf}`
before each word splits the line into runs, so each word is shaped
correctly but the words go left→right.
- `\kf` always wipes left→right within a word
([libass#406](https://github.com/libass/libass/issues/406)).
- PIL (title/end cards) has no per-glyph font fallback; the theme font
(Avenir Next) has no Hebrew glyphs.
**Fixes**
- `build_karaoke_styles` forces `Encoding=-1` (libass auto paragraph
direction; LTR unchanged).
- RTL lines get `{\frz(180+angle)\frx180\fry180}` — a visual no-op that
makes libass ≥0.15 sweep fills right→left without splitting words
(Arabic still joins). Theme angle preserved; re-emitted after duet
`{\r}` resets.
- Lead-in indicator mirrored for RTL lines.
- Text measurement mirrors libass: per-glyph fontconfig fallback, size
scaled from each font's metrics (old fixed 0.70 was 20%+ off for
fallback fonts).
- New `karaoke_gen/utils/font_fallback.py` (fontTools cmap +
fc-match/fc-list) used by title/end cards, portrait headers and ASS
measurement.
- Found by the new tests: inter-word gaps ≤0.1s were dropped, so the
highlight ran up to ~0.2s early by line end (all languages). Karaoke
tags now derive from one absolute centisecond timeline.
- Fixed pre-existing `test_render_video_worker_gce` failure on main
(MagicMock generation → `_g<mock>` id).
## Testing
- **Pixel-level render tests** (`test_ass_render_highlight.py` +
`ass_render_harness.py`): real `SubtitlesGenerator` output rendered with
ffmpeg/libass in a pure-colour palette, asserting on frames for English,
Hebrew, Arabic, Chinese, Japanese, Korean + an RTL duet line: highlight
starts at the reading-start edge, sung pixels always precede unsung in
reading order (word order + in-word fill direction), monotonic and in
time with the words, lead-in on the correct side, no missing glyphs, RTL
tags don't move the line (incl. rotated themes).
- 32 fail on pre-fix main; mutation checks ("Encoding only", "no re-emit
after `\r`", "ignore theme angle") each fail exactly the expected cases.
- 123/123 pass on Linux against **both** Debian ffmpeg (CI-equivalent)
and **prod's johnvansickle static ffmpeg** via
`scripts/run-render-tests-linux.sh`.
- CI now installs `fonts-noto-core fonts-noto-cjk`; under `CI=true`
these tests fail rather than skip on missing preconditions. (macOS
libass uses CoreText → font-sensitive checks skip locally.)
- Title-card tests (`test_font_fallback_render.py`): no `.notdef` tofu
for Hebrew/Arabic/CJK titles; Hebrew title reads right-to-left.
- Unit tests: `test_rtl_ass_output.py`, `test_font_fallback.py`.
- `make test`: unit 1972 passed; emulator 150 passed; frontend 1491
passed.
- Full local re-render of job 5710831e reviewed by Andrew: RTL highlight
smooth and correct, title card correct.
## Follow-ups (not in this PR)
- CDG output still renders RTL left-to-right / tofu (separate renderer).
- Separate investigation: transcription dropped 3 sung lines in this
job, which the section detector then labelled INSTRUMENTAL.
No local CodeRabbit review (CLI has no seat for this org), so leaving
this open for the CodeRabbit bot. A local code review ran; its findings
are addressed in the second commit.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Karaoke lyrics now render more reliably for Hebrew, Arabic, Chinese,
Japanese, and Korean, with improved reading-direction highlighting,
timing, and lead-in placement.
* Titles and portrait text can use fallback fonts when the selected font
lacks the necessary characters.
* **Bug Fixes**
* Improved lyric centering, word highlighting, and glyph rendering
across supported scripts.
* **Documentation**
* Added guidance on lyric rendering and multilingual test coverage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
v0.255.0
|
||
|
|
e15a485007 |
Re-render: refresh tenant Dropbox archive in place instead of rejecting (v0.254.1) (#1093)
## Summary Follow-up to #1091, found by testing it in prod. randy-vild jobs are archived to Dropbox under a brand code (`RVILD-0001 - Randy Vild - What Goes Up`). The "never distributed" guard therefore rejected the very jobs the Re-render button is for (`POST /api/jobs/2579a1ea/rerender` → 400 "published to YouTube/Dropbox/Drive"). - Dropbox-archived tenant jobs can now be re-rendered. The re-render stores the job's `brand_code` in `state_data.theme_rerender`, and the orchestrator config and native distribution use it as `keep_brand_code`. The same folder is overwritten (Dropbox `WriteMode.overwrite`) and no new code is allocated. - The code is scoped to the re-render marker (cleared on success), not written to the job's `keep_brand_code` field. That way a later Edit, which recycles the brand code, can't reuse it. - YouTube/GDrive-published jobs are still rejected. The frontend hides the button only when a YouTube link shows. ## Testing - Backend 5158 passed, 36 skipped; `test_theme_rerender.py` 35 cases, including Dropbox-archived jobs being re-renderable, the marker recording the brand code, and the orchestrator keeping it. - Frontend `OutputLinks.test.tsx` 28 passed. - CodeRabbit CLI isn't available on this org, and the bot has been rate-limited. This is a small change on top of the reviewed #1091. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.254.1 |
||
|
|
9085a3005e |
feat(tenants): configure Dropbox output folder in tenant setup (v0.254.0) (#1092)
## Summary The Randy Vild tenant's completion email said "Here's the dropbox folder…" then `[Dropbox URL not available]`: the tenant was created with the (optional) Dropbox path left blank, so every job was download-only. The admin form already had the fields, but nothing guided, validated, or provisioned them. ### Changes - **Tenant create/update** (`tenant_admin_service`): `dropbox_path` + `brand_prefix` must be set together (the worker silently skips upload otherwise); path normalized, prefix uppercased/validated (2-12 alnum) and rejected if it's the consumer default (`NOMAD`/`NOMADNP`) or another tenant's (brand-code counters are keyed by prefix alone). The Dropbox folder is **created up front**; `features.dropbox_upload` always follows the path. - **DropboxService**: `ensure_folder()` (idempotent); `list_folders()` treats a missing path as empty — previously the *first* brand-code allocation for a new tenant would raise `not_found`. - **Admin UI** (`/admin/tenants`): pre-fills `/MediaUnsynced/Karaoke/Tracks-<Name>` + prefix from the name (Vocal Star→VSTAR, Singa→SINGA, Randy Vild→RVILD), blocks half-set delivery, warns that download-only tenants get no email link; table shows `PREFIX → path`. - Update route maps conflict→409, Dropbox provisioning failure→502. ### Prod data fix (already applied, outside this PR) - `randy-vild` tenant → `/MediaUnsynced/Karaoke/Tracks-RandyVild`, `RVILD`; `brand_code_counters/RVILD` seeded at 3. - 10 in-review Randy Vild jobs backfilled with `dropbox_path`/`brand_prefix` (jobs snapshot these at creation). - 2 completed jobs copied into `RVILD-0001 - Randy Vild - What Goes Up` / `RVILD-0002 - Randy Vild - Embers in the wind` (24 files each via Dropbox `save_url` from signed GCS URLs), `state_data.brand_code`/`dropbox_link` set. ## Testing - `make test`: backend 5121 + 1846 + 150 passed, frontend 1476 passed - New unit tests: tenant delivery validation/folder creation/prefix conflicts, DropboxService ensure_folder + missing-path listing (real SDK error types), admin page pre-fill + half-set block + download-only warning, `suggestDelivery`/`deliveryError`. - Local CodeRabbit review: 2 minor findings, both fixed. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.254.0 |
||
|
|
31a8838b07 |
Tenant: re-render tracks with the current theme; edited tenant tracks keep their downloads (v0.253.0) (#1091)
## Summary
- **Bug: finished tenant track showed no download buttons** (randy-vild
job 2579a1ea). The Edit flow sets `outputs_deleted_at`, and it was only
cleared when the re-render uploaded to YouTube/Dropbox/GDrive. Tenant
jobs never distribute, so every download stayed hidden after the
re-render. A customer's private job (6452888e) had the same problem for
a month. Every successful render now clears the flag
(`video_worker.py`). Both prod jobs were cleared by hand.
- **New: "Re-render" button** on finished tenant-portal tracks (`POST
/api/jobs/{id}/rerender`). It re-applies the tenant's **current** theme
and rebuilds the title/end screens, lyrics video, all encodes and
CDG/TXT. It reuses the reviewed lyrics and instrumental selection, so
there's no review step. Same mechanism as the private→public visibility
flow (`regen_restore_status=review_complete` → screens → render →
encode). Owner (still on the tenant allowlist) or admin. Free. The theme
editor now points to the button.
- **Encoding-worker cache:** the worker caches finished jobs by id, so a
re-run submitted while the VM was still up could get the previous render
or encode back. Render and encode now submit as
`{job_id}_g{worker_generation}`.
- **Old title/end cards:** the encoder's `**/*Title*.mov` lookup would
have picked up the old `finals/… (Title).mov`. The re-render now deletes
the MOVs in `finals/`.
- **Failed re-render:** it can be re-run. Retry re-runs it if the new
screens weren't made yet, and otherwise resumes from render/video as
usual.
## Testing
- Backend: 5128 passed, 36 skipped. New `test_theme_rerender.py` (31
cases: validation, claim/conflict, artifact cleanup incl. finals MOVs,
trigger rollback, route auth/allowlist, retry branch, generation-keyed
ids) and `test_video_worker_outputs_deleted_clear.py` (fails without the
fix).
- Frontend: `OutputLinks.test.tsx` 28 passed (button visibility, confirm
→ API → refresh). Strings translated to all 33 locales.
- Review: CodeRabbit CLI isn't available on this org. An independent
reviewer agent went over the diff: 1 must-fix (stale finals MOVs) fixed;
the failed-re-render recovery fixed.
## Not done (follow-ups)
- Re-render re-sends the completion email and push notification (same as
the visibility flow).
- No per-tenant rate limit. One re-render per job at a time is enforced
by the status claim.
- The status briefly shows "Lyrics ready" before "Generating screens".
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
v0.253.0
|
||
|
|
eaa70e04ea |
feat(review): hold a word's resize handle at the row edge to keep extending it (v0.252.0) (#1090)
## Summary In the lyrics review Waveforms rows, the view is the segment ±1s and it was fixed during a drag (and leaving the row ended the drag). So a word ending 5s+ too early took several drag-release cycles to fix. Now, holding a word's resize handle within 24px of the edge it's heading toward (or past it) keeps growing the word at 1–4 s/s (faster nearer the edge). The view zooms out so the handle stays under the cursor. It stops when: - the pointer leaves the edge zone (then the handle tracks the pointer normally in the zoomed view), - the mouse is released (commits once, as before), or - it reaches the neighbouring word, including the **next line's first word** even when that's beyond the drawn context padding (new `computeNeighbourBoundsBySegment`), else the audio end/start. Works for both handles and also in the Edit Segment modal timeline (same `TimelineEditor`). ## Changes - `TimelineEditor.tsx`: absolute pointer→time mapping for resizes (grab offset), a rAF auto-extend loop, a `dragView` override that widens the view mid-drag, and window-level mousemove/mouseup (so drags survive leaving the row, with text selection prevented). Resizes now clamp to the neighbour instead of freezing short of it on overshoot, and never grow into the adjacent line (pre-existing overlaps aren't snapped back). - `contextWords.ts`: `computeNeighbourBoundsBySegment` (prev line end / next line start per segment), wired through `TranscriptionView` → `WaveformSegmentRow`. - Version 0.251.1 → 0.252.0; docs/README.md recent-changes entry. ## Testing - 7 new `TimelineEditor` tests (fake rAF): extends while held at the edge, stops at the next line / same-segment neighbour, stops when leaving the zone, respects the drag threshold, left handle bounded by the previous line, commits once when released outside the row. Plus 2 new util tests. - `make test`: all backend + 1476 frontend tests pass. - Checked in the browser against the local fixture harness: the last line's "shoes" extended with the handle glued to the edge while the row zoomed out; "alright" stopped exactly at the next line's "Because". 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * In lyrics review waveforms and the Edit Segment timeline, dragging a resize handle within 24 pixels of the direction of travel now automatically extends the word. The timeline zooms to keep the handle under the cursor as the word grows. * Resizing stops at neighboring words, adjacent segment boundaries, or the audio limit. You can continue dragging outside the row, and releasing completes the resize. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.252.0 |
||
|
|
69986bb844 |
fix(youtube): canonical description template wins over stale theme youtube_description.txt (v0.251.1) (#1088)
## Summary
The daily YouTube description backfill finished its full-channel rewrite
on 2026-09-15, but kept rewriting 1-4 **brand-new** customer videos most
days. Root cause: the `nomad` theme's legacy
`gs://karaoke-gen-storage-nomadkaraoke/themes/nomad/youtube_description.txt`
(old Fiverr text: "...my cheapest package is only \$10!
fiverr.com/share/...") was returned by `_prepare_theme_for_job` and
written into `job.youtube_description_template` by every themed
job-creation path (made-for-you, bulk search, file upload x4, guided
search), overriding `Settings.default_youtube_description`. New videos
were published with the stale description and only corrected ~1 day
later by the backfill worker.
## Change
- `_prepare_theme_for_job` now only returns a theme's
`youtube_description.txt` when no canonical template is configured. All
callers already handle `None` (they keep the canonical template set at
creation, or leave it `None`, which the orchestrator renders with the
default).
- Tests: 2 new unit tests covering both precedence branches.
- Version 0.251.0 → 0.251.1.
## Evidence
Post-completion backfill updates (e.g. `l8xUGZqCYSY`, `jCyfwQorgOY`,
`44Qa2Pt-gpU`, `hMQEV2vCgLY`, `MFu7t-F0VR0`) all map to customer jobs on
theme `nomad` whose `youtube_description_template` starts "This is a
karaoke (instrumental) version of the song, with vocals removed using
AI...".
## Testing
- `pytest backend/tests/ --ignore=backend/tests/emulator`: 5080 passed,
36 skipped
- Local review (subagent fallback; CodeRabbit unavailable): no
must/should-fix findings
## Follow-ups (optional)
- The stale GCS `themes/nomad/youtube_description.txt` can be deleted
(also served by the unused `GET /api/themes/{id}/youtube-description`).
- Verify after deploy: backfill worker should report 0 pending on
subsequent days.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
v0.251.1
|
||
|
|
780241ad03 |
feat(dashboard): show other accounts' awaiting-audio jobs + admin toggle (v0.251.0) (#1089)
## Why fiverrbot (v0.3.0) parks Fiverr orders with no recommended audio at `awaiting_audio_selection` (owner `fiverr@nomadkaraoke.com`) and tells the admin to click **Select Audio** in `/app`. But `shouldShowJobOnDashboard` hid every job at that status except `made_for_you` orders, so the card never appeared (e.g. job `bdf35771`, order FO62A31A27AC6). The API returned the job; only the frontend filter dropped it. ## Changes - `shouldShowJobOnDashboard(job, { viewerEmail, showOthersAwaitingAudio })`: every `awaiting_audio_selection` job is now shown. The old wizard reason for hiding them is gone, since the guided flow searches via `searchStandalone` and never creates a job at this status. This also fixes a bug already on main: "Choose different audio" (v0.250.0) moved your own job to this status, and the card with its open picker disappeared. - With the admin switch off, only **other accounts'** non-made-for-you jobs are hidden. Your own jobs and made-for-you orders stay visible. - New admin-only **Show Jobs Awaiting Audio** switch in the account menu, next to Show Test Jobs. It's persisted in `admin-settings` (`showAwaitingAudioJobs`, default on). Turning it off hides other accounts' jobs at this status. - i18n: `auth.status.showAwaitingAudioJobs` in all 33 locales. - LESSONS-LEARNED: added a recurrence note to the Aug 2026 made-for-you entry. - Version 0.250.0 → 0.251.0. `user_email` is already in `SUMMARY_FIELD_PATHS`, so no backend change is needed. Non-admin users only ever receive their own jobs, so nothing changes for them. ## Testing - New unit tests for the filter: shown by default, own job shown, toggle off hides others but keeps own (case-insensitive) and made-for-you, unknown owner hidden when off, other statuses unaffected. Plus a store test for the new setting. - Full frontend Jest: 124 suites / 1459 tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
c831bd5e89 |
feat(uploads): line up mismatched user instrumentals + tenant bulk upload modal (v0.251.0) (#1087)
## Summary
A Randy Vild tenant bulk upload lost 3 of 10 tracks to the 0.5s
mix/instrumental duration check, and the bulk UI showed only
"Submitting…" while uploading. All three rejected tracks were valid
edits of the same song:
| Track | Difference | Now |
|---|---|---|
| Keep Smiling | instrumental has 2.97s extra lead-in | start trimmed →
matches mix exactly |
| Simulation | instrumental 6.5s longer outro | cut at mix end with 2s
fade |
| Faithful and True | instrumental stops 4.45s early | end padded with
silence |
## Changes
**Backend: conform instead of cancel**
(`backend/services/instrumental_conform.py`, `uploads-complete`)
- On a >0.5s mismatch, the start offset is found by FFT
cross-correlation (reusing `derived_vocals.find_offset`) at **25% and
75%** of the song, searching ±15s. The start is then trimmed or padded,
and the end is faded and cut or padded to the mix length. All of this is
one ffmpeg pass on the original file, keeping its sample rate and bit
depth, and written as FLAC.
- Stored at `uploads/{job}/conformed/existing_instrumental.flac`, which
becomes `existing_instrumental_gcs_path`. The original upload is kept,
and details go in `state_data.instrumental_conformed`.
- Still rejected (job cancelled and refunded, 400):
- correlation < 0.3 (wrong file; real pairs score 0.78–0.90, wrong pairs
about 0.01);
- offsets at 25% and 75% differ by more than 20 ms (edited differently
mid-song).
- Conforming runs fully off the event loop, with at most 2 at a time per
instance. GCS or ffmpeg failures return a retryable 503 instead of
cancelling the job.
- The orchestrator now stages the recorded
`existing_instrumental_gcs_path` for the GCE encoder, so the conformed
copy is what gets rendered.
**Frontend**
- Tenant bulk shows `UploadProgressModal` for the whole batch: total
bytes, speed and ETA, tracks submitted and failed, and an offline or
retrying notice. The modal gained optional `title`, `detail` and
`notice` props and uses the tenant's primary colour.
- Bulk retry bug:
- **Before:** a server-cancelled job kept its ID. The retry's 400 ("not
pending") was read as "already processing", so the row showed
**Submitted** for a dead job.
- **Now:** any 400 checks the job's status. A processing job counts as
done. A cancelled or failed job is dropped, so the retry creates a fresh
job. Otherwise the row keeps the job for retry, so no duplicate is
created.
- The single-track tenant form and the own-instrumental field now check
only the size cap in the browser; the server handles length differences.
## Testing
- Backend: 5086 passed (new: synthetic lead-in, long/short outro, late
start, wrong-file and mid-song-edit cases; route and helper guards;
orchestrator staging). Frontend jest: 1459 passed (new: batch modal
progress and offline notice, failed rows, retry and no-duplicate cases).
- Ran `conform_instrumental` on Randy's real files:
- offsets at 25% and 75% agree to the sample;
- output is exactly the mix length at 24-bit / 44.1 kHz;
- after conforming, each file lines up with its mix at 0 offset;
- a wrong pairing is rejected (correlation 0.02).
- Real frontend (`next dev`, randy-vild tenant, mocked API, slowed
upload server), 8 tracks / 192 MB: the modal showed progress, track
counts, speed and ETA, then closed onto the "All tracks submitted"
screen.
- Review: an independent reviewer agent found 1 must-fix and 4
should-fix issues, all addressed in the last commit. The CodeRabbit CLI
has no seat here, so this PR is left open to the CodeRabbit bot.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Uploaded instrumentals with different durations can now be aligned and
adjusted to match the mix when the audio matches.
* Bulk uploads now display overall progress, completed and failed track
counts, connection updates, and retry guidance.
* Failed bulk-upload tracks can be retried, with clearer handling for
submissions already in progress.
* **Bug Fixes**
* Instrumental duration checks now occur during upload processing
instead of blocking the upload in the browser. Files that cannot be
matched are reported as a mismatch.
* **Documentation**
* Updated upload instructions and related messages across supported
languages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
v0.251.0
|
||
|
|
d182151626 |
infra: remove legacy us-central1 CPU Cloud Run Jobs (Pulumi already applied) (#1086)
## Summary Follow-up to #1077 and #1083. This removes the four legacy **us-central1** Cloud Run Jobs, which were kept only as a rollback path while the us-east4 copies were verified: - audio-download-job - lyrics-transcription-job - bulk-search-job - video-encoding-job The us-east4 jobs were verified in prod on 2026-09-29: - A normal job's download and lyrics jobs started in about 20 s, and a kjbox make-it worked end to end. - Five render and final-video runs started in 10–24 s, and all three jobs completed with YouTube, Dropbox and GDrive delivery (the tenant job skips those, as expected). - The legacy jobs have had no runs since each cutover. ## Pulumi **Already applied** (`pulumi up`: 4 deleted). A preview afterwards shows **286 unchanged**, so this merge introduces no drift. The us-east4 jobs keep their suffixed logical names so they aren't replaced. ## Changes - `infrastructure/__main__.py`: removed the `legacy_*` job resources; comment updated. - `infrastructure/modules/cloud_run.py`: comment-only updates. - `backend/config.py`: comment-only update to the rollback guidance. - `docs/TROUBLESHOOTING.md`: rollback runbook now describes recreating the jobs via Pulumi, and notes that rolling the service back to before v0.247.1 needs them. ## Testing - `test_worker_service_cpu_jobs_region.py` and `test_audio_download_worker.py` (which parses `cloud_run.py`) pass. - The Pulumi preview and apply both match the expected changes exactly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Infrastructure** * Removed the legacy Cloud Run job copies in `us-central1`; active jobs continue to run in their configured region. * **Documentation** * Updated rollback guidance to cover recreating the `us-central1` jobs before rolling back the backend to a version before v0.247.1. The instructions now describe the staged transition between regions, including updates to backend and CI job settings, region defaults, and the rollback script. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
f4eca80551 |
feat(audio-download): stalled torrent → 'Keep trying?' prompt (v0.250.0) (#1085)
## Why Job 64be8a87 (a Fiverr order): the only seeder of a 1-seeder RED single was offline. The torrent sat at **0 peers through 5 re-announces for 10 min**, flacfetch aborted, the job showed a raw error, and a Discord "Job failed" alert fired even though a Cloud Run auto-retry was already queued. That retry got the file in 17s. Andrew's ask: 10 min is too short, and a stalled download should explain itself and offer "Keep trying?" for up to another hour. ## What - **Longer first wait:** 20 min with no progress (was 10), sent to flacfetch per request as `max_stall_seconds` (needs flacfetch v0.31.0, nomadkaraoke/flacfetch#51, already deployed to flacup). The worker's wait and the Cloud Run **per-execution timeout override** come from the same budget (`backend/services/audio_download_limits.py`), so the job's Pulumi config is untouched. - **Stall → user prompt, no silent retry:** flacfetch now reports `error_code: torrent_stalled` (older versions are detected from the error text). The worker fails the job with `error_details = {stage: audio_download, code: audio_download_stalled, stall_minutes, keep_trying}` and exits 0, so Cloud Run doesn't retry the same torrent. Other download failures keep their auto-retries. - **Job card:** `StalledDownloadNotice` replaces the raw error with an amber explanation and two buttons: - **Keep trying (up to 1 hour)**: `POST /api/jobs/{id}/retry {"keep_trying": true}` (the body is optional and backward compatible). This gives a 3600s stall budget and a 1h25m execution timeout. - **Choose different audio**: new `POST /api/jobs/{id}/choose-different-audio` (owner-checked). It moves FAILED to `awaiting_audio_selection` (new transition) using the saved search results and opens the Select Audio dialog. - The generic Retry button is hidden for stalled jobs. `error_details.{code,stage,stall_minutes,keep_trying}` were added to the summary fields. Strings are translated into all 33 locales. - **Heartbeat:** while waiting on a torrent, the worker writes `state_data.audio_download_progress` (progress, peers) every 2 min. Without this, recover-stuck-jobs (which parks a job with no update for 10 min) would park a live 20–60 min wait and start a duplicate download. - **Ops alert:** the alert is skipped while a Cloud Run auto-retry is pending (the final attempt clears the marker and alerts then). Stall alerts get a 🟠 "Audio download stalled" headline. - Docs updated: API.md, TROUBLESHOOTING.md, ARCHITECTURE.md. ## Tests - `make test`: backend 1846 + 5078 + 150 passed; frontend 1451 passed. - New tests: - `test_audio_download_limits.py`, `test_routes_jobs_stalled_download.py` (real HTTP via TestClient), `test_worker_service_audio_download_timeout.py`. - Stall, keep-trying, heartbeat and `main()` exit-code cases in `test_audio_download_worker.py`. - Stall detection and payload in `test_flacfetch_client.py`; retry-pending suppression in `test_ops_alerts.py`. - Frontend: `StalledDownloadNotice.test.tsx`, plus JobCard/JobActions cases. ## Notes - Credits are unchanged: a failed job is still auto-refunded and retries don't re-charge. That's existing behaviour for every retry. - A local CodeRabbit review couldn't run (the CLI seat is on another org), so this PR has no ignore line and the GitHub bot will review it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Failed audio downloads now offer options to keep trying for up to an hour or choose a different audio source. * Retry progress and clearer stalled-download guidance are available in the job view, with messages translated across supported languages. * **Bug Fixes** * Stalled downloads can be recovered without triggering unnecessary automatic retries or duplicate failure alerts. * Choosing different audio returns eligible failed jobs to audio selection. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.250.0 |
||
|
|
fc05571779 |
fix(theme-preview): skip ffprobe for no-audio ASS generation (v0.249.1) (#1084)
## Summary Error-monitor alert: `Failed to get audio duration: ... /tmp/theme-preview-*/no-audio.flac`. The tenant theme editor (#1080) passed a nonexistent audio path to `SubtitlesGenerator.generate_ass`, so every uncached preview ran ffprobe on a missing file and logged at ERROR before falling back to the segment-derived duration. Previews rendered fine (200); this was pure log noise. ## Changes - `generate_ass` / `_get_audio_duration` accept `audio_filepath=None` → skip ffprobe, use `last segment end + 30s` silently. Real-audio callers unchanged (a failing probe still logs ERROR). - `theme_preview_service` passes `None`. - Version 0.249.0 → 0.249.1. ## Testing - New `tests/unit/lyrics_transcriber/output/test_subtitles_audio_duration.py` (None path: no probe, no error log; failed probe still logs; `generate_ass(..., None)` end-to-end). - `make test-unit test-backend-unit`: 1846 + 5016 passed. - Local CodeRabbit review: no findings. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.249.1 |
||
|
|
badd845a1d |
perf(jobs): move video-encoding-job (video + render) to us-east4 (v0.249.1) (#1083)
## Summary Follow-up to #1077. `video-encoding-job` runs both the post-review render (`render_video_worker`) and the final video (`video_worker`). In us-central1 each run waited **2–5 min** for Cloud Run Jobs to start it, so every customer job paid that twice. This moves it to us-east4 alongside the download, lyrics and bulk-search jobs. ## Cost (measured, not estimated) - Over the last 30 days the job read **156 GB** from Google services, mostly the us-central1 bucket, across 494 runs (Cloud Monitoring `container/network/received_bytes_count`, kind=google). That's about 0.32 GB per run. - Pricing from the Billing Catalog: GCS inter-region within North America is **free for the first 100 GB/month, then $0.02/GB**. Cloud Run writes to Google APIs are **$0**. - Internet egress (224 GB/month, Dropbox etc.) and Cloud Run CPU cost the same in either region. - Existing cross-region reads are about 24 GB/month (GPU job and lyrics), which the billing export shows at about $0. - **Net: about $1.60/month**, or about $3/month worst case. Time saved: roughly 4–8 min per customer job. ## Changes - **backend**: `trigger_video_worker` (via `_trigger_cloud_run_job`) and `trigger_render_video_worker` now use `cpu_jobs_region`. `GCP_REGION` stays us-central1; encoding VM and zone logic are unchanged. - **infra (Pulumi, already applied)**: new us-east4 `video-encoding-job`, with the legacy us-central1 copy kept for rollback and its deletion protection turned off. A preview after applying showed 290 unchanged. - **ci**: the video job image update now targets us-east4, using the us-east4 CPU image copy. - **Also updated**: rollback.sh, tests (video and render routing), TROUBLESHOOTING/ARCHITECTURE/LESSONS-LEARNED. ## Testing - `make test` passes: backend tests plus 1443 frontend tests. - Prod smoke test of the new us-east4 job: it imports `video_worker` and `render_video_worker` and calls the encoding VM's `/health` on its public IP (port 8080 is open to 0.0.0.0/0 with API-key auth). **Started in 25 s, exit 0.** 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Updates** - Video encoding and render jobs now run in the `us-east4` region, alongside other CPU jobs. - Video encoding continues to access storage and encoding resources in `us-central1`; cross-region transfer may incur costs. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
cb5ce7f162 |
chore(pypi): prune keep-window default 60→30 days (#1082)
Ran the PyPI prune runbook today (2026-09-29): publishing was blocked (`Project size too large`). The 60-day policy would only have freed 1.08 GB (→9.64 GB), refilling within ~a week, so pruned with `--keep-days 30` instead (69 releases deleted, 0.188.5–0.207.0; newest release of each older month retained) → ~6.8 GB. Re-ran the failed publish; 0.249.0 is live on PyPI. This PR makes 30 days the default in the planner script, the monthly reminder workflow, and the runbook. Unit tests pass (13). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
385a997aba |
test(e2e): fix tenant theme editor prod spec selectors (#1081)
Follow-up to #1080. The production spec failed to find its targets on the live `randy-vild` portal: - `AuthStatus` renders twice, once in the desktop header and once in the mobile header, so `user-menu-trigger` matched two elements. The spec now targets the visible one. - The dialog's corner ✕ is also labelled "Close". The spec now targets the footer button: `exact: true`, then `.first()`. I verified this against production with `DEBUG=pw:api`: every step and assertion passes (both previews render, only the lyrics preview changes after a sung-colour edit, Discard and Close work). Test-only change, so no version bump. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Updated the tenant theme editor end-to-end test to open the user menu and close the editor using more specific controls. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.249.0 |
||
|
|
80f6d34007 |
feat(tenant): self-service theme editor with exact server-rendered previews (#1080)
## Summary Tenant portal users (anyone on the tenant's allowlist) can now edit their tenant's theme from **Theme & style** in the user menu. - **Guided form:** - backgrounds (upload or solid colour) for the title card, lyrics video and end screen - font (bundled or uploaded TTF/OTF) - title and artist colours, positions and sizes - sung, upcoming and outline lyric colours - lyric size, top position and max line length - the end-screen message - **Advanced:** validated full-theme JSON. - **Exact previews:** the title card and a lyrics-video frame (first line half-sung) are rendered **by the production renderers** from the unsaved draft, in about 2 s, debounced. The plan is in `docs/archive/2026-09-29-tenant-theme-editor-plan.md`. - **Save** applies to jobs created afterwards. ## Backend - New router: `GET/PUT /api/tenant/theme`, `POST /api/tenant/theme/assets`, `POST /api/tenant/theme/preview`. - **Membership guard:** `X-Tenant-ID` is client-controlled, so every call re-checks the tenant allowlist (admins always pass). Non-members and spoofed headers get 403. The tenant bulk router only checks that the tenant exists, so this pattern wasn't copied from there. - `theme_preview_service`: - title card via `VideoGenerator.create_title_video(duration=0)` - karaoke frame via `SegmentResizer` → `SubtitlesGenerator.generate_ass` → one ffmpeg frame, using `setpts` to jump straight to time T and scaling inside the same pass (11.8 s → 0.66 s) - bounded concurrency, and a cache for identical drafts - `tenant_theme_service`, stricter than the admin console: - asset fields must be basenames that exist in the theme (no paths or `gs://`) - bounds, colours and regions are validated - **font consistency with the render pipeline:** the single `font` asset is applied everywhere, `karaoke.font` is set to the TTF family name libass matches, and a chosen bundled font is copied into the theme's assets - required ASS keys are filled in - uploads are content-addressed, so they never overwrite an asset that live jobs use - tenants can't change their config or allowlist through this API ## Frontend - `components/tenant-theme/TenantThemeEditorDialog` and `theme-utils`. - `tenantThemeApi` in `lib/api.ts`. - A menu item in `AuthStatus`, shown only on tenant portals. - `tenantTheme.*` strings in all 33 locales. ## Tests - **pytest:** - real renders (the background colour appears in the frames; only the karaoke frame changes when the sung colour changes; caching) - sanitisation (paths, un-uploaded assets, bounds, colours, regions, sections) - uploads (content addressing, `no-store`, rejecting non-images and bad fonts) - save (font family, bundled font copy, no config changes) - the route guard for spoofed, other-tenant, inactive and no-tenant cases, on every endpoint - **Jest:** load, debounced preview, hex→rgba conversion, upload, region sliders, Advanced JSON validation, save, discard, the unsaved-changes guard, and preview errors. - **Prod E2E:** `e2e/production/tenant-theme-editor.spec.ts`. - Results: backend 4998 passed; frontend 1434 passed. No local CodeRabbit review was run (no CLI seat in this org), so this PR is left open for the GitHub bot. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a theme editor to the signed-in menu, with controls for title cards, karaoke videos, end screens, and advanced settings. * Preview theme drafts, upload images and fonts, and save or discard changes. Saved themes apply to future videos. * Added translations for the editor across supported languages. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
835fdf6096 |
kn-data-sync: ListenBrainz popularity + inclusion in the kjbox song-id index (v0.248.1) (#1078)
## Summary Adds ListenBrainz (karaoke-decide `lb-refresh` → `lb_recording_popularity` / `lb_artist_popularity`) to `SONG_ID_SQL`, fixing the index's freshness gap (design: kjbox `docs/SONG-IDENTIFICATION.md` D5a/D12). - **Score:** each stats range maps onto Spotify's 0–100 as `offset + 19·log10(listeners)`. Offsets: all_time 8, this_year 11, year/half_yearly 12, month 14, this_month 20. They were fitted to the median Spotify popularity of songs that have both. Ranges under 3 listeners are ignored. - **Song popularity** = max(Spotify track score, ListenBrainz score). The capped artist estimate stays the fallback, and it can now also use the ListenBrainz artist score. - **Inclusion:** a MusicBrainz song is also kept with ≥ 5 listeners this/last year or ≥ 3 last/this month. That's how brand-new artists get in (Ninajirachi, Geese, CORTIS). ListenBrainz artist popularity deliberately does **not** gate inclusion: it would add ~440K unlistened songs. - **Examples:** Olivia Dean "Man I Need" 50 → 75; "The Fate of Ophelia" 51 → 77; Tame Impala "Dracula" 0 → 80. - **Size / cost:** 5.90M → 6.30M rows exported (+7%); 8.0 GB scanned (~$0.05); `EXPORT DATA` ~23 s, well under the 300 s function timeout. - `deploy.sh` now runs `gcloud functions deploy` itself. Pulumi's source object has no generation pin, so a new upload never rebuilt the function, and the script's "updated on next Pulumi deploy" message was wrong. - Also: an SQL-contract test, a blank-title filter, and the ARCHITECTURE.md row. ## Evaluation The candidate SQL was exported to a scratch prefix (`gs://nomadkaraoke-kn-data/song-id-eval/lb1/`), then built and evaluated on the NomadPC with kjbox #261: - Labelled: 72% → 80% auto-identified, wrong 2 → 0. - Freshness cases: 18 → 26 of 26. - Frozen held-out: 84% unchanged, wrong 8 → 7. ## Deploy After merge: run `infrastructure/functions/kn_data_sync/deploy.sh`, then `gcloud scheduler jobs run kn-data-sync-full-daily --location us-central1`. The NomadPC picks the new run up at its next sync. ## Testing `cd infrastructure/functions/kn_data_sync && python -m pytest test_main.py`: 6 pass. The SQL was dry-run and executed in prod BigQuery (read-only queries plus the scratch export). 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * The song-identification index now includes eligible MusicBrainz songs using ListenBrainz listener activity and popularity alongside Spotify popularity. This expands coverage beyond the previous Spotify-popularity-based selection and includes all KaraokeNerds entries. * Song popularity can reflect recording-level ListenBrainz scores, with artist popularity used as a fallback. * **Documentation** * Updated the documented index size to approximately 6.6 million rows. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
48e5b6bfe2 |
feat(review): Edit Segment timeline — click waveform to play, Ctrl/Cmd-click word to delete (v0.248.0) (#1079)
## Summary Brings the Edit Segment modal's timeline to parity with the main Waveforms review rows: 1. **Click the waveform to play from that point** — previously only the ruler/timeline strip was a click target in the modal; the waveform only was in compact rows. 2. **Ctrl/Cmd-click any word bar to delete it** — no more scrolling the word list to hit delete. ## Changes - `TimelineEditor`: waveform wrapper always handles click-to-play (pointer cursor when `onPlaySegment` is set); new `onWordCtrlDelete` prop fires on Ctrl/Cmd+mousedown on a word bar (incl. its resize handles). Modifier is read from the event, so no key-state tracking. - `EditTimelineSection` → `EditModal`: wired to the existing `handleRemoveWord`; disabled during Tap To Sync so indices don't shift mid-sync. ## Testing - New `TimelineEditor.test.tsx` (waveform click → time mapping in compact + non-compact; Ctrl/Meta delete; plain click doesn't delete; handles delete) + `EditModal` test (Ctrl-delete removes word, recomputes text/start on save). 5 of the new tests fail without the source change. - Full frontend jest: 120 suites / 1430 tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Ctrl/Cmd-click a word or its resize handle to delete that word from the timeline. * Click the waveform to play from the selected point in both standard and compact layouts. * **Bug Fixes** * Word deletion updates the saved segment’s text and start time to reflect the remaining words. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
3e855ff65d |
perf(jobs): run download/lyrics/bulk-search Cloud Run Jobs in us-east4 — cut 2-5 min job start latency (v0.247.1) (#1077)
## Summary Every gen job waited **2–5 min** before `audio-download-job` (and then `lyrics-transcription-job`) even started. That blocked kjbox's goal of singing within about 4 minutes of making a song. **Root cause: Cloud Run Jobs in us-central1 are slow to start on Google's side.** Our image, CPU and code aren't the cause. - The `Started` condition fires about 10 s before the first Python log, so our process startup is fine. - decide's 203 MB image on 4 CPU is just as slow in us-central1. - A throwaway job running Google's tiny sample image, 4 runs per region: **us-central1 4m–5m07s**, **us-east4 8–19 s**, **us-east1 8–25 s**. All four us-central1 runs started in the same second, as if released from a queue. Quotas are far from their limits. ## Changes - **backend**: new `CPU_JOBS_REGION` setting (default `us-east4`) for triggering `audio-download-job`, `lyrics-transcription-job` and `bulk-search-job`. `GCP_REGION` is unchanged. `video-encoding-job` stays in us-central1, because it pulls multi-GB outputs from the us-central1 bucket. - **infra (Pulumi, already applied)**: the three jobs are created in us-east4 as new resources. The legacy us-central1 copies are kept as the rollback path, with deletion protection turned off so a follow-up can remove them. The preview after applying showed 289 unchanged, so there's no drift. - **ci**: the CPU app image is also pushed to `us-east4-docker.pkg.dev/nomadkaraoke/karaoke-backend-gpu/karaoke-backend-cpu`, so jobs pull from their own region. The lyrics job image update now targets us-east4. After each deploy, CI starts a no-op `audio-download-job` execution so the one-time ~2.5 min image import doesn't land on the first customer. - `scripts/rollback.sh`: lyrics job moves to the us-east4 image, and jobs whose target tag is missing are skipped instead of aborting the loop. - Docs: TROUBLESHOOTING runbook (measure + rollback), ARCHITECTURE, LESSONS-LEARNED. ## Testing - `make test`: all backend tests pass, plus 1422 frontend tests. - New tests in `backend/tests/test_worker_service_cpu_jobs_region.py`: routing of each moved job, video job stays in `GCP_REGION`, env override for rollback, and the Pulumi/backend region constants must match. - Prod smoke test of the new us-east4 `audio-download-job` with the real image (`import backend.workers.audio_download_worker`): **Started in 7.0 s, 8.6 s, 10.0 s** (first run of a new digest adds a one-time ~2.5 min import, which the CI step covers). - Local review via a reviewer agent: no must-fix findings; the rollback-path suggestions are addressed. ## Rollout One-time image seed (Cloud Build gcrane copy) and `pulumi up` are done. After merge, CI deploys the backend, which starts triggering the us-east4 jobs. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Performance** * Audio downloads, bulk searches, and lyric transcriptions now run in a region with faster job startup times, reducing delays before processing begins. Video encoding continues to run in its existing region. * **Reliability** * Rollbacks now skip jobs when the target image is unavailable and continue with the remaining jobs. Other image lookup errors still stop the rollback. * **Documentation** * Updated architecture and troubleshooting guides with regional job details, startup-delay guidance, and rollback instructions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
518126727b |
feat(tenant): tenant users never see or use consumer credits (#1076)
## Problem A tenant user's first login on `randy-vild.nomadkaraoke.com` showed the consumer "We weren't able to grant free credits" screen with **Buy Credits**. The backend also sent a credits-denied email. Tenant jobs were being charged consumer credits too, so the client's first submit would fail with "out of credits". Tenants are companies or bulk clients on a separate commercial agreement. ## Changes - **Sign-in:** - Tenant magic links skip the whole welcome-credit flow: no AI eval, no denial or welcome email, no referral attribution. `credit_status` comes back as `not_applicable`. - Sending a tenant magic link no longer precomputes a credit evaluation. - **Jobs:** `create_job` never credit-checks or charges jobs with a `tenant_id`. They're marked `payment_bypassed`, so duration reconciliation never pauses them for credits and refunds never mint credits. - **Access:** an empty tenant allowlist now means **admins only**. Free jobs must never be open to arbitrary sign-ups. All 3 live tenants already have allowlists. The admin UI copy is updated. - **Frontend:** on tenant portals, the credits badge and menu, Buy Credits, and Referrals are hidden. The verify page never shows credit screens for tenant logins. ## Tests - `test_tenant_no_consumer_credits.py`: a tenant job isn't charged and is marked `payment_bypassed`; consumer jobs are still charged and still rejected without credits. - Verify route: a tenant login returns `not_applicable` and the grant is never called. - Model: an empty allowlist means admins only. - `AuthStatus`: tenant mode hides credits. - Results: backend 4953 passed; frontend 1423 passed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Access** * Tenant portals with no configured email addresses or domains are restricted to administrator sign-ins. Configured allowlists continue to limit access to listed users and administrators. * **Tenant Portal** * Tenant sign-ins no longer receive consumer welcome-credit evaluations or related credit emails. * Tenant jobs bypass consumer credit checks and charges. * Credit balances, credit-purchase options, and referral links are hidden in tenant portals. * Successful tenant sign-ins proceed to the app regardless of credit status. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
2b8439a0ae |
fix(auth): own domain never disposable; tenant-allowlisted emails skip disposable check (#1075)
## Problem Signing in as `randy@nomadkaraoke.com` on the Randy Vild portal showed "We don't support disposable email addresses". `nomadkaraoke.com` had been **auto-learned into the disposable-domain blocklist** (`blocklists/config.manual_domains`). Its `verified_clean_domains` entry dated from 2026-07-04 and has a 7-day TTL. After it expired, an external API (DeBounce/verifymail) flagged the domain. The result: **every `@nomadkaraoke.com` magic-link sign-in was rejected.** ## Fix - **Prod data (already done through the admin API):** removed `nomadkaraoke.com` from `manual_domains` and added it to `allowlisted_domains`. - **Code:** - New `NEVER_DISPOSABLE_DOMAINS = {"nomadkaraoke.com"}`: a tier-0 short-circuit before any list or API check, and `_auto_learn_domain` refuses it (and well-known providers). - On tenant portals, an email the tenant explicitly allowlisted (`allowed_emails` or an allowed domain) skips the disposable checks, because an admin vouched for it. Everyone else is still checked. ## Tests - Regression tests: our domain isn't flagged even when it's blocklisted and the API says it's disposable; auto-learn refuses it; a tenant-vouched email skips the check while other emails on the same portal still get 422. - `pytest backend/tests --ignore=backend/tests/emulator`: 4940 passed. No local CodeRabbit review was run, so this PR is left open for the GitHub bot. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Explicitly allowlisted email addresses can now request magic links even if their domain is flagged as disposable. Other addresses on the same tenant portal remain subject to the existing check. - `nomadkaraoke.com` is no longer treated as a disposable domain or automatically recorded as one. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.246.2 |
||
|
|
fe217fda1f |
kjbox singer search: Gemini fallback with the job flow's catalog tidy + descriptive queries (v0.246.2) (#1065)
## Summary The Gemini fallback for kjbox's singer song identification (kjbox `docs/SONG-IDENTIFICATION.md`). kjbox's on-device matcher answers most searches; this endpoint handles what it can't. - `POST /api/kjbox/catalog/resolve` now runs `resolve_and_tidy`: 1. One Gemini call (`gemini-3.8-flash`) splits the singer's one-line query into artist/title and fixes typos. 2. It now also resolves **descriptions**: "that song from titanic" → Céline Dion — My Heart Will Go On; "is this the real life" → Queen — Bohemian Rhapsody; "song from top gun" → ambiguous: Danger Zone / Take My Breath Away / Hold My Hand (live-checked). 3. The chosen song goes through the job flow's own `judge_match(stage="fast")` catalogue tidy, so formatting matches gen's job form. - AI kind `none` keeps its typed split, so kjbox can pre-fill its make-it form. A failed catalogue pass is returned uncached. - kjbox calls it only when on-device identification and karaoke search both found nothing, or when the singer uses "Describe it". It's cached and rate-limited as before. - Docs: `docs/API.md` notes that the match judge's corrections are Gemini in practice (its catalogue pass is exact-match only and decide's catalogue is prefix-only). ## Tests - `test_match_judge_free_text.py`: resolve_and_tidy cases, descriptive prompt coverage - `TestCatalogResolve` - Local CodeRabbit: findings fixed, re-review clean. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
5a9d0024a5 |
fix(quick-version): ffmpeg 4.4 never-ending render + fast ONNX model (v0.246.1) (#1074)
## Summary First production make-it quick version (job `405eed71`, Rihanna "Push Up On Me") failed with `render timed out`: - **Render never ended**: the GPU image ships ffmpeg **4.4** (Ubuntu 22.04). With the guide-vocals `amix` in the filtergraph plus the infinite lavfi colour + looped PNG inputs, it ignores `-shortest` — reproduced in `ubuntu:22.04`: a 20 s song produced 540 s+ of video until killed. ffmpeg 8 (local dev) stops fine, which is why local tests passed. Fix: every input and the output are bounded with `-t <duration>` (colour source `d=`). Verified in `ubuntu:22.04`: 3.7 s render, exactly 20.0 s audio+video. - **Quick separation too slow**: the baked roformer took 14 s load + 124 s for a 3:15 song on the L4. Switched to fastgen's `UVR-MDX-NET-Inst_HQ_4.onnx`. It's downloaded on load until the next GPU base rebuild bakes it (`download_models.py` `EXTRA_MODELS`), with a fallback to the baked roformer if the load fails. Prod timeline observed for that job (for the <4 min goal): job created 05:22:12 → **audio-download-job container took 3m46s to start** (existing issue, 1m40–4m across recent executions; separate follow-up) → audio in GCS 05:26:46 → GPU worker started 05:27:10 → quick separation done 05:29:49 → render hung. ## Testing - `backend/tests/test_quick_version.py`: 40 pass (new: every stream bounded to the duration; model fallback; both-fail raises). - Docker `ubuntu:22.04` ffmpeg 4.4.2 reproduction before/after (see above). 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Quick audio separation now uses a faster model by default, with an alternate model available if the primary model cannot be loaded. * **Bug Fixes** * Quick video rendering is now limited to the song’s duration, helping prevent renders from continuing beyond the audio or failing to finish. * Model loading failures are handled more reliably: if both the primary and alternate models fail to load, the error is reported. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>v0.246.1 |
||
|
|
37f7d578ee |
kn-data-sync: daily export of the kjbox song-identification index (#1066)
## Summary After the daily KN `full` refresh, `kn-data-sync` now also exports the **kjbox song-identification index**: every song a singer might mean. - **Contents:** `spotify_tracks_normalized` (popularity ≥ 30) FULL OUTER JOIN `karaokenerds_raw`, deduped on normalised artist+title. About 2.0M rows: artist, title, popularity (blank = KN-only), karaoke (0/1). - **Export:** `EXPORT DATA` to `gs://nomadkaraoke-kn-data/song-id/<run>/songs-*.tsv.gz`, then a manifest at `song-id/latest.json` naming that run's shards, so a device never mixes runs. The last 3 runs are kept. - **Cost and safety:** it's a server-side export; no rows pass through the function (512M memory is fine). The dry run scans about 160 MB (~$0.001/day). A failed export is reported in the response and doesn't fail the KN sync. - No IAM change: the SA already has project `bigquery.dataEditor` + `jobUser` and write access to the bucket. Design and reasoning: kjbox `docs/SONG-IDENTIFICATION.md` (on-device matcher first, Gemini fallback). The NomadPC side (download → build `song_id.db`) is in the kjbox PR. ## ⚠️ Deploy (manual, needs GCP write) 1. `infrastructure/functions/kn_data_sync/deploy.sh` (uploads the source zip) 2. `pulumi up`, or redeploy the function. The source object name doesn't change, so force the function to pick up the new zip. 3. Optionally trigger it once: `gcloud scheduler jobs run kn-data-sync-full-daily --location us-central1`, then check `gs://nomadkaraoke-kn-data/song-id/latest.json`. ## Tests - `infrastructure/functions/kn_data_sync/test_main.py` (5): manifest names only this run's shards, prune keeps 3 runs, empty export raises, full sync survives an export failure, community mode doesn't export. Run manually (function tests aren't in the repo testpaths). - The rendered SQL passes a BigQuery dry run. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Full catalog syncs now generate a song-identification index covering popular Spotify tracks and KaraokeNerds entries. The index is exported to cloud storage, with a manifest pointing to the latest run and the three most recent runs retained. * **Bug Fixes** * If index export fails, the catalog sync still completes and reports the export failure. Community syncs do not generate an index. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |