mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
main
113
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
1840645faa |
feat(youtube): Firestore catalogue of every channel video with change history (v0.297.3) (#1270)
## Summary Follows #1269. The daily description backfill already fetches every channel video's snippet, status and content details, then throws them away. This saves them to Firestore as a lasting record of the channel, so we can investigate YouTube video activity later. It uses no extra YouTube quota. ## Data model - `youtube_videos/{video_id}`, one doc per video: - from YouTube: title, description, tags, published_at, privacy_status, upload_status, category_id, duration, default_language, made_for_kids, license, embeddable; - parsed from the title: artist, song_title, brand_code, kind, backfill_target; - bookkeeping: first_seen_at, last_seen_at, last_changed_at, in_uploads_playlist, deleted, deleted_at. - `youtube_videos/{id}/history/{auto}`: `{at, event, changes: {field: {old, new}}, source}`. - `event`: first_seen, changed, deleted or restored. - `source`: `backfill` for this job's own rewrites, otherwise `observed`. - If a Studio edit and a backfill rewrite land in the same run, they're logged as two separate entries. - **Soft-delete only.** A known video YouTube stops returning gets `deleted=True` and a `deleted_at`. It stays in the known-ID list and keeps being checked, so if it comes back that's logged as `restored`. - **Writes only on change.** `last_seen_at` is refreshed at most once every 20 hours. A video's doc and its history entries always go in one Firestore batch. - **Best effort:** if the catalogue sync fails, the run logs a WARNING and still succeeds. A dry run skips it. ## Changes - New `backend/services/youtube_video_catalog.py` (100% covered). - `fetch_snippets` also requests `contentDetails`, at no extra quota cost. - The worker passes the metadata from before each rewrite (for attribution), keeps gone IDs in `known_video_ids`, and adds `catalog` stats to the run summary. - README `scripts/youtube-descriptions` documents the catalogue. Version 0.297.3. ## Testing - `pytest backend/tests -k "youtube_comment or backfill or video_catalog"`: 129 passed. - 12 new tests use an in-memory Firestore fake. They cover first seen, unchanged same day and next day, Studio edits, backfill attribution, both kinds of change in one run, soft-delete once, a stub for a video never catalogued, restore, playlist-only changes, and batches never splitting a doc from its history. - After deploy: one manual run on prod to fill the catalogue (expect about 1.9k new docs), then check it. Local CodeRabbit: 2 rounds, 3 minor issues fixed (source attribution, doc and history in one batch, a video first seen and rewritten in the same run). @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_012RYLVVbErmEFhmiW48uuFK --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
4a23398c04 |
fix(youtube-backfill): check known video IDs the uploads playlist drops (v0.297.2) (#1269)
## Summary Andrew got "[YouTube Descriptions] 5 updated, 0 remaining" on 2026-11-14. The cause: the channel's uploads playlist (`playlistItems.list` on `UU…`) isn't a complete list of the channel. - Its pages overlap: 1914 items, but 11 are duplicates. - It leaves out public videos for weeks at a time. Five 2023 This Is Me Smiling videos were missing for the whole Oct 22 template cycle, so the backfill reported "complete" while they still had the old description. They came back on 11-14 and were rewritten. - `hSPiNfrqf5w` (Alive in the Chase) is still on the pre-10-22 template, and `QP2lKxkfeqE` is missing from the playlist again right now. ## Changes - `youtube_backfill.iter_all_video_ids` removes duplicate IDs. - New `youtube_backfill.enumerate_channel(youtube, known_ids)`: lists the playlist, then looks up known IDs it left out directly with `videos.list` (about 1 unit per 50 IDs). Returns `recovered_ids` and `gone_ids`. - The worker keeps every ID it has seen in Firestore `youtube_backfill/known_video_ids`. - The first run seeds the set from the quota ledger's `desc-backfill:<id>` entries (1800 IDs on prod, read-only check). - If seeding fails, nothing is saved, so the next run retries. - Deleted videos are dropped from the set. - A dry run saves nothing. - The run summary and progress email report `recovered_from_known`. A WARNING log names the IDs the playlist omitted. - Docs: `scripts/youtube-descriptions/README.md`, `docs/LESSONS-LEARNED.md`. Version 0.297.2. ## Testing - `pytest backend/tests -k "youtube_comment or backfill"`: 114 passed. - New tests cover duplicate removal, recovering videos the playlist omits, gone IDs, seeding from the ledger, a failed seed being retried, no saves on a dry run, and the email count. - Seeding checked against prod Firestore (read-only): 1800 IDs, including all 5 affected videos and Alive in the Chase. - The first prod run after deploy should log "uploads playlist omitted N known videos" and rewrite Alive in the Chase. Local CodeRabbit: 2 rounds. Fixed 1 major (failed seed was never retried) and 1 minor (loose test assertion). @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_012RYLVVbErmEFhmiW48uuFK --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
2db5407a5d |
feat: archive deleted jobs (doc + logs) instead of losing them (v0.295.0) (#1261)
## Why
When NOMAD-1875 was deleted (2026-11-09), its Firestore record and logs
were gone for good, so we couldn't use them to debug the delete-outputs
result afterwards. Andrew's call: job records are **system data**. Keep
them forever, and on account deletion redact the user's identity rather
than deleting the record.
## What
- `FirestoreService.archive_job()` copies `jobs/{id}` to
`jobs_archive/{id}`, adding `archived_at`, `archived_by` and
`archive_reason`. It also copies the `logs` subcollection to
`archived_logs` without `ttl_expiry` (the TTL policy is on the `logs`
collection group, so otherwise they'd expire after 30 days), and the
`audio_edit_sessions` subcollection.
- `JobManager.delete_job()` archives first and **aborts the delete if
the archive fails**.
- Callers pass who deleted the job and why: `admin_delete` /
`owner_delete` from `DELETE /api/jobs/{id}`, and
`cleanup_distribution_delete`.
- **Account deletion** now also redacts archived jobs: owner/customer
email becomes `deleted:<uuid>`, and IPs, notes and contact fields are
removed. These are single-field queries, so no new composite index is
needed. Archived jobs are never deleted.
- Bulk test cleanup (`delete_jobs_by_filter`) still hard-deletes.
- `scripts/get_job.py` falls back to the archive. Documented in
`docs/ARCHITECTURE.md`.
No infra change: Firestore creates the collection on first write, and no
TTL policy is set because archived jobs are kept forever.
## Testing
- Unit: archive happens before delete; the delete is aborted when the
archive fails; archived jobs are redacted on account deletion (including
made-for-you customer jobs).
- Emulator: `backend/tests/emulator/test_job_archive_emulator.py` checks
the doc, logs (no `ttl_expiry`) and audio-edit sessions are copied, and
that a missing job returns False.
- `test_account_deletion`, `test_job_manager`, `test_jobs_api`,
`test_services`, `test_api_integration`: 195 passed.
- Local CodeRabbit: no findings.
@coderabbitai ignore
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_012RYLVUwKapcjF8czVQrPD5
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
ce57b4b598 |
fix(review): Tap To Sync partial pass keeps word order; local stack never reaches prod (v0.288.1) (#1237)
## Summary
### 1. Tap To Sync: a pass that stops before the last word
(customer-facing)
If a pass stopped before the last word (auto-stop about a second after
the line end, or the user pressing Stop), the untapped words after the
last tapped one kept their **old** timings, and those could now start
*before* the last tapped word. On gen's fake fixture line ("Nothing here
was ever transcribed", words 15.5 to 19.5, next line at 20.0), taps at
16.0, 17.2, 18.4 and 19.6 followed by auto-stop left "ever" at 19.6 and
"transcribed" at a stale 18.74 to 19.5. The two words were out of order,
and the video rendered them out of order.
- `frontend/hooks/useManualSync.ts`: new pure
`commitPartialSyncPass(words, synced)` runs in `cleanupManualSync`,
which is both the auto-stop path and the Stop toggle. It clears (sets
start/end to null) every untapped word after the last tapped one that
starts before it, then caps the last tapped word's end before the next
still-timed word in the line. The existing untimed-lyrics guard then
blocks finishing and points the user at those words. This is the same
rule as nomad-mobile's review-core `commitTapSync`
(`packages/review-core/src/tapSync.ts`, branch
`feat/sess-20261030-1930-e2e-health`). It uses the hook's existing
`syncedIndicesRef`.
- Backend backstop, `backend/services/timing_sanitizer.py`
`sanitize_corrections`: within a line, a word that starts before the
previous word's **start** is now treated like any other invalid start.
Small overlaps (a start a few ms before the previous word's end, common
in AI transcriptions) are left exactly as on main, so they get no move,
no WARNING and no edit-modal toast. It is **moved** to the previous
word's end, not cleared. I chose moving because that is how the module
already repairs bad or missing starts, because a cleared word would just
be refilled at prev_end on the next sanitize, and because nulls would
crash the render (`find_missing_timings` only guards the preview). The
module logs one aggregated `WARNING` per submission (never ERROR, since
this is an expected and repaired condition).
- `frontend/lib/lyrics-review/sanitizeWordTimings.ts` (the frontend
mirror, run when the edit modal opens) applies the same ordering rule,
with the same exemption for small overlaps.
### 2. The local stack must not reach production
nomad-mobile's E2E runs gen web inside an Android emulator's Chrome.
- `components/google-analytics.tsx`: GA (`G-D0E63LB9FY`) is no longer
loaded when the page host is localhost, 127.0.0.1 or 10.0.2.2. The check
runs at runtime after mount, so it also covers CI, and the server render
and first client render match.
- `lib/api/autoCorrect.ts` and `lib/api/customLyrics.ts` now use
`API_BASE_URL` from `lib/api.ts` instead of `NEXT_PUBLIC_API_BASE ??
'https://api.nomadkaraoke.com'`. That env var is set nowhere. A local
page now calls its own backend through relative URLs, so it never sends
its bearer token to production.
- New `lib/local-host.ts` is the single list of local hostnames, shared
by `lib/api.ts` and GA. `lib/api.ts` now also treats 10.0.2.2 as local.
- `scripts/local-stack.sh` backend_env: blank `FLACFETCH_API_URL=` and
`FLACFETCH_API_KEY=`, so direnv can't point the local stack at the real
flacfetch.
Version is bumped to 0.288.1.
## Testing
Each change had a failing test written first.
- Frontend jest (hook, sanitizer, edit modals, GA, API clients,
api/local-mode, auto-correct, custom lyrics): 13 suites, **150 passed**.
- New tests: 3 partial-pass hook tests (auto-stop, Stop, and the
keep-and-cap case), 2 sanitizer tests (the stale-word move, and a 20 ms
overlap left untouched), 4 GA tests, and 8 API-base tests (local hosts
use relative URLs, the production host uses the production API).
- Backend pytest: `test_timing_sanitizer.py` +
`test_jobs_corrections.py` **20 passed** (3 new, including a 20 ms
overlap that is left untouched), and `test_routes_review.py` **114
passed**.
- `customLyrics.test.ts`'s AbortSignal test now asserts that the
caller's abort reaches fetch, not that the signal object is identical. A
relative URL makes `apiFetch` manage the request and wrap the signal.
## Notes
- The web hook has no view of the next line, so the cap only uses words
in the same line. The backend sanitizer and segment bounds handle the
rest.
- The render-time `SegmentResizer._sanitize_segment_timings` is
unchanged (it clamps to the window only).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_012RYLVSLS3eVDqPctfYBMTt
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved lyric timing cleanup when recording or editing stops before
the final word. Out-of-order timings are corrected while small overlaps
are preserved, helping keep lyrics synchronized without changing valid
timings.
* Corrected out-of-order word timings during lyric cleanup and editing.
* **Improvements**
* Local app use now works more consistently across supported addresses,
including Android emulators, with API requests routed appropriately.
* Analytics scripts are now omitted on local pages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
65940c6f57 |
fix(local-stack): decide partner calls go to local decide, never production (#1235)
The local stack's backend env had no `KARAOKE_DECIDE_API_URL`, so gen's decide partner client (account deletion) used the production default and sent `DELETE /api/partner/gen/users` to decide.nomadkaraoke.com. Production answered 401, because the local secret differs, so nothing was touched. It surfaced in nomad-mobile's E2E account-1 flow. The fix defaults it to decide's local stack on :8030, and callers can still override it. Script only, no runtime code. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Configuration** * The local environment can now use a custom karaoke decision service address, with a default address provided when none is configured. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
437b34ee64 |
fix(ci): wheel over PyPI's 100 MiB limit — drop unused RSC payloads + size guard (v0.287.2) (#1233)
## Summary
Every `Deploy - Publish to PyPI` since v0.279.4 (about 10-22) has failed
with `400 File too large. Limit for project 'karaoke-gen' is 100 MB`.
The wheel had grown to 110 MB (105.1 MiB), so PyPI is still on 0.279.4.
Backend deploys were not affected. I found this while watching deploys
during an alerts-triage session.
In the wheel, 66 MB of the bundled Next.js export is RSC `.txt` payloads
(about 3 MB per locale × 33). The only thing that uses the export is the
CLI local-review server
(`karaoke_gen/lyrics_transcriber/review/server.py`). It mounts only
`/_next` and `/static`, and every other route returns an HTML entry
point. Next's RSC fetch paths (`/{locale}/app/.../index.txt`) already
get HTML from those routes, so the `.txt` files are never served.
- `make build-frontend` deletes `*.txt` from the copied export. On the
real CI wheel from run 40826404322 this takes it from 105.1 MiB to 41.1
MiB.
- `scripts/verify_wheel_frontend.py`, which already runs in both
package-build jobs, now fails above 95 MiB. Growth will show up on the
PR instead of as a failed publish after merge.
## Testing
- New `tests/unit/test_verify_wheel_frontend.py`: 5 tests covering the
pass case, missing entrypoint, too few files, the size guard, and
headroom below 100 MiB.
- Re-zipped the real 0.286.0 CI wheel without the `.txt` files: 41.1
MiB, and the verifier passes. The verifier rejects the original wheel
(105.1 MiB) with the new error.
- CodeRabbit: 1 finding ("deleting RSC payloads can break client
navigation"). Skipped, because the local server never serves those files
(see above), so it can't change runtime behaviour.
@coderabbitai ignore
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_012RYLVTiUPKxn3sQPyp3Teh
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
1715a07010 |
local stack: env overrides for parallel E2E lanes (defaults unchanged) (#1222)
Lets several local stacks run on one machine for nomad-mobile's parallel Maestro lanes (the spare MacBook Pro runner): \`LOCAL_STACK_PROJECT\` (per-lane Firestore project on the shared emulator, so global fake-config does not leak between lanes) and \`LOCAL_STACK_STATE_DIR\`, plus \`LOCAL_STACK_KEEP_EMULATORS=1\` for \`down\`. Every default is unchanged, so existing local and CI runs behave exactly as before. Docs: docs/TESTING.md § Local stack. Context: nomadkaraoke/nomad-mobile#21 (E2E off Andrew's Mac). 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Local development stacks can now use configurable state directories and Firestore project IDs, making it easier to run isolated stacks side by side. * You can keep shared emulators running when stopping a stack, and backend log locations reflect the selected state directory. * **Documentation** * Updated local setup guidance with recommendations for ports, projects, state directories, and sharing emulators across stacks. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
d76a136ee6 |
feat: mobile Phase 6 — RevenueCat IAP webhook, Expo push, self-serve account deletion (flagged off), App Review demo login (v0.284.0) (#1220)
Phase 6 (mobile release readiness) — gen side. Spec: nomad-mobile
`docs/superpowers/specs/2026-10-25-mobile-phase6-release-readiness-design.md`
(§4 payments, §5 push, §6 deletion, §9 demo account).
## What
- **`/me` app fields:** opaque `app_user_id` (UUID, used for
RevenueCat/Sentry — no email to third parties),
`capabilities.account_deletion`, `push_prefs`.
- **In-app purchases:** `POST /api/users/webhooks/revenuecat` (shared
Authorization header). Exactly-once grants keyed by
`{store}:{transaction_id}` inside the same Firestore transaction as the
balance change; refunds (REFUND / CANCELLATION with CUSTOMER_SUPPORT or
negative price) debit the granted account, refund tombstones block late
grants; other cancellations ignored. `store_payments` for accounting +
revenue sums; referral earnings on **net** proceeds; sandbox grants but
excluded from revenue/referrals. Credit packages gain `store_product_id`
+ server-driven `web_link` (iOS US only for now).
- **Push:** `POST/DELETE /api/push/expo-token` (token moves between
users atomically), `PATCH /api/users/me/push-prefs` (ready / review /
problems, gates Expo **and** Web Push), Expo sender (batched,
DeviceNotRegistered cleanup), new FAILED / AWAITING_AUDIO_SELECTION
triggers.
- **Account deletion** (`ACCOUNT_DELETION_ENABLED`, **off in prod**
until Andrew's manual pass): `GET /api/users/me/deletion-preview`,
`DELETE /api/users/me`. Refuses admin / unpaid referral balance / Stripe
Connect / made-for-you in progress (409). Cancels active jobs, deletes
decide data (partner call, twice), RevenueCat subscriber, revokes Sign
in with Apple (when a code is given), queues kjbox
(`/api/kjbox/account-deletions` pull + ack), deletes **inputs only**
(uploads, input/, audio_edit/, style/, staged instrumentals) — **never
published outputs** — strips personal fields, anonymises
jobs/referrals/song requests/show-credit records, keeps payment records.
Every step idempotent; failures → 503 retry; sessions revoked last.
- **Web:** `/account/delete` page (Google Play web deletion path),
hidden while the flag is off.
- **App Review demo login:** fixed code for one configured email — inert
until `APP_REVIEW_DEMO_EMAIL` is set (not set in prod).
- **CI:** mounts `REVENUECAT_WEBHOOK_AUTH`, `REVENUECAT_SECRET_API_KEY`,
`APP_REVIEW_DEMO_CODE` (secrets exist; runtime SA already has accessor).
## Tests
6838 backend unit tests pass (emulator suite needs emulators;
iap/deletion emulator tests pass locally), frontend jest 1785/1785.
Per-task reviews + local CodeRabbit review (5 minor findings fixed).
@coderabbitai ignore
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01CZf1cM9vefnNw5djARRhyA
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
53bed95e76 |
feat(local-stack): kjbox partner secret + fake quick version (v0.280.1) (#1206)
## Summary Local-stack only (no prod impact). For mobile Phase 5 "Tonight": the kjbox local stack points at gen's local stack for make-it. - `scripts/local-stack.sh`: `KJBOX_PARTNER_SECRET` and `DECIDE_GEN_PARTNER_SECRET` default to `local-stack-secret`, external values win. - Fake workers emit a quick version (`state_data.quick_version` ready + `file_urls.quick.video_mp4`) for kjbox-sourced jobs, using prod's `should_build_quick_version` gate and quick-version helpers. `FAKE_WORKERS` is still refused in production. - docs/TESTING.md § Local stack updated. ## Testing `test_kjbox_partner_api.py`, `test_fake_workers.py`, `test_kjbox_resolve_partner.py`, `test_fake_match_judge.py`: all pass. Reviewed by a task reviewer (1 fix round). @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_013Bkasn6trBadg1Zb39vSVR --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
6c53f0471a |
feat(youtube): auto-reply to song requests in YouTube comments with requests-board links (v0.280.0) (#1205)
## Summary Backlog Frog #6 (YouTube channel → gen funnel). Andrew's spec: *"we could probably also solve that by building some automation to detect youtube comments asking for songs and auto-reply with a link to the requests board"*. An hourly bot reads new top-level comments on the Nomad Karaoke channel and, when someone asks for a song: - **Song already on the channel:** replies with that video's link. - **Otherwise:** adds the song to the requests board (or finds the existing request) and replies with a link straight to it, e.g. `Added "Tiny Hitman" by Bear Ghost to our requests board 🥳 upvote it here and we'll email you when it's made: <link>`. - **Vague asks** ("more Johnnie please") get a generic board link. - **Non-English comments** get a translated reply, which falls back to English if a URL doesn't survive translation. The reply promotes only the board, not gen (Andrew's call). The goal is turning anonymous YouTubers into board users with an email address. **Andrew signed this off on a 6-month local replay** (`scripts/youtube-comments/replay.py`, read-only): - 265 threads, 33 song requests. - It flagged all 29 requests Andrew had answered by hand, and all 7 vague asks. - It posted no replies to non-requests. - His calls on borderline cases became prompt rules: yes to Playa Fly, no to "Joe Cuervo Karaoke 🎤" on an unrelated video. ## Changes - **Decision core:** `backend/services/youtube_comment_bot.py`, a pure module with injectable ports (Gemini classifier/translator, board, channel index), shared by the replay and the worker. - **Worker:** `backend/workers/youtube_comment_bot_worker.py`, called through `POST /api/internal/youtube-comments/run` (`?dry_run=true`). - Creates a create-only action doc per comment **before** posting, so nothing gets a double reply. - Keeps a high-water mark with a 6h overlap. A Gemini error causes a retry, not a permanent skip. - Per-run cap and quota coordination. - Posts to #noise per reply; failures go to #alerts, throttled. - **Kill switch:** `YOUTUBE_COMMENT_AUTOREPLY_ENABLED` defaults to **false**, and this PR leaves it off. A follow-up PR switches it on after a prod dry run. `YOUTUBE_COMMENT_AUTOREPLY_START` enables a one-time backfill of unanswered requests. - **Requests board:** - New `source="youtube_comment"`; bot-added requests are owned by the placeholder `youtube-comment@nomadkaraoke.com` with no votes. - The **first real up-voter claims the request**, inside the vote transaction. The daily picker skips unclaimed requests, so a free credit never goes to the placeholder. - Frontend `?request=<id>` deep link: highlight, scroll, banner (33 locales). - Signing in from the link uses purpose `requests_board:<id>`, so the magic link returns to the request with `&upvote=1` and the upvote is cast once. - **OAuth:** the bot uses its **own** Desktop OAuth client and secret `youtube-comment-bot-credentials` (`youtube` + `youtube.force-ssl`), because Google refuses `force-ssl` in the device flow. The upload token is untouched. - **Infra (Pulumi, already applied locally):** secret container + Cloud Scheduler `youtube-comment-autoreply-hourly`. - **Description template:** a make-your-own line moves above YouTube's "...more" fold (`utm_campaign=make_cta_top`). The daily backfill re-drains the channel over about 11 days. - **Docs:** `docs/REQUESTS-BOARD.md` § YouTube comment auto-reply, API.md, README.md, plus the plan in `docs/archive/2026-10-22-youtube-comment-autoreply-plan.md`. ## Testing - `backend/tests/test_youtube_comment_bot.py` (33): skips, routing, vague asks, translation fallback, malformed classifier output, API helpers. - `backend/tests/test_youtube_comment_bot_worker.py` (12): kill switch, dry run, idempotency (second run and concurrent claim), go-live start, cap, quota, missing scope, post failure, classifier-error retry. - `backend/tests/emulator/test_song_request_service.py`: bot add/dedupe, claim on first upvote, downvote doesn't claim, picker skips unclaimed, human resubmit claims, `find_published_by_key`. - `backend/tests/test_admin_login_link.py`: `requests_board:<id>` redirect and purpose helper. - `frontend/e2e/regression/requests-board-deep-link.spec.ts` (3): highlight + sign-in purpose, upvote-once on return + param dropped, no toggle-off when already upvoted. - Full `make test`: backend 6584 passed; the only failures are the local-environment `test_emulator_integration` cases, which fail identically on `origin/main` (port 8080 is held by another session's emulator). Frontend Jest: 1760 passed. - Behavioural acceptance: the 6-month replay above. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_016E3ibN2EcKxnGQvEf6T4V8 --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9d6aa3df7d |
feat(medley): per-section backing_vocals_db for the karaoke track (v0.278.6) (#1184)
## Summary The Fiverr medley customer said the backing singers in two songs were too loud to sing over in tune, and asked for them as a faint background. - **New option:** a per-section `backing_vocals_db` in the medley spec. For the karaoke track only, the section is rebuilt from the clean instrumental plus the backing-vocal stem at that level, e.g. `-14` for a faint background. The rest of the karaoke track, the clean instrumental and the guide are unchanged. - **Cached mix:** the mix is cached per job/level, written to a `.partial.flac` and renamed, so an interrupted run never leaves a truncated mix to be reused. If the job has no backing stem, it stops with a clear error. - **Docs:** `docs/MEDLEYS.md` spec table. ## Testing - `test_make_medley.py`: parsing, the ffmpeg graph, only the karaoke variant changing, the atomic write and the cache. 20 + chord-chart tests pass. - Real data: the customer's medley with `-14` on Remember When and On the Other Hand. - Only those two sections change; the others differ by ≤ -43 dB, which is crossfade overlap. - Overall section loudness is unchanged. - A local CodeRabbit review found 1 issue (stale or partial cache). It is fixed. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
03ee693f28 |
feat(medley): customer chord charts + static-gain master (v0.278.5) (#1183)
## Summary Two follow-ups on the medley tooling, both from the Fiverr medley customer's feedback. - **Fixes the robotic artifact at the start of the karaoke audio.** The final single-pass `loudnorm` ran in dynamic mode. On a song that opens a cappella (HSLHT sings from 0.4s, band at 3s), the instrumental's first 3s is near-silent separation residue at -70 dB, and `loudnorm` boosted it by about 50 dB into a loud warble. The master is now one static gain to -14 LUFS plus a 4×-oversampled limiter at -1 dB (`master_filter`). After the fix, the first 3s measure -61 dB (were -15 dB). Sync is sample-identical and true peak is -0.9 dB. - **Imports the customer's own chord chart.** `scripts/medley/chord_chart.py` plus a new `make_medley.py chart` step read a chords-over-lyrics chart (.txt or .docx in Courier, capo shapes, `Capo N` / `Key change Capo 1` lines). They align it word-by-word to the reviewed lyrics, so chords come out timed on the words the chart puts them on. Detected chords fill intros, breaks and spoken verses, transposed into the capo shapes. - **Shows the capo in the chord box.** New `ChordChange.capo` field, shown as "capo N" under the chord. - **Docs:** `docs/MEDLEYS.md` gets a new section on importing a customer chart, and the mastering guidance now says never to use single-pass `loudnorm`. `docs/README.md` has an entry. ## Testing - Unit tests: - New: `tests/unit/test_chord_chart.py` (9). - Added: capo tests in `test_guitar_chords.py`, plus master and capo tests in `test_make_medley.py`. - Full `tests/unit` suite: 2100 passed, 65 skipped (run before the last CodeRabbit fix; only the 9 chord-chart tests were re-run after it). - Real data: the customer's 4-song chart against the medley. The lyrics-with-chords output matched the chart line by line, and 4K frames rendered with capo labels. - A local CodeRabbit review found 1 issue (two chord groups after the same word could come out in the wrong order). It is fixed and has a test. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01QV2TQwyC7pwVkGPCk98Bqg --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
7cf5beede8 |
feat: guitar chords in karaoke videos + ChordMini ensemble chord detection (v0.277.0) (#1175)
## Summary Adds optional guitar chords to karaoke videos, first used for a Fiverr medley customer who wants chords on their video. - **Rendering (gen library):** `OutputGenerator.generate_outputs(chords=[ChordChange(start, end, label)])`. - Each chord change during a sung line is drawn above the word it lands on, in a row reserved above every line (the same slot mechanism as translated lyrics). - Chords light up when played, using the same `\k` highlight as lyrics: gold while upcoming (theme `chord_color`), sung-lyric colour once played. Quick changes on one word never overlap. - A top-right "now / next" chord box runs for the whole song, covering intros and instrumental breaks. - With no chords supplied, output is unchanged. A test asserts the ASS is byte-identical. - **Detection (`scripts/medley/detect_chords.py`):** - Ensembles both ChordMini checkpoints (BTC + 2E1D, MIT; about 80% majmin/MIREX, the best open weights we found) over the full mix, gen's clean instrumental, and an htdemucs_6s harmonic mix (guitar + piano + bass + other). - Weighted per-frame vote, then merges short blips, snaps changes to beats and spells chords for the key (Ab/Db/Eb7, F#m, borrowed bIII/bVI/bVII as flats). - **Medley script:** new `chords` step plus a `"chords": true` spec option. Chords are clipped per section and hand over mid-crossfade, then shifted for the countdown. - **Docs:** `docs/MEDLEYS.md` § Guitar chords (setup, method, accuracy caveats, theme keys) and a docs/README.md entry. There's no job or UI option yet. The medley pipeline uses this for now; a product version would need a chords toggle plus detection in a worker. ## Testing - New: `test_guitar_chords.py` (12: chord timeline helpers, word placement measured against the font, layout geometry, highlight timing, now/next box, theme colour, no-chords path byte-identical), `test_detect_chords.py` (7), and 2 medley chord-shifting tests. - `tests/unit`: 2087 passed, 65 skipped. - End to end on the 5 real medley songs: models/inputs agree on 91–99% of major/minor labels, and the output matches the expected I–IV–V progressions. Rendered 1080p and 4K medley videos with 146 chord changes; spot-checked frames (verse, instrumental screen, chorus). - Local CodeRabbit: 1 finding (doc section reference), fixed. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01QV2TQwyC7pwVkGPCk98Bqg --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
85390fd12b |
docs: medley-making guide + spec-driven make_medley script (#1174)
## Summary Documents the multi-song medley process used for the first classic-country medley, and turns it into a reusable script so the next one only needs a spec file. - `docs/MEDLEYS.md`: the full process. One gen job per original song, then identical cuts on every audio variant, and the reviewed lyrics shifted onto the medley timeline and rendered with gen's own `OutputGenerator`. Also covers how to choose cut points (crossfades, pot-out/pot-in, keys/tempos, checking the brief against the recording) and gotchas. - `scripts/medley/make_medley.py`: `fetch` (Firestore + gcloud CLI) → `audio` (per-section LUFS gain, crossfade/gap/pot fades, `loudnorm` to -14 LUFS / -1 dBTP, three variants) → `video` (shifted reviewed lyrics + countdown + themed title/end screens + karaoke and guide MP4s). - `scripts/medley/example-spec.json`: the country medley as a worked example. - Linked from `CLAUDE.md` Quick Reference and `docs/README.md`. Ops tooling only. No backend or frontend change, so no version bump. ## Testing - `tests/unit/test_make_medley.py`: 12 unit tests covering spec validation, the timeline offset maths, lyric-segment shifting (including the same song used twice) and the ffmpeg filtergraph. - End-to-end run against the real prod jobs: `fetch` → `audio` reproduces the hand-built v1 medley **bit-for-bit** (max sample diff 0.0 on both the instrumental and guide WAVs). `video --resolution 360p` renders both MP4s and the LRC. - Local CodeRabbit review: 1 finding (unchecked ffmpeg exit code in `measure_lufs`), fixed. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01QV2TQwyC7pwVkGPCk98Bqg --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c277bc8ee3 |
feat: local E2E stack (fake workers + test API) + email-code sign-in (v0.275.0) (#1165)
## Summary - **Local stack:** `scripts/local-stack.sh up --frontend` runs the **real** backend on the Firestore + GCS emulators with `FAKE_WORKERS=1`. In-process fake workers move jobs through their real states in seconds, using tiny real media, so web Playwright and mobile Maestro full-journey tests can run offline against the same endpoints. - **Email-code sign-in** (nomad-mobile spec §4.1): `POST /api/users/auth/send-code` + `verify-code`. The code logic is shared with kjbox via `LoginCodeStore`, and sends are throttled per email and per IP. - **Bug fix found by the new suite:** `confirm-duration` moved jobs to `separating_stage1`, from which they can never reach review. It now resumes at `downloading`. No prod job had taken this path yet. ## Changes - `backend/services/fake_workers/`: dispatch seam in `WorkerService.trigger_*`, plus fake search, download, checkout and outbox, scenarios, and per-user config. The real stem upload, transcode, backing-vocals analysis and preview render are reused. Every check is strict (`is True`), so a MagicMock settings object never enables the fakes, and startup **refuses FAKE_WORKERS in production**. - `/api/test/*` router (mounted only in that mode): `status`, `scenario`, `fake-config`, `emails`, `reset`, plus the fake Stripe checkout page. Paying it runs the real webhook handler. - `frontend/e2e/local-stack/` (9 specs, ~50s): - sign-up → make → review → instrumental → download - buy and cancel credits - duration confirm, failed, complete and review states - code sign-in and lockout - New required CI job `local-stack-e2e`. - Docs: TESTING.md § Local stack, DEVELOPMENT.md, API.md, README, LESSONS-LEARNED. - `MAX_SIGNUPS_PER_IP` can now be set by env. Only the local stack raises it; production keeps the default of 2. ## Testing - [x] Backend unit: 6175 passed - [x] Emulator: 152 passed - [x] Frontend Jest: 1706 passed - [x] Local-stack Playwright: 9/9 passed (CI mode) ## Review - [x] CodeRabbit CLI review completed locally. 1 minor finding declined: it suggested swallowing infrastructure errors in send-code as fake "sent" responses, which would hide real outages from users. @coderabbitai ignore --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01AGuxtgoscF6RJGtpjHPg3W --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
39dcf2598e |
fix: PyPI prune snippet + force marker only in PR title (v0.268.10) (#1146)
## Summary Two follow-ups to #1145, both found while doing today's prune. 1. **Prune console snippet silently did nothing.** PyPI's delete-version form now requires five `acknowledge_*` checkboxes. A submit without them gets a **200 re-render**, which the old snippet counted as "deleted". The snippet now reads each release's real form (CSRF plus checkboxes), submits it, and counts a version deleted only when its manage page 404s. This is the exact approach used for today's prune (88 releases, 10.73 → ~4.1 GB). 2. **The force marker matched the commit body.** A squash commit's body is the PR description. #1145's description just *mentioned* the marker, which forced a publish on merge. That publish ran before the prune, so it 400'd and uploaded nothing. The marker now only counts in the commit subject (the PR title). Docs: the runbook covers the new form, the per-version JSON cache lag, and a prune log. ## Testing - Unit tests: 30 pass across both scripts, with new tests for checkbox/404 handling and for a marker in the body not forcing a publish - `node --check` on the generated snippet - Replayed #1145's real merge message through the gate: no publish - Local CodeRabbit review: 0 findings - Full `make test` not re-run (no app code touched); PR CI covers the package suites @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01MSW9yjwGnzz1S8MgCGm6t5 --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
5005d3cb80 |
ci: publish to PyPI only when CLI code changed (v0.268.9) (#1145)
## Summary PyPI publishing failed on every deploy since v0.268.4. karaoke-gen hit PyPI's 10 GB project cap for the third time (June, Sept 29, Oct 12). Each wheel is ~62 MiB and we published on every merge (~100/month), so each prune only bought a couple of weeks. PyPI is only the public `pip install` channel for the CLIs. Cloud Run builds from source, and the encoding worker pulls the wheel from GCS. So we now publish only when something CLI users get has changed. ## Changes - `scripts/pypi_publish_gate.py` (stdlib): diffs `scripts/pypi_publish_paths.txt` (`karaoke_gen/` plus the `backend/` modules it imports) and `pyproject.toml` (ignoring the package `version =` line only) against **the latest version on PyPI**. A skipped or failed publish therefore never loses a CLI change. Fails open. - `ci.yml` `package-publish`: new "Decide PyPI publish" step gates `poetry publish` (now `--skip-existing`). **Tags and GitHub releases still happen on every version**, because the encoding-worker gate diffs against tags. Override with `[publish-pypi]` in the PR title/commit, or the new workflow_dispatch input "Publish to PyPI". - Not gated on purpose: `frontend/**` (bundled review UI) and `backend/config.py` (cloud settings). A test keeps the backend import list in sync, with `config.py` as the one documented exemption. - Docs: PYPI-STORAGE-PRUNE.md, TROUBLESHOOTING.md, prune script and reminder workflow comments. ## Impact Replayed over v0.200 to v0.268: **24 of 136** releases would have published (~0.7 GB/month instead of ~4 GB/month). Run against current main, it decides "no CLI-affecting changes since v0.268.3", which is correct for v0.268.4 to v0.268.8. ## Testing - 15 new unit tests in `tests/unit/test_pypi_publish_gate.py` - `make test` all green - actionlint clean on the new step - Local CodeRabbit review: 1 finding (doc recovery path), fixed @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01MSW9yjwGnzz1S8MgCGm6t5 --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0329d47e38 |
fix: YouTube description backfill reverted translated-lyrics notices (v0.268.9) (#1144)
## Summary
The daily YouTube description backfill worker was undoing the
translated-lyrics notice on videos with translated lyrics.
**Root cause:** `classify()` parsed `Artist - Title (With Translation
into Spanish) (Karaoke)` with the translation label left inside the song
title, and `render_for()` never passed a `translation_language_name`.
Every translated upload therefore looked "pending". The worker re-checks
the whole channel every day, even after a cycle completes, so it rewrote
each new translated video. The rewrite dropped the `This karaoke video
includes translated lyrics (…)` notice, put the label into `{title}`,
and added it to the tags as well.
## Changes
- `backend/services/youtube_backfill.py`: `classify()` strips the `(With
Translation into X)` label from the parsed song title and records
`translation_language_name`. `render_for()` passes it to
`render_youtube_description`, which is the same thing the upload
pipeline does. Tags now use the clean song title.
- On its next run the worker restores the notice on the translated
videos it already reverted, because their titles still carry the label.
- README: added a safety note that any per-video description variant has
to be recoverable from the live title.
- Version 0.268.9
## Testing
- 5 new tests in `backend/tests/test_youtube_backfill.py`: parsing,
untranslated case, matching upload left alone, reverted description
restored, title preserved + clean tags.
- `make test-backend-unit`: 5961 passed.
- Local CodeRabbit review: no findings.
@coderabbitai ignore
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_013RzGSdPUXLn1nK5eMEZZTW
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
3ecea6f2e7 |
feat(storage): enable real purge with deletion audit log and orphan cleanup mode (v0.266.0) (#1127)
## Summary Andrew approved this follow-up to #1125 and #1126. It turns on real deletion, adds an audit log of every deleted object, and adds an orphan-folder cleanup mode. ### Real deletion on `STORAGE_RETENTION_DRY_RUN=false` and `STORAGE_RETENTION_MAX_JOBS_PER_RUN=50` are added to the backend deploy's `--set-env-vars`. The API service is the only thing that reads them: it passes `--dry-run` to `storage-retention-job`. That keeps the setting in one place, so a later deploy can't silently revert it. ### Deletion audit log - `storage-retention/deletion-logs/<ts>-<mode>.jsonl` has one line per deleted or failed object: timestamp, run_id, mode, job_id, path, size_bytes, generation, category and result. - The log is rewritten after every job or folder, so a crash still leaves a complete record. Each run also writes `<ts>-<mode>.summary.json`. - Deletes are pinned to the logged generation (`if_generation_match`). - `assert_deletable` refuses any path outside `jobs/{id}/`, so `storage-retention/` is never touched. No lifecycle rule matches live objects there. - The Firestore manifest on each job doc now also records generations and the log path. ### Orphan cleanup - `POST /api/internal/storage-retention/orphans` runs on the same Cloud Run job with `--mode orphans`. - It targets `jobs/{id}/` folders with no doc in `jobs`, `jobs-dev` or `youtube_upload_queue`, re-checked live per folder against a fresh listing. - It skips any folder with an object modified in the last 30 days, keeps `input/`, and deletes the rest. - `encoding_worker_jobs` doesn't count as a record: it's the encoder's expiring result cache, which deleted jobs leave behind. - It is a dry-run unless called with `dry_run=false&confirm=delete-orphan-job-folders`, and the scheduler never runs it. ### Docs - `docs/STORAGE-RETENTION.md`: run modes, log locations, jq review commands and restore one-liners (noncurrent `cp #gen` and soft-delete `restore`). - `scripts/storage_retention_report.py`: list runs, summarise a run, or find which run deleted a path or job. ## Testing New tests cover the log contents, generation-pinned deletes, per-job flushing across a crash, the path guard, orphan planning (input kept, 30-day skip, live record re-check, batch limit), and the endpoint and worker modes. Full backend suite: 5774 passed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added admin-triggered orphan-folder cleanup, with dry-run enabled by default, optional folder limits, and explicit confirmation required for deletion. * Added deletion reports and audit logs covering removed objects, reclaimed space, and failures, plus a read-only tool to review runs and search records. * **Bug Fixes** * Added safeguards to prevent deleting changed objects or files outside their intended job folders. * **Documentation** * Expanded storage-retention guidance with recovery details, review commands, and cleanup instructions. * **Chores** * Set the production cleanup limit to 50 jobs per run and updated the package version. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
4e43449559 |
fix(backend): stop API event-loop freezes + identify users in degradation telemetry (v0.264.0) (#1120)
## Summary The "Reconnecting" pill and the "servers unavailable" banner (164 / 49 events, 2026-09-19 → 10-02) were mostly caused by the single API instance's **event loop freezing for 20–45s**. Every request on the instance, `/api/health` included, completed at the same moment. The full evidence and plan are in `docs/archive/2026-10-03-backend-loop-freezes-telemetry-plan.md`. ### What froze the loop, and the fix for each - **Screens worker:** it ran as a post-response `async def` BackgroundTask doing 4K PIL renders, GCS transfers and ffmpeg transcodes on the loop. Under `--cpu-throttling` it was also CPU-starved. **Fix:** it now runs inline in its Cloud Tasks request (CPU stays allocated, failures still return 200 so Cloud Tasks doesn't double-run it), with the heavy steps in `asyncio.to_thread`. Also offloaded: style load, timing signals, pre-apply. - **Matplotlib font cache:** it was rebuilt on every new instance (17/28 episodes). **Fix:** it's pre-built in `Dockerfile.base` (`MPLCONFIGDIR=/opt/mplconfig`), and pyplot is imported lazily. - **Hourly stale-review cron:** an `async def` with zero awaits doing ≤1500 reads and SMTP sends. **Fix:** it runs in a thread. - **166 `async def` routes that never awaited** (sync Firestore/GCS/email/Gemini on the loop, e.g. magic-link verify). **Fix:** they're now plain `def`, so they run in FastAPI's threadpool. The new AST guard `test_no_blocking_async_routes.py` fails CI on new ones. It also flags sync routes that call `asyncio.create_task`/`get_running_loop`; that caught `confirm_duration` and `trigger_youtube_queue_processing`, which stay async. ### Safety net - `backend/services/loop_watchdog.py`: a heartbeat plus a daemon thread. Any stall over 1s logs `EVENT_LOOP_STALL` with the loop thread's stack and a `culprit=file:line`. Stalls of 5s or more are written to `client_events` as `server_loop_stall`. - Pulumi: the log metric `backend/event_loop_stalls` plus an alert on ≥2 stalls of ≥10s in 30 min. **This still needs a local `pulumi up`.** Pulumi is never applied by CI, and the admin gcloud login has expired. ### Telemetry gaps fixed - **Identity:** `user_email` used to be hard-coded to null. The frontend now sends its session token, and the server resolves `user_email`, `is_admin`, `is_internal` and `is_test`. A client-sent email is ignored. - **New fields:** `device_fingerprint` (the existing FingerprintJS visitorId), `tab_id`, `episode_id`, and `tenant`. - **New event:** `banner_recovered`, carrying `duration_ms` and `peak_status`. - **Report:** `scripts/client_events_report.py` summarises the data, excluding admin/internal/test accounts by default. ## Testing - `make test` passes: backend 5529, plus the emulator suite, karaoke_gen unit tests, and frontend Jest 1623. - New tests: - Watchdog: a deterministic state machine plus detection of a real blocking call. - Event-loop responsiveness: a ticker keeps ticking while the heavy steps run. - Screens inline and failure semantics. - `client_events` identity and fields. - The AST guard. - The report summary. - Frontend reporter and banner episode tests. - Local CodeRabbit CLI review: 0 findings. 🤖 Generated with [Claude Code](https://claude.com/claude-code) @coderabbitai ignore --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
8490ab75eb |
chore(gemini): move every Gemini call from Vertex AI to the Gemini Developer API (#1108)
## Summary Moves **every** Gemini call in gen off **Vertex AI** (billed to the `nomadkaraoke` GCP project, ~$85/mo) onto the **Gemini Developer API** using the `gemini-api-key` Secret Manager secret (an AI Studio key billed outside this project). Once deployed and verified, `aiplatform.googleapis.com` / `generativelanguage.googleapis.com` get disabled in nomadkaraoke. ### Key injection - `backend/services/gemini_client.get_genai_client(timeout_ms)` — key from `GEMINI_API_KEY`, else Secret Manager `gemini-api-key` via `settings.get_secret` (runtime SA already has project-level `secretAccessor`). No `project`/`location`. - Service: CI `--set-secrets ... GEMINI_API_KEY=gemini-api-key:latest`. - Jobs: Pulumi secret env on `lyrics-transcription-job`, `video-encoding-job` (renders + translations), `nomad-error-monitor` — **already applied** (targeted `pulumi up`, only that env added). Other jobs fall back to Secret Manager. - Local scripts (`frontend/scripts/translate.py`, `scripts/measure_autocorrect_cost.py`, `scripts/test_agentic_correction.py`): env, else `gcloud secrets versions access` (in-process only). - **Rotation:** add a new secret version; new instances / job executions pick it up. No code change. ### Degradation (quota / billing / invalid key; per-minute 429s stay retryable) | Caller | Behaviour | |---|---| | auto-correct | non-retryable 503 → compare mode drops the Gemini leg, keeps Opus | | match-judge / free-text / parse-titles / tenant bulk | existing fallbacks (no suggestion / blanks / regex) | | credit eval | `pending_review` (fail-closed, unchanged) | | lyrics translation | no retry; render proceeds without translation, status recorded | | custom lyrics | clean 503 "temporarily unavailable…" (+ 5-min timeout so it can't hang) | | error-monitor LLM | no retry; alerts sent without AI summary | | translate.py | exit 2 "Gemini quota/credit exhausted — top up AI Studio or rotate gemini-api-key" | `note_gemini_failure()` also sends one Discord ops alert per 6h (Firestore-throttled across instances via new `ops_alerts.send_throttled_ops_alert`). ### Infra - Pulumi: removes `roles/aiplatform.user` from `karaoke-backend` — **will be applied after this deploys and Vertex traffic is confirmed at zero**. - Agentic correction default model `gemini/gemini-3.8-flash` (`vertexai/` prefix still accepted, routed to the Developer API). - Version 0.261.1 → 0.262.0. Docs: CLAUDE.md, LESSONS-LEARNED, README, ARCHITECTURE, API, etc. ## Testing - New: `backend/tests/services/test_backend_gemini_client.py` (classifier, key resolution, no-Vertex client kwargs, alert throttle, per-caller degradation), `tests/unit/lyrics_transcriber/test_agentic_gemini_provider.py`, `frontend/scripts/test_gemini_client.py`, lyrics-translation quota test. - `pytest backend/tests --ignore=backend/tests/emulator` — all pass; `pytest tests/unit` — 2021 passed. - Verified `gemini-3.8-flash` is listed on the Developer API and answered a 5-token test call with the key. - `translate.py --dry-run` on `frontend/messages` and `backend/translations` OK. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Updates** * AI-powered lyric translation, custom lyrics, auto-correction, title parsing, match judging, and related features now use the Gemini Developer API. * AI features use the configured Gemini API key. Quota, billing, or key issues may show an unavailable message or use an existing fallback instead of repeated attempts. * **Documentation** * Updated product, API, setup, and recovery guidance to reflect Gemini API key configuration and usage. * Updated guidance for handling AI service availability and quota issues. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
f8d0febcf5 |
Translated lyrics beneath the karaoke lyrics for language learners (v0.260.0) (#1101)
## Summary
New job-submission option: **Add translated lyrics** → pick a language
(33 UI locales) → the karaoke videos show an AI translation of each
lyric line in smaller text beneath the synced lyrics, so language
learners can sing songs in another language while learning what the
words mean.
- **Job creation**: toggle + language picker (default = UI language) on
the Customize step, with a **live preview** of a real rendered frame
(`GET /api/themes/translation-preview?language=xx`; pre-generated sample
translations, no LLM call at preview time). Any job, public or private,
free for now. `translation_language` accepted by every creation route;
admin can edit it.
- **Post-review translation**: right after the job enters
RENDERING_VIDEO, the render worker translates the *final reviewed*
lyrics line by line with Gemini
(`backend/services/lyrics_translation.py`, `LYRICS_TRANSLATION_MODEL`,
GCS-cached so theme/admin re-renders are free) into
`jobs/{id}/lyrics/translations.json`. Not editable. A failure never
blocks the render (renders without translations, recorded in
`state_data.lyrics_translation`); a song already in the target language
gets no row.
- **Rendering** (landscape + portrait, GCE worker + local fallback):
`LyricsSegment.translation`; when present, slots get taller and screens
show 3 lines; a `Karaoke.Translation` style (unsung colour, ~½ size)
draws one static event beneath each line, sanitized, wrapped/scaled to
fit. Jobs without translations render identically.
- **RTL + LTR together**: the translation is its own ASS event with auto
paragraph direction, so e.g. a Hebrew song keeps its right-to-left
highlight with an English translation beneath (and vice versa). Long
lines split by the resizer share their translation across the pieces
(grapheme-safe for Thai/CJK).
Design: `docs/archive/2026-10-01-translated-lyrics-plan.md`.
## Testing
- `make test` all green (root 2014, backend 5416, emulator 150, frontend
1583).
- New: `tests/unit/lyrics_transcriber/output/test_translated_lyrics.py`
(layout/positions/style/sanitizing/fit/mixed RTL-LTR/Thai/portrait),
`test_translated_lyrics_render.py` (libass pixel tests incl.
Hebrew/Arabic lyrics with English/Arabic translations and CJK),
`backend/tests/test_lyrics_translation.py`,
`backend/tests/test_render_video_worker_translations.py`, GCE endpoint
test, frontend CustomizeStep/GuidedJobFlow/api tests, prod E2E
`e2e/production/translated-lyrics.spec.ts`.
- `scripts/run-render-tests-linux.sh`: 135 passed on Debian ffmpeg AND
prod's static ffmpeg (no skips).
- Real-data check: job 5710831e (Hebrew) translated to English by Gemini
and rendered with the nomad theme — RTL highlight intact, English
beneath, split line shares translation.
Local CodeRabbit CLI unavailable in this workspace (no seat), so leaving
this PR for the CodeRabbit bot; an independent local review was done and
its findings fixed in the second commit.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an opt-in translated-lyrics option to job setup, with language
selection and a preview.
* Translations are generated after lyric review and appear beneath
lyrics in landscape and portrait videos, with text layout for multiple
languages and writing directions.
* Translation settings can be changed or cleared by admins.
* **Bug Fixes**
* If translation fails or is unavailable, video rendering continues
without translated lyrics.
* **Documentation**
* Updated API and product documentation with translated-lyrics setup and
behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
||
|
|
777daaf25f |
Missing-lyrics gate: block auto-approval when singing has no lyrics and the reference has lines there (v0.258.0) (#1097)
## Summary Phase 2a of the missing-lyrics work (phase 1: #1095, shadow detector + backfill). **Audit** (262 completed jobs, 60 days, server-side `scripts/audit_vocal_gaps.py`): - Longest unlyricked vocal run per job: p50 0.7s, p90 5.1s, p99 22.4s. 47 jobs (18%) have a run of 3s or more, far too many to gate on audio alone. - Cross-checked against **synced** reference lyrics (LRCLIB line timestamps): of 57 suspect gaps that have a synced reference, only 15 contain a reference line. Audio-only is ~25% precise. The rest are vocal chops, samples, ad-libs and synth bleed, mostly in drum & bass tracks. - **Evidence-based rule:** a gap is reliable when the reference lyrics place lines in it. **Changes** - `vocal_gaps` 0.2.0: records the synced reference lines inside each gap. `evidenced` = suspect vocal run AND reference lines, either synced or between the neighbouring anchors. - Scorer 0.5.0: new never-auto `missing-lyrics-gate` with a readable reason, e.g. *"possible missing lyrics: ~10s of singing at 0:20-0:32 has no transcribed words, and the reference lyrics have 3 line(s) there"*. Audio-only gaps don't gate. - The executor analyzes vocal gaps *before* scoring (off the event loop, fingerprinted cache) and passes them to both scoring passes. It fails open on pending audio or analysis errors. - `GET /api/review/{id}/correction-data` now includes `vocal_gaps`, for the upcoming review-UI marker (phase 2b). On job 5710831e, the synced LRCLIB lines inside the gap are exactly the three dropped lines. The genuine instrumentals have none. ## Testing - `test_vocal_gaps.py`: synced lines with edge tolerance; section headers ignored; evidence requires both singing and reference (instrumental with misplaced reference lines doesn't gate; singing without reference doesn't gate). - `test_scorer.py`: reasons only for evidenced gaps; the gate overrides confident lyrics; non-evidenced gaps don't gate. - `test_executor.py`: evidenced gaps block auto-approval with the reason and skip the timing check; cached evidenced gaps still gate. - `test_combined_review_flow.py`: `vocal_gaps` is included in correction-data. - Full backend suite: 5211 passed. Versioned 0.258.0 to follow #1096 (0.257.0). No local CodeRabbit review (the CLI has no seat for this org), so this is left open for the CodeRabbit bot. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
e2f7dbd13c |
Detect sung stretches with no transcribed lyrics (shadow) + backfill endpoint (v0.256.0) (#1095)
## Summary
Transcription can silently **drop whole sung lines**. On job 5710831e
(Hebrew), the words stop at 20.66s and resume at 32.40s while the lead
vocal is ~96% active, and both reference sources have three pre-chorus
lines there. `SectionDetector` (gap ≥ 10s, no audio check) then rendered
"♪ INSTRUMENTAL ♪" over the singing. Nothing caught it:
- the scorer's anchor coverage measures how much of the *transcribed*
text matches the reference, so omissions don't lower it (98.8% here);
- `timing_check` G3 (`max_unclaimed_run_s`) does fire on this job but is
shadow-only and only computed for AUTO-bound jobs;
- the review UI shows nothing at the gap.
This PR is **phase 1: a shadow signal plus server-side backfill for
calibration**. It changes no verdicts or rendering.
- `backend/services/auto_approval/vocal_gaps.py`: for every
transcription gap ≥ 3s, records the lead-vocal active fraction, the
longest breath-bridged vocal run (with a 1.5s allowance for a held note
carried over from the previous word), and the reference lines between
the neighbouring words' reference positions. Section headers are dropped
and ±1-word anchor drift at line boundaries is snapped. `suspect` = run
≥ 3s.
- The executor analyzes every job once stems exist (any verdict), off
the event loop, and stores `state_data.vocal_gaps` keyed by an
`input_key` fingerprint so resets/edits recompute. A summary goes in
`processing_metadata.auto_approval.vocal_gaps`.
- `POST /api/internal/jobs/{id}/vocal-gaps[?dry_run=true]` (admin) plus
`scripts/audit_vocal_gaps.py` for backfill/calibration. Cloud Run does
the stem work.
**On the real job:** one suspect gap, 20.66–32.40s (10.26s run after the
allowance), whose reference lines are exactly the three missing Genius
lines. The three genuine instrumentals have ≤ 0.16s runs.
## Testing
- `test_vocal_gaps.py` (25 tests, synthesized audio + synthetic
correction data):
- dropped lines vs instrumental vs ad-lib vs held note
- a short dropped line diluted by a long gap
- breath bridging
- silent stem = error
- malformed reference data
- reference snapping (off-by-one, single-word line, headers, reversed
chorus mapping)
- Executor tests: stored without changing the outcome; analyzed for
review-bound jobs; cache hit only for the same `input_key`, stale
recompute; pending audio / no stem / error / exception all non-fatal;
backfill uses the reviewed lyrics + original references; dry run.
- Internal endpoint tests (store / dry_run / admin).
- Full backend suite: 5199 passed.
- A local code review's findings were addressed in the second commit:
event loop, stale cache, silent stem, held notes, reference robustness,
audit ordering.
## Next (after deploy)
Run the audit → hand-check suspects → choose thresholds → phase 2:
scorer REVIEW gate with an explicit reason, no fake INSTRUMENTAL over
vocals, review-UI marker with one-click insert of the reference lines.
Also noted: anchor reference ids look shifted by one word at `[Section]`
header boundaries. Separate issue, documented in the plan doc.
No local CodeRabbit review (the CLI has no seat for this org), so this
is left open for the CodeRabbit bot.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added vocal-gap analysis to identify stretches of missing lyrics that
overlap with lead vocals. Results are saved with job data and can be
requested through an admin-only internal API.
* Analysis also runs automatically during auto-approval when vocal stems
are available. It does not change job status or approval decisions.
* **Documentation**
* Documented the internal API, its dry-run option, and the analysis
result statuses.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
||
|
|
354e43bab2 |
Render Hebrew/Arabic karaoke right-to-left; fix title-card tofu (v0.255.0) (#1094)
## Summary
A Hebrew KaraokeHunt job (5710831e, Omer Adam – "שני משוגעים") shipped
with its lyrics **laid out and highlighted left-to-right** and a **title
card of "?" boxes**. Diagnosed by measuring rendered pixels (not by
eye), reproduced locally and with prod's static ffmpeg 7.0.2 in Docker.
**Root causes**
- ASS style `Encoding=0` makes libass force LTR paragraphs; the `{\kf}`
before each word splits the line into runs, so each word is shaped
correctly but the words go left→right.
- `\kf` always wipes left→right within a word
([libass#406](https://github.com/libass/libass/issues/406)).
- PIL (title/end cards) has no per-glyph font fallback; the theme font
(Avenir Next) has no Hebrew glyphs.
**Fixes**
- `build_karaoke_styles` forces `Encoding=-1` (libass auto paragraph
direction; LTR unchanged).
- RTL lines get `{\frz(180+angle)\frx180\fry180}` — a visual no-op that
makes libass ≥0.15 sweep fills right→left without splitting words
(Arabic still joins). Theme angle preserved; re-emitted after duet
`{\r}` resets.
- Lead-in indicator mirrored for RTL lines.
- Text measurement mirrors libass: per-glyph fontconfig fallback, size
scaled from each font's metrics (old fixed 0.70 was 20%+ off for
fallback fonts).
- New `karaoke_gen/utils/font_fallback.py` (fontTools cmap +
fc-match/fc-list) used by title/end cards, portrait headers and ASS
measurement.
- Found by the new tests: inter-word gaps ≤0.1s were dropped, so the
highlight ran up to ~0.2s early by line end (all languages). Karaoke
tags now derive from one absolute centisecond timeline.
- Fixed pre-existing `test_render_video_worker_gce` failure on main
(MagicMock generation → `_g<mock>` id).
## Testing
- **Pixel-level render tests** (`test_ass_render_highlight.py` +
`ass_render_harness.py`): real `SubtitlesGenerator` output rendered with
ffmpeg/libass in a pure-colour palette, asserting on frames for English,
Hebrew, Arabic, Chinese, Japanese, Korean + an RTL duet line: highlight
starts at the reading-start edge, sung pixels always precede unsung in
reading order (word order + in-word fill direction), monotonic and in
time with the words, lead-in on the correct side, no missing glyphs, RTL
tags don't move the line (incl. rotated themes).
- 32 fail on pre-fix main; mutation checks ("Encoding only", "no re-emit
after `\r`", "ignore theme angle") each fail exactly the expected cases.
- 123/123 pass on Linux against **both** Debian ffmpeg (CI-equivalent)
and **prod's johnvansickle static ffmpeg** via
`scripts/run-render-tests-linux.sh`.
- CI now installs `fonts-noto-core fonts-noto-cjk`; under `CI=true`
these tests fail rather than skip on missing preconditions. (macOS
libass uses CoreText → font-sensitive checks skip locally.)
- Title-card tests (`test_font_fallback_render.py`): no `.notdef` tofu
for Hebrew/Arabic/CJK titles; Hebrew title reads right-to-left.
- Unit tests: `test_rtl_ass_output.py`, `test_font_fallback.py`.
- `make test`: unit 1972 passed; emulator 150 passed; frontend 1491
passed.
- Full local re-render of job 5710831e reviewed by Andrew: RTL highlight
smooth and correct, title card correct.
## Follow-ups (not in this PR)
- CDG output still renders RTL left-to-right / tofu (separate renderer).
- Separate investigation: transcription dropped 3 sung lines in this
job, which the section detector then labelled INSTRUMENTAL.
No local CodeRabbit review (CLI has no seat for this org), so leaving
this open for the CodeRabbit bot. A local code review ran; its findings
are addressed in the second commit.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Karaoke lyrics now render more reliably for Hebrew, Arabic, Chinese,
Japanese, and Korean, with improved reading-direction highlighting,
timing, and lead-in placement.
* Titles and portrait text can use fallback fonts when the selected font
lacks the necessary characters.
* **Bug Fixes**
* Improved lyric centering, word highlighting, and glyph rendering
across supported scripts.
* **Documentation**
* Added guidance on lyric rendering and multilingual test coverage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
||
|
|
badd845a1d |
perf(jobs): move video-encoding-job (video + render) to us-east4 (v0.249.1) (#1083)
## Summary Follow-up to #1077. `video-encoding-job` runs both the post-review render (`render_video_worker`) and the final video (`video_worker`). In us-central1 each run waited **2–5 min** for Cloud Run Jobs to start it, so every customer job paid that twice. This moves it to us-east4 alongside the download, lyrics and bulk-search jobs. ## Cost (measured, not estimated) - Over the last 30 days the job read **156 GB** from Google services, mostly the us-central1 bucket, across 494 runs (Cloud Monitoring `container/network/received_bytes_count`, kind=google). That's about 0.32 GB per run. - Pricing from the Billing Catalog: GCS inter-region within North America is **free for the first 100 GB/month, then $0.02/GB**. Cloud Run writes to Google APIs are **$0**. - Internet egress (224 GB/month, Dropbox etc.) and Cloud Run CPU cost the same in either region. - Existing cross-region reads are about 24 GB/month (GPU job and lyrics), which the billing export shows at about $0. - **Net: about $1.60/month**, or about $3/month worst case. Time saved: roughly 4–8 min per customer job. ## Changes - **backend**: `trigger_video_worker` (via `_trigger_cloud_run_job`) and `trigger_render_video_worker` now use `cpu_jobs_region`. `GCP_REGION` stays us-central1; encoding VM and zone logic are unchanged. - **infra (Pulumi, already applied)**: new us-east4 `video-encoding-job`, with the legacy us-central1 copy kept for rollback and its deletion protection turned off. A preview after applying showed 290 unchanged. - **ci**: the video job image update now targets us-east4, using the us-east4 CPU image copy. - **Also updated**: rollback.sh, tests (video and render routing), TROUBLESHOOTING/ARCHITECTURE/LESSONS-LEARNED. ## Testing - `make test` passes: backend tests plus 1443 frontend tests. - Prod smoke test of the new us-east4 job: it imports `video_worker` and `render_video_worker` and calls the encoding VM's `/health` on its public IP (port 8080 is open to 0.0.0.0/0 with API-key auth). **Started in 25 s, exit 0.** 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Updates** - Video encoding and render jobs now run in the `us-east4` region, alongside other CPU jobs. - Video encoding continues to access storage and encoding resources in `us-central1`; cross-region transfer may incur costs. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
cb5ce7f162 |
chore(pypi): prune keep-window default 60→30 days (#1082)
Ran the PyPI prune runbook today (2026-09-29): publishing was blocked (`Project size too large`). The 60-day policy would only have freed 1.08 GB (→9.64 GB), refilling within ~a week, so pruned with `--keep-days 30` instead (69 releases deleted, 0.188.5–0.207.0; newest release of each older month retained) → ~6.8 GB. Re-ran the failed publish; 0.249.0 is live on PyPI. This PR makes 30 days the default in the planner script, the monthly reminder workflow, and the runbook. Unit tests pass (13). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
3e855ff65d |
perf(jobs): run download/lyrics/bulk-search Cloud Run Jobs in us-east4 — cut 2-5 min job start latency (v0.247.1) (#1077)
## Summary Every gen job waited **2–5 min** before `audio-download-job` (and then `lyrics-transcription-job`) even started. That blocked kjbox's goal of singing within about 4 minutes of making a song. **Root cause: Cloud Run Jobs in us-central1 are slow to start on Google's side.** Our image, CPU and code aren't the cause. - The `Started` condition fires about 10 s before the first Python log, so our process startup is fine. - decide's 203 MB image on 4 CPU is just as slow in us-central1. - A throwaway job running Google's tiny sample image, 4 runs per region: **us-central1 4m–5m07s**, **us-east4 8–19 s**, **us-east1 8–25 s**. All four us-central1 runs started in the same second, as if released from a queue. Quotas are far from their limits. ## Changes - **backend**: new `CPU_JOBS_REGION` setting (default `us-east4`) for triggering `audio-download-job`, `lyrics-transcription-job` and `bulk-search-job`. `GCP_REGION` is unchanged. `video-encoding-job` stays in us-central1, because it pulls multi-GB outputs from the us-central1 bucket. - **infra (Pulumi, already applied)**: the three jobs are created in us-east4 as new resources. The legacy us-central1 copies are kept as the rollback path, with deletion protection turned off so a follow-up can remove them. The preview after applying showed 289 unchanged, so there's no drift. - **ci**: the CPU app image is also pushed to `us-east4-docker.pkg.dev/nomadkaraoke/karaoke-backend-gpu/karaoke-backend-cpu`, so jobs pull from their own region. The lyrics job image update now targets us-east4. After each deploy, CI starts a no-op `audio-download-job` execution so the one-time ~2.5 min image import doesn't land on the first customer. - `scripts/rollback.sh`: lyrics job moves to the us-east4 image, and jobs whose target tag is missing are skipped instead of aborting the loop. - Docs: TROUBLESHOOTING runbook (measure + rollback), ARCHITECTURE, LESSONS-LEARNED. ## Testing - `make test`: all backend tests pass, plus 1422 frontend tests. - New tests in `backend/tests/test_worker_service_cpu_jobs_region.py`: routing of each moved job, video job stays in `GCP_REGION`, env override for rollback, and the Pulumi/backend region constants must match. - Prod smoke test of the new us-east4 `audio-download-job` with the real image (`import backend.workers.audio_download_worker`): **Started in 7.0 s, 8.6 s, 10.0 s** (first run of a new digest adds a one-time ~2.5 min import, which the CI step covers). - Local review via a reviewer agent: no must-fix findings; the rollback-path suggestions are addressed. ## Rollout One-time image seed (Cloud Build gcrane copy) and `pulumi up` are done. After merge, CI deploys the backend, which starts triggering the us-east4 jobs. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Performance** * Audio downloads, bulk searches, and lyric transcriptions now run in a region with faster job startup times, reducing delays before processing begins. Video encoding continues to run in its existing region. * **Reliability** * Rollbacks now skip jobs when the target image is unavailable and continue with the remaining jobs. Other image lookup errors still stop the rollback. * **Documentation** * Updated architecture and troubleshooting guides with regional job details, startup-delay guidance, and rollback instructions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
e9465c73d4 |
chore: gemini-3.8-flash everywhere + auto-correct on Opus 5.5 (cost) (#1054)
## Why GCP credits expired 2026-09-19; Vertex Gemini 3.1 Pro spend had grown $9 → $37 → $105/30d. `gemini-3.8-flash` is $0.75/$3.75 per 1M (vs $2/$12 for 3.1 Pro; rises to $1.50/$7.50 on 2027-01-01). Opus 5.5 is $4/$20 (vs $5/$25 for 4.8). ## Changes - All Gemini defaults → `gemini-3.8-flash`: credit eval, custom lyrics, auto-correct (default + compare leg), match judge, parse titles, tenant bulk analyze, agentic AI router, error-monitor LLM analysis (also `us-central1` → `global`, required for Gemini 3.x), frontend `translate.py`. - Auto-correct Claude leg `claude-opus-4-8` → `claude-opus-5-5` (ci.yml env), with `effort: "high"` pinned (5.5 defaults to medium). Pricing table gains both new models. - Docs updated; version 0.241.1. ## Evidence (12 recent prod jobs, pre-review lyrics scored against the human-reviewed final) | Model | Precision | Word-level recall | $/job | |---|---|---|---| | gemini-3.1-pro-preview | 91% | 74% | $0.067 | | gemini-3.8-flash | 98% | 38–41% | $0.020 | | gemini-3.8-flash @ high thinking | 97% | 60% (4/12 timed out, 504) | $0.041 | | claude-opus-4-8 | 94% | 33% | $0.080 | | claude-opus-5-5 (effort high) | 93% | 58% | $0.048 | Flash for the Gemini leg is a deliberate cost/recall trade-off (Andrew chose it); Opus 5.5 more than compensates. Also verified: `gemini-3.8-flash` works on Vertex `global` (rejects `thinking_level=minimal`; nothing here uses it), match-judge latency median 3.3s / max 6.2s (< 12s timeout), Japanese translation smoke test preserves ICU placeholders. Translation cache is keyed on English text + locale, so no re-translation churn. Tests: auto_correct, credit eval, custom lyrics, parse_titles, error_monitor, agentic router suites pass locally. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * AI-powered correction, translation, matching, title parsing, and evaluation now use updated models, including Gemini 3.8 Flash and Claude Opus 5.5. * **Updates** * AI service pricing reflects scheduled rate changes, including a Gemini 3.8 Flash rate change beginning January 1, 2027. * Application version updated to 0.241.1. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
|
30cf41e7d6 |
feat: concurrent-reliability phases 2-5 — Cloud Run shape, degradation telemetry, banner tuning, load-test guard (#1028)
## Summary Phases 2–5 of the concurrent-load reliability initiative (Phase 1 = #1024, verified live: the 10-tab meltdown went from 9/30 failed requests to 30/30 OK). Verbatim spec: `docs/archive/2026-09-17-concurrent-review-reliability-verbatim-record.md`; plan: `docs/archive/2026-09-18-concurrent-review-reliability-investigation-and-plan.md`. ## P2 — Cloud Run shape - **`--concurrency 80 → 16`** for karaoke-backend: the service is one uvicorn process per 2-vCPU instance, and review loads are heavy; at 80 a burst never triggers scale-out and all lands on the single min-instance. 16 makes bursts fan out while staying well above normal per-instance load. (Deploy flags live in ci.yml — the service is not Pulumi-managed.) - **Local preview ffmpeg fallback bounded** (`LOCAL_PREVIEW_MAX_CONCURRENCY=1`, 900s queue timeout, fast RuntimeError after that): the GCE-outage fallback runs ffmpeg ON the API instance; unbounded, a few concurrent Preview clicks during an encoder outage pegged both vCPUs, stalled the health probe, and showed everyone the orange banner (Andrew's banner-during-preview screenshot). ## P3 — degradation telemetry (the explicit ask) - New **`POST /api/client-events`** (unauthenticated so review-token users report too; 30/min/IP rate limit; bot-filtered; detail capped; URLs query-stripped). Each event → Firestore `client_events` collection **and** a structured `client_event type=…` log line, so "how often are users seeing this?" is reviewable via Firestore aggregation or Cloud Logging correlation. - Frontend reporter `lib/degradation-events.ts` (60s per-type throttle, localhost no-op, never throws), wired into: - `BackendStatusBanner` → `banner_reconnecting` / `banner_unavailable` with stall age, in-flight count, probe state - review lyrics-load failure → `lyrics_load_failed` with message/status - `VocalsAudioDataLoader` → `waveform_slow` (>20s to first strips) and `waveform_failed` (gave up) with poll/retry counts ## P4 — banner tuning - Escalating to the full "We're having trouble reaching our servers" card now requires **two consecutive failed health probes** (`UNAVAILABLE_PROBE_FAILURES=2`). One failed 4s probe against a briefly-pegged instance shows only the calm "Reconnecting…" pill. Single-failure behavior for the pill is unchanged, as is instant clearing on recovery. ## P5 — regression guard - **`scripts/load-test-review.py`** checked in: replays the 10-tab incident (Firestore-ADC review tokens, curl request trios, pass/fail thresholds, exit code). Documented in `docs/TESTING.md`. Validated against prod post-Phase-1: 9/9 requests OK, tabs < 3s. ## Testing - Backend: 6 new tests for client-events (persist/validate/rate-limit/bot/fail-soft/detail-cap) + local-preview bound test — full suite **4517 passed**. - Frontend: new degradation-events suite, two-probe escalation + debug-snapshot tests; one existing api-fetch test updated to the new escalation contract — full suite **1298 passed**. - Live validation of the load-test tool against prod (small 3-tab run): PASS. ## Notes for review - Cloud Run concurrency change means bursts now spin up extra instances (billable during requests only; max-instances unchanged at 20). - Firestore `client_events` has no TTL policy yet (needs a gcloud/console action our read-only ADC can't do); volume is tiny (throttled per client) — can add later if it ever matters. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added automatic reporting for degraded experiences, including reconnecting or unavailable backend status, failed lyrics loads, and slow or failed waveform loads. - Added concurrent review-page load testing to help verify performance under multiple open tabs. - **Improvements** - Backend status now remains in “reconnecting” until multiple health checks fail, reducing premature unavailable messages. - Local preview processing is limited during heavy demand to improve stability. - **Documentation** - Added API documentation for client telemetry and guidance for running concurrent-load regression tests. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
912ca37a01 |
Lyrics review: Waveforms mode + always-show backing vocals in preview (#1010)
## Summary
Two review-UI improvements, both driven from real reviewing pain:
### 1. New "Waveforms" mode in Synced Lyrics
A third view (alongside Simple/Advanced) that renders a **compact inline
copy of the Edit Segment timeline for every segment** — so you can spot
mis-timed words (e.g. an over-long trailing word whose vocal energy died
earlier) at a glance without opening a modal per line.
- Reuses the modal's `TimelineEditor`: resizable/draggable word bars
over each segment's vocal waveform, with the 1s buffer padding,
start/end boundary lines, and greyed read-only neighbour words.
- **Drag** a bar body moves timing; **drag** an edge resizes; both
commit on release (a 4px threshold distinguishes drag from click). A
plain **click** on a word — or the index/✎ — opens the Edit modal.
**Trash** deletes the segment (matches Advanced).
- Bars colour-code with the **exact `HIGHLIGHT_CLASSES` tints** the
Advanced pills use (anchor/gap/corrected/AI/edited) and show the
struck-through **AI-correction ghost text** above corrected bars.
- Tight vertical density (~39px/segment: 20px bars + 14px waveform + 5px
gap) to stay close to Advanced.
- View mode persists to `localStorage` (`lyricsReviewViewMode`,
migrating the old `lyricsReviewAdvancedMode` boolean).
- Advanced + Waveforms now size the **Reference column to its longest
line** so Synced gets the freed width (no more ~25% wasted space).
### 2. Preview modal always shows the backing-vocals waveform
Previously the backing-vocals waveform only appeared *after* switching
away from the recommended "Clean instrumental", making it easy to accept
clean without ever seeing/hearing the backing vocals you'd lose. Now
it's **always shown** whenever a playable backing stem exists, with
selection-aware wording ("kept" vs "removed — click to hear what you'd
be leaving out"). Clicking still auditions the backing track without
changing the final choice.
### Also
- Fixes a pre-existing `Header` `FORMATTING_ERROR` (`autoCorrectedDesc`
rendered without its `count` param on jobs with ≥1 auto-correction).
- `scripts/list-review-jobs.py` — lists in-review jobs with
ready-to-open local review URLs.
## Testing
- New unit tests: `contextWords`, `segmentTiming` (recompute + view-mode
migration), `wordDecorations` (classifier + palette),
`WaveformSegmentRow` (control wiring + untimed fallback).
- Updated `ReviewChangesModal` tests for the always-show behavior +
reworded copy.
- Full frontend Jest suite green (104 suites / 1283 tests).
- i18n: all 33 locales translated; stale key pruned; parity validated.
- Verified live against a real in-review prod job (Playwright):
coloring, ghost text, click-to-open, compact layout, dynamic columns.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added a Waveforms view to Synced Lyrics for inline word-timing
adjustments.
- Added controls to edit segments, play timed segments, delete segments,
and handle untimed lyrics.
- Waveform bars now visually distinguish corrections, edits, gaps, and
related word states.
- Backing-vocal waveforms remain available for review even when the
clean instrumental is selected.
- **Improvements**
- Auto-correction messaging now displays the number of applied
corrections.
- Clarified backing-vocal and clean-instrumental guidance across
supported languages.
- Updated the application version to 0.227.0.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
462f246768 |
internal: admin endpoint to re-run auto-approval eval on parked review jobs (#1005)
## Summary
Adds `POST /api/internal/jobs/{job_id}/auto-approval-eval` (admin-only):
re-runs the auto-approval evaluator on a job parked at AWAITING_REVIEW.
The KaraokeHunt outreach batch runs with `made_for_you: true`
(stale-review exemption), which is also an enforcement blocker at
scoring time — this endpoint lets us lift the flag and re-evaluate so
confident jobs full-auto through review.
## Testing
- `test_internal_api.py::TestAutoApprovalEval` — outcome passthrough +
auth required; file green in full-collection run.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Administrators can manually re-run auto-approval evaluation for jobs
awaiting review.
- Jobs remain in review unless the evaluation is fully confident,
providing a fail-safe review process.
- **Bug Fixes**
- Improved batch job processing so an issue updating an optional job
setting no longer stops the remaining workflow.
- Prevented missing cached results from causing failures during
recanonicalization tasks.
- **Tests**
- Added coverage for authorized evaluation requests and unauthorized
access.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
6800c14e57 |
admin: make made_for_you editable on jobs (stale-review exemption) + KaraokeHunt outreach tooling (#1004)
## Summary - Add `made_for_you` to `EDITABLE_JOB_FIELDS` (with a boolean guard) so admin batch flows can exempt jobs from the 48h stale-review auto-expiry (`stale_review_processor` skips MFY jobs). Needed for the KaraokeHunt outreach batch: ~90 jobs held at review under a holding account until coordinated emails go out. - KaraokeHunt outreach tooling under `scripts/karaokehunt_outreach/` (phase1b/1c analysis, phase2 executor, batch generator, torrent review) — standalone scripts, no runtime impact. - Session handoff docs in `docs/archive/`. ## Testing - `backend/tests/test_admin_job_update.py`: new `TestUpdateJobMadeForYou` (editable + non-bool rejected) — 20/20 pass. - Full backend suite: 4417 passed (unit) + emulator suite green via `make test-e2e`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Administrators can mark jobs as “made for you,” keeping them available for review indefinitely instead of automatically expiring after 48 hours. - Invalid values for this setting are rejected with a clear validation error. - **Documentation** - Added operational documentation for the KaraokeHunt outreach and migration workflow, including request review, song matching, account credits, and generation stages. - **Chores** - Updated the application version to 0.225.4. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
5b5264dc54 |
Incident hardening: deploy/CI/runner resilience (R1/R2/R3/R4/C1/C2) (#987)
Part 3 of 3 of the **Incident hardening (post-NOMAD-1632)** initiative (plan: `docs/archive/2026-09-12-incident-hardening-plan.md`). Closes the deploy/CI/runner gaps: no functional gate, no codified rollback, bad revision serves 100% immediately, fire-and-forget runner stranding, and a flaky smoke. **No backend app code changed → no version bump.** ## ⚠️ Reviewer note — read first **R3/R4 rewrites the live `deploy-backend` step.** It's designed to be *auto-revert-safe* (a failed candidate smoke means we simply never migrate traffic, so the prior revision keeps serving 100% — a bad revision serves **0%**, not an outage). But it changes how **every** backend deploy works and **can't be fully exercised in CI**. This is the one to watch on first merge. If you'd rather de-risk, R3/R4 is the only item here that's safe to split out — the rest (R1/R2/C1/C2) are independent and lower-risk. ## R2 — codified fast rollback (`scripts/rollback.sh` + runbook) Dry-run-able, explicit-target-required rollback for the Cloud Run **service** (traffic-shift to a prior revision — instant, no rebuild) and the three **jobs** (image re-pin, correct per-job region/repo — verified against ci.yml). `--list` shows available revisions/tags; `--dry-run` prints every gcloud command. `docs/TROUBLESHOOTING.md` "Fast rollback" runbook added as the first section. ## R3/R4 — service traffic-canary `deploy-backend` now deploys `--no-traffic --tag candidate`, smokes the tagged candidate URL (health + expected version, with retries), then `update-traffic --to-latest` **only on pass**. Cloud Run Jobs can't traffic-split (noted inline) → they rely on R1 + R2. ## R1 — post-deploy functional canary, version-change-gated New `deploy-verify` job runs the real production happy-path (`happy-path-real-user.spec.ts`, admin-credit variant — reuses e2e-daily's impersonate token refresh, secret names verified identical) after a backend deploy, **only when the backend version actually changed** (new "Detect backend version change" step compares `poetry version` to the currently-served `/api/health/detailed`). Frontend/docs/infra-only merges short-circuit. On failure → Discord alert + loud fail printing the exact `scripts/rollback.sh` invocation. Per the plan's caveat, a fresh happy-path won't reproduce every incident class (e.g. Failure B's stale-cache) — an admin reset+retry variant is a noted follow-up. ## C1 — ephemeral-runner stranding fix `infrastructure/functions/runner_manager/ephemeral.py` now confirms the VM insert **operation** succeeded instead of fire-and-forget: a late stockout/quota/transient error raises → existing zone-fallback / webhook redelivery kicks in instead of silently stranding a queued job for ~15 min. An insert-confirm *timeout* is treated as "still provisioning" (not a failure); guarded for older clients. **+4 tests (46 pass).** ## C2 — de-flake the E2E smoke `networkidle` → `domcontentloaded` + explicit element waits (the marketing page's background requests made `networkidle` time out and redden real deploys). ## Validation (local only — no deploy) - `ci.yml` + `e2e-daily.yml` parse; **actionlint clean** on new code. - `rollback.sh`: `bash -n` + `shellcheck` clean; constants cross-checked against ci.yml. - `test_ephemeral.py`: **46 passed**. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
a344783376 |
gen: unify YouTube descriptions + automated back-catalogue rewrite (#978)
## Summary
Unifies the descriptions across all ~1.6k videos on
youtube.com/@nomadkaraoke to one consistent, conversion-focused
template, makes new uploads use it automatically, and adds a
**reusable** automated system to re-apply any future template change
across the whole channel.
Old descriptions were inconsistent: pre-2024 ones linked a dead $10
Fiverr gig; newer ones led with "AI-powered" (bad rep among viewers) and
linked nowhere useful.
## New template
Single source of truth: `Settings.default_youtube_description` rendered
by `backend/services/youtube_description.py` (placeholders `{title}
{artist} {artist_hashtag} {brand_code}`). Drops all "AI" framing; leads
with the craft; CTA to the requests board then the referral link
`nomadkaraoke.com/r/youtube` (50% off, tracked via referrals); Nomad
Discord + decide + KaraokeNerds; fair-use line; preserves the brand
code.
## Live pipeline (new uploads)
Both upload sites (`video_worker_orchestrator.py`,
`youtube_queue_processor.py`) now render via the shared module +
enriched tags — new uploads use the new template automatically.
## Bulk rewrite of the back catalogue
- Shared logic `backend/services/youtube_backfill.py` (classify / target
/ render / update-body / enumeration). **High-confidence-only**
eligibility (title ends in `(Karaoke)`) + curated `skip_ids.txt` /
`include_ids.txt` lists shipped in the package.
- **Manual CLI** `scripts/youtube-descriptions/` (`analyze` / `apply` /
`status`) — read-only report + full old→new diffs before any write.
- **Automated daily drain**
`backend/workers/youtube_description_backfill_worker.py` + `POST
/api/internal/youtube-backfill/run` + Cloud Scheduler (Pulumi). Progress
in Firestore keyed by a **template fingerprint**, so any future template
change auto-re-drains the whole channel with no manual reset.
Quota-coordinated with uploads (reserves headroom via
`YouTubeQuotaService`); Postmark progress + completion emails.
## Verified
- Analyzed all 1,631 videos → **1,614 to rewrite** (1,610 clean + 4
rescued truncated titles), **17 skipped** (lyrics-with-vocals, demos,
tutorials), zero false positives.
- One video (`PmwRXIZoCpg`) updated live and verified on YouTube (new
description + enriched tags; title/category preserved).
## Tests
42 new unit tests (renderer, classifier/targeting/update-body, worker:
full drain, budget cap, quota reserve, dry-run, completion/idempotency,
template-change cycle, quotaExceeded stop). Local review done via the
local code-review fallback (CodeRabbit CLI was rate-limited) — 3
findings fixed, 2 documented trade-offs.
## Deploy notes
- Backend auto-deploys on merge (worker + endpoint + config + data
files).
- **`pulumi up` required** to create the
`youtube-description-backfill-daily` Cloud Scheduler job.
- New config env (all defaulted): `YOUTUBE_BACKFILL_ENABLED`,
`..._DAILY_MAX_UPDATES`, `..._QUOTA_RESERVE`, `..._ENRICH_TAGS`,
`..._REPORT_EMAIL`.
Plan/architecture:
`docs/archive/2026-09-03-youtube-descriptions-plan.md`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- YouTube uploads now use standardized descriptions with artist, title,
brand-code, and hashtag details.
- Tags are generated consistently from karaoke, artist, and title
information.
- Existing channel videos can be analyzed and updated in quota-limited
batches, with dry-run support and progress reporting.
- Added scheduled daily backfills and administrative controls for update
limits, exclusions, and overrides.
- Added safeguards for low-confidence titles and explicitly skipped
videos.
- **Documentation**
- Added guidance for automated and manual YouTube description management
workflows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
37b9e6088b |
ci(pypi): recurring storage-cap prune reminder (fix failing PyPI publish) (#960)
## Problem The `Deploy - Publish to PyPI` step on `main` has been failing: ``` HTTP Error 400: Project size too large. Limit for project 'karaoke-gen' total size is 10 GB. ``` This is a recurrence of the known 10 GB PyPI project cap. The project is at **10.71 GB / 479 releases**; each release is a ~62 MiB universal wheel (bundled 33-locale frontend) auto-published on every merge (~3 GB/month), so it refills the cap every few months. **Only the public `pip install` publish step fails** — Cloud Run and the GCE encoding worker (pulls the wheel from GCS) are unaffected. The prior fix (v0.187.5) was wheel-only publishing + a one-time manual deletion, which only bought ~2 months. This makes the cleanup **recurring and durable**. ## Why not just automate deletion in CI? PyPI has **no deletion API** (upload tokens / Trusted Publishing are upload-only, and 2FA is mandatory). Deletion is only possible from a logged-in browser session, which can't be safely/durably stored in CI. So we automate the *decision + reminder* and keep the *deletion* a ~1-minute human step. ## What this adds - **`scripts/prune_pypi_releases.py`** (stdlib-only) — reads the public JSON API and computes a retention plan: **keep last 60 days + newest release per older calendar month + latest overall**. Renders a table / JSON / Markdown report and a **paste-into-browser-console** deletion snippet. Never deletes anything itself. Today's plan: keep 82, delete 397, **10.71 GB → 4.17 GB**. - **`.github/workflows/pypi-prune-reminder.yml`** — monthly cron (+ manual dispatch) that runs the planner and, once the project crosses a 7 GB watermark, opens/updates a single tracking issue with the plan + snippet. No secrets. - **`docs/PYPI-STORAGE-PRUNE.md`** runbook + a `TROUBLESHOOTING.md` entry. - **`tests/unit/test_prune_pypi_releases.py`** — 13 tests covering the policy (recency window, per-month keeper, latest-overall, same-day timestamp ranking, empty/undated releases, sizes, rendering). ## Testing - `pytest tests/unit/test_prune_pypi_releases.py` → 13 passed. - Ran the planner against live PyPI data; generated JS validated with `node --check`. - `black` clean; workflow YAML validated; `should_nag` gate verified. ## Notes - **No version bump** — tooling/CI/docs only, nothing in the wheel changes. Merging therefore **skips** the publish step (tag `v0.209.1` already exists), so this PR does not itself add another fat wheel while we're over the cap. - **To unblock publishing now:** after merge, run the prune (locally via the script or trigger the workflow with `workflow_dispatch`), paste the snippet on the logged-in PyPI releases page, then re-run the publish. The monthly cron keeps it under the cap going forward. - Related: filed [pypi/support#12050](https://github.com/pypi/support/issues/12050) requesting a 50 GB limit as a one-off buffer; the recurring prune is the durable fix. @coderabbitai ignore --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
8ce0d39faa |
fix(lyrics-review): Tap To Sync timing corruption + padded edit-segment timeline (#956)
## Summary Fixes **Tap To Sync timing corruption** in the Edit Segment modal and reworks the segment timeline for easier re-timing of the first/last words. ### Tap To Sync fixes (`useManualSync`) - **Negative durations on re-sync** — the overlap-correction effect pulled a just-tapped word's end back to the *next* word's **stale** start (it hadn't been re-tapped yet), producing `end_time < start_time`. The sanitizer then reported *"Fixed N word timing(s) that fell outside this segment"* on reopen. Now guarded + clamped so it never inverts. - **First word squished / segment start ratcheting** — `clampSyncTime` forced every tap into the tight `[segStart, segEnd]`, so a legit tap in the 3s playback lead-in got shoved up to `segStart` and the start could never be re-synced earlier. Now allows taps in `[segStart − leadIn, segEnd + leadOut]` (floored at 0). - **Last word cut off** — symmetric lead-out + auto-stop no longer fires while the spacebar is held, so the last word can extend the segment (hold past the end / default-duration overshoot) instead of being truncated. - Both keyboard **and touch** sync paths use the widened window. ### Edit Segment timeline UX - ±1s **greyed context padding** on each side (dashed boundary), **faded waveform** outside the segment, and **neighbouring segments' words** as read-only context blocks. Word bars, drag, ruler, cursor and click-to-scrub all map over the padded view; words can be dragged/synced into the padding to extend the segment. - Modal widened `960px → 95vw / max 1400px`. - Header **play button** starts 1s before and auto-stops 1s after the segment. ### Dev tooling (dev-gated, no prod impact) - `&edit=1` makes a replay session editable locally (guarded by `NODE_ENV !== 'production'`) so the real review UI can be exercised against real job data + audio. Documented in `docs/REPLAY.md`. - `review_test_fixtures.py` now serves a synthetic **click-track** so Tap To Sync is exercisable offline. ## Testing - New/expanded unit tests in `hooks/__tests__/useManualSync*.ts(x)`: negative-duration regression (fails on old code, passes on fix), lead-in/lead-out clamp cases, and segment-end extension. All 13 pass; 178 lyrics-review + hook tests green. - Typecheck clean on touched files. - Verified end-to-end in the browser against a real completed job (replay + real vocals): tap-sync produces valid monotonic timings, no sanitizer banner, first/last words extend correctly, timeline padding + fade + neighbour words render as intended. ## Notes - CodeRabbit reviewed locally; its one finding (touch path missing the lead window) is fixed. @coderabbitai ignore --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
9b6c4f236e |
fix: Cloud Tasks queue region from GPU workers + REVIEW_COMPLETE lost-trigger recovery (v0.202.1) (#948)
## Summary Follow-up to #947, found by live-testing auto-approval in prod (job `b8bda9c2` — the first real fully-auto-approved job). **Bug:** the GPU `audio-separation-job` sets `GCP_REGION` to its GPU region, but every Cloud Tasks queue lives in `us-central1`. Any queue enqueue from inside that worker fails with `404 Queue does not exist` — which silently dropped the auto-approval render trigger (job auto-approved perfectly, then sat at `review_complete` forever). This is also a latent pre-existing bug for any other enqueue attempted from the GPU job. **Fixes:** 1. New `settings.cloud_tasks_region` (default `us-central1`, overridable via `CLOUD_TASKS_REGION`) used for all Cloud Tasks queue paths. `gcp_region` is still used for Cloud Run Job locations (correct). 2. `recover-stuck-jobs` cron gains a sweep for `REVIEW_COMPLETE` jobs stalled >10 min with no `render_progress` — re-triggers the render worker (idempotent via worker-generation fence). This closes the lost-trigger orphan gap for BOTH the human complete-review fire-and-forget task and the auto-approval path, and will auto-rescue `b8bda9c2` on the first tick after deploy. ## Testing - New unit tests: queue path uses `us-central1` even when `GCP_REGION=us-east4`; stall detector (stalled / fresh / render-underway). - Note: `test_internal_api.py` auth tests fail identically on pristine `main` in this local env (green in CI) — pre-existing, unrelated. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
f01213f5cb |
Fully-auto review for confident jobs: review_mode framework + auto-approval scorer/executor + replay tool (v0.202.0) (#947)
## Summary Ships the **fully-automated review framework**: jobs now default to `review_mode="auto"` — when the auto-approval scorer is fully confident on BOTH axes, the combined review screens are skipped entirely and the job renders straight through. Everything else falls back to human review exactly as before. **The confident class (deliberately narrow to start, ~5% of jobs expected):** - **Lyrics**: synced reference present, ≥90-99% anchor coverage, and every gap word covered by the pre-generated AI auto-correct suggestions (which the executor applies server-side — a faithful port of the review UI's on-load auto-apply, including conflict-group resolution) — OR zero gaps outright. - **Backing vocals**: non-subjective only — no audible backing content → clean instrumental. - **Never-auto gates**: vocalization sections (da-da-dun/woo-woo — P3), phantom/absurd-duration lines (P8), too-few-words, no/unsynced reference, errored backing analysis. **Calibrated against a 20-job human-review corpus** (all recent admin reviews replayed + annotated): safety 100% (every job that truly needed a human stays in review), lyrics-auto 3/20, overall-auto 1/20. ## How it works - `screens_worker` (pre-AWAITING_REVIEW) and `audio_worker` (post-stems, for the audio-lags-lyrics ordering) both call `maybe_auto_complete_review()`. Every job gets a verdict recorded in `processing_metadata.auto_approval` (shadow data keeps accumulating for calibration); enforcement additionally requires: flag on + `review_mode="auto"` + not made-for-you + no existing/custom instrumental + audio complete. - Enforce path mirrors a human "Complete Review": apply cached AI suggestions → sanity checks → `corrections_updated.json` → `instrumental_selection="clean"` → clear progress keys → `GENERATING_SCREENS→REVIEW_COMPLETE` (new legal edge; no review token or "ready for review" notification is ever issued) → trigger render worker. - **Fail-safe everywhere**: stale suggestions, duplicate-word artifacts (P1 self-conflict signature), state changed mid-enforce (human opened editor), any exception → normal human review. Auto-approval can never fail a job. - Kill switch: `AUTO_APPROVAL_ENFORCE_ENABLED=false` (scoring continues). Per-job: `review_mode="always_review"` (admin PATCH, validated). - Excluded from enforcement (still shadow-scored): made-for-you orders (human QC is part of that product), custom-instrumental jobs. ## Also in this PR (sessions 1-3 of the initiative) - **Review replay tool** (admin-only, prod-inert): `?replay=1` reopens the real lyrics/instrumental review UIs read-only for completed jobs — queue nav, Post-AI/Final toggle, edit-log action panel, dev-only audio byte-proxy (env-gated `REVIEW_AUDIO_PROXY`, path-anchored). This is what produced the calibration corpus. - `auto_approval` service package: scorer (anchor/gap/AI-coverage signals + P3/P8 gate detectors), apply engine, executor, lyrics-diff, offline capture script. ## Testing - 100+ new unit tests across scorer/apply/executor/replay routes; full backend suite 4000+ green locally (8 pre-existing emulator failures reproduce identically on pristine `main` — local-env only, green in CI). Frontend: 1196 Jest tests green. - Scorer validated read-only against the 20 real prod jobs in the private calibration corpus. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
70ea964ca9 |
feat(dev): full local backend for external contributors (no GCP creds needed) (#946)
## Summary Motivated by external contributor Martín (netux) asking "is there an easy way to run the full backend without GCS?" — he couldn't test PR #895's backend changes locally. This PR makes the whole backend runnable locally against emulators with **zero GCP credentials and zero paid API keys**. ### What was broken for contributors - Workers migrated to Cloud Run Jobs for prod robustness (audio-download, audio-separation, lyrics-transcription, bulk-search) silently bypassed the direct-HTTP dev mode → local jobs stalled at PENDING forever - Signed URL generation called `google.auth.default()` → crashed with no credentials (review audio, downloads, theme previews, upload URLs) - Magic-link login 503'd locally (console email provider counted as "not configured") - Fresh emulator had no default theme → every job create returned 422 - `docs/DEVELOPMENT.md` told people to `cp .env.example .env` but the file didn't exist — and nothing loaded `.env` anyway - `run-backend-local.sh` defaulted to the production GCP project ### Changes - **worker_service**: CRJ-based workers run as local subprocesses (`python -m backend.workers.X --job-id Y`, exact CRJ parity) when `ENABLE_CLOUD_TASKS=false` **and** not `is_production()` (regression-tested both ways) - **storage_service**: signed URLs → plain fake-gcs-server URLs when `STORAGE_EMULATOR_HOST` set (GET via JSON API download path; PUT via unvalidated-signature path, enabled by `-public-host` on the emulator) - **users route**: magic-link 503 only in production; dev prints the link to the console - **file_upload bugfix**: a theme with style params but no asset files never got `style_params_gcs_path` written (empty-dict falsiness in `elif theme_style_assets:`) — surfaced by the minimal seed theme, latent in prod - **scripts/seed-local-data.py** (new): creates bucket + default theme; wired into `run-backend-local.sh --with-emulators`, which now uses `test-project` - **config**: `load_dotenv()` so `.env` actually works; new root `.env.example` - **CONTRIBUTING.md** (new): no-credentials quickstart; README + DEVELOPMENT.md link to it ### Verified end-to-end locally (no GCP credentials) Upload job → local audio separation (real models) → transcription → screens → `awaiting_review`; review endpoints all working including `/api/review/{job}/audio/vocals` (the PR #895 endpoint) and 302 → emulator audio URLs. Magic-link login round-trip via console link. ### Tests - 224 tests across affected files pass; full backend unit suite: 3969 passed - `backend/tests/test_emulator_integration.py` has 8 pre-existing failures that reproduce identically on unmodified `main` locally (broken theme-service mock in that file) — unrelated to this PR - New tests: dev-mode subprocess dispatch (audio + bulk-search), prod-mode never-Popen guard, emulator URL generation (GET + PUT) @coderabbitai ignore --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
7729d30937 |
build: build frontend into the wheel in CI instead of committing out/ to git (supersedes #892) (#926)
Build the Next.js frontend into the wheel in CI instead of committing the generated out/ to git (fixes #876; supersedes #892). Validated: package-build-test green (npm ci → make build-frontend → poetry build → verify_wheel_frontend guard → install). Thanks @netux for surfacing it. |
||
|
|
6f86b3d901 |
test(e2e): rewrite tenant daily E2E as a faithful frontend journey (#873)
## Why The daily tenant E2E (`scripts/e2e/tenant_e2e.py`) drove the whole flow via **direct backend API calls**, bypassing the frontend. Two problems: 1. **Not representative** — it never exercised the tenant portal UI a real operator uses. 2. **Intermittent stuck-render orphans** (Run #31 vocalstar, #32 singa) — because it never loaded the lyrics-review page, it never fired `encoding-worker/warmup`, so the encoder was cold when the render started and the job orphaned in `rendering_video` (no recovery). Root-caused this session; the fix is to make the test faithful, which fires the warmup a real user fires. ## What Replaces `tenant_e2e.py` with **`frontend/e2e/production/tenant-happy-path.spec.ts`** — a Playwright journey mirroring `happy-path-real-user.spec.ts`, run per-tenant via the existing `e2e-tenant-daily.yml` matrix (vocalstar, singa; `max-parallel: 1`): login (real magic-link UI) → submit track (tenant form + real audio upload) → wait for transcription → **open review UI (fires warmup)** → approve → wait for render → assert tenant distribution (Dropbox present, no YouTube, downloads, `is_private`) → cleanup. - **Tenant auth**: added `inbox.testmail.app` to both tenants' `allowed_email_domains` (setup scripts + live GCS config via `scripts/e2e/update_tenant_allowlist.py`) so the test signs in through the real magic-link UI. - **Test audio**: short 60s clips fetched from private GCS in CI (`scripts/e2e/fetch_test_audio.sh`) — fast, reliable uploads and quick renders. - Workflow reworked to Node/Playwright; `tenant_e2e.py` deleted; version `0.190.3 → 0.190.4`. ## Findings surfaced (validating the exercise) - **Browser upload was CORS-blocked in Playwright** by the shared config's global `x-playwright-test` header (it made the GCS signed-URL PUT preflight request a header the bucket's CORS doesn't allow). Verified the same PUT succeeds in a real browser — a **test-harness artifact, not a product bug**. Worked around by stripping the header for `storage.googleapis.com` requests. (Note: any prod E2E doing a browser signed-URL upload is affected by this global header.) ## Verification Both tenants run **green end-to-end against prod** locally (login → submit → review → render → distribution; render completed without orphaning). Reworked workflow dispatched on this branch to confirm CI wiring (Node setup + private-audio fetch + secrets). Test-only + CI + tenant-config change. No product code. 🤖 Generated with [Claude Code](https://claude.com/claude-code) @coderabbitai ignore --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
21e8a05c61 |
test(tenant): retry daily E2E once on transient encoder failure (#834)
@coderabbitai ignore
The shared GCE encoder intermittently drops a job mid-encode ("lost
contact with worker ... not found") — a pre-existing,
**non-tenant-specific** transient (it hit the daily E2E twice but not
the manual runs; both tenants complete when the encoder cooperates). The
daily health check now detects that specific error and retries the whole
job once, so it doesn't false-alarm on encoder flakiness. Other failures
still fail immediately. Test-script only.
Separately worth a look (out of scope here): the encoder "lost contact /
job not found" reliability affects consumer renders too.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
|
1218337346 |
test(tenant): daily E2E asserts portal downloads via file_urls (#831)
@coderabbitai ignore The daily tenant E2E asserted on `job.download_urls` (backed by `output_files`, which the orchestrator pipeline never populates), so it failed even on fully-successful jobs. The `OutputLinks` UI reads `job.file_urls` (finals/packages) — assert that instead. Confirmed in prod: **both vocalstar and singa pass the full E2E** — job completes, Dropbox link present, no YouTube, and all downloads available (4K / 720p / with-vocals MP4 + CDG + TXT). Doc updated with the final outcome. Test/docs only — no production code change. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
66dcfd7d19 |
fix(tenant): stage instrumental in the orchestrator encode path (#829)
@coderabbitai ignore Follow-up to #828. That PR staged the instrumental in `_encode_via_gce` (legacy `video_worker.py`), but production jobs run the **orchestrator** pipeline (`video_worker_orchestrator.py`), whose `_run_encoding` builds its own `EncodingInput` and never made the user-supplied instrumental available under `jobs/{job_id}/`. Tenant jobs still failed at the GCE encoder with "No instrumental audio found" (confirmed — the "Staged existing instrumental" log never appeared). **Fix:** in the orchestrator's `_run_encoding`, when using the GCE backend, upload the local instrumental (existing/custom — identified by the `(Instrumental User)` / `(Instrumental Custom)` filename) into `jobs/{job_id}/` so the encoder's recursive `find_file` locates it. Consumer `clean`/`backing` jobs are untouched (their stems already live under `jobs/{job_id}/`). Also: the daily E2E uses a tenant-specific title so the two matrix tenants don't share a `base_name` (the GCE encoder caches by base_name). Validated by re-running the full prod E2E to completion after deploy. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
40c7960854 |
fix(tenant): stage instrumental for the GCE encoder + daily tenant E2E (#828)
@coderabbitai ignore ## Final render blocker (bug #8) With #824/#825/#826/#827, tenant jobs now render the karaoke video — but the GCE encoder failed with **"No instrumental audio found"**. **Root cause:** the user-supplied instrumental lives under `uploads/{job}/audio/`, but the encoder only pulls input from `jobs/{job}/` (`input_gcs_path`) and locates the instrumental via `find_file("*existing_instrumental*", "*Instrumental User*")`. On top of that, the running encoder VM (`encoding-worker-b`) boots a **2026-01-23 image** that predates the encoder's own `existing_instrumental` config-download path (added 2026-03-04), so it could never fetch it from `uploads/` either. **Fix (image-agnostic — no risky shared-VM recreation, no consumer impact):** `_encode_via_gce` server-side copies the instrumental into `jobs/{job_id}/existing_instrumental.<ext>` before submitting. The encoder downloads `jobs/{job}/` and its **recursive** `find_file` then locates it on any image version. Added `StorageService.copy_blob` + unit test. > Note: the stale encoder VM image is a separate infra item — the `encoding-worker` VMs should be recreated from the image family's newest image (which has 3 months of encoder fixes). Tracked for a Pulumi pass; not required for this fix. ## Daily tenant E2E (second ask) - `scripts/e2e/tenant_e2e.py` — full per-tenant prod health check: submit → approve review → render → assert (completes, Dropbox link present, no YouTube, downloads available) → clean up job + uploads. - `.github/workflows/e2e-tenant-daily.yml` — daily matrix (vocalstar, singa), WIF auth, Discord + email on failure. - `docs/archive/2026-06-15-tenant-portals-e2e.md` — the full story (8 bugs, all fixed). ## Testing - `copy_blob` unit test; storage suite green (20). - Full tenant E2E re-run to completion after deploy. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
d6bd2f1787 |
fix(tenant): complete the render for jobs that supply their own instrumental (#827)
@coderabbitai ignore ## Two final render blockers (found by the Vocal Star E2E) With attribution (#824), the audio slot (#825), and transcription padding (#826) fixed, the first real tenant job advanced through transcription → screens → review → render, surfacing the last two never-exercised issues: ### 1. Render prerequisites — "No instrumental selected" existing-instrumental jobs (every tenant submission) have no instrumental-selection UI step, so `instrumental_selection` was never written to state_data. The video orchestrator then defaults to `'clean'` (a separated stem that was never produced for these jobs), and `_validate_prerequisites` rejects the job, leaving it stuck retrying. **Fix:** `complete-review` now defaults `instrumental_selection` to `'custom'` (use the provided instrumental) when `existing_instrumental_gcs_path` is set. An explicit request value still wins. The render path already supports `'custom'`. ### 2. Theme asset paths (reproducibility) The tenant setup scripts wrote full `gs://bucket/themes/.../assets/X` URLs, but `theme_service._build_style_assets_mapping` only resolves bare basenames (the consumer convention). With `gs://` URLs, `style_assets` came back empty → assets never downloaded → encoder failed with "Video background image not found". Scripts now emit basenames, and add the newly-required `extra_text_text_transform` field the strict theme validator expects. (The two live GCS themes — vocalstar, singa — were corrected out-of-band to match.) ## Testing - New regression tests: complete-review defaults to `custom` for existing-instrumental jobs; explicit selection still wins. - Review suites green (102). Full E2E re-run to completion after deploy. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
fea6ebec1a |
fix(tenant): make white-label B2B portals attribute jobs to the tenant (Vocal Star + Singa) (#824)
@coderabbitai ignore ## Problem The white-label tenant portals (`vocalstar.nomadkaraoke.com`, `singa.nomadkaraoke.com`) looked "done" but **had never worked end-to-end**: 0 of the last 484 production jobs carried a `tenant_id`. No tenant job had ever run. **Root cause:** the frontend API client never sent the `X-Tenant-ID` header. Tenant portals call the shared `api.nomadkaraoke.com`, whose Host isn't the tenant subdomain, and the `?tenant=` query param is disabled in prod — so the backend could never detect the tenant. Every tenant sign-in and job submission silently fell back to consumer behavior (default theme, no tenant Dropbox, etc.). The `getTenantHeaders()` helper existed and was unit-tested, but was never wired into requests. ## Changes **Frontend** (`lib/api.ts`, `lib/tenant.ts`) - Attach `X-Tenant-ID` on all authenticated requests **and** the magic-link request, sourced from `window.__TENANT_CONFIG__` (edge-injected by the Cloudflare function; mirrored from the tenant store for dev/preview). Consumer build resolves no tenant → no header → **behavior unchanged**. **Backend** (`_apply_tenant_overrides` in `file_upload.py`) - Make tenant config **authoritative** over global distribution defaults. Prod sets `DEFAULT_DROPBOX_PATH` / `DEFAULT_GDRIVE_FOLDER_ID` / `DEFAULT_BRAND_PREFIX` globally; these seed `dist`, and the old "fill only when unset" overlay let them leak — so every tenant job would have landed in the **shared consumer Dropbox folder** and uploaded to the **global Google Drive**. Now Dropbox/GDrive routing is gated by the tenant feature flags and cleared when disabled. **Tenant configs** (GCS + setup scripts, Vocal Star + Singa) - Disable Google Drive (`gdrive_upload=false`, `gdrive_folder_id=null`), keep YouTube off, enable Dropbox to each tenant's own folder. B2B = locked theme, no YouTube/GDrive, Dropbox + portal download only. ## Why it's consumer-safe - Frontend: header only attached when a tenant is resolved (never on `gen.nomadkaraoke.com`). - Backend: `_apply_tenant_overrides` early-returns when `tenant_config is None` (all consumer jobs). ## Testing - New: frontend header-attachment tests (tenant build vs consumer); backend authoritative-override + feature-flag-gating regression guards (incl. the global-default-leak case). - Full suites green: **3466 backend**, **1013 frontend**. - Verified live tenant configs via API after update. Manual prod E2E (real track through each portal) + a daily per-tenant E2E workflow follow in a second PR, once this deploys and is verified in prod. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
39fb93653f |
fix(auto-correct): switch active model from Fable 5 to Opus 4.8 (#822)
## Summary Claude **Fable 5** (`claude-fable-5`) is no longer available on the Anthropic API. Verified live against `api.anthropic.com`: ``` 404 not_found_error: "Claude Fable 5 is not available. Please use Opus 4.8. Learn more: https://www.anthropic.com/news/fable-mythos-access" ``` `claude-opus-4-8` responds normally and is the API's own prescribed replacement. Opus 4.8 was also ≈ Gemini 3.1 Pro in our 30-job offline eval, so switching to it preserves the two-model compare quality. ## Changes - **`.github/workflows/ci.yml`** (load-bearing): `AUTO_CORRECT_MODEL=claude-opus-4-8` and `AUTO_CORRECT_COMPARE_MODELS=claude-opus-4-8;gemini-3.1-pro-preview`. Compare mode now runs **Opus 4.8 + Gemini 3.1 Pro** instead of Fable + Gemini. - **`scripts/measure_autocorrect_cost.py`**: default models + docstring example → Opus, so the cost tool measures what prod runs. - **`docs/API.md`**: example auto-correct response reflects Opus. - **`pyproject.toml`**: `0.182.1` → `0.182.2`. ## Re-enable path Fable is **disabled, not deleted** — its pricing entry in `pricing.py` and all Fable unit tests are intentionally left intact. If access returns, flip it back by reverting the two env-var values in `ci.yml`. ## Testing - Auto-correct suite: 73 passed (`backend/tests/services/auto_correct/` + `test_proactive_auto_correct.py`). - Live API probe confirmed Fable 404s and Opus 4.8 works. - CodeRabbit (uncommitted): 0 findings. @coderabbitai ignore Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
364e4ebd5f |
feat(auto-correct): proactive multi-model suggestions + cost logging + de-AI-ified review UI (#816)
## Summary Makes AI lyrics auto-correction run **automatically on every job** (multi-model), so suggestions are cached and ready the moment the reviewer opens the lyrics UI — plus token/cost instrumentation and a UI cleanup that removes all "AI"/"model" wording. Measured cost first (`scripts/measure_autocorrect_cost.py`): **~$0.13/track** multi-model on normal songs — cheap enough to run proactively. ## Changes **Cost instrumentation** - Capture per-model `usage` from Anthropic + Gemini; log greppable `auto-correct usage` / `auto-correct cost … total_est_cost_usd=…` lines from `suggest()`. - `backend/services/auto_correct/pricing.py` (Anthropic rates authoritative; Gemini estimated) + `scripts/measure_autocorrect_cost.py` offline harness. **Proactive generation (every job)** - `lyrics_worker._run_proactive_auto_correct()` runs the multi-model path after `mark_lyrics_complete` (downstream already triggered → non-blocking), gated by `AUTO_CORRECT_PROACTIVE_ENABLED`, bounded by a 180s timeout, and **swallows every error** — it never blocks or fails the karaoke job (flaky AI / out of credits / timeout all degrade gracefully). - Cache-key aligned with the UI request, so the review page's on-load call is an instant cache hit (no double spend). - `ci.yml`: wires `AUTO_CORRECT_*` env + the `anthropic-api-key` secret onto `lyrics-transcription-job` (same SA as the API service — no IAM change). **Review UI** - Always multi-model (removed the model-selection toggle); button renamed **"Auto-correct"**; stripped model/consensus/cached attribution badges; reworded strings to drop "AI"/"model" (translated to all 32 locales). - Auto-runs once on review load so suggestions are ready without a click; the button stays for the paste-references-later flow. ## Testing - Backend: **3460 passed** (`backend/tests/`, emulator excluded) — incl. new `test_pricing.py` + `test_proactive_auto_correct.py` (gating, graceful failure, cache alignment). - Frontend: **1006 jest tests pass**; type-clean; 33-locale key parity verified. ## Notes / follow-ups - Proactive activates on the **next deploy** (the lyrics-job env/secret apply via this PR's ci.yml on merge). - Left "AI" wording in two *separate* features untouched (different subsystems): `lyricsReview.agenticMetrics` and `lyricsReview.modals.correctionAnnotation` — flagged for a separate decision. - The production E2E spec's click-to-open flow may want a tweak now that suggestions auto-run on load. @coderabbitai ignore --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |