mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
main
23
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
42a7793681 |
AI-powered credit gating with Gemini evaluation (#584)
## Summary - Replace automatic welcome/feedback credit grants with AI-powered abuse evaluation using Gemini 3.1 Pro - **Fail-closed**: errors/uncertainty → credits withheld, admin notified via email with user admin link - Clean users (no correlations) get instant grant with no AI call latency - Frontend verify page shows preparing account → celebratory/denied/pending interstitial - Comprehensive docs at `docs/AI-CREDIT-GATING.md` ## Changes | File | What | |------|------| | `backend/services/credit_evaluation_service.py` | **New**: Signal collection, Gemini prompt, response parsing, Firestore audit | | `backend/config.py` | `CREDIT_EVAL_ENABLED`, `CREDIT_EVAL_MODEL` config | | `backend/services/user_service.py` | Welcome credit flow integration (returns tuple) | | `backend/api/routes/users.py` | Feedback credit + verify response integration | | `backend/services/email_service.py` | `send_credit_denied_email()`, `send_credit_review_needed_email()` | | `backend/models/user.py` | `credit_status` field on verify response | | `frontend/app/auth/verify/page.tsx` | 5-state interstitial flow | | `frontend/lib/types.ts` + `auth.ts` | Type + state updates | | `backend/tests/test_credit_evaluation.py` | **New**: 17 tests | | `backend/tests/test_anti_abuse.py` | Updated for new return type | | `docs/AI-CREDIT-GATING.md` | **New**: Comprehensive documentation | | `pyproject.toml` | `google-generativeai` dep, version bump | ## Test plan - [x] 50 backend tests pass (17 new + 33 updated) - [ ] Manual: test with new account signup in prod (see docs for expected flow) - [ ] Manual: verify admin review email received for error cases ## Rollback Set `CREDIT_EVAL_ENABLED=false` in Cloud Run env vars → instant revert to auto-grant. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
c9e434037b |
feat: admin feedback visibility — page + email notification (#581)
<!-- @coderabbitai ignore --> ## Summary - **Admin feedback page** at `/admin/feedback` — browse all user feedback submissions with star ratings, text fields, search, pagination, and aggregate stats - **Email notification** to admin@nomadkaraoke.com whenever a user submits new feedback (configurable via `ADMIN_NOTIFICATION_EMAIL` env var) - **Admin API endpoint** `GET /api/admin/feedback` with pagination, search, test-data filtering, and average ratings ## Context User feedback was stored in Firestore (`user_feedback` collection) but completely invisible to the admin — the admin panel only showed a boolean "Feedback Submitted: Yes/No" badge per user. This PR closes that gap. ## Changes **Backend:** - `GET /api/admin/feedback` endpoint with pagination, search, `exclude_test` filter, aggregate rating averages - `send_feedback_notification()` method on `EmailService` — sends HTML email with star ratings, all text fields, link to admin page - Wired into `submit_user_feedback` as fire-and-forget (won't block user response) **Frontend:** - New admin page with aggregate stat cards, searchable/paginated table, visual star ratings, expandable rows for text detail - "Feedback" sidebar nav item with MessageSquare icon - `adminApi.listFeedback()` client method + TypeScript types **Tests:** 14 tests covering endpoint (pagination, search, filtering, aggregates, empty state, datetime edge cases), email (content, subject, custom admin, failure), and caller-callee contract integration ## Test plan - [x] `make test` — 712 tests pass (39 suites) - [x] 14 new tests in `test_admin_feedback.py` - [ ] Manual: verify admin page renders at `/admin/feedback` after deploy - [ ] Manual: submit feedback → verify email arrives at admin@nomadkaraoke.com 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
b8239dc691 |
Block disposable email registration with friendly messaging (#542)
## Summary - Wire existing `EmailValidationService` into the magic link signup flow to reject disposable email domains (yopmail, tempmail, emailhook.site, etc.) before creating users - Return 422 with machine-readable `disposable_email_not_allowed` error code for disposable domains - Silent rejection for explicitly blocked emails/IPs (anti-enumeration) - Frontend shows a warm amber message box encouraging real email use, promising no spam - Add `emailhook.site` to default disposable domains list (160+ domains total) ## Context Users were signing up with disposable emails like `anything@yopmail.com` to get unlimited free welcome credits (2 per account). The `EmailValidationService` existed with a comprehensive blocklist but was never called during signup. This PR wires it in. The admin UI for managing the blocklist already exists at `/admin/rate-limits/` (Blocklists tab) with full CRUD for domains, emails, and IPs. ## Test plan - [x] 8 new backend integration tests (disposable rejection, blocked email/IP, email lowercasing, domain list) - [x] 2 new frontend tests (friendly message rendering, generic error fallback) - [x] All 2562 backend tests pass - [x] All 16 frontend auth component tests pass - [x] CodeRabbit review: no findings @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
9d8bf3a952 |
Improve job observability with timeline metadata and log_to_job (#508)
@coderabbitai ignore ## Summary - Add `metadata` field to `TimelineEvent` for structured output details (brand codes, YouTube video IDs, Dropbox paths, GDrive file IDs) - Add `log_to_job()` helper for API endpoints to write to Firestore job log subcollections - Plumb `timeline_metadata` through `job_manager.transition_to_state()` and `firestore_service.update_job_status()` - Capture output details in completion timeline events for full traceability - Log edit/delete cleanup operations to both Cloud Logging and Firestore job logs - Snapshot previous outputs before cleanup for audit trail - Enhance distribution logging with video_id, folder path, and file IDs ## Context After shipping "edit completed tracks," tracing edited jobs was difficult because timeline events lacked output details, cleanup operations weren't logged to Firestore, and previous outputs were overwritten with no history. These changes make the job's Firestore document self-documenting — any agent or human can read the timeline and understand the full lifecycle. ## Test plan - [x] 7 new unit tests for `TimelineEvent.metadata` and `log_to_job()` helper - [x] All 103 backend tests pass - [ ] Verify in production after deploy: edit a test job and confirm timeline metadata appears in Firestore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
83e353438a |
feat: admin lyrics review dashboard (#492)
@coderabbitai ignore
## Summary
- Add "Edit Reviews" section to admin dashboard for reviewing user
lyrics corrections
- New backend endpoints: `GET /api/admin/edit-reviews` (list) and `GET
/api/admin/edit-reviews/{job_id}` (detail snapshot)
- Three frontend views: read-only lyrics review, edit log replay with
step-through, raw data JSON viewer
- All endpoints are strictly read-only (no state mutations, no
Firestore/GCS writes)
- 7 backend unit tests covering filtering, search, pagination, test
exclusion, error handling
## Motivation
Users are now making lyrics corrections during the review phase, and
we're collecting edit logs. This dashboard lets admins:
1. See what users experienced (original vs corrected lyrics in read-only
LyricsAnalyzer)
2. Replay user edits step-by-step to evaluate quality
3. Review raw human edits data for potential AudioShake data sharing
## Test plan
- [x] 7 backend unit tests pass (list filtering, search, pagination,
test exclusion, 404, full snapshot, missing files)
- [ ] Manual: navigate to /admin/edit-reviews, verify list loads
- [ ] Manual: click View Review, verify read-only LyricsAnalyzer renders
- [ ] Manual: click View Edit Log, verify step-through works
- [ ] Manual: click View Raw Data, verify JSON tabs render
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
|
9f85ae4221 |
feat: credit notifications and refund on job deletion (#474)
<!-- @coderabbitai ignore --> ## Summary - **Backend**: Refund credits when jobs are deleted (if not already complete/failed/cancelled) - **Frontend**: Show toast notifications for credit changes — "Credit used" on job creation, "Credit refunded" on eligible job deletion - Refreshes user data from backend after credit-changing operations so header counter stays accurate ## Changes | File | What | |------|------| | `backend/services/job_manager.py` | Refund credit on job deletion (non-complete jobs) | | `frontend/components/job/JobSubmission.tsx` | "Credit used — X remaining" toast after job creation | | `frontend/components/job/JobActions.tsx` | "Credit refunded — X remaining" toast after eligible deletion | | `frontend/components/__tests__/JobActions.test.tsx` | Updated test mock for auth module | ## Test plan - [x] All 178 Playwright + Jest tests pass - [ ] Create a job → see "Credit used" toast with correct remaining count - [ ] Delete a pending/processing job → see "Credit refunded" toast - [ ] Delete a completed job → see "Job deleted" toast (no refund mention) - [ ] Header credit counter updates immediately after both operations 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
186b3dd7a9 |
feat: song lookup autocomplete and community version detection (#451)
## Summary - Add track autocomplete on the title field using karaoke-decide catalog API (MusicBrainz + Spotify data) for canonical artist/title styling - Add community karaoke version detection via karaokenerds.com scraping — warns users when existing high-quality versions are available on YouTube, saving credits - Both features are optional and non-blocking — users can still type freely and submit without selecting from autocomplete ## Changes ### Backend - **`backend/services/catalog_proxy_service.py`** — HTTP proxy to karaoke-decide API with 5-min in-memory TTL cache - **`backend/services/karaokenerds_service.py`** — Karaokenerds.com HTML scraper (ported from kjbox) for community version detection - **`backend/api/routes/catalog.py`** — 3 new endpoints: `GET /api/catalog/artists`, `GET /api/catalog/tracks`, `POST /api/catalog/community-check` with per-user rate limiting (20 req/min) - **`backend/config.py`** — Added `karaoke_decide_api_url` setting ### Frontend - **`frontend/components/ui/autocomplete-input.tsx`** — Reusable autocomplete input component (shadcn Command + Popover) - **`frontend/components/job/CommunityVersionBanner.tsx`** — Green dismissible banner showing community karaoke versions with YouTube links - **`frontend/components/job/steps/SongInfoStep.tsx`** — Title field now has autocomplete; selecting a track overwrites both artist and title with canonical names - **`frontend/lib/api.ts`** — Added catalog API client functions ### Tests - 9 backend route tests (artist/track search, community check, rate limiting) - 9 karaokenerds service tests (HTML parsing, community detection, YouTube URL cleanup) ## How it works 1. User types artist name in the plain text field 2. User starts typing in the title field → autocomplete dropdown shows tracks from Spotify/MusicBrainz, filtered by artist 3. Selecting a track overwrites both fields with canonical names (e.g., "fleetwood mac" → "Fleetwood Mac") 4. When both fields are filled, a community check fires — if existing karaoke versions exist on YouTube, a green banner appears with links ## Test plan - [x] 18 backend tests pass (catalog routes + karaokenerds service) - [x] Tested locally with dev server against real karaoke-decide API - [x] Verified autocomplete shows results, selection overwrites both fields - [x] Verified community banner appears for known songs (e.g., Fleetwood Mac - Dreams) - [x] Verified graceful degradation when API is unavailable @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
fa9bc8eeeb |
Add private (non-published) tracks option (#403)
@coderabbitai ignore ## Summary - Add `is_private` flag to jobs that changes distribution behavior for non-published tracks - Private jobs upload to Dropbox `Tracks-NonPublished` folder with `NOMADNP` brand prefix - Private jobs skip YouTube and Google Drive uploads entirely - Completion emails omit YouTube section for private jobs - Available to all users at job creation; admin can toggle on existing jobs - When admin toggles completed job to private, existing outputs are auto-deleted ## Changes ### Backend - **Job model**: Added `is_private: bool = False` to `Job` and `JobCreate` - **Config**: Added `DEFAULT_PRIVATE_DROPBOX_PATH` and `DEFAULT_PRIVATE_BRAND_PREFIX` settings - **Distribution helper**: New `get_effective_distribution_for_job()` in `job_defaults_service.py` applies private overrides at runtime - **Orchestrator + native paths**: Both distribution code paths use the helper (per "Fix Both Sides" lesson) - **Email template**: YouTube section conditionally removed when `is_private=True` - **Credential pre-validation**: Skip YouTube/GDrive credential checks for private jobs - **API endpoints**: All 4 job creation endpoints accept `is_private`; admin can PATCH it - **Admin auto-delete**: Toggling completed job to private triggers output deletion ### Frontend - **Job creation form**: "Private (Non-Published)" checkbox on all 3 tabs (upload, URL, search) - **Admin dashboard**: Private toggle in job configuration section - **API interfaces**: Updated TypeScript interfaces and API methods ### Tests - 23 unit tests covering model, distribution overrides, email template, notification service, request models, admin toggle, and orchestrator config - Fixed MagicMock truthiness issue in existing tests ## Test plan - [x] All 23 private tracks tests pass - [x] All 1794 existing backend tests pass (1 flaky unrelated test) - [x] CodeRabbit review: credential pre-validation issue found and fixed - [ ] Manual: Create private job end-to-end, verify Dropbox folder - [ ] Manual: Verify completion email has Dropbox link but no YouTube section - [ ] Manual: Admin toggle on completed job deletes outputs 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
716936c8a9 |
Add retry logic for flacfetch download failures and audio search recovery (#382)
## Summary Adds comprehensive retry logic to handle transient flacfetch network failures, preventing permanent job failures from temporary service issues. Includes state persistence for retry capability, enhanced retry endpoint for audio search jobs, and user-facing error feedback. ## Key Changes **Retry Logic (backend/services/flacfetch_client.py):** - Manual exponential backoff: 9 attempts over ~6 minutes (10s → 20s → 40s → 60s capped) - Selective retry: retries on connection errors and 5xx, immediate fail on 4xx - Runtime-configurable via environment variables (FLACFETCH_RETRY_MAX_ATTEMPTS, etc.) **State Persistence (backend/api/routes/audio_search.py):** - Save download parameters (audio_source_type, source_name, source_id, target_file, download_url) BEFORE attempting download - Enables retry even if download fails completely - Added 5 new fields to Job model (backend/models/job.py) **Enhanced Retry Endpoint (backend/api/routes/jobs.py):** - Added ~150 lines handling audio search retry cases - Re-downloads using saved parameters for YouTube, RED, OPS, Spotify sources - Clears error state and transitions to DOWNLOADING before triggering workers **User Feedback (frontend/components/job/JobActions.tsx):** - Error toast shows actual API error message on retry failure - Success toast confirms retry started - No more silent failures **Configuration (backend/config.py):** - Added retry config: max_attempts (9), min_wait (10s), max_wait (60s) ## Testing - [x] Added 5 comprehensive retry tests in `backend/tests/test_flacfetch_client.py` - Network error retry with exponential backoff - No retry on 4xx client errors - Retry on 5xx server errors - Max retries exhausted - Wrapper consistency - [x] All backend tests pass: 103 passed - [x] Frontend E2E tests: 142 passed (21 pre-existing failures unrelated to changes) ## Review - [x] CodeRabbit CLI review completed locally - [x] Fixed unused import identified in review - [x] Documentation updated (README.md Recent Changes, LESSONS-LEARNED.md retry patterns) - [x] Implementation plan archived to docs/archive/2026-02-10-flacfetch-retry-logic.md ## Root Cause Job 749141f8 failed because flacfetch network call failed during audio search, and no source info was saved, making retry impossible. This PR fixes all four related issues identified during investigation. ## Version Bumped 0.115.5 → 0.116.0 (minor version for new feature) @coderabbitai ignore --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com> |
||
|
|
29be83cb56 |
Combined review flow: lyrics + instrumental selection in one session (#351)
## Summary Merges the two-step human review process (lyrics review → instrumental selection) into a single unified flow. Users now review lyrics AND select instrumental track in one session. **Benefits:** - Better UX (one interaction vs two) - Reduced drop-off between steps - Single email notification instead of two - Simpler workflow ## Changes ### Backend - `complete_review` endpoint now accepts optional `instrumental_selection` in request body - Backing vocals analysis moved from `render_video_worker` to `screens_worker` (runs before review) - Removed standalone instrumental selection endpoints (now part of combined review) - Added `CompleteReviewRequest` model with validation ### Frontend - New `InstrumentalSelectorEmbedded` component for combined review UI - `LyricsAnalyzer` updated to proceed to instrumental review - `ReviewChangesModal` shows "Proceed to Instrumental Review" button - Mock data updated in E2E tests ### Local CLI - `gen_cli.py` refactored to use `run_combined_review()` instead of separate `run_instrumental_review()` - `ReviewServer` extended with instrumental options and analysis - Deleted standalone `InstrumentalReviewServer` ### Documentation - Updated `docs/README.md` with recent changes entry - Updated `docs/ARCHITECTURE.md` with new pipeline diagram - Updated `docs/API.md` with endpoint changes - Added lesson to `docs/LESSONS-LEARNED.md` ## Test plan - [x] Backend unit tests: 1742 passed - [x] Frontend unit tests: 234 passed - [x] New integration tests: 5 tests for combined flow - [x] New backend tests: 7 tests for CompleteReviewRequest and state_data - [x] New frontend E2E tests: 6 tests for combined flow UI 🤖 Generated with [Claude Code](https://claude.com/claude-code) @coderabbitai ignore --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
|
b68a46d174 |
feat(infra): Implement immutable deployment pattern for encoding worker (#333)
## Summary - Implements robust deployment pattern for GCE encoding worker to eliminate recurring version mismatch issues - Fixed wheel path (`karaoke_gen-current.whl`) eliminates version sorting bugs - Self-updating startup script downloaded from GCS on every service start - Version manifest with strict CI verification (deployment fails on mismatch) ## Problem The encoding worker had recurring version issues because: - `startup.sh` was baked into the Packer image (required rebuild + VM recreation) - Wheel selection used version sorting across 51 files - CI only warned on version mismatch, didn't fail - Multiple deployment vectors (CI, Pulumi, manual SSH) caused drift ## Solution 1. **Fixed wheel path**: CI uploads to `karaoke_gen-current.whl` (no sorting needed) 2. **Self-updating startup**: `bootstrap.sh` downloads `startup.sh` from GCS on every start 3. **Version manifest**: CI writes expected version, service verifies before starting 4. **Strict CI verification**: Deployment fails if version doesn't match (was just warning) ## Changes - **New**: `infrastructure/encoding-worker/` with `bootstrap.sh`, `startup.sh`, README - **Modified**: CI workflow for fixed paths and strict verification - **Modified**: Packer `provision.sh` for bootstrap pattern - **Updated**: `LESSONS-LEARNED.md` with immutable deployment pattern - **Updated**: `docs/README.md` and `docs/ARCHITECTURE.md` ## Test plan - [x] Shell scripts validated (`bash -n`) - [x] Backend tests pass (1703 passed) - [x] CodeRabbit review passed - [ ] After merge: Rebuild Packer image - [ ] After merge: Recreate VM to use new image - [ ] After merge: Verify first deployment uses new pattern ## Post-Merge Steps Required 1. Rebuild Packer image: `packer build infrastructure/packer/encoding-worker.pkr.hcl` 2. Recreate VM: Delete encoding-worker, run `pulumi up` 🤖 Generated with [Claude Code](https://claude.com/claude-code) @coderabbitai ignore --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
|
2947dd05bc |
refactor: consolidate YouTube downloads into YouTubeDownloadService (#322)
## Summary - Consolidates all YouTube download paths into a single `YouTubeDownloadService` - Ensures YouTube downloads always route through flacfetch remote when configured (avoids bot detection on Cloud Run) - Both entry points (audio search selection AND direct URL submission) now use the same service - Clear error messaging when remote is unavailable ## Problem YouTube downloads were failing in cloud because: 1. Direct YouTube downloads from Cloud Run get blocked by bot detection 2. There were 3 different code paths for YouTube downloads with inconsistent handling 3. Some paths weren't passing `gcs_path` to flacfetch, causing file access issues ## Changes ### New Files - `backend/services/youtube_download_service.py` - Consolidated service for all YouTube downloads - `backend/tests/test_youtube_download_service.py` - 20 unit tests - `backend/tests/integration/test_youtube_flow.py` - 8 integration tests ### Modified Files - `backend/api/routes/audio_search.py` - Route YouTube to YouTubeDownloadService - `backend/api/routes/file_upload.py` - Download YouTube before triggering workers - `backend/workers/audio_worker.py` - Mark legacy fallback as deprecated - `backend/services/audio_search_service.py` - Fix source_id extraction - `docs/ARCHITECTURE.md` - Add audio source download flow diagram ## Test Plan - [x] 20 unit tests for YouTubeDownloadService - [x] 8 integration tests for YouTube flow routing - [x] Updated existing tests to mock new service - [x] All 1630+ backend tests pass @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
|
83c54803ef |
feat: disable auto-correction during lyrics transcription (#321)
## Summary - Disabled all auto-correction during lyrics transcription (both agentic AI via Gemini and heuristic rule-based handlers) - Raw transcription now goes directly to human review without automatic fixes - This change was made because auto-correction was creating more work for reviewers by introducing errors that needed manual correction ## Changes - Added `SKIP_CORRECTION` env var (default: `true`) in `backend/config.py` - Updated `LyricsProcessor` to check skip_correction and set `run_correction=False` - Added `_create_uncorrected_result()` method to controller for review UI compatibility - Updated docs to reflect disabled status ## How to Re-enable Set `SKIP_CORRECTION=false` environment variable to re-enable auto-correction if quality improves in the future. ## Test plan - [x] All 1605 tests pass - [x] Test review: Changes are configuration-driven, existing code paths remain tested - [x] CodeRabbit review: Passed clean @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
|
170acfd508 |
fix: pass gcs_path for YouTube downloads via remote flacfetch (#320)
## Summary - Fix YouTube audio downloads failing when remote flacfetch is enabled - Add dynamic MIME type detection for audio uploads (fixes hardcoded `audio/flac` for `.webm`/`.opus` files) - Add documentation example to LESSONS-LEARNED.md ## Problem When remote flacfetch is enabled and a user selects a YouTube audio source, the backend was calling `download_by_id()` without the `gcs_path` parameter. This caused flacfetch to download the file locally on the VM instead of uploading to GCS. The backend then tried to open the VM's local path (`/var/lib/transmission-daemon/downloads/...`) which doesn't exist on Cloud Run, causing the job to fail. **Production impact:** Job `4c266875` failed with "No such file or directory" ## Root Cause This is the same class of bug that was fixed in December 2025 for RED/OPS torrent sources. However, that fix only addressed the torrent code path, leaving YouTube downloads broken when remote is enabled. ## Changes ### `backend/api/routes/audio_search.py` - Add new `elif` branch to handle remote YouTube downloads with `gcs_path` parameter - Use `mimetypes.guess_type()` to detect correct content type instead of hardcoded `audio/flac` ### `backend/tests/test_audio_search.py` - Add `test_youtube_remote_download_passes_gcs_path` to verify the fix - Add `test_youtube_local_download_when_remote_disabled` for coverage ### `docs/LESSONS-LEARNED.md` - Add specific example of the `gcs_path` bug to the "Fix Both Sides of Dual Code Paths" lesson ## Test plan - [x] All 50 audio_search tests pass - [x] New test verifies `gcs_path` is passed to `download_by_id()` for YouTube sources - [x] New test verifies local download path when remote is disabled - [ ] Production verification: retry previously failed job after deployment <!-- @coderabbitai ignore --> 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
|
404c6e6133 |
fix: make lyrics_transcriber a subpackage of karaoke_gen (#299)
## Summary Fixes the deployment failure from PR #297 by making lyrics_transcriber a proper subpackage of karaoke_gen instead of a separate vendored library. ### Changes - Move `libs/lyrics_transcriber/` to `karaoke_gen/lyrics_transcriber/` - Update all imports from `from lyrics_transcriber.X` to `from karaoke_gen.lyrics_transcriber.X` - Simplify Dockerfile (no separate `libs/` COPY needed) - Update pyproject.toml to remove separate package entry - Update test mocks and logger names ### Why this is better - Cleaner project structure - one package containing everything - Simpler Dockerfile - just COPY karaoke_gen/ - No confusion about vendored vs internal code ## Test plan - [x] `poetry install` succeeds - [x] All imports work correctly - [x] All 2614 tests pass - [ ] Backend deployment succeeds @coderabbitai ignore Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
|
a52ea006cb |
refactor: deprecate python-lyrics-transcriber, consolidate into karaoke-gen (#297)
## Summary - Moved lyrics_transcriber from lyrics_transcriber_temp/ to libs/ - Removed obsolete standalone package files - Updated pyproject.toml paths - Archived python-lyrics-transcriber repo ## Test plan - [x] poetry install succeeds - [x] Backend tests pass (2614) - [x] python-lyrics-transcriber archived @coderabbitai ignore --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
|
7cfac407e3 |
feat(admin): Enhanced admin job detail page with files, edit, and reset (#261)
## Summary
- Add comprehensive job management features to admin job detail view
- Enable file downloads with signed GCS URLs grouped by category
- Add inline editing of job fields (artist, title, theme, etc.)
- Add job reset actions to re-process from workflow checkpoints
## Changes
### Backend (39 new tests)
- `GET /api/admin/jobs/{job_id}/files` - Get all job files with signed
URLs (2-hour expiry)
- `PATCH /api/admin/jobs/{job_id}` - Update editable job fields with
whitelist validation
- `POST /api/admin/jobs/{job_id}/reset` - Reset job to specific workflow
checkpoint
### Frontend (13 E2E tests)
- **Files accordion** - Download job files grouped by category (input,
stems, lyrics, outputs)
- **Configuration section** - Inline editing for artist, title,
theme_id, brand_prefix, customer_email
- **Admin Actions section** - Reset buttons for pending, audio
selection, review, instrumental selection
- **Enhanced timeline** - Color-coded stages (green=complete,
blue=processing, orange=waiting, red=failed)
### Documentation
- Updated `docs/API.md` with new endpoint documentation
- Updated `docs/README.md` with feature summary
- Added lesson learned about AuthResult tuple unpacking gotcha
## Test Plan
- [x] Backend tests pass (39 new tests for files, update, reset
endpoints)
- [x] Frontend E2E tests pass (13 tests for admin job detail page)
- [x] Frontend unit tests pass (150 tests)
- [x] Frontend builds successfully
- [x] CodeRabbit review passed (2 cycles, all issues fixed)
## Screenshots
See updated admin job detail page at `/admin/jobs?id={job_id}`
@coderabbitai ignore
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
|
42c304dcdf |
feat(multitenancy): Implement white-label B2B portal infrastructure (#241)
## Summary
Adds multi-tenant white-label portal infrastructure for B2B customers.
Vocal Star is the first tenant at `vocalstar.nomadkaraoke.com`.
## Changes
### Backend
- **Tenant Models** (`backend/models/tenant.py`): Pydantic models for
TenantConfig with branding, features, defaults, and auth settings
- **Tenant Service** (`backend/services/tenant_service.py`): GCS-backed
config loading with 5-min cache, signed URL generation for logos
- **Tenant API** (`backend/api/routes/tenant.py`): `GET
/api/tenant/config` and `GET /api/tenant/config/{tenant_id}` endpoints
- **Tenant Middleware** (`backend/middleware/tenant.py`): Extracts
tenant from X-Tenant-ID header, query param (dev only), or Host
subdomain
- **Auth Updates**: Email domain validation for tenant-restricted auth,
sender email override
- **Feature Enforcement**: Audio search endpoint checks
`features.audio_search`, returns 403 if disabled
- **Job Scoping**: Added `tenant_id` field to JobCreate
### Frontend
- **Tenant Store** (`frontend/lib/tenant.ts`): Zustand store with tenant
state, `useTenant()` hook, CSS variable application
- **TenantProvider** (`frontend/components/tenant-provider.tsx`):
Hydration-safe initialization via useEffect
- **TenantLogo** (`frontend/components/tenant-logo.tsx`): Dynamic logo
with fallback to Nomad Karaoke
- **Feature-Flagged UI**: Tabs conditionally rendered based on tenant
features, active tab sync
- **Locked Themes**: `defaults.locked_theme` hides theme selector and
enforces specific theme
- **Output Links**: Filtered based on distribution features
### Security
- Query param tenant detection disabled in production (prevents
spoofing)
- Strict subdomain patterns: only `{tenant}.nomadkaraoke.com` or
`{tenant}.gen.nomadkaraoke.com`
- PII masking in logs (`an***@vo***.com`)
- `is_active` check prevents inactive tenants from being used
### Documentation
- ARCHITECTURE.md: New Multitenancy section
- API.md: Tenant Config endpoints
- LESSONS-LEARNED.md: 3 new lessons on multitenancy patterns
- Archive: Comprehensive implementation document
### Setup
- `scripts/setup-vocalstar-tenant.py`: Uploads theme assets, creates
tenant/theme configs in GCS
## Vocal Star Configuration
- **Subdomain**: `vocalstar.nomadkaraoke.com`
- **Features**: File upload only (no audio search, no YouTube URL)
- **Distribution**: Download only (no YouTube/Dropbox/GDrive)
- **Theme**: Locked to "vocalstar" (yellow/blue)
- **Auth**: Restricted to `@vocal-star.com` and `@vocalstarmusic.com`
emails
## Testing
- [x] Frontend TypeScript build passes
- [x] CodeRabbit CLI review completed (3 cycles, all issues fixed)
- [x] Manual testing with `?tenant=vocalstar` query param
## Next Steps (post-merge)
1. Configure DNS: CNAME for `vocalstar.nomadkaraoke.com`
2. Update Cloudflare Pages for subdomain
3. Run setup script to upload Vocal Star assets to GCS
4. Test at `https://vocalstar.nomadkaraoke.com`
@coderabbitai ignore
---
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
|
||
|
|
560c3e4904 |
feat: Add email notification system with admin controls (#220)
## Summary
- Add automated email notification system for job completion and action
reminders
- Admin UI buttons to copy completion message or send email directly
- Comprehensive test suite (81 new tests)
- **Email subjects include release ID and song info** for easy
identification
## Changes
### Backend
- **Email Service** (`email_service.py`): Extended with SendGrid CC
support, `send_job_completion()` and `send_action_reminder()` methods
- **Job Notification Service** (`job_notification_service.py`): New
service orchestrating completion emails and action reminder scheduling
- **Template Service** (`template_service.py`): GCS-backed HTML email
templates with fallback defaults
- **Worker Service** (`worker_service.py`): Cloud Tasks integration for
delayed idle reminders (5-min delay)
- **Job Manager** (`job_manager.py`): State transition triggers for
automated notifications
- **Admin Routes** (`admin.py`): New endpoints `GET
/api/admin/jobs/{id}/completion-message` and `POST
/api/admin/jobs/{id}/send-completion-email`
- **Internal Routes** (`internal.py`): Idle reminder check endpoint for
Cloud Tasks
### Email Subject Format
Email subjects now include brand code and song info for easy
identification:
- With brand code: `NOMAD-1178: Seether - Tonight (Your karaoke video is
ready!)`
- Without brand code: `Seether - Tonight (Your karaoke video is ready!)`
- Without song info: `Your karaoke video is ready!`
### Frontend
- **OutputLinks Component**: Admin-only "Copy Message" and "Send Email"
buttons with dialog UI
- **E2E Helpers** (`email-testing.ts`): `waitForCompletionEmail()` and
`isCompletionEmail()` methods for testmail.app integration
- **Happy Path Test**: Step 9.5 verifies completion email arrives after
job completion
### Documentation
- Updated API.md with new endpoints
- Updated ARCHITECTURE.md with notification system architecture
- Updated README.md Recent Changes
## Testing
- [x] 81 new unit tests covering:
- `test_email_service.py` (27 tests) - providers, CC support, job
completion/reminder methods
- `test_job_notification_service.py` (21 tests) - URL building, message
generation
- `test_job_manager_notifications.py` (15 tests) - state transition
triggers
- `test_admin_email_endpoints.py` (12 tests) - admin API endpoints
- `test_template_service.py` (6 tests) - GCS template loading, fallbacks
- [x] E2E test extended to verify completion email delivery
- [x] All backend tests pass (`make test`)
- [x] TypeScript compiles
## Feature Flags
- `ENABLE_AUTO_EMAILS` (default: `false`) - Controls automatic email
sending on job state changes
## Review
- [x] Local CodeRabbit review completed (3 cycles)
- [x] Review feedback addressed:
- Fixed potential None state_data in internal.py
- Masked user emails in logs (PII protection)
- Fixed React setTimeout memory leak
- Changed placeholder URL default to empty
- Added CC email deduplication
@coderabbitai ignore
---
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
|
ae85ec0a09 |
feat: Move worker logs to Firestore subcollection to avoid 1MB limit (#195)
## Summary
Move `worker_logs` from embedded array in job documents to a Firestore
subcollection (`jobs/{job_id}/logs`) to avoid the 1MB document size
limit. This fixes job failures when logs exceed the limit during
long-running operations like video encoding.
**Problem:** Job `501258e1` failed when logs reached 1.26MB (2,547 log
entries), representing 98.6% of the maximum document size.
## Changes
### New Files
- `backend/models/worker_log.py` - WorkerLogEntry dataclass with factory
methods, serialization (to_dict, to_legacy_dict, from_dict)
- `backend/tests/test_worker_log_subcollection.py` - 24 unit tests
- `backend/tests/emulator/test_worker_logs_subcollection.py` - 11
integration tests using Firestore emulator
### Modified Files
- `backend/services/firestore_service.py` - Added subcollection methods
(append, get, count, delete)
- `backend/services/job_manager.py` - Use subcollection with fallback to
legacy embedded arrays
- `backend/api/routes/jobs.py` - Updated logs endpoint to use new
methods
- `backend/config.py` - Added `USE_LOG_SUBCOLLECTION` feature flag
(default: true)
- `infrastructure/__main__.py` - Added TTL policy (30 days) and
composite index for logs subcollection
- `pyproject.toml` - Version bump 0.89.6 → 0.90.0
### Documentation
- `docs/README.md` - Added to Recent Changes
- `docs/ARCHITECTURE.md` - Added logs subcollection to Firestore
Collections table
- `docs/LESSONS-LEARNED.md` - Added "Firestore Document 1MB Limit with
Embedded Arrays" section
## Testing
- [x] 24 unit tests pass
(`backend/tests/test_worker_log_subcollection.py`)
- [x] 11 emulator integration tests pass
(`backend/tests/emulator/test_worker_logs_subcollection.py`)
- [x] All 68 existing emulator tests pass
## Review
- [x] CodeRabbit CLI review completed locally
- [x] Review feedback addressed (timestamp formatting fix)
## Rollback Plan
Set `USE_LOG_SUBCOLLECTION=false` environment variable and redeploy. New
logs will go back to embedded array; subcollection logs will remain but
won't be read.
@coderabbitai ignore
---
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
|
16dc451bdf |
docs: Add API auth guide and worker logs rearchitecture plan (#192)
## Summary - Add API authentication documentation to CLAUDE.md for future agents - Add comprehensive plan for rearchitecting worker logs storage ## Changes ### CLAUDE.md - New "API Authentication (for Agents)" section covering: - Internal worker endpoints with X-Admin-Token - Firestore direct access - GCE encoding worker SSH commands - Common pitfalls to avoid ### .claude/plans/worker-logs-rearchitecture.plan.md - Plan to move worker_logs from embedded array to Firestore subcollection - Fixes 1MB document size limit issue (job 501258e1 hit 1.26 MB) - Includes TTL policy for automatic cleanup (30 days) ## Testing - [x] Documentation review - No code changes @coderabbitai ignore --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
|
c1588992ce |
refactor: Unify video worker pipeline with new orchestrator (#182)
Resolves GCE vs KaraokeFinalise code path divergence that caused features like YouTube upload, Discord notifications, and CDG/TXT packaging to be missing from the GCE encoding path. - New services: youtube_upload_service, discord_service, packaging_service, local_encoding_service, encoding_interface - New VideoWorkerOrchestrator coordinates all stages (packaging, encoding, distribution, notifications) - Works with any encoding backend (GCE or local) - 139 new tests, 976 total tests passing - Feature flag USE_NEW_ORCHESTRATOR (default: true) with rollback capability |
||
|
|
dfb2ebc184 |
feat(frontend): migrate from MailSlurp to testmail.app for E2E testing (#159)
## Summary
- Replace MailSlurp email testing service with testmail.app for E2E
magic link authentication tests
- Simpler API with namespace-based email addresses
(`{namespace}.{tag}@inbox.testmail.app`)
- No new dependencies - uses native `fetch` instead of mailslurp-client
## Changes
- **`frontend/e2e/helpers/email-testing.ts`** - Complete rewrite using
testmail.app JSON API
- Uses `livequery=true` to wait for emails (no polling needed)
- Generates unique tags per test run to avoid collisions
- Same `EmailHelper` interface maintained for minimal test changes
- **`frontend/e2e/production/full-user-journey.spec.ts`** - Update env
var references and messages
- **`frontend/playwright.record.config.ts`** - Update env var comment
- **`frontend/package.json`** - Remove `mailslurp-client` dependency,
bump version to 0.79.0
- **`frontend/.env.local.example`** - New file documenting required env
vars for local testing
- **`docs/DEVELOPMENT.md`** - Add testmail.app env vars to frontend
section
## Environment Variables
| Old | New |
|-----|-----|
| `MAILSLURP_API_KEY` | `TESTMAIL_API_KEY` |
| (none) | `TESTMAIL_NAMESPACE` |
**Note:** GitHub Actions org secrets already configured with new env
vars.
## Testing
- [x] Backend tests pass (`make test` - 57 tests)
- [x] Frontend unit tests pass (139 tests)
- [x] Frontend E2E tests pass (64 tests)
- [x] TypeScript compilation clean for changed files
## Review
- [x] Local CodeRabbit CLI review completed
- [x] Review feedback addressed (removed exposed credentials from plan
file)
@coderabbitai ignore
---
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude <noreply@anthropic.com>
|