23 Commits
Author SHA1 Message Date
Andrew Beveridge ☄️andClaude Opus 4.6 42a7793681 AI-powered credit gating with Gemini evaluation (#584)
## Summary
- Replace automatic welcome/feedback credit grants with AI-powered abuse
evaluation using Gemini 3.1 Pro
- **Fail-closed**: errors/uncertainty → credits withheld, admin notified
via email with user admin link
- Clean users (no correlations) get instant grant with no AI call
latency
- Frontend verify page shows preparing account →
celebratory/denied/pending interstitial
- Comprehensive docs at `docs/AI-CREDIT-GATING.md`

## Changes
| File | What |
|------|------|
| `backend/services/credit_evaluation_service.py` | **New**: Signal
collection, Gemini prompt, response parsing, Firestore audit |
| `backend/config.py` | `CREDIT_EVAL_ENABLED`, `CREDIT_EVAL_MODEL`
config |
| `backend/services/user_service.py` | Welcome credit flow integration
(returns tuple) |
| `backend/api/routes/users.py` | Feedback credit + verify response
integration |
| `backend/services/email_service.py` | `send_credit_denied_email()`,
`send_credit_review_needed_email()` |
| `backend/models/user.py` | `credit_status` field on verify response |
| `frontend/app/auth/verify/page.tsx` | 5-state interstitial flow |
| `frontend/lib/types.ts` + `auth.ts` | Type + state updates |
| `backend/tests/test_credit_evaluation.py` | **New**: 17 tests |
| `backend/tests/test_anti_abuse.py` | Updated for new return type |
| `docs/AI-CREDIT-GATING.md` | **New**: Comprehensive documentation |
| `pyproject.toml` | `google-generativeai` dep, version bump |

## Test plan
- [x] 50 backend tests pass (17 new + 33 updated)
- [ ] Manual: test with new account signup in prod (see docs for
expected flow)
- [ ] Manual: verify admin review email received for error cases

## Rollback
Set `CREDIT_EVAL_ENABLED=false` in Cloud Run env vars → instant revert
to auto-grant.

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 21:57:46 -04:00
Andrew Beveridge ☄️andClaude Opus 4.6 c9e434037b feat: admin feedback visibility — page + email notification (#581)
<!-- @coderabbitai ignore -->

## Summary

- **Admin feedback page** at `/admin/feedback` — browse all user
feedback submissions with star ratings, text fields, search, pagination,
and aggregate stats
- **Email notification** to admin@nomadkaraoke.com whenever a user
submits new feedback (configurable via `ADMIN_NOTIFICATION_EMAIL` env
var)
- **Admin API endpoint** `GET /api/admin/feedback` with pagination,
search, test-data filtering, and average ratings

## Context

User feedback was stored in Firestore (`user_feedback` collection) but
completely invisible to the admin — the admin panel only showed a
boolean "Feedback Submitted: Yes/No" badge per user. This PR closes that
gap.

## Changes

**Backend:**
- `GET /api/admin/feedback` endpoint with pagination, search,
`exclude_test` filter, aggregate rating averages
- `send_feedback_notification()` method on `EmailService` — sends HTML
email with star ratings, all text fields, link to admin page
- Wired into `submit_user_feedback` as fire-and-forget (won't block user
response)

**Frontend:**
- New admin page with aggregate stat cards, searchable/paginated table,
visual star ratings, expandable rows for text detail
- "Feedback" sidebar nav item with MessageSquare icon
- `adminApi.listFeedback()` client method + TypeScript types

**Tests:** 14 tests covering endpoint (pagination, search, filtering,
aggregates, empty state, datetime edge cases), email (content, subject,
custom admin, failure), and caller-callee contract integration

## Test plan

- [x] `make test` — 712 tests pass (39 suites)
- [x] 14 new tests in `test_admin_feedback.py`
- [ ] Manual: verify admin page renders at `/admin/feedback` after
deploy
- [ ] Manual: submit feedback → verify email arrives at
admin@nomadkaraoke.com

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 16:18:32 -04:00
Andrew Beveridge ☄️andClaude Opus 4.6 b8239dc691 Block disposable email registration with friendly messaging (#542)
## Summary

- Wire existing `EmailValidationService` into the magic link signup flow
to reject disposable email domains (yopmail, tempmail, emailhook.site,
etc.) before creating users
- Return 422 with machine-readable `disposable_email_not_allowed` error
code for disposable domains
- Silent rejection for explicitly blocked emails/IPs (anti-enumeration)
- Frontend shows a warm amber message box encouraging real email use,
promising no spam
- Add `emailhook.site` to default disposable domains list (160+ domains
total)

## Context

Users were signing up with disposable emails like `anything@yopmail.com`
to get unlimited free welcome credits (2 per account). The
`EmailValidationService` existed with a comprehensive blocklist but was
never called during signup. This PR wires it in.

The admin UI for managing the blocklist already exists at
`/admin/rate-limits/` (Blocklists tab) with full CRUD for domains,
emails, and IPs.

## Test plan

- [x] 8 new backend integration tests (disposable rejection, blocked
email/IP, email lowercasing, domain list)
- [x] 2 new frontend tests (friendly message rendering, generic error
fallback)
- [x] All 2562 backend tests pass
- [x] All 16 frontend auth component tests pass
- [x] CodeRabbit review: no findings

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 14:33:41 -04:00
Andrew Beveridge ☄️andClaude Opus 4.6 9d8bf3a952 Improve job observability with timeline metadata and log_to_job (#508)
@coderabbitai ignore

## Summary
- Add `metadata` field to `TimelineEvent` for structured output details
(brand codes, YouTube video IDs, Dropbox paths, GDrive file IDs)
- Add `log_to_job()` helper for API endpoints to write to Firestore job
log subcollections
- Plumb `timeline_metadata` through `job_manager.transition_to_state()`
and `firestore_service.update_job_status()`
- Capture output details in completion timeline events for full
traceability
- Log edit/delete cleanup operations to both Cloud Logging and Firestore
job logs
- Snapshot previous outputs before cleanup for audit trail
- Enhance distribution logging with video_id, folder path, and file IDs

## Context
After shipping "edit completed tracks," tracing edited jobs was
difficult because timeline events lacked output details, cleanup
operations weren't logged to Firestore, and previous outputs were
overwritten with no history. These changes make the job's Firestore
document self-documenting — any agent or human can read the timeline and
understand the full lifecycle.

## Test plan
- [x] 7 new unit tests for `TimelineEvent.metadata` and `log_to_job()`
helper
- [x] All 103 backend tests pass
- [ ] Verify in production after deploy: edit a test job and confirm
timeline metadata appears in Firestore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 23:08:17 -05:00
Andrew Beveridge ☄️andClaude Opus 4.6 83e353438a feat: admin lyrics review dashboard (#492)
@coderabbitai ignore

## Summary
- Add "Edit Reviews" section to admin dashboard for reviewing user
lyrics corrections
- New backend endpoints: `GET /api/admin/edit-reviews` (list) and `GET
/api/admin/edit-reviews/{job_id}` (detail snapshot)
- Three frontend views: read-only lyrics review, edit log replay with
step-through, raw data JSON viewer
- All endpoints are strictly read-only (no state mutations, no
Firestore/GCS writes)
- 7 backend unit tests covering filtering, search, pagination, test
exclusion, error handling

## Motivation
Users are now making lyrics corrections during the review phase, and
we're collecting edit logs. This dashboard lets admins:
1. See what users experienced (original vs corrected lyrics in read-only
LyricsAnalyzer)
2. Replay user edits step-by-step to evaluate quality
3. Review raw human edits data for potential AudioShake data sharing

## Test plan
- [x] 7 backend unit tests pass (list filtering, search, pagination,
test exclusion, 404, full snapshot, missing files)
- [ ] Manual: navigate to /admin/edit-reviews, verify list loads
- [ ] Manual: click View Review, verify read-only LyricsAnalyzer renders
- [ ] Manual: click View Edit Log, verify step-through works
- [ ] Manual: click View Raw Data, verify JSON tabs render

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 00:40:09 -05:00
Andrew Beveridge ☄️andClaude Opus 4.6 9f85ae4221 feat: credit notifications and refund on job deletion (#474)
<!-- @coderabbitai ignore -->

## Summary

- **Backend**: Refund credits when jobs are deleted (if not already
complete/failed/cancelled)
- **Frontend**: Show toast notifications for credit changes — "Credit
used" on job creation, "Credit refunded" on eligible job deletion
- Refreshes user data from backend after credit-changing operations so
header counter stays accurate

## Changes

| File | What |
|------|------|
| `backend/services/job_manager.py` | Refund credit on job deletion
(non-complete jobs) |
| `frontend/components/job/JobSubmission.tsx` | "Credit used — X
remaining" toast after job creation |
| `frontend/components/job/JobActions.tsx` | "Credit refunded — X
remaining" toast after eligible deletion |
| `frontend/components/__tests__/JobActions.test.tsx` | Updated test
mock for auth module |

## Test plan

- [x] All 178 Playwright + Jest tests pass
- [ ] Create a job → see "Credit used" toast with correct remaining
count
- [ ] Delete a pending/processing job → see "Credit refunded" toast
- [ ] Delete a completed job → see "Job deleted" toast (no refund
mention)
- [ ] Header credit counter updates immediately after both operations

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 14:18:10 -05:00
Andrew Beveridge ☄️andClaude Opus 4.6 186b3dd7a9 feat: song lookup autocomplete and community version detection (#451)
## Summary

- Add track autocomplete on the title field using karaoke-decide catalog
API (MusicBrainz + Spotify data) for canonical artist/title styling
- Add community karaoke version detection via karaokenerds.com scraping
— warns users when existing high-quality versions are available on
YouTube, saving credits
- Both features are optional and non-blocking — users can still type
freely and submit without selecting from autocomplete

## Changes

### Backend
- **`backend/services/catalog_proxy_service.py`** — HTTP proxy to
karaoke-decide API with 5-min in-memory TTL cache
- **`backend/services/karaokenerds_service.py`** — Karaokenerds.com HTML
scraper (ported from kjbox) for community version detection
- **`backend/api/routes/catalog.py`** — 3 new endpoints: `GET
/api/catalog/artists`, `GET /api/catalog/tracks`, `POST
/api/catalog/community-check` with per-user rate limiting (20 req/min)
- **`backend/config.py`** — Added `karaoke_decide_api_url` setting

### Frontend
- **`frontend/components/ui/autocomplete-input.tsx`** — Reusable
autocomplete input component (shadcn Command + Popover)
- **`frontend/components/job/CommunityVersionBanner.tsx`** — Green
dismissible banner showing community karaoke versions with YouTube links
- **`frontend/components/job/steps/SongInfoStep.tsx`** — Title field now
has autocomplete; selecting a track overwrites both artist and title
with canonical names
- **`frontend/lib/api.ts`** — Added catalog API client functions

### Tests
- 9 backend route tests (artist/track search, community check, rate
limiting)
- 9 karaokenerds service tests (HTML parsing, community detection,
YouTube URL cleanup)

## How it works

1. User types artist name in the plain text field
2. User starts typing in the title field → autocomplete dropdown shows
tracks from Spotify/MusicBrainz, filtered by artist
3. Selecting a track overwrites both fields with canonical names (e.g.,
"fleetwood mac" → "Fleetwood Mac")
4. When both fields are filled, a community check fires — if existing
karaoke versions exist on YouTube, a green banner appears with links

## Test plan

- [x] 18 backend tests pass (catalog routes + karaokenerds service)
- [x] Tested locally with dev server against real karaoke-decide API
- [x] Verified autocomplete shows results, selection overwrites both
fields
- [x] Verified community banner appears for known songs (e.g., Fleetwood
Mac - Dreams)
- [x] Verified graceful degradation when API is unavailable

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 11:28:01 -05:00
Andrew Beveridge ☄️andClaude Opus 4.6 fa9bc8eeeb Add private (non-published) tracks option (#403)
@coderabbitai ignore

## Summary

- Add `is_private` flag to jobs that changes distribution behavior for
non-published tracks
- Private jobs upload to Dropbox `Tracks-NonPublished` folder with
`NOMADNP` brand prefix
- Private jobs skip YouTube and Google Drive uploads entirely
- Completion emails omit YouTube section for private jobs
- Available to all users at job creation; admin can toggle on existing
jobs
- When admin toggles completed job to private, existing outputs are
auto-deleted

## Changes

### Backend
- **Job model**: Added `is_private: bool = False` to `Job` and
`JobCreate`
- **Config**: Added `DEFAULT_PRIVATE_DROPBOX_PATH` and
`DEFAULT_PRIVATE_BRAND_PREFIX` settings
- **Distribution helper**: New `get_effective_distribution_for_job()` in
`job_defaults_service.py` applies private overrides at runtime
- **Orchestrator + native paths**: Both distribution code paths use the
helper (per "Fix Both Sides" lesson)
- **Email template**: YouTube section conditionally removed when
`is_private=True`
- **Credential pre-validation**: Skip YouTube/GDrive credential checks
for private jobs
- **API endpoints**: All 4 job creation endpoints accept `is_private`;
admin can PATCH it
- **Admin auto-delete**: Toggling completed job to private triggers
output deletion

### Frontend
- **Job creation form**: "Private (Non-Published)" checkbox on all 3
tabs (upload, URL, search)
- **Admin dashboard**: Private toggle in job configuration section
- **API interfaces**: Updated TypeScript interfaces and API methods

### Tests
- 23 unit tests covering model, distribution overrides, email template,
notification service, request models, admin toggle, and orchestrator
config
- Fixed MagicMock truthiness issue in existing tests

## Test plan

- [x] All 23 private tracks tests pass
- [x] All 1794 existing backend tests pass (1 flaky unrelated test)
- [x] CodeRabbit review: credential pre-validation issue found and fixed
- [ ] Manual: Create private job end-to-end, verify Dropbox folder
- [ ] Manual: Verify completion email has Dropbox link but no YouTube
section
- [ ] Manual: Admin toggle on completed job deletes outputs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 18:07:57 -05:00
Andrew Beveridge ☄️andClaude Sonnet 4.5 716936c8a9 Add retry logic for flacfetch download failures and audio search recovery (#382)
## Summary

Adds comprehensive retry logic to handle transient flacfetch network
failures, preventing permanent job failures from temporary service
issues. Includes state persistence for retry capability, enhanced retry
endpoint for audio search jobs, and user-facing error feedback.

## Key Changes

**Retry Logic (backend/services/flacfetch_client.py):**
- Manual exponential backoff: 9 attempts over ~6 minutes (10s → 20s →
40s → 60s capped)
- Selective retry: retries on connection errors and 5xx, immediate fail
on 4xx
- Runtime-configurable via environment variables
(FLACFETCH_RETRY_MAX_ATTEMPTS, etc.)

**State Persistence (backend/api/routes/audio_search.py):**
- Save download parameters (audio_source_type, source_name, source_id,
target_file, download_url) BEFORE attempting download
- Enables retry even if download fails completely
- Added 5 new fields to Job model (backend/models/job.py)

**Enhanced Retry Endpoint (backend/api/routes/jobs.py):**
- Added ~150 lines handling audio search retry cases
- Re-downloads using saved parameters for YouTube, RED, OPS, Spotify
sources
- Clears error state and transitions to DOWNLOADING before triggering
workers

**User Feedback (frontend/components/job/JobActions.tsx):**
- Error toast shows actual API error message on retry failure
- Success toast confirms retry started
- No more silent failures

**Configuration (backend/config.py):**
- Added retry config: max_attempts (9), min_wait (10s), max_wait (60s)

## Testing

- [x] Added 5 comprehensive retry tests in
`backend/tests/test_flacfetch_client.py`
  - Network error retry with exponential backoff
  - No retry on 4xx client errors
  - Retry on 5xx server errors
  - Max retries exhausted
  - Wrapper consistency
- [x] All backend tests pass: 103 passed
- [x] Frontend E2E tests: 142 passed (21 pre-existing failures unrelated
to changes)

## Review

- [x] CodeRabbit CLI review completed locally
- [x] Fixed unused import identified in review
- [x] Documentation updated (README.md Recent Changes,
LESSONS-LEARNED.md retry patterns)
- [x] Implementation plan archived to
docs/archive/2026-02-10-flacfetch-retry-logic.md

## Root Cause

Job 749141f8 failed because flacfetch network call failed during audio
search, and no source info was saved, making retry impossible. This PR
fixes all four related issues identified during investigation.

## Version

Bumped 0.115.5 → 0.116.0 (minor version for new feature)

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 14:21:14 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 29be83cb56 Combined review flow: lyrics + instrumental selection in one session (#351)
## Summary

Merges the two-step human review process (lyrics review → instrumental
selection) into a single unified flow. Users now review lyrics AND
select instrumental track in one session.

**Benefits:**
- Better UX (one interaction vs two)
- Reduced drop-off between steps
- Single email notification instead of two
- Simpler workflow

## Changes

### Backend
- `complete_review` endpoint now accepts optional
`instrumental_selection` in request body
- Backing vocals analysis moved from `render_video_worker` to
`screens_worker` (runs before review)
- Removed standalone instrumental selection endpoints (now part of
combined review)
- Added `CompleteReviewRequest` model with validation

### Frontend
- New `InstrumentalSelectorEmbedded` component for combined review UI
- `LyricsAnalyzer` updated to proceed to instrumental review
- `ReviewChangesModal` shows "Proceed to Instrumental Review" button
- Mock data updated in E2E tests

### Local CLI
- `gen_cli.py` refactored to use `run_combined_review()` instead of
separate `run_instrumental_review()`
- `ReviewServer` extended with instrumental options and analysis
- Deleted standalone `InstrumentalReviewServer`

### Documentation
- Updated `docs/README.md` with recent changes entry
- Updated `docs/ARCHITECTURE.md` with new pipeline diagram
- Updated `docs/API.md` with endpoint changes
- Added lesson to `docs/LESSONS-LEARNED.md`

## Test plan

- [x] Backend unit tests: 1742 passed
- [x] Frontend unit tests: 234 passed
- [x] New integration tests: 5 tests for combined flow
- [x] New backend tests: 7 tests for CompleteReviewRequest and
state_data
- [x] New frontend E2E tests: 6 tests for combined flow UI

🤖 Generated with [Claude Code](https://claude.com/claude-code)

@coderabbitai ignore

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 11:49:51 -08:00
Andrew Beveridge ☄️andClaude Opus 4.5 b68a46d174 feat(infra): Implement immutable deployment pattern for encoding worker (#333)
## Summary
- Implements robust deployment pattern for GCE encoding worker to
eliminate recurring version mismatch issues
- Fixed wheel path (`karaoke_gen-current.whl`) eliminates version
sorting bugs
- Self-updating startup script downloaded from GCS on every service
start
- Version manifest with strict CI verification (deployment fails on
mismatch)

## Problem
The encoding worker had recurring version issues because:
- `startup.sh` was baked into the Packer image (required rebuild + VM
recreation)
- Wheel selection used version sorting across 51 files
- CI only warned on version mismatch, didn't fail
- Multiple deployment vectors (CI, Pulumi, manual SSH) caused drift

## Solution
1. **Fixed wheel path**: CI uploads to `karaoke_gen-current.whl` (no
sorting needed)
2. **Self-updating startup**: `bootstrap.sh` downloads `startup.sh` from
GCS on every start
3. **Version manifest**: CI writes expected version, service verifies
before starting
4. **Strict CI verification**: Deployment fails if version doesn't match
(was just warning)

## Changes
- **New**: `infrastructure/encoding-worker/` with `bootstrap.sh`,
`startup.sh`, README
- **Modified**: CI workflow for fixed paths and strict verification
- **Modified**: Packer `provision.sh` for bootstrap pattern
- **Updated**: `LESSONS-LEARNED.md` with immutable deployment pattern
- **Updated**: `docs/README.md` and `docs/ARCHITECTURE.md`

## Test plan
- [x] Shell scripts validated (`bash -n`)
- [x] Backend tests pass (1703 passed)
- [x] CodeRabbit review passed
- [ ] After merge: Rebuild Packer image
- [ ] After merge: Recreate VM to use new image
- [ ] After merge: Verify first deployment uses new pattern

## Post-Merge Steps Required
1. Rebuild Packer image: `packer build
infrastructure/packer/encoding-worker.pkr.hcl`
2. Recreate VM: Delete encoding-worker, run `pulumi up`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

@coderabbitai ignore

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 23:56:53 -08:00
Andrew Beveridge ☄️andClaude Opus 4.5 2947dd05bc refactor: consolidate YouTube downloads into YouTubeDownloadService (#322)
## Summary

- Consolidates all YouTube download paths into a single
`YouTubeDownloadService`
- Ensures YouTube downloads always route through flacfetch remote when
configured (avoids bot detection on Cloud Run)
- Both entry points (audio search selection AND direct URL submission)
now use the same service
- Clear error messaging when remote is unavailable

## Problem

YouTube downloads were failing in cloud because:
1. Direct YouTube downloads from Cloud Run get blocked by bot detection
2. There were 3 different code paths for YouTube downloads with
inconsistent handling
3. Some paths weren't passing `gcs_path` to flacfetch, causing file
access issues

## Changes

### New Files
- `backend/services/youtube_download_service.py` - Consolidated service
for all YouTube downloads
- `backend/tests/test_youtube_download_service.py` - 20 unit tests
- `backend/tests/integration/test_youtube_flow.py` - 8 integration tests

### Modified Files
- `backend/api/routes/audio_search.py` - Route YouTube to
YouTubeDownloadService
- `backend/api/routes/file_upload.py` - Download YouTube before
triggering workers
- `backend/workers/audio_worker.py` - Mark legacy fallback as deprecated
- `backend/services/audio_search_service.py` - Fix source_id extraction
- `docs/ARCHITECTURE.md` - Add audio source download flow diagram

## Test Plan

- [x] 20 unit tests for YouTubeDownloadService
- [x] 8 integration tests for YouTube flow routing
- [x] Updated existing tests to mock new service
- [x] All 1630+ backend tests pass

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-20 19:54:07 -08:00
Andrew Beveridge ☄️andClaude Opus 4.5 83c54803ef feat: disable auto-correction during lyrics transcription (#321)
## Summary

- Disabled all auto-correction during lyrics transcription (both agentic
AI via Gemini and heuristic rule-based handlers)
- Raw transcription now goes directly to human review without automatic
fixes
- This change was made because auto-correction was creating more work
for reviewers by introducing errors that needed manual correction

## Changes

- Added `SKIP_CORRECTION` env var (default: `true`) in
`backend/config.py`
- Updated `LyricsProcessor` to check skip_correction and set
`run_correction=False`
- Added `_create_uncorrected_result()` method to controller for review
UI compatibility
- Updated docs to reflect disabled status

## How to Re-enable

Set `SKIP_CORRECTION=false` environment variable to re-enable
auto-correction if quality improves in the future.

## Test plan

- [x] All 1605 tests pass
- [x] Test review: Changes are configuration-driven, existing code paths
remain tested
- [x] CodeRabbit review: Passed clean

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-20 19:48:01 -08:00
Andrew Beveridge ☄️andClaude Opus 4.5 170acfd508 fix: pass gcs_path for YouTube downloads via remote flacfetch (#320)
## Summary

- Fix YouTube audio downloads failing when remote flacfetch is enabled
- Add dynamic MIME type detection for audio uploads (fixes hardcoded
`audio/flac` for `.webm`/`.opus` files)
- Add documentation example to LESSONS-LEARNED.md

## Problem

When remote flacfetch is enabled and a user selects a YouTube audio
source, the backend was calling `download_by_id()` without the
`gcs_path` parameter. This caused flacfetch to download the file locally
on the VM instead of uploading to GCS. The backend then tried to open
the VM's local path (`/var/lib/transmission-daemon/downloads/...`) which
doesn't exist on Cloud Run, causing the job to fail.

**Production impact:** Job `4c266875` failed with "No such file or
directory"

## Root Cause

This is the same class of bug that was fixed in December 2025 for
RED/OPS torrent sources. However, that fix only addressed the torrent
code path, leaving YouTube downloads broken when remote is enabled.

## Changes

### `backend/api/routes/audio_search.py`
- Add new `elif` branch to handle remote YouTube downloads with
`gcs_path` parameter
- Use `mimetypes.guess_type()` to detect correct content type instead of
hardcoded `audio/flac`

### `backend/tests/test_audio_search.py`
- Add `test_youtube_remote_download_passes_gcs_path` to verify the fix
- Add `test_youtube_local_download_when_remote_disabled` for coverage

### `docs/LESSONS-LEARNED.md`
- Add specific example of the `gcs_path` bug to the "Fix Both Sides of
Dual Code Paths" lesson

## Test plan

- [x] All 50 audio_search tests pass
- [x] New test verifies `gcs_path` is passed to `download_by_id()` for
YouTube sources
- [x] New test verifies local download path when remote is disabled
- [ ] Production verification: retry previously failed job after
deployment

<!-- @coderabbitai ignore -->

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-20 18:33:42 -08:00
Andrew Beveridge ☄️andClaude Opus 4.5 404c6e6133 fix: make lyrics_transcriber a subpackage of karaoke_gen (#299)
## Summary
Fixes the deployment failure from PR #297 by making lyrics_transcriber a
proper subpackage of karaoke_gen instead of a separate vendored library.

### Changes
- Move `libs/lyrics_transcriber/` to `karaoke_gen/lyrics_transcriber/`
- Update all imports from `from lyrics_transcriber.X` to `from
karaoke_gen.lyrics_transcriber.X`
- Simplify Dockerfile (no separate `libs/` COPY needed)
- Update pyproject.toml to remove separate package entry
- Update test mocks and logger names

### Why this is better
- Cleaner project structure - one package containing everything
- Simpler Dockerfile - just COPY karaoke_gen/
- No confusion about vendored vs internal code

## Test plan
- [x] `poetry install` succeeds
- [x] All imports work correctly
- [x] All 2614 tests pass
- [ ] Backend deployment succeeds

@coderabbitai ignore

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-19 16:53:34 +00:00
Andrew Beveridge ☄️andClaude Opus 4.5 a52ea006cb refactor: deprecate python-lyrics-transcriber, consolidate into karaoke-gen (#297)
## Summary
- Moved lyrics_transcriber from lyrics_transcriber_temp/ to libs/
- Removed obsolete standalone package files
- Updated pyproject.toml paths
- Archived python-lyrics-transcriber repo

## Test plan
- [x] poetry install succeeds
- [x] Backend tests pass (2614)
- [x] python-lyrics-transcriber archived

@coderabbitai ignore

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-19 08:13:52 -08:00
Andrew Beveridge ☄️andClaude 7cfac407e3 feat(admin): Enhanced admin job detail page with files, edit, and reset (#261)
## Summary
- Add comprehensive job management features to admin job detail view
- Enable file downloads with signed GCS URLs grouped by category
- Add inline editing of job fields (artist, title, theme, etc.)
- Add job reset actions to re-process from workflow checkpoints

## Changes

### Backend (39 new tests)
- `GET /api/admin/jobs/{job_id}/files` - Get all job files with signed
URLs (2-hour expiry)
- `PATCH /api/admin/jobs/{job_id}` - Update editable job fields with
whitelist validation
- `POST /api/admin/jobs/{job_id}/reset` - Reset job to specific workflow
checkpoint

### Frontend (13 E2E tests)
- **Files accordion** - Download job files grouped by category (input,
stems, lyrics, outputs)
- **Configuration section** - Inline editing for artist, title,
theme_id, brand_prefix, customer_email
- **Admin Actions section** - Reset buttons for pending, audio
selection, review, instrumental selection
- **Enhanced timeline** - Color-coded stages (green=complete,
blue=processing, orange=waiting, red=failed)

### Documentation
- Updated `docs/API.md` with new endpoint documentation
- Updated `docs/README.md` with feature summary
- Added lesson learned about AuthResult tuple unpacking gotcha

## Test Plan
- [x] Backend tests pass (39 new tests for files, update, reset
endpoints)
- [x] Frontend E2E tests pass (13 tests for admin job detail page)
- [x] Frontend unit tests pass (150 tests)
- [x] Frontend builds successfully
- [x] CodeRabbit review passed (2 cycles, all issues fixed)

## Screenshots
See updated admin job detail page at `/admin/jobs?id={job_id}`

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-09 21:11:58 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 42c304dcdf feat(multitenancy): Implement white-label B2B portal infrastructure (#241)
## Summary

Adds multi-tenant white-label portal infrastructure for B2B customers.
Vocal Star is the first tenant at `vocalstar.nomadkaraoke.com`.

## Changes

### Backend
- **Tenant Models** (`backend/models/tenant.py`): Pydantic models for
TenantConfig with branding, features, defaults, and auth settings
- **Tenant Service** (`backend/services/tenant_service.py`): GCS-backed
config loading with 5-min cache, signed URL generation for logos
- **Tenant API** (`backend/api/routes/tenant.py`): `GET
/api/tenant/config` and `GET /api/tenant/config/{tenant_id}` endpoints
- **Tenant Middleware** (`backend/middleware/tenant.py`): Extracts
tenant from X-Tenant-ID header, query param (dev only), or Host
subdomain
- **Auth Updates**: Email domain validation for tenant-restricted auth,
sender email override
- **Feature Enforcement**: Audio search endpoint checks
`features.audio_search`, returns 403 if disabled
- **Job Scoping**: Added `tenant_id` field to JobCreate

### Frontend
- **Tenant Store** (`frontend/lib/tenant.ts`): Zustand store with tenant
state, `useTenant()` hook, CSS variable application
- **TenantProvider** (`frontend/components/tenant-provider.tsx`):
Hydration-safe initialization via useEffect
- **TenantLogo** (`frontend/components/tenant-logo.tsx`): Dynamic logo
with fallback to Nomad Karaoke
- **Feature-Flagged UI**: Tabs conditionally rendered based on tenant
features, active tab sync
- **Locked Themes**: `defaults.locked_theme` hides theme selector and
enforces specific theme
- **Output Links**: Filtered based on distribution features

### Security
- Query param tenant detection disabled in production (prevents
spoofing)
- Strict subdomain patterns: only `{tenant}.nomadkaraoke.com` or
`{tenant}.gen.nomadkaraoke.com`
- PII masking in logs (`an***@vo***.com`)
- `is_active` check prevents inactive tenants from being used

### Documentation
- ARCHITECTURE.md: New Multitenancy section
- API.md: Tenant Config endpoints
- LESSONS-LEARNED.md: 3 new lessons on multitenancy patterns
- Archive: Comprehensive implementation document

### Setup
- `scripts/setup-vocalstar-tenant.py`: Uploads theme assets, creates
tenant/theme configs in GCS

## Vocal Star Configuration

- **Subdomain**: `vocalstar.nomadkaraoke.com`
- **Features**: File upload only (no audio search, no YouTube URL)
- **Distribution**: Download only (no YouTube/Dropbox/GDrive)
- **Theme**: Locked to "vocalstar" (yellow/blue)
- **Auth**: Restricted to `@vocal-star.com` and `@vocalstarmusic.com`
emails

## Testing

- [x] Frontend TypeScript build passes
- [x] CodeRabbit CLI review completed (3 cycles, all issues fixed)
- [x] Manual testing with `?tenant=vocalstar` query param

## Next Steps (post-merge)

1. Configure DNS: CNAME for `vocalstar.nomadkaraoke.com`
2. Update Cloudflare Pages for subdomain
3. Run setup script to upload Vocal Star assets to GCS
4. Test at `https://vocalstar.nomadkaraoke.com`

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 16:39:56 -05:00
Andrew Beveridge ☄️andClaude 560c3e4904 feat: Add email notification system with admin controls (#220)
## Summary

- Add automated email notification system for job completion and action
reminders
- Admin UI buttons to copy completion message or send email directly
- Comprehensive test suite (81 new tests)
- **Email subjects include release ID and song info** for easy
identification

## Changes

### Backend
- **Email Service** (`email_service.py`): Extended with SendGrid CC
support, `send_job_completion()` and `send_action_reminder()` methods
- **Job Notification Service** (`job_notification_service.py`): New
service orchestrating completion emails and action reminder scheduling
- **Template Service** (`template_service.py`): GCS-backed HTML email
templates with fallback defaults
- **Worker Service** (`worker_service.py`): Cloud Tasks integration for
delayed idle reminders (5-min delay)
- **Job Manager** (`job_manager.py`): State transition triggers for
automated notifications
- **Admin Routes** (`admin.py`): New endpoints `GET
/api/admin/jobs/{id}/completion-message` and `POST
/api/admin/jobs/{id}/send-completion-email`
- **Internal Routes** (`internal.py`): Idle reminder check endpoint for
Cloud Tasks

### Email Subject Format
Email subjects now include brand code and song info for easy
identification:
- With brand code: `NOMAD-1178: Seether - Tonight (Your karaoke video is
ready!)`
- Without brand code: `Seether - Tonight (Your karaoke video is ready!)`
- Without song info: `Your karaoke video is ready!`

### Frontend
- **OutputLinks Component**: Admin-only "Copy Message" and "Send Email"
buttons with dialog UI
- **E2E Helpers** (`email-testing.ts`): `waitForCompletionEmail()` and
`isCompletionEmail()` methods for testmail.app integration
- **Happy Path Test**: Step 9.5 verifies completion email arrives after
job completion

### Documentation
- Updated API.md with new endpoints
- Updated ARCHITECTURE.md with notification system architecture
- Updated README.md Recent Changes

## Testing

- [x] 81 new unit tests covering:
- `test_email_service.py` (27 tests) - providers, CC support, job
completion/reminder methods
- `test_job_notification_service.py` (21 tests) - URL building, message
generation
- `test_job_manager_notifications.py` (15 tests) - state transition
triggers
  - `test_admin_email_endpoints.py` (12 tests) - admin API endpoints
- `test_template_service.py` (6 tests) - GCS template loading, fallbacks
- [x] E2E test extended to verify completion email delivery
- [x] All backend tests pass (`make test`)
- [x] TypeScript compiles

## Feature Flags

- `ENABLE_AUTO_EMAILS` (default: `false`) - Controls automatic email
sending on job state changes

## Review

- [x] Local CodeRabbit review completed (3 cycles)
- [x] Review feedback addressed:
  - Fixed potential None state_data in internal.py
  - Masked user emails in logs (PII protection)
  - Fixed React setTimeout memory leak
  - Changed placeholder URL default to empty
  - Added CC email deduplication

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-06 11:53:11 -05:00
Andrew Beveridge ☄️andClaude ae85ec0a09 feat: Move worker logs to Firestore subcollection to avoid 1MB limit (#195)
## Summary

Move `worker_logs` from embedded array in job documents to a Firestore
subcollection (`jobs/{job_id}/logs`) to avoid the 1MB document size
limit. This fixes job failures when logs exceed the limit during
long-running operations like video encoding.

**Problem:** Job `501258e1` failed when logs reached 1.26MB (2,547 log
entries), representing 98.6% of the maximum document size.

## Changes

### New Files
- `backend/models/worker_log.py` - WorkerLogEntry dataclass with factory
methods, serialization (to_dict, to_legacy_dict, from_dict)
- `backend/tests/test_worker_log_subcollection.py` - 24 unit tests
- `backend/tests/emulator/test_worker_logs_subcollection.py` - 11
integration tests using Firestore emulator

### Modified Files
- `backend/services/firestore_service.py` - Added subcollection methods
(append, get, count, delete)
- `backend/services/job_manager.py` - Use subcollection with fallback to
legacy embedded arrays
- `backend/api/routes/jobs.py` - Updated logs endpoint to use new
methods
- `backend/config.py` - Added `USE_LOG_SUBCOLLECTION` feature flag
(default: true)
- `infrastructure/__main__.py` - Added TTL policy (30 days) and
composite index for logs subcollection
- `pyproject.toml` - Version bump 0.89.6 → 0.90.0

### Documentation
- `docs/README.md` - Added to Recent Changes
- `docs/ARCHITECTURE.md` - Added logs subcollection to Firestore
Collections table
- `docs/LESSONS-LEARNED.md` - Added "Firestore Document 1MB Limit with
Embedded Arrays" section

## Testing

- [x] 24 unit tests pass
(`backend/tests/test_worker_log_subcollection.py`)
- [x] 11 emulator integration tests pass
(`backend/tests/emulator/test_worker_logs_subcollection.py`)
- [x] All 68 existing emulator tests pass

## Review

- [x] CodeRabbit CLI review completed locally
- [x] Review feedback addressed (timestamp formatting fix)

## Rollback Plan

Set `USE_LOG_SUBCOLLECTION=false` environment variable and redeploy. New
logs will go back to embedded array; subcollection logs will remain but
won't be read.

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-04 20:14:25 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 16dc451bdf docs: Add API auth guide and worker logs rearchitecture plan (#192)
## Summary
- Add API authentication documentation to CLAUDE.md for future agents
- Add comprehensive plan for rearchitecting worker logs storage

## Changes

### CLAUDE.md
- New "API Authentication (for Agents)" section covering:
  - Internal worker endpoints with X-Admin-Token
  - Firestore direct access
  - GCE encoding worker SSH commands
  - Common pitfalls to avoid

### .claude/plans/worker-logs-rearchitecture.plan.md
- Plan to move worker_logs from embedded array to Firestore
subcollection
- Fixes 1MB document size limit issue (job 501258e1 hit 1.26 MB)
- Includes TTL policy for automatic cleanup (30 days)

## Testing
- [x] Documentation review
- No code changes

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 22:37:17 +00:00
Andrew Beveridge ☄️ c1588992ce refactor: Unify video worker pipeline with new orchestrator (#182)
Resolves GCE vs KaraokeFinalise code path divergence that caused features like YouTube upload, Discord notifications, and CDG/TXT packaging to be missing from the GCE encoding path.

- New services: youtube_upload_service, discord_service, packaging_service, local_encoding_service, encoding_interface
- New VideoWorkerOrchestrator coordinates all stages (packaging, encoding, distribution, notifications)
- Works with any encoding backend (GCE or local)
- 139 new tests, 976 total tests passing
- Feature flag USE_NEW_ORCHESTRATOR (default: true) with rollback capability
2026-01-04 00:25:23 -05:00
Andrew Beveridge ☄️andClaude dfb2ebc184 feat(frontend): migrate from MailSlurp to testmail.app for E2E testing (#159)
## Summary

- Replace MailSlurp email testing service with testmail.app for E2E
magic link authentication tests
- Simpler API with namespace-based email addresses
(`{namespace}.{tag}@inbox.testmail.app`)
- No new dependencies - uses native `fetch` instead of mailslurp-client

## Changes

- **`frontend/e2e/helpers/email-testing.ts`** - Complete rewrite using
testmail.app JSON API
  - Uses `livequery=true` to wait for emails (no polling needed)
  - Generates unique tags per test run to avoid collisions
  - Same `EmailHelper` interface maintained for minimal test changes
- **`frontend/e2e/production/full-user-journey.spec.ts`** - Update env
var references and messages
- **`frontend/playwright.record.config.ts`** - Update env var comment
- **`frontend/package.json`** - Remove `mailslurp-client` dependency,
bump version to 0.79.0
- **`frontend/.env.local.example`** - New file documenting required env
vars for local testing
- **`docs/DEVELOPMENT.md`** - Add testmail.app env vars to frontend
section

## Environment Variables

| Old | New |
|-----|-----|
| `MAILSLURP_API_KEY` | `TESTMAIL_API_KEY` |
| (none) | `TESTMAIL_NAMESPACE` |

**Note:** GitHub Actions org secrets already configured with new env
vars.

## Testing

- [x] Backend tests pass (`make test` - 57 tests)
- [x] Frontend unit tests pass (139 tests)
- [x] Frontend E2E tests pass (64 tests)
- [x] TypeScript compilation clean for changed files

## Review

- [x] Local CodeRabbit CLI review completed
- [x] Review feedback addressed (removed exposed credentials from plan
file)

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-02 22:24:19 -05:00