mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
## Summary Alert hygiene, part of keeping Discord #alerts to actionable signal only. ERROR is page-worthy: the error monitor turns every new ERROR pattern into an #alerts message. This PR moves expected conditions down to INFO/WARNING and fixes the one real bug among them, which was a Payment Link checkout crashing the Stripe webhook. It covers the BACKLOG items "Payment Link checkouts crash the Stripe webhook" and part of "Alert-hygiene bundle". ## Changes 1. **Stripe webhook, non-gen checkouts** (`users.py`, `stripe_service.py`, `stripe_admin_service.py`) - A session that gen didn't create (a Stripe Payment Link: `payment_link` is set, there's no gen metadata, and `customer_email=None`) used to log ERROR "No user email in checkout session". `store_payment` then crashed on `None.lower()`, so the payment never reached `stripe_payments` or revenue. - New `StripeService.is_gen_checkout()`. A non-gen checkout now logs INFO "non-gen checkout (payment link ...)" and grants no credits. It is marked processed for idempotency and is still stored, with `customer_details.email`, `order_type="payment_link"`, `payment_link_id` and the line-item description. - `store_payment` now handles None for metadata, customer_details, total_details and payment-method fields. - **Not backfilled:** session `cs_live_a1ENsua3w6mDH04PPc…` (2026-10-13, a $36 tip, no customer action needed). It could be backfilled through an admin path that calls `store_payment` on the expanded session. GCP writes from agents are restricted, so this PR doesn't do it. 2. **Lyrics providers** (`base_lyrics_provider._log_request_exception`): a 404 means "no lyrics for this track" and now logs INFO. 429 moves to WARNING alongside 5xx, connection errors and timeouts. Auth errors and other 4xx stay ERROR. After 5 404s from one provider in a process, it logs a single WARNING saying the provider may be dead (TODO points at the Musixmatch backlog item). 3. **research_audio** (`audio_search.py`): no more `AWAITING_AUDIO_SELECTION -> SEARCHING_AUDIO -> AWAITING_AUDIO_SELECTION` transitions, which were invalid and logged 2 ERRORs per re-search. The job stays in AWAITING_AUDIO_SELECTION with no status writes, so a crashed request can't strand it. After the search the job is re-read. If a selection landed in the meantime, the fresh results are discarded with a 409 instead of replacing the results that selection indexed into. 4. **audio-download-job on a deleted job** (`audio_download_worker.py`): if `get_job()` returns None at the start or after the download, or a failure happens and a fresh read confirms the doc is gone, the worker logs INFO and returns success, so it exits 0. That means no `fail_job`, no further writes, no worker triggers and no Cloud Run retry. A GCS NotFound on a job that still exists stays an ERROR. Seen with job 6ffe88ad on 2026-10-14. 5. **Loop watchdog**: "Failed to record loop stall (non-fatal)" now logs WARNING with exc_info. Only that line changed, to keep the conflict with feat/alert-noise-monitor trivial. 6. **Job-list queries** (`firestore_service.list_jobs`, `list_jobs_summary`, `list_jobs_by_batch_id`): `query.stream()` now gets `retry=` and `timeout=` of 30s. The client default is 300s for both, which caused the "Error listing jobs summary: Timeout of 300.0s exceeded" errors. Version 0.278.9 (0.278.8 is taken by the still-open #1186). ## Testing - New/updated unit tests: - `test_stripe_webhook_non_gen_checkout.py`: webhook stores the payment, logs INFO, logs nothing at ERROR, and marks the session processed. - `test_stripe_admin_service.py`: Payment Link session and null fields. - `tests/unit/test_base_lyrics_provider.py`: log levels for 404, 429 and 401, plus the dead-provider WARNING. - `test_audio_search_research_attach.py`: no status writes, and a selection made mid-search gets a 409. - `test_audio_download_worker.py`: job deleted at start, mid-download, or with NotFound on write. - `test_loop_watchdog.py`: recorder failure logs WARNING. - `test_firestore_list_query_timeout.py`: bounded retry/timeout on all three list queries. - `pytest backend/tests --ignore=backend/tests/emulator`: 6434 passed. `pytest tests/unit`: 2108 passed. - Local CodeRabbit review: cycle 1 raised 2 findings (non-gen session idempotency, re-search race), both fixed. Cycle 2 was clean. @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_019pmG2XAgtUguEJWLFpkKHN --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Backend Development Setup
One-Time Setup
1. Create Virtual Environment
cd /Users/andrew/Projects/karaoke-gen
python3 -m venv backend/venv
2. Activate Virtual Environment
source backend/venv/bin/activate
3. Install Dependencies
IMPORTANT: Use Python 3.12 (not 3.13/3.14 - pydantic has compatibility issues)
# Backend dependencies
pip install -r backend/requirements.txt
# Install karaoke_gen in development mode
pip install -e .
# Install lyrics_transcriber submodule
pip install -e lyrics_transcriber_local/
# Install Google API client (for YouTube upload)
pip install google-api-python-client google-auth-httplib2 google-auth-oauthlib
4. Add Shell Alias (Optional)
Add to ~/.zshrc:
alias backend='cd /Users/andrew/Projects/karaoke-gen && source backend/venv/bin/activate'
Then you can just run:
backend
# Now you're in the project directory with venv activated!
Daily Workflow
Before Making Changes
# Activate venv
source backend/venv/bin/activate
# Or if you set up the alias:
backend
After Making Changes
# 1. Run full validation (catches import errors!)
python3 backend/validate.py
# 2. If validation passes, deploy
./scripts/deploy.sh
Validation Options
Full Validation (Recommended - Requires venv)
source backend/venv/bin/activate
python3 backend/validate.py
Catches:
- ✅ Syntax errors
- ✅ Import errors (like the processing_service issue!)
- ✅ Configuration issues
- ✅ FastAPI app creation problems
- ✅ Missing dependencies
This would have caught the ModuleNotFoundError!
Quick Check (No venv needed)
./backend/quick-check.sh
Catches:
- ✅ Syntax errors
- ✅ Missing files
- ✅ Some obvious import issues
Good for quick checks, but won't catch all import errors
Example: Complete Development Session
# 1. Activate environment
backend # or: source backend/venv/bin/activate
# 2. Make your changes
vim backend/api/routes/jobs.py
# 3. Validate (catches import errors!)
python3 backend/validate.py
# ✅ All validations passed!
# 4. Deploy
./scripts/deploy.sh
# 5. Test
export BACKEND_URL="https://karaoke-backend-ipzqd2k4yq-uc.a.run.app"
export AUTH_TOKEN=$(gcloud auth print-identity-token)
curl -H "Authorization: Bearer $AUTH_TOKEN" $BACKEND_URL/api/health
What About karaoke_gen Imports?
The backend imports from karaoke_gen (the CLI package):
from karaoke_gen.karaoke_gen import KaraokePrep
from karaoke_gen.audio_processor import AudioProcessor
Options:
Option 1: Install in development mode (Recommended)
source backend/venv/bin/activate
pip install -e .
This makes karaoke_gen available in the venv without copying files.
Option 2: Use system Python
Your system Python already has karaoke_gen installed, so validation will work even if the venv doesn't have it (Python will fall back to system packages).
Running the Backend Locally
# Activate venv
source backend/venv/bin/activate
# Set required environment variables
export GOOGLE_CLOUD_PROJECT="nomadkaraoke"
export GCS_BUCKET_NAME="karaoke-gen-storage-nomadkaraoke"
# Optional: Point to local credentials
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account-key.json"
# Run the server
cd backend
uvicorn main:app --reload --port 8080
Visit: http://localhost:8080/api/health
Troubleshooting
"No module named 'fastapi'"
Fix:
source backend/venv/bin/activate
pip install -r backend/requirements.txt
"No module named 'karaoke_gen'"
Fix:
source backend/venv/bin/activate
cd /Users/andrew/Projects/karaoke-gen
pip install -e .
Validation passes locally but fails in Cloud Run
Check:
- Is
requirements.txtup to date? - Did you add a new dependency without adding it to
requirements.txt? - Run:
pip freeze | grep <package-name>
Updating Dependencies
When you add new packages:
# 1. Install in venv
source backend/venv/bin/activate
pip install new-package
# 2. Update requirements.txt
pip freeze > backend/requirements.txt
# 3. Or manually add just the new package:
echo "new-package==1.2.3" >> backend/requirements.txt
# 4. Validate
python3 backend/validate.py
# 5. Deploy
./scripts/deploy.sh
Summary
Best practice workflow:
- One-time setup: Create venv and install dependencies
- Daily: Activate venv when working on backend
- Before deploy: Run
python3 backend/validate.py(catches import errors!) - Deploy: Use
./scripts/deploy.sh
The import error would have been caught if you ran validation in a venv! ✅