2 Commits
Author SHA1 Message Date
Andrew Beveridge ☄️andClaude Fable 5 70ea964ca9 feat(dev): full local backend for external contributors (no GCP creds needed) (#946)
## Summary

Motivated by external contributor Martín (netux) asking "is there an
easy way to run the full backend without GCS?" — he couldn't test PR
#895's backend changes locally. This PR makes the whole backend runnable
locally against emulators with **zero GCP credentials and zero paid API
keys**.

### What was broken for contributors
- Workers migrated to Cloud Run Jobs for prod robustness
(audio-download, audio-separation, lyrics-transcription, bulk-search)
silently bypassed the direct-HTTP dev mode → local jobs stalled at
PENDING forever
- Signed URL generation called `google.auth.default()` → crashed with no
credentials (review audio, downloads, theme previews, upload URLs)
- Magic-link login 503'd locally (console email provider counted as "not
configured")
- Fresh emulator had no default theme → every job create returned 422
- `docs/DEVELOPMENT.md` told people to `cp .env.example .env` but the
file didn't exist — and nothing loaded `.env` anyway
- `run-backend-local.sh` defaulted to the production GCP project

### Changes
- **worker_service**: CRJ-based workers run as local subprocesses
(`python -m backend.workers.X --job-id Y`, exact CRJ parity) when
`ENABLE_CLOUD_TASKS=false` **and** not `is_production()`
(regression-tested both ways)
- **storage_service**: signed URLs → plain fake-gcs-server URLs when
`STORAGE_EMULATOR_HOST` set (GET via JSON API download path; PUT via
unvalidated-signature path, enabled by `-public-host` on the emulator)
- **users route**: magic-link 503 only in production; dev prints the
link to the console
- **file_upload bugfix**: a theme with style params but no asset files
never got `style_params_gcs_path` written (empty-dict falsiness in `elif
theme_style_assets:`) — surfaced by the minimal seed theme, latent in
prod
- **scripts/seed-local-data.py** (new): creates bucket + default theme;
wired into `run-backend-local.sh --with-emulators`, which now uses
`test-project`
- **config**: `load_dotenv()` so `.env` actually works; new root
`.env.example`
- **CONTRIBUTING.md** (new): no-credentials quickstart; README +
DEVELOPMENT.md link to it

### Verified end-to-end locally (no GCP credentials)
Upload job → local audio separation (real models) → transcription →
screens → `awaiting_review`; review endpoints all working including
`/api/review/{job}/audio/vocals` (the PR #895 endpoint) and 302 →
emulator audio URLs. Magic-link login round-trip via console link.

### Tests
- 224 tests across affected files pass; full backend unit suite: 3969
passed
- `backend/tests/test_emulator_integration.py` has 8 pre-existing
failures that reproduce identically on unmodified `main` locally (broken
theme-service mock in that file) — unrelated to this PR
- New tests: dev-mode subprocess dispatch (audio + bulk-search),
prod-mode never-Popen guard, emulator URL generation (GET + PUT)

@coderabbitai ignore

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 06:48:00 +00:00
Andrew Beveridge ☄️ 321750a8e7 security: add gitleaks pre-commit hook and CI scan (#724)
## Summary

Adds org-wide gitleaks secret-scanning in two layers:

1. **Local pre-commit hook** (`.githooks/pre-commit`) — blocks commits
containing potential secrets before they're even created. Enable with
`git config core.hooksPath .githooks`.
2. **CI backstop** (`.github/workflows/security.yml`) — calls the
reusable gitleaks workflow in `nomadkaraoke/.github`, scans every PR and
push to `main`.

Config: `.gitleaks.toml` extends the default gitleaks rules with a
dedicated Discord-webhook rule and an allowlist for known false-positive
paths (lockfiles, vendor/, node_modules/, Flutter `.dart_tool/`,
WordPress `wp-content/`, Python test fixtures, Firebase/GCP client API
keys).

## Why

Following the 2026-04-17 Discord webhook URL leak — a real webhook URL
sat in `nomadkaraoke/karaoke-gen`'s archived setup docs for ~9 months,
was scraped, and was used to push malware into the `#releases` channel.

Existing defenses: GitHub push protection (enabled on all public repos).
This PR adds the two missing layers (local pre-commit, CI).

## Test plan

- [ ] After merge, create a test commit with a fake Discord webhook URL
— the CI job should fail.
- [ ] Run `brew install gitleaks && git config core.hooksPath .githooks`
locally, then attempt the same commit — the pre-commit hook should block
it.

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-04-19 21:14:58 -04:00