Files
Andrew Beveridge ☄️andClaude Opus 5.5 c4c4f9265c feat(bulk): album batches search the album once; one flacfetch search in flight per batch (#1214)
## Why
A real Bulk "By album" batch (Chase & Status, *No More Idols*, 12
tracks):
- It took about 5 minutes to reach audio selection.
- 4 of the 12 searches timed out.
- All 12 were parked for manual choice, each showing the same album
torrents.

Causes:
- The album was dropped at submit, so each track was searched
separately.
- 4 parallel searches ran against flacfetch's single search worker, and
the queued ones ran out the 95s budget.
- The auto-pick bar was 50 seeders (lowered to 10 in #1211).
- A title-only file search misses the album for common titles ("Time").

## What
- **Submit carries the album:** `POST /api/bulk/submit` takes `album`
{artist, title, release_group_mbid, release_mbid} plus each song's
`position`. These are stamped as `state_data.bulk_album` /
`bulk_track_position`. The frontend album mode sends them, using the
release-group title.
- **One album search per album:** the worker makes one flacfetch `POST
/search/album` call (nomadkaraoke/flacfetch#56).
  - Each track's file is resolved inside the album's torrents.
- The whole album comes from **one torrent**: the first BEST CHOICE
(lossless, non-vinyl, 10+ seeders).
- A track is auto-selected when that torrent's file names it and the
artist matches.
- A track the batch torrent doesn't cover gets the normal per-track pick
among the album torrents that do.
  - Parked jobs show the album torrents in the selection modal.
- **Fallbacks are one search at a time** (`SEARCH_CONCURRENCY = 1`):
uncovered tracks, text batches, and failed or unavailable album searches
(404 from an older flacfetch). A batch search deadline (3000s) parks any
leftovers before the 3600s job timeout could strand them.
- **Auto-pick checks the artist** on the backend too: a Unicode-aware
port of `isArtistMatch`, used by bulk and KaraokeHunt. A 10+-seeder
cover or tribute torrent is no longer auto-picked.
- Fake album search for the local stack; docs in API.md and
LESSONS-LEARNED.

The money path is unchanged. Album results have no duration, so nothing
is charged at select time. The real duration charge or refund still
happens after download (`reconcile_and_maybe_pause`).

## Testing
- Backend: 6677 passed. The only failures are the 8
`test_emulator_integration` tests, which hit a stale local Firestore
emulator from another session; they passed in CI on #1211.
- Frontend: Jest 1765/1765.
- New tests cover:
- album worker path: one search, all auto-selected, uncovered →
fallback, search failure → fallback, low seeders → park, auto disabled
  - `process_batch` routing, with at most one search in flight
  - the deadline
- album helpers, artist match (incl. CJK and &), client `search_album`
(incl. 404 and 502), submit stamping, the fake album response, and the
frontend submit payload
- Code review: CodeRabbit CLI unavailable (invalid token / org
mismatch), so a local Superpowers review ran instead. Its fixes are in
the second commit.

## Rollout
flacfetch v0.34.0 (#56) deploys to flacup first. Then: one album search,
12 auto-selected jobs, one torrent; this is verified in prod after
merge.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_019MixBuvzxB1c5AYc3mbDyW

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
* Album-mode bulk submissions now include album details and track
positions, enabling one album-wide search for matching tracks.
* Search results are checked against both track title and artist before
automatic selection, with tolerance for minor name variations.
* **Improvements**
* Tracks not covered by an album result, or affected by an unavailable
or failed album search, fall back to individual searches.
* Searches still awaiting selection after the batch deadline are parked
for manual review.
* **Documentation**
* Updated bulk submission guidance to describe album searches and
fallback behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 17:06:57 -04:00

225 lines
8.3 KiB
Python

"""Tests for POST /api/bulk/submit — upfront credit gate + batch job creation."""
from unittest.mock import MagicMock
import pytest
from fastapi.testclient import TestClient
from backend.main import app
from backend.api.dependencies import require_auth, AuthResult, UserType
from backend.exceptions import InsufficientCreditsError
def _auth(is_admin=False, email="t@example.com"):
def _dep():
return AuthResult(
is_valid=True,
user_type=UserType.ADMIN if is_admin else UserType.STRIPE,
remaining_uses=999,
message="ok",
user_email=email,
is_admin=is_admin,
)
return _dep
@pytest.fixture(autouse=True)
def _patch_deps(monkeypatch):
"""Patch the heavy collaborators used by bulk_submit."""
import backend.api.routes.bulk as bulk
# Theme + settings + cdg/txt
theme = MagicMock()
theme.get_default_theme_id.return_value = "nomad"
monkeypatch.setattr("backend.services.theme_service.get_theme_service", lambda: theme)
monkeypatch.setattr("backend.services.job_defaults_service.resolve_cdg_txt_defaults", lambda *a, **k: (False, False))
settings = MagicMock(
default_brand_prefix="NOMAD", default_enable_youtube_upload=False,
default_youtube_description="", default_discord_webhook_url=None,
default_dropbox_path=None, default_gdrive_folder_id=None,
)
monkeypatch.setattr("backend.config.get_settings", lambda: settings)
# Worker trigger
worker = MagicMock()
async def _trigger(batch_id):
return True
worker.trigger_bulk_search_worker = _trigger
monkeypatch.setattr("backend.services.worker_service.get_worker_service", lambda: worker)
yield
def _patch_user_service(monkeypatch, credits):
us = MagicMock()
us.check_credits.return_value = credits
monkeypatch.setattr("backend.services.user_service.get_user_service", lambda: us)
return us
def _patch_job_manager(monkeypatch, fail_after=None):
"""JobManager whose create_job returns sequential ids, optionally failing after N."""
created = {"count": 0}
jm = MagicMock()
def _create_job(job_create, is_admin=False):
if fail_after is not None and created["count"] >= fail_after:
raise InsufficientCreditsError(message="out", credits_available=0, credits_required=1)
created["count"] += 1
job = MagicMock()
job.job_id = f"job{created['count']}"
return job
jm.create_job.side_effect = _create_job
jm.update_job.return_value = None
monkeypatch.setattr("backend.services.job_manager.JobManager", lambda: jm)
return jm
@pytest.fixture
def client():
c = TestClient(app)
yield c
app.dependency_overrides.clear()
def test_happy_path_creates_jobs_and_dispatches(client, monkeypatch):
app.dependency_overrides[require_auth] = _auth()
_patch_user_service(monkeypatch, credits=10)
jm = _patch_job_manager(monkeypatch)
resp = client.post("/api/bulk/submit", json={
"songs": [{"artist": "Daft Punk", "title": "One More Time"},
{"artist": "Daft Punk", "title": "Aerodynamic"}],
"settings": {"auto_select_if_lossless": True, "is_private": False},
})
assert resp.status_code == 200, resp.text
data = resp.json()
assert data["total"] == 2
assert len(data["job_ids"]) == 2
assert jm.create_job.call_count == 2
# batch_id + bulk settings stamped via update_job
assert jm.update_job.call_count == 2
_, kwargs_call = jm.update_job.call_args
stamped = jm.update_job.call_args[0][1]
assert "state_data.batch_id" in stamped
assert stamped["state_data.bulk_pending_search"] is True
def test_insufficient_credits_creates_nothing(client, monkeypatch):
app.dependency_overrides[require_auth] = _auth()
_patch_user_service(monkeypatch, credits=1)
jm = _patch_job_manager(monkeypatch)
resp = client.post("/api/bulk/submit", json={
"songs": [{"artist": "A", "title": "B"}, {"artist": "C", "title": "D"}],
})
assert resp.status_code == 402
assert jm.create_job.call_count == 0
def test_admin_bypasses_credit_gate(client, monkeypatch):
app.dependency_overrides[require_auth] = _auth(is_admin=True)
us = _patch_user_service(monkeypatch, credits=0)
jm = _patch_job_manager(monkeypatch)
resp = client.post("/api/bulk/submit", json={"songs": [{"artist": "A", "title": "B"}]})
assert resp.status_code == 200
us.check_credits.assert_not_called()
assert jm.create_job.call_count == 1
def test_rejects_over_100(client, monkeypatch):
app.dependency_overrides[require_auth] = _auth()
_patch_user_service(monkeypatch, credits=1000)
_patch_job_manager(monkeypatch)
songs = [{"artist": f"A{i}", "title": f"T{i}"} for i in range(101)]
resp = client.post("/api/bulk/submit", json={"songs": songs})
assert resp.status_code == 422
def test_partial_failure_returns_created_jobs(client, monkeypatch):
app.dependency_overrides[require_auth] = _auth()
_patch_user_service(monkeypatch, credits=3) # gate passes for 3
jm = _patch_job_manager(monkeypatch, fail_after=2) # but only 2 succeed
resp = client.post("/api/bulk/submit", json={
"songs": [{"artist": "A", "title": "1"}, {"artist": "A", "title": "2"}, {"artist": "A", "title": "3"}],
})
assert resp.status_code == 200
assert resp.json()["total"] == 2
def test_album_batch_stamps_album_and_track_positions(client, monkeypatch):
app.dependency_overrides[require_auth] = _auth()
_patch_user_service(monkeypatch, credits=10)
jm = _patch_job_manager(monkeypatch)
resp = client.post("/api/bulk/submit", json={
"songs": [{"artist": "Daft Punk", "title": "One More Time", "position": 1},
{"artist": "Daft Punk", "title": "Aerodynamic", "position": 2}],
"settings": {},
"album": {"artist": "Daft Punk", "title": "Discovery", "release_group_mbid": "rg-1"},
})
assert resp.status_code == 200, resp.text
stamps = [c[0][1] for c in jm.update_job.call_args_list]
assert [s["state_data.bulk_track_position"] for s in stamps] == [1, 2]
assert stamps[0]["state_data.bulk_album"] == {
"artist": "Daft Punk", "title": "Discovery", "release_group_mbid": "rg-1", "release_mbid": None,
}
def test_text_batch_has_no_album_state(client, monkeypatch):
app.dependency_overrides[require_auth] = _auth()
_patch_user_service(monkeypatch, credits=10)
jm = _patch_job_manager(monkeypatch)
client.post("/api/bulk/submit", json={"songs": [{"artist": "Daft Punk", "title": "One More Time"}]})
stamped = jm.update_job.call_args[0][1]
assert "state_data.bulk_album" not in stamped
assert "state_data.bulk_track_position" not in stamped
@pytest.mark.asyncio
async def test_cancelled_request_still_dispatches_created_batch(monkeypatch):
"""A client disconnect mid-creation must not strand created (and charged) jobs:
the create+dispatch step is shielded, so dispatch still runs after cancellation."""
import asyncio
import threading
import backend.api.routes.bulk as bulk
from starlette.requests import Request
_patch_user_service(monkeypatch, credits=10)
jm = _patch_job_manager(monkeypatch)
started, release = threading.Event(), threading.Event()
inner_create = jm.create_job.side_effect
def _slow_create(job_create, is_admin=False):
started.set()
release.wait(5)
return inner_create(job_create, is_admin=is_admin)
jm.create_job.side_effect = _slow_create
dispatched = []
async def _trigger(batch_id):
dispatched.append(batch_id)
return True
worker = MagicMock()
worker.trigger_bulk_search_worker = _trigger
monkeypatch.setattr("backend.services.worker_service.get_worker_service", lambda: worker)
request = Request({"type": "http", "method": "POST", "path": "/api/bulk/submit", "headers": []})
body = bulk.BulkSubmitRequest(songs=[{"artist": "A", "title": "B"}], settings={})
route = asyncio.create_task(bulk.bulk_submit(request, body, _auth()()))
await asyncio.to_thread(started.wait, 5)
route.cancel()
release.set()
with pytest.raises(asyncio.CancelledError):
await route
for _ in range(100): # let the shielded task finish
if dispatched:
break
await asyncio.sleep(0.01)
assert len(dispatched) == 1