mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
## Why A real Bulk "By album" batch (Chase & Status, *No More Idols*, 12 tracks): - It took about 5 minutes to reach audio selection. - 4 of the 12 searches timed out. - All 12 were parked for manual choice, each showing the same album torrents. Causes: - The album was dropped at submit, so each track was searched separately. - 4 parallel searches ran against flacfetch's single search worker, and the queued ones ran out the 95s budget. - The auto-pick bar was 50 seeders (lowered to 10 in #1211). - A title-only file search misses the album for common titles ("Time"). ## What - **Submit carries the album:** `POST /api/bulk/submit` takes `album` {artist, title, release_group_mbid, release_mbid} plus each song's `position`. These are stamped as `state_data.bulk_album` / `bulk_track_position`. The frontend album mode sends them, using the release-group title. - **One album search per album:** the worker makes one flacfetch `POST /search/album` call (nomadkaraoke/flacfetch#56). - Each track's file is resolved inside the album's torrents. - The whole album comes from **one torrent**: the first BEST CHOICE (lossless, non-vinyl, 10+ seeders). - A track is auto-selected when that torrent's file names it and the artist matches. - A track the batch torrent doesn't cover gets the normal per-track pick among the album torrents that do. - Parked jobs show the album torrents in the selection modal. - **Fallbacks are one search at a time** (`SEARCH_CONCURRENCY = 1`): uncovered tracks, text batches, and failed or unavailable album searches (404 from an older flacfetch). A batch search deadline (3000s) parks any leftovers before the 3600s job timeout could strand them. - **Auto-pick checks the artist** on the backend too: a Unicode-aware port of `isArtistMatch`, used by bulk and KaraokeHunt. A 10+-seeder cover or tribute torrent is no longer auto-picked. - Fake album search for the local stack; docs in API.md and LESSONS-LEARNED. The money path is unchanged. Album results have no duration, so nothing is charged at select time. The real duration charge or refund still happens after download (`reconcile_and_maybe_pause`). ## Testing - Backend: 6677 passed. The only failures are the 8 `test_emulator_integration` tests, which hit a stale local Firestore emulator from another session; they passed in CI on #1211. - Frontend: Jest 1765/1765. - New tests cover: - album worker path: one search, all auto-selected, uncovered → fallback, search failure → fallback, low seeders → park, auto disabled - `process_batch` routing, with at most one search in flight - the deadline - album helpers, artist match (incl. CJK and &), client `search_album` (incl. 404 and 502), submit stamping, the fake album response, and the frontend submit payload - Code review: CodeRabbit CLI unavailable (invalid token / org mismatch), so a local Superpowers review ran instead. Its fixes are in the second commit. ## Rollout flacfetch v0.34.0 (#56) deploys to flacup first. Then: one album search, 12 auto-selected jobs, one torrent; this is verified in prod after merge. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_019MixBuvzxB1c5AYc3mbDyW <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Album-mode bulk submissions now include album details and track positions, enabling one album-wide search for matching tracks. * Search results are checked against both track title and artist before automatic selection, with tolerance for minor name variations. * **Improvements** * Tracks not covered by an album result, or affected by an unavailable or failed album search, fall back to individual searches. * Searches still awaiting selection after the batch deadline are parked for manual review. * **Documentation** * Updated bulk submission guidance to describe album searches and fallback behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
225 lines
8.3 KiB
Python
225 lines
8.3 KiB
Python
"""Tests for POST /api/bulk/submit — upfront credit gate + batch job creation."""
|
|
|
|
from unittest.mock import MagicMock
|
|
|
|
import pytest
|
|
from fastapi.testclient import TestClient
|
|
|
|
from backend.main import app
|
|
from backend.api.dependencies import require_auth, AuthResult, UserType
|
|
from backend.exceptions import InsufficientCreditsError
|
|
|
|
|
|
def _auth(is_admin=False, email="t@example.com"):
|
|
def _dep():
|
|
return AuthResult(
|
|
is_valid=True,
|
|
user_type=UserType.ADMIN if is_admin else UserType.STRIPE,
|
|
remaining_uses=999,
|
|
message="ok",
|
|
user_email=email,
|
|
is_admin=is_admin,
|
|
)
|
|
return _dep
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _patch_deps(monkeypatch):
|
|
"""Patch the heavy collaborators used by bulk_submit."""
|
|
import backend.api.routes.bulk as bulk
|
|
|
|
# Theme + settings + cdg/txt
|
|
theme = MagicMock()
|
|
theme.get_default_theme_id.return_value = "nomad"
|
|
monkeypatch.setattr("backend.services.theme_service.get_theme_service", lambda: theme)
|
|
monkeypatch.setattr("backend.services.job_defaults_service.resolve_cdg_txt_defaults", lambda *a, **k: (False, False))
|
|
settings = MagicMock(
|
|
default_brand_prefix="NOMAD", default_enable_youtube_upload=False,
|
|
default_youtube_description="", default_discord_webhook_url=None,
|
|
default_dropbox_path=None, default_gdrive_folder_id=None,
|
|
)
|
|
monkeypatch.setattr("backend.config.get_settings", lambda: settings)
|
|
|
|
# Worker trigger
|
|
worker = MagicMock()
|
|
async def _trigger(batch_id):
|
|
return True
|
|
worker.trigger_bulk_search_worker = _trigger
|
|
monkeypatch.setattr("backend.services.worker_service.get_worker_service", lambda: worker)
|
|
|
|
yield
|
|
|
|
|
|
def _patch_user_service(monkeypatch, credits):
|
|
us = MagicMock()
|
|
us.check_credits.return_value = credits
|
|
monkeypatch.setattr("backend.services.user_service.get_user_service", lambda: us)
|
|
return us
|
|
|
|
|
|
def _patch_job_manager(monkeypatch, fail_after=None):
|
|
"""JobManager whose create_job returns sequential ids, optionally failing after N."""
|
|
created = {"count": 0}
|
|
jm = MagicMock()
|
|
|
|
def _create_job(job_create, is_admin=False):
|
|
if fail_after is not None and created["count"] >= fail_after:
|
|
raise InsufficientCreditsError(message="out", credits_available=0, credits_required=1)
|
|
created["count"] += 1
|
|
job = MagicMock()
|
|
job.job_id = f"job{created['count']}"
|
|
return job
|
|
|
|
jm.create_job.side_effect = _create_job
|
|
jm.update_job.return_value = None
|
|
monkeypatch.setattr("backend.services.job_manager.JobManager", lambda: jm)
|
|
return jm
|
|
|
|
|
|
@pytest.fixture
|
|
def client():
|
|
c = TestClient(app)
|
|
yield c
|
|
app.dependency_overrides.clear()
|
|
|
|
|
|
def test_happy_path_creates_jobs_and_dispatches(client, monkeypatch):
|
|
app.dependency_overrides[require_auth] = _auth()
|
|
_patch_user_service(monkeypatch, credits=10)
|
|
jm = _patch_job_manager(monkeypatch)
|
|
|
|
resp = client.post("/api/bulk/submit", json={
|
|
"songs": [{"artist": "Daft Punk", "title": "One More Time"},
|
|
{"artist": "Daft Punk", "title": "Aerodynamic"}],
|
|
"settings": {"auto_select_if_lossless": True, "is_private": False},
|
|
})
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
assert data["total"] == 2
|
|
assert len(data["job_ids"]) == 2
|
|
assert jm.create_job.call_count == 2
|
|
# batch_id + bulk settings stamped via update_job
|
|
assert jm.update_job.call_count == 2
|
|
_, kwargs_call = jm.update_job.call_args
|
|
stamped = jm.update_job.call_args[0][1]
|
|
assert "state_data.batch_id" in stamped
|
|
assert stamped["state_data.bulk_pending_search"] is True
|
|
|
|
|
|
def test_insufficient_credits_creates_nothing(client, monkeypatch):
|
|
app.dependency_overrides[require_auth] = _auth()
|
|
_patch_user_service(monkeypatch, credits=1)
|
|
jm = _patch_job_manager(monkeypatch)
|
|
|
|
resp = client.post("/api/bulk/submit", json={
|
|
"songs": [{"artist": "A", "title": "B"}, {"artist": "C", "title": "D"}],
|
|
})
|
|
assert resp.status_code == 402
|
|
assert jm.create_job.call_count == 0
|
|
|
|
|
|
def test_admin_bypasses_credit_gate(client, monkeypatch):
|
|
app.dependency_overrides[require_auth] = _auth(is_admin=True)
|
|
us = _patch_user_service(monkeypatch, credits=0)
|
|
jm = _patch_job_manager(monkeypatch)
|
|
|
|
resp = client.post("/api/bulk/submit", json={"songs": [{"artist": "A", "title": "B"}]})
|
|
assert resp.status_code == 200
|
|
us.check_credits.assert_not_called()
|
|
assert jm.create_job.call_count == 1
|
|
|
|
|
|
def test_rejects_over_100(client, monkeypatch):
|
|
app.dependency_overrides[require_auth] = _auth()
|
|
_patch_user_service(monkeypatch, credits=1000)
|
|
_patch_job_manager(monkeypatch)
|
|
songs = [{"artist": f"A{i}", "title": f"T{i}"} for i in range(101)]
|
|
resp = client.post("/api/bulk/submit", json={"songs": songs})
|
|
assert resp.status_code == 422
|
|
|
|
|
|
def test_partial_failure_returns_created_jobs(client, monkeypatch):
|
|
app.dependency_overrides[require_auth] = _auth()
|
|
_patch_user_service(monkeypatch, credits=3) # gate passes for 3
|
|
jm = _patch_job_manager(monkeypatch, fail_after=2) # but only 2 succeed
|
|
|
|
resp = client.post("/api/bulk/submit", json={
|
|
"songs": [{"artist": "A", "title": "1"}, {"artist": "A", "title": "2"}, {"artist": "A", "title": "3"}],
|
|
})
|
|
assert resp.status_code == 200
|
|
assert resp.json()["total"] == 2
|
|
|
|
|
|
def test_album_batch_stamps_album_and_track_positions(client, monkeypatch):
|
|
app.dependency_overrides[require_auth] = _auth()
|
|
_patch_user_service(monkeypatch, credits=10)
|
|
jm = _patch_job_manager(monkeypatch)
|
|
|
|
resp = client.post("/api/bulk/submit", json={
|
|
"songs": [{"artist": "Daft Punk", "title": "One More Time", "position": 1},
|
|
{"artist": "Daft Punk", "title": "Aerodynamic", "position": 2}],
|
|
"settings": {},
|
|
"album": {"artist": "Daft Punk", "title": "Discovery", "release_group_mbid": "rg-1"},
|
|
})
|
|
assert resp.status_code == 200, resp.text
|
|
stamps = [c[0][1] for c in jm.update_job.call_args_list]
|
|
assert [s["state_data.bulk_track_position"] for s in stamps] == [1, 2]
|
|
assert stamps[0]["state_data.bulk_album"] == {
|
|
"artist": "Daft Punk", "title": "Discovery", "release_group_mbid": "rg-1", "release_mbid": None,
|
|
}
|
|
|
|
|
|
def test_text_batch_has_no_album_state(client, monkeypatch):
|
|
app.dependency_overrides[require_auth] = _auth()
|
|
_patch_user_service(monkeypatch, credits=10)
|
|
jm = _patch_job_manager(monkeypatch)
|
|
client.post("/api/bulk/submit", json={"songs": [{"artist": "Daft Punk", "title": "One More Time"}]})
|
|
stamped = jm.update_job.call_args[0][1]
|
|
assert "state_data.bulk_album" not in stamped
|
|
assert "state_data.bulk_track_position" not in stamped
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_cancelled_request_still_dispatches_created_batch(monkeypatch):
|
|
"""A client disconnect mid-creation must not strand created (and charged) jobs:
|
|
the create+dispatch step is shielded, so dispatch still runs after cancellation."""
|
|
import asyncio
|
|
import threading
|
|
import backend.api.routes.bulk as bulk
|
|
from starlette.requests import Request
|
|
|
|
_patch_user_service(monkeypatch, credits=10)
|
|
jm = _patch_job_manager(monkeypatch)
|
|
started, release = threading.Event(), threading.Event()
|
|
inner_create = jm.create_job.side_effect
|
|
|
|
def _slow_create(job_create, is_admin=False):
|
|
started.set()
|
|
release.wait(5)
|
|
return inner_create(job_create, is_admin=is_admin)
|
|
|
|
jm.create_job.side_effect = _slow_create
|
|
dispatched = []
|
|
|
|
async def _trigger(batch_id):
|
|
dispatched.append(batch_id)
|
|
return True
|
|
|
|
worker = MagicMock()
|
|
worker.trigger_bulk_search_worker = _trigger
|
|
monkeypatch.setattr("backend.services.worker_service.get_worker_service", lambda: worker)
|
|
|
|
request = Request({"type": "http", "method": "POST", "path": "/api/bulk/submit", "headers": []})
|
|
body = bulk.BulkSubmitRequest(songs=[{"artist": "A", "title": "B"}], settings={})
|
|
route = asyncio.create_task(bulk.bulk_submit(request, body, _auth()()))
|
|
await asyncio.to_thread(started.wait, 5)
|
|
route.cancel()
|
|
release.set()
|
|
with pytest.raises(asyncio.CancelledError):
|
|
await route
|
|
for _ in range(100): # let the shielded task finish
|
|
if dispatched:
|
|
break
|
|
await asyncio.sleep(0.01)
|
|
assert len(dispatched) == 1
|