mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
fix: PyPI prune snippet + force marker only in PR title (v0.268.10) (#1146)
## Summary Two follow-ups to #1145, both found while doing today's prune. 1. **Prune console snippet silently did nothing.** PyPI's delete-version form now requires five `acknowledge_*` checkboxes. A submit without them gets a **200 re-render**, which the old snippet counted as "deleted". The snippet now reads each release's real form (CSRF plus checkboxes), submits it, and counts a version deleted only when its manage page 404s. This is the exact approach used for today's prune (88 releases, 10.73 → ~4.1 GB). 2. **The force marker matched the commit body.** A squash commit's body is the PR description. #1145's description just *mentioned* the marker, which forced a publish on merge. That publish ran before the prune, so it 400'd and uploaded nothing. The marker now only counts in the commit subject (the PR title). Docs: the runbook covers the new form, the per-version JSON cache lag, and a prune log. ## Testing - Unit tests: 30 pass across both scripts, with new tests for checkbox/404 handling and for a marker in the body not forcing a publish - `node --check` on the generated snippet - Replayed #1145's real merge message through the gate: no publish - Local CodeRabbit review: 0 findings - Full `make test` not re-run (no app code touched); PR CI covers the package suites @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01MSW9yjwGnzz1S8MgCGm6t5 --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
5005d3cb80
commit
39dcf2598e
8 files changed
+67
-29
No files matched your search
@@ -986,8 +986,8 @@ jobs:
|
||||
# the version bump). Each wheel is ~62 MiB and PyPI caps the project at
|
||||
# 10 GB; publishing on every merge filled it every few weeks. PyPI is only
|
||||
# the public `pip install` channel — Cloud Run and the encoding worker
|
||||
# (GCS wheel) don't use it. Force: "[publish-pypi]" in the commit message
|
||||
# or the workflow_dispatch input. The tag + GitHub release below still
|
||||
# (GCS wheel) don't use it. Force: "[publish-pypi]" in the commit subject
|
||||
# (PR title) or the workflow_dispatch input. The tag + GitHub release below still
|
||||
# happen on every release (the encoding-worker gate diffs against tags).
|
||||
- name: Decide PyPI publish
|
||||
id: pypi_gate
|
||||
|
||||
@@ -30,7 +30,7 @@ version on PyPI (git tag `v<version>`):
|
||||
|
||||
Frontend changes (the bundled review UI) and `backend/config.py` are deliberately
|
||||
not gated; they reach CLI users with the next CLI release. Force a publish with
|
||||
**`[publish-pypi]`** in the merge commit message (PR title) or Actions → CI → Run
|
||||
**`[publish-pypi]`** in the PR title (only the squash commit's subject line counts) or Actions → CI → Run
|
||||
workflow → **"Publish to PyPI"**. The git tag and GitHub release (with the wheel
|
||||
attached) are still created for every version. If PyPI or git can't be read, the
|
||||
gate publishes (fails open). Replayed over v0.200–v0.268 it would have published
|
||||
@@ -73,14 +73,23 @@ public `pip install` channel, whose users want recent versions).
|
||||
1. Open the tracking issue (or run the script locally, below) to get the snippet.
|
||||
2. Go to **<https://pypi.org/manage/project/karaoke-gen/releases/>** and log in.
|
||||
3. Open browser devtools → **Console**.
|
||||
4. Paste the snippet and press Enter. It reads the page's CSRF token and POSTs a
|
||||
whole-version delete for each planned version, ~0.5s apart, logging progress.
|
||||
4. Paste the snippet and press Enter. For each planned version it reads that
|
||||
release's delete form (CSRF token plus the required `acknowledge_*`
|
||||
checkboxes), submits it, and counts it deleted only once the release's manage
|
||||
page 404s. If every version logs "still exists", PyPI changed the form again:
|
||||
inspect the form fields on one release page and update `render_console_js`.
|
||||
5. Reload to confirm. If a publish was blocked, re-run the failed CI
|
||||
`Deploy - Publish to PyPI` job (within a day; its build artifact expires), or
|
||||
run Actions → CI → Run workflow on `main` with **"Publish to PyPI"** ticked.
|
||||
|
||||
> The aggregate `pypi.org/pypi/karaoke-gen/json` is Fastly-cached and lags a few
|
||||
> minutes after deletes — trust the logged-in manage page for live state.
|
||||
> Even the per-version `pypi.org/pypi/karaoke-gen/<v>/json` stayed 200 for minutes
|
||||
> after deletes on 2026-10-14; the manage page 404 is the reliable check.
|
||||
|
||||
**Prune log:** 2026-06-22 (42 releases, → 6.36 GB) · 2026-09-29 (69, → ~6.8 GB) ·
|
||||
2026-10-14 (88 at `--keep-days 14`, 10.73 → ~4.1 GB; 53 kept, done alongside the
|
||||
publish gate).
|
||||
|
||||
## Running the planner locally
|
||||
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[tool.poetry]
|
||||
name = "karaoke-gen"
|
||||
version = "0.268.9"
|
||||
version = "0.268.10"
|
||||
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
|
||||
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
|
||||
license = "MIT"
|
||||
|
||||
@@ -162,32 +162,40 @@ def render_console_js(project: str, versions: list[str]) -> str:
|
||||
"""A snippet to paste into the browser console on the logged-in PyPI page:
|
||||
https://pypi.org/manage/project/<project>/releases/
|
||||
|
||||
It reads the session CSRF token from the page and POSTs a whole-version
|
||||
delete for each listed version, ~500ms apart, logging progress.
|
||||
For each version it fetches that release's manage page, reads the real
|
||||
delete-version form (CSRF token plus every required "acknowledge_*"
|
||||
checkbox; PyPI added those in 2026 and silently re-renders the page with a
|
||||
200 if any is missing), POSTs it, then counts the version as deleted only if
|
||||
its manage page now 404s. ~400ms apart, logging progress.
|
||||
"""
|
||||
versions_json = json.dumps(versions)
|
||||
return f"""// Paste on https://pypi.org/manage/project/{project}/releases/ while logged in.
|
||||
(async () => {{
|
||||
const project = {json.dumps(project)};
|
||||
const versions = {versions_json};
|
||||
const el = document.querySelector('input[name="csrf_token"]');
|
||||
if (!el) {{ console.error("No csrf_token on page — are you on the releases management page, logged in?"); return; }}
|
||||
const csrf = el.value;
|
||||
let ok = 0, fail = 0;
|
||||
for (const v of versions) {{
|
||||
const page = `/manage/project/${{project}}/release/${{encodeURIComponent(v)}}/`;
|
||||
try {{
|
||||
const res = await fetch(`/manage/project/${{project}}/release/${{encodeURIComponent(v)}}/`, {{
|
||||
method: "POST",
|
||||
credentials: "include",
|
||||
headers: {{ "Content-Type": "application/x-www-form-urlencoded" }},
|
||||
body: `csrf_token=${{encodeURIComponent(csrf)}}&confirm_delete_version=${{encodeURIComponent(v)}}`,
|
||||
}});
|
||||
if (res.ok || res.redirected) {{ ok++; console.log(`✓ deleted ${{v}} (${{ok}}/${{versions.length}})`); }}
|
||||
else {{ fail++; console.warn(`✗ ${{v}} → HTTP ${{res.status}}`); }}
|
||||
const res = await fetch(page, {{ credentials: "include" }});
|
||||
if (res.status === 404) {{ ok++; console.log(`✓ ${{v}} already gone`); continue; }}
|
||||
const doc = new DOMParser().parseFromString(await res.text(), "text/html");
|
||||
const form = [...doc.querySelectorAll("form")].find(f => f.querySelector('[name="confirm_delete_version"]'));
|
||||
if (!form) {{ fail++; console.warn(`✗ ${{v}}: no delete form — logged in as an owner?`); continue; }}
|
||||
const body = new URLSearchParams();
|
||||
for (const i of form.querySelectorAll("input")) {{
|
||||
if (i.type === "checkbox") body.append(i.name, i.value || "on");
|
||||
else if (i.type === "hidden") body.append(i.name, i.value);
|
||||
}}
|
||||
body.set("confirm_delete_version", v);
|
||||
await fetch(page, {{ method: "POST", credentials: "include", body }});
|
||||
const check = await fetch(page, {{ credentials: "include" }});
|
||||
if (check.status === 404) {{ ok++; console.log(`✓ deleted ${{v}} (${{ok}}/${{versions.length}})`); }}
|
||||
else {{ fail++; console.warn(`✗ ${{v}} still exists (HTTP ${{check.status}}) — the delete form may have changed`); }}
|
||||
}} catch (e) {{ fail++; console.warn(`✗ ${{v}}`, e); }}
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
await new Promise(r => setTimeout(r, 400));
|
||||
}}
|
||||
console.log(`Done. deleted=${{ok}} failed=${{fail}}. Reload to confirm.`);
|
||||
console.log(`Done. deleted=${{ok}} failed=${{fail}}.`);
|
||||
}})();
|
||||
"""
|
||||
|
||||
|
||||
@@ -10,8 +10,8 @@ something CLI users get has changed since the latest version on PyPI:
|
||||
* a path in scripts/pypi_publish_paths.txt changed, or
|
||||
* pyproject.toml changed in anything other than its `version = ...` line
|
||||
(dependencies, entry points, package includes), or
|
||||
* a publish was forced ("[publish-pypi]" in the commit message, or the
|
||||
workflow_dispatch input).
|
||||
* a publish was forced ("[publish-pypi]" in the commit subject, i.e. the PR
|
||||
title, or the workflow_dispatch input).
|
||||
|
||||
The diff base is the version currently on PyPI, not the previous commit, so a
|
||||
skipped or failed publish can never lose a CLI change: the next push still sees
|
||||
@@ -93,8 +93,11 @@ def decide(
|
||||
) -> Decision:
|
||||
if force:
|
||||
return Decision(True, "forced by workflow_dispatch input")
|
||||
if FORCE_MARKER in commit_message:
|
||||
return Decision(True, f"forced by {FORCE_MARKER} in the commit message")
|
||||
# Subject line only (= the PR title on a squash merge). The body carries the
|
||||
# whole PR description, which may just mention the marker (PR #1145 did).
|
||||
subject = commit_message.splitlines()[0] if commit_message else ""
|
||||
if FORCE_MARKER in subject:
|
||||
return Decision(True, f"forced by {FORCE_MARKER} in the commit subject")
|
||||
|
||||
try:
|
||||
pypi_version = get_pypi_version()
|
||||
|
||||
@@ -13,8 +13,8 @@
|
||||
# created. pyproject.toml is checked separately with its version line ignored,
|
||||
# because every PR bumps it.
|
||||
#
|
||||
# Force a publish anyway with "[publish-pypi]" in the merge commit message (e.g.
|
||||
# in the PR title) or Actions → CI → Run workflow → "Publish to PyPI".
|
||||
# Force a publish anyway with "[publish-pypi]" in the PR title (the squash
|
||||
# commit's subject line; the body is ignored) or Actions → CI → Run workflow → "Publish to PyPI".
|
||||
#
|
||||
# tests/unit/test_pypi_publish_gate.py derives the backend.* import
|
||||
# closure of karaoke_gen and fails if a module the CLI imports is not listed.
|
||||
|
||||
@@ -145,11 +145,21 @@ def test_same_day_uploads_pick_latest_overall():
|
||||
def test_console_js_is_wellformed_and_contains_versions():
|
||||
js = prune.render_console_js("karaoke-gen", ["1.2.3", "4.5.6"])
|
||||
assert "confirm_delete_version" in js
|
||||
assert "csrf_token" in js
|
||||
assert json.dumps(["1.2.3", "4.5.6"]) in js
|
||||
assert "/manage/project/${project}/release/" in js
|
||||
|
||||
|
||||
def test_console_js_submits_every_form_field_and_verifies_by_404():
|
||||
# PyPI's delete form requires acknowledge_* checkboxes (2026) and answers a
|
||||
# rejected submit with a 200 re-render, so the snippet must copy the form's
|
||||
# checkbox + hidden (CSRF) inputs and only count a 404 afterwards as deleted.
|
||||
js = prune.render_console_js("karaoke-gen", ["1.2.3"])
|
||||
assert 'i.type === "checkbox"' in js
|
||||
assert 'i.type === "hidden"' in js
|
||||
assert "check.status === 404" in js
|
||||
assert "res.ok" not in js
|
||||
|
||||
|
||||
def test_render_json_shape():
|
||||
releases = {
|
||||
"0.1.0": _release("2026-01-05", size=100), # deleted (superseded in Jan)
|
||||
|
||||
@@ -100,13 +100,21 @@ def test_already_on_pypi_skips_without_touching_git():
|
||||
assert git.calls == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("kwargs", [{"force": True}, {"message": "fix: cli flag [publish-pypi]\n\nbody"}])
|
||||
@pytest.mark.parametrize("kwargs", [{"force": True}, {"message": "fix: cli flag [publish-pypi] (#12)\n\nbody"}])
|
||||
def test_force_publishes_even_without_changes(kwargs):
|
||||
git = FakeGit(changed=[])
|
||||
assert _decide(git, **kwargs).publish is True
|
||||
assert git.calls == []
|
||||
|
||||
|
||||
def test_marker_in_commit_body_does_not_force():
|
||||
# A squash merge's body is the PR description, which may just *mention* the
|
||||
# marker (PR #1145's did, forcing an unwanted publish).
|
||||
git = FakeGit(changed=[])
|
||||
d = _decide(git, message="ci: gate PyPI publish (#1145)\n\nForce with `[publish-pypi]` in the PR title.")
|
||||
assert d.publish is False
|
||||
|
||||
|
||||
def test_fails_open_when_pypi_unreachable():
|
||||
d = _decide(pypi=OSError("timeout"))
|
||||
assert d.publish is True
|
||||
|
||||
Reference in new issue
Block a user