fix: PyPI prune snippet + force marker only in PR title (v0.268.10) (#1146)

## Summary
Two follow-ups to #1145, both found while doing today's prune.

1. **Prune console snippet silently did nothing.** PyPI's delete-version
form now requires five `acknowledge_*` checkboxes. A submit without them
gets a **200 re-render**, which the old snippet counted as "deleted".
The snippet now reads each release's real form (CSRF plus checkboxes),
submits it, and counts a version deleted only when its manage page 404s.
This is the exact approach used for today's prune (88 releases, 10.73 →
~4.1 GB).
2. **The force marker matched the commit body.** A squash commit's body
is the PR description. #1145's description just *mentioned* the marker,
which forced a publish on merge. That publish ran before the prune, so
it 400'd and uploaded nothing. The marker now only counts in the commit
subject (the PR title).

Docs: the runbook covers the new form, the per-version JSON cache lag,
and a prune log.

## Testing
- Unit tests: 30 pass across both scripts, with new tests for
checkbox/404 handling and for a marker in the body not forcing a publish
- `node --check` on the generated snippet
- Replayed #1145's real merge message through the gate: no publish
- Local CodeRabbit review: 0 findings
- Full `make test` not re-run (no app code touched); PR CI covers the
package suites

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01MSW9yjwGnzz1S8MgCGm6t5

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Andrew Beveridge ☄️andClaude Opus 5.5 authored and GitHub committed 2026-10-14 13:34:57 -04:00
1 parent 5005d3cb80
commit 39dcf2598e
8 files changed
+67 -29

No files matched your search

+2 -2
View File
@@ -986,8 +986,8 @@ jobs:
# the version bump). Each wheel is ~62 MiB and PyPI caps the project at
# 10 GB; publishing on every merge filled it every few weeks. PyPI is only
# the public `pip install` channel — Cloud Run and the encoding worker
# (GCS wheel) don't use it. Force: "[publish-pypi]" in the commit message
# or the workflow_dispatch input. The tag + GitHub release below still
# (GCS wheel) don't use it. Force: "[publish-pypi]" in the commit subject
# (PR title) or the workflow_dispatch input. The tag + GitHub release below still
# happen on every release (the encoding-worker gate diffs against tags).
- name: Decide PyPI publish
id: pypi_gate
+12 -3
View File
@@ -30,7 +30,7 @@ version on PyPI (git tag `v<version>`):
Frontend changes (the bundled review UI) and `backend/config.py` are deliberately
not gated; they reach CLI users with the next CLI release. Force a publish with
**`[publish-pypi]`** in the merge commit message (PR title) or Actions → CI → Run
**`[publish-pypi]`** in the PR title (only the squash commit's subject line counts) or Actions → CI → Run
workflow → **"Publish to PyPI"**. The git tag and GitHub release (with the wheel
attached) are still created for every version. If PyPI or git can't be read, the
gate publishes (fails open). Replayed over v0.200–v0.268 it would have published
@@ -73,14 +73,23 @@ public `pip install` channel, whose users want recent versions).
1. Open the tracking issue (or run the script locally, below) to get the snippet.
2. Go to **<https://pypi.org/manage/project/karaoke-gen/releases/>** and log in.
3. Open browser devtools → **Console**.
4. Paste the snippet and press Enter. It reads the page's CSRF token and POSTs a
whole-version delete for each planned version, ~0.5s apart, logging progress.
4. Paste the snippet and press Enter. For each planned version it reads that
release's delete form (CSRF token plus the required `acknowledge_*`
checkboxes), submits it, and counts it deleted only once the release's manage
page 404s. If every version logs "still exists", PyPI changed the form again:
inspect the form fields on one release page and update `render_console_js`.
5. Reload to confirm. If a publish was blocked, re-run the failed CI
`Deploy - Publish to PyPI` job (within a day; its build artifact expires), or
run Actions → CI → Run workflow on `main` with **"Publish to PyPI"** ticked.
> The aggregate `pypi.org/pypi/karaoke-gen/json` is Fastly-cached and lags a few
> minutes after deletes — trust the logged-in manage page for live state.
> Even the per-version `pypi.org/pypi/karaoke-gen/<v>/json` stayed 200 for minutes
> after deletes on 2026-10-14; the manage page 404 is the reliable check.
**Prune log:** 2026-06-22 (42 releases, → 6.36 GB) · 2026-09-29 (69, → ~6.8 GB) ·
2026-10-14 (88 at `--keep-days 14`, 10.73 → ~4.1 GB; 53 kept, done alongside the
publish gate).
## Running the planner locally
+1 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "karaoke-gen"
version = "0.268.9"
version = "0.268.10"
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
license = "MIT"
+23 -15
View File
@@ -162,32 +162,40 @@ def render_console_js(project: str, versions: list[str]) -> str:
"""A snippet to paste into the browser console on the logged-in PyPI page:
https://pypi.org/manage/project/<project>/releases/
It reads the session CSRF token from the page and POSTs a whole-version
delete for each listed version, ~500ms apart, logging progress.
For each version it fetches that release's manage page, reads the real
delete-version form (CSRF token plus every required "acknowledge_*"
checkbox; PyPI added those in 2026 and silently re-renders the page with a
200 if any is missing), POSTs it, then counts the version as deleted only if
its manage page now 404s. ~400ms apart, logging progress.
"""
versions_json = json.dumps(versions)
return f"""// Paste on https://pypi.org/manage/project/{project}/releases/ while logged in.
(async () => {{
const project = {json.dumps(project)};
const versions = {versions_json};
const el = document.querySelector('input[name="csrf_token"]');
if (!el) {{ console.error("No csrf_token on page — are you on the releases management page, logged in?"); return; }}
const csrf = el.value;
let ok = 0, fail = 0;
for (const v of versions) {{
const page = `/manage/project/${{project}}/release/${{encodeURIComponent(v)}}/`;
try {{
const res = await fetch(`/manage/project/${{project}}/release/${{encodeURIComponent(v)}}/`, {{
method: "POST",
credentials: "include",
headers: {{ "Content-Type": "application/x-www-form-urlencoded" }},
body: `csrf_token=${{encodeURIComponent(csrf)}}&confirm_delete_version=${{encodeURIComponent(v)}}`,
}});
if (res.ok || res.redirected) {{ ok++; console.log(`✓ deleted ${{v}} (${{ok}}/${{versions.length}})`); }}
else {{ fail++; console.warn(`✗ ${{v}} → HTTP ${{res.status}}`); }}
const res = await fetch(page, {{ credentials: "include" }});
if (res.status === 404) {{ ok++; console.log(`✓ ${{v}} already gone`); continue; }}
const doc = new DOMParser().parseFromString(await res.text(), "text/html");
const form = [...doc.querySelectorAll("form")].find(f => f.querySelector('[name="confirm_delete_version"]'));
if (!form) {{ fail++; console.warn(`✗ ${{v}}: no delete form — logged in as an owner?`); continue; }}
const body = new URLSearchParams();
for (const i of form.querySelectorAll("input")) {{
if (i.type === "checkbox") body.append(i.name, i.value || "on");
else if (i.type === "hidden") body.append(i.name, i.value);
}}
body.set("confirm_delete_version", v);
await fetch(page, {{ method: "POST", credentials: "include", body }});
const check = await fetch(page, {{ credentials: "include" }});
if (check.status === 404) {{ ok++; console.log(`✓ deleted ${{v}} (${{ok}}/${{versions.length}})`); }}
else {{ fail++; console.warn(`✗ ${{v}} still exists (HTTP ${{check.status}}) — the delete form may have changed`); }}
}} catch (e) {{ fail++; console.warn(`✗ ${{v}}`, e); }}
await new Promise(r => setTimeout(r, 500));
await new Promise(r => setTimeout(r, 400));
}}
console.log(`Done. deleted=${{ok}} failed=${{fail}}. Reload to confirm.`);
console.log(`Done. deleted=${{ok}} failed=${{fail}}.`);
}})();
"""
+7 -4
View File
@@ -10,8 +10,8 @@ something CLI users get has changed since the latest version on PyPI:
* a path in scripts/pypi_publish_paths.txt changed, or
* pyproject.toml changed in anything other than its `version = ...` line
(dependencies, entry points, package includes), or
* a publish was forced ("[publish-pypi]" in the commit message, or the
workflow_dispatch input).
* a publish was forced ("[publish-pypi]" in the commit subject, i.e. the PR
title, or the workflow_dispatch input).
The diff base is the version currently on PyPI, not the previous commit, so a
skipped or failed publish can never lose a CLI change: the next push still sees
@@ -93,8 +93,11 @@ def decide(
) -> Decision:
if force:
return Decision(True, "forced by workflow_dispatch input")
if FORCE_MARKER in commit_message:
return Decision(True, f"forced by {FORCE_MARKER} in the commit message")
# Subject line only (= the PR title on a squash merge). The body carries the
# whole PR description, which may just mention the marker (PR #1145 did).
subject = commit_message.splitlines()[0] if commit_message else ""
if FORCE_MARKER in subject:
return Decision(True, f"forced by {FORCE_MARKER} in the commit subject")
try:
pypi_version = get_pypi_version()
+2 -2
View File
@@ -13,8 +13,8 @@
# created. pyproject.toml is checked separately with its version line ignored,
# because every PR bumps it.
#
# Force a publish anyway with "[publish-pypi]" in the merge commit message (e.g.
# in the PR title) or Actions → CI → Run workflow → "Publish to PyPI".
# Force a publish anyway with "[publish-pypi]" in the PR title (the squash
# commit's subject line; the body is ignored) or Actions → CI → Run workflow → "Publish to PyPI".
#
# tests/unit/test_pypi_publish_gate.py derives the backend.* import
# closure of karaoke_gen and fails if a module the CLI imports is not listed.
+11 -1
View File
@@ -145,11 +145,21 @@ def test_same_day_uploads_pick_latest_overall():
def test_console_js_is_wellformed_and_contains_versions():
js = prune.render_console_js("karaoke-gen", ["1.2.3", "4.5.6"])
assert "confirm_delete_version" in js
assert "csrf_token" in js
assert json.dumps(["1.2.3", "4.5.6"]) in js
assert "/manage/project/${project}/release/" in js
def test_console_js_submits_every_form_field_and_verifies_by_404():
# PyPI's delete form requires acknowledge_* checkboxes (2026) and answers a
# rejected submit with a 200 re-render, so the snippet must copy the form's
# checkbox + hidden (CSRF) inputs and only count a 404 afterwards as deleted.
js = prune.render_console_js("karaoke-gen", ["1.2.3"])
assert 'i.type === "checkbox"' in js
assert 'i.type === "hidden"' in js
assert "check.status === 404" in js
assert "res.ok" not in js
def test_render_json_shape():
releases = {
"0.1.0": _release("2026-01-05", size=100), # deleted (superseded in Jan)
+9 -1
View File
@@ -100,13 +100,21 @@ def test_already_on_pypi_skips_without_touching_git():
assert git.calls == []
@pytest.mark.parametrize("kwargs", [{"force": True}, {"message": "fix: cli flag [publish-pypi]\n\nbody"}])
@pytest.mark.parametrize("kwargs", [{"force": True}, {"message": "fix: cli flag [publish-pypi] (#12)\n\nbody"}])
def test_force_publishes_even_without_changes(kwargs):
git = FakeGit(changed=[])
assert _decide(git, **kwargs).publish is True
assert git.calls == []
def test_marker_in_commit_body_does_not_force():
# A squash merge's body is the PR description, which may just *mention* the
# marker (PR #1145's did, forcing an unwanted publish).
git = FakeGit(changed=[])
d = _decide(git, message="ci: gate PyPI publish (#1145)\n\nForce with `[publish-pypi]` in the PR title.")
assert d.publish is False
def test_fails_open_when_pypi_unreachable():
d = _decide(pypi=OSError("timeout"))
assert d.publish is True