Files
karaoke-gen/docs/PYPI-STORAGE-PRUNE.md
T
Andrew Beveridge ☄️andClaude Opus 5.5 39dcf2598e fix: PyPI prune snippet + force marker only in PR title (v0.268.10) (#1146)
## Summary
Two follow-ups to #1145, both found while doing today's prune.

1. **Prune console snippet silently did nothing.** PyPI's delete-version
form now requires five `acknowledge_*` checkboxes. A submit without them
gets a **200 re-render**, which the old snippet counted as "deleted".
The snippet now reads each release's real form (CSRF plus checkboxes),
submits it, and counts a version deleted only when its manage page 404s.
This is the exact approach used for today's prune (88 releases, 10.73 →
~4.1 GB).
2. **The force marker matched the commit body.** A squash commit's body
is the PR description. #1145's description just *mentioned* the marker,
which forced a publish on merge. That publish ran before the prune, so
it 400'd and uploaded nothing. The marker now only counts in the commit
subject (the PR title).

Docs: the runbook covers the new form, the per-version JSON cache lag,
and a prune log.

## Testing
- Unit tests: 30 pass across both scripts, with new tests for
checkbox/404 handling and for a marker in the body not forcing a publish
- `node --check` on the generated snippet
- Replayed #1145's real merge message through the gate: no publish
- Local CodeRabbit review: 0 findings
- Full `make test` not re-run (no app code touched); PR CI covers the
package suites

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01MSW9yjwGnzz1S8MgCGm6t5

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:34:57 -04:00

5.7 KiB
Raw Blame History

PyPI storage-cap prune runbook

The problem

PyPI caps total project size at 10 GB. karaoke-gen publishes a universal py3-none-any wheel from CI on merges to main. Since the i18n initiative the wheel bundles a Next.js static export pre-rendered into 33 locales (the CLI local lyrics-review UI serves it at runtime), so each release is ~62 MiB. Publishing on every merge (~100 releases/month by Sep 2026, ~6 GB/month) refilled the cap every few weeks and the publish step failed:

HTTP Error 400: Project size too large. Limit for project 'karaoke-gen'
total size is 10 GB.

Only the public pip install publish step fails — the Cloud Run backend and the GCE encoding worker (which pulls the wheel from GCS, not PyPI) are unaffected.

Prevention: publish only when the CLI changed (since v0.268.9)

PyPI is only the public pip install karaoke-gen channel for the CLIs; nothing in production installs from it. So the Deploy - Publish to PyPI job runs scripts/pypi_publish_gate.py first and uploads only if, since the latest version on PyPI (git tag v<version>):

  • a path in scripts/pypi_publish_paths.txt changed (karaoke_gen/ plus the backend/ modules it imports; a unit test keeps that list in sync), or
  • pyproject.toml changed beyond its version = line (dependencies, entry points).

Frontend changes (the bundled review UI) and backend/config.py are deliberately not gated; they reach CLI users with the next CLI release. Force a publish with [publish-pypi] in the PR title (only the squash commit's subject line counts) or Actions → CI → Run workflow → "Publish to PyPI". The git tag and GitHub release (with the wheel attached) are still created for every version. If PyPI or git can't be read, the gate publishes (fails open). Replayed over v0.200–v0.268 it would have published 24 of 136 releases (~0.7 GB/month instead of ~4 GB).

The monthly prune below stays as the backstop.

Why we can't just automate deletion

PyPI has no deletion API. Upload tokens and Trusted Publishing are upload-only, and PyPI mandates 2FA, so deletion is only possible from a logged-in browser session — which can't be safely or durably stored in CI. So we automate the decision and the reminder, and keep the deletion a ~1-minute human step.

Retention policy

Keep every release from the last 30 days, plus the newest release of each older calendar month, plus the latest release overall. Delete the rest. Run recurringly, this holds steady-state around 6–7 GB — permanently under the cap. (The window was 60 days until 2026-09-29; at the current release rate a 60-day window alone is ~9.6 GB, so a prune freed only ~1 GB and the cap refilled within a week.) Deletion is irreversible and a deleted version/filename can never be re-uploaded; that's acceptable here (nothing builds from source; PyPI is only the public pip install channel, whose users want recent versions).

How it runs

  • .github/workflows/pypi-prune-reminder.yml — monthly cron (1st, 09:00 UTC) + manual workflow_dispatch. Computes the plan and, when there's anything to prune, opens or updates a single tracking issue titled "🧹 PyPI storage prune due: karaoke-gen" with the plan + a paste-in deletion snippet. No secrets.
  • scripts/prune_pypi_releases.py — the planner. Reads the public JSON API, applies the policy, and renders a table / JSON / Markdown / browser-console JS. It never deletes anything itself.

How to prune (≈1 minute)

  1. Open the tracking issue (or run the script locally, below) to get the snippet.
  2. Go to https://pypi.org/manage/project/karaoke-gen/releases/ and log in.
  3. Open browser devtools → Console.
  4. Paste the snippet and press Enter. For each planned version it reads that release's delete form (CSRF token plus the required acknowledge_* checkboxes), submits it, and counts it deleted only once the release's manage page 404s. If every version logs "still exists", PyPI changed the form again: inspect the form fields on one release page and update render_console_js.
  5. Reload to confirm. If a publish was blocked, re-run the failed CI Deploy - Publish to PyPI job (within a day; its build artifact expires), or run Actions → CI → Run workflow on main with "Publish to PyPI" ticked.

The aggregate pypi.org/pypi/karaoke-gen/json is Fastly-cached and lags a few minutes after deletes — trust the logged-in manage page for live state. Even the per-version pypi.org/pypi/karaoke-gen/<v>/json stayed 200 for minutes after deletes on 2026-10-14; the manage page 404 is the reliable check.

Prune log: 2026-06-22 (42 releases, → 6.36 GB) · 2026-09-29 (69, → ~6.8 GB) · 2026-10-14 (88 at --keep-days 14, 10.73 → ~4.1 GB; 53 kept, done alongside the publish gate).

Running the planner locally

# Dry-run table (what would be deleted, and the resulting size)
python scripts/prune_pypi_releases.py

# Just the paste-in deletion snippet
python scripts/prune_pypi_releases.py --format console-js

# Machine-readable / the Markdown the workflow posts
python scripts/prune_pypi_releases.py --format json
python scripts/prune_pypi_releases.py --format markdown

# Tune the window (default 30 days) or target another project
python scripts/prune_pypi_releases.py --keep-days 60 --project karaoke-gen

Stdlib only — no dependencies to install.

  • One-off escape hatch: request a PyPI storage-limit increase via the limit-request-project template on pypi/support (we filed pypi/support#12050 for 50 GB). A larger cap just buys time; the recurring prune is the durable fix.
  • History of the first time this bit us: docs/LESSONS-LEARNED.md.