Files
Andrew Beveridge ☄️andClaude Opus 5.5 feef49a588 fix(e2e): stop publishing card data/admin token in artifacts; fail fast on card decline (v0.266.1) (#1128)
## Summary
E2E Daily **#200** failed in Stage 1. The cause was **the issuer
declining the test card**: Stripe `confirm` returned 402 `card_error /
card_declined / generic_decline` on all 3 attempts. It was not a Stripe
Checkout change. The self-healing runner still treated it as one: a 90s
verify wait, then 6 Gemini calls "re-entering" a form that was already
correct.

The investigation also found that **every run's artifacts published
secrets** in this public repo:
- the full test card (number, expiry, CVC) in screenshots, videos,
`error-context.md` and the DOM dump;
- `X-Admin-Token` in the trace network logs.

All 140 unexpired `e2e-*` artifacts have already been deleted via the
API. The admin token and the test card still need rotating (Andrew).

## Changes
- **Fail fast on decline:** `stripe-checkout.ts` watches
`/v1/payment_pages/<id>/confirm`. A `card_error` aborts `submitPayment`
immediately with `STRIPE_CARD_DECLINED: …`: no LLM calls, no second Pay
click.
- New `CheckoutStep.abortReason`, honoured by the runner, `pollVerify`
and the LLM loop.
  - Non-card errors (e.g. a missing field) still go to the LLM fallback.
- **Alerts say why:** Stage 1 exposes a `failure_reason` output ("Test
card declined, not a checkout bug (…)"). It appears in Discord and in
the daily email. The Discord payload is now built with `jq` from env, so
Stripe's text never reaches the shell.
- **No secrets in artifacts:**
- The Stripe helper's screenshots mask every input, and the DOM dump
redacts input values.
- The credit-purchase spec turns off trace, video and Playwright's
automatic screenshots.
- `playwright.production.config.ts` turns traces off on CI
(`E2E_TRACE=on` overrides).
  - New `frontend/e2e/scripts/scrub-artifacts.mjs`:
- deletes `*.zip` traces and `e2e-session-token.txt` (and videos with
`--drop-videos`);
    - redacts secret env values and card-number-shaped digit runs;
- with `--blank-inputs`, blanks `<input value>` and ARIA textbox values;
    - deletes any file that still contains a secret.
- Every production-E2E upload (`e2e-daily` ×2, `e2e-tenant-daily`, the
`ci.yml` happy path) now runs the scrubber first and **uploads only if
the scrub succeeded**.
- Docs: `TESTING.md` (decline behaviour, a "E2E Artifacts Are Public"
section), `LESSONS-LEARNED.md`.

## Testing
- `npx playwright test --config=playwright.selfheal.config.ts`: **30
passed**. New tests:
- a declined card fails in ~7s with 0 planner calls and exactly 1
confirm request;
  - the DOM dump contains no typed card values;
  - card-error parsing and the confirm-URL matcher;
- scrubber units (literal/grouped/PAN redaction, short secrets only
blanked inside inputs, deletion rules).
- Ran the scrubber against a copy of the real #200 artifacts: no card
data remains, and traces and videos were removed.
- actionlint: no new findings (only shellcheck info notes; ci.yml:996 is
pre-existing).
- Local CodeRabbit review: 2 findings (shell interpolation of the
reason; extension allowlist in the scrubber), both fixed.
- The full `make test` suite was not run: the change is limited to E2E
helpers, workflows and docs, with no backend or app code.

## Follow-ups (Andrew)
- Rotate the `admin-tokens` secret and the `E2E_ADMIN_TOKEN` repo
secret.
- Replace or unlock the test card; ideally a low-limit virtual card.
Then update the `E2E_STRIPE_CARD_*` secrets and re-run E2E Daily.

@coderabbitai ignore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 22:25:52 -04:00
..