Compare commits

...
Author SHA1 Message Date
Andrew BeveridgeandClaude Opus 4.5 97f1a7640b feat: add production debug script template for agents
Add infrastructure for ad-hoc production testing:
- Debug script template at frontend/e2e/helpers/debug-prod-template.mjs
- Gitignore patterns for test-*.local.* and debug-*.local.* files
- Documentation in CLAUDE.md, TESTING.md, and DEVELOPMENT.md

This allows agents to quickly test production issues when asked to
"test it yourself in prod" without risking token exposure in commits.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-20 10:15:25 -08:00
5 changed files with 283 additions and 0 deletions

No files matched your search

+24
View File
@@ -60,6 +60,30 @@ This single command:
**All tests must pass.** Don't dismiss failures as "pre-existing" - investigate and fix them.
### Testing in Production (for Agents)
When asked to "test it yourself in prod" or verify a fix in production:
```bash
# Get admin token
export KARAOKE_ADMIN_TOKEN=$(gcloud secrets versions access latest --secret=admin-tokens --project=nomadkaraoke | cut -d',' -f1)
# Option 1: Use the debug template directly
cd frontend && node e2e/helpers/debug-prod-template.mjs
# Option 2: Copy template for customization (gitignored)
cp frontend/e2e/helpers/debug-prod-template.mjs frontend/test-my-issue.local.mjs
# Edit the script, then run it
node test-my-issue.local.mjs
# Option 3: Run existing production E2E tests
KARAOKE_ADMIN_TOKEN=$KARAOKE_ADMIN_TOKEN npx playwright test e2e/production/admin-dashboard.spec.ts
```
Files matching `test-*.local.*` and `debug-*.local.*` are gitignored - you can hardcode tokens in them safely.
See `docs/TESTING.md` § "Ad-Hoc Production Debugging" for full details.
### Version Bumping
- Bump `tool.poetry.version` in `pyproject.toml` for code changes
- Skip for docs-only changes
+2
View File
@@ -76,6 +76,8 @@ For faster iteration, run subsets:
TESTING.md covers: test types, CI requirements, Playwright usage, mocking guidelines, and coverage enforcement.
**For ad-hoc production debugging**, see the "Ad-Hoc Production Debugging" section in TESTING.md. Use the template at `frontend/e2e/helpers/debug-prod-template.mjs`.
## Deployment
### Automatic (CI/CD)
+80
View File
@@ -402,6 +402,86 @@ The production tests cover:
- Final encoding and download
- Admin dashboard functionality
## Ad-Hoc Production Debugging (for Agents)
When asked to "test it yourself in prod" or debug a specific production issue, use the debug script template:
### Quick Start
```bash
# Copy the template (the .local suffix is gitignored)
cp frontend/e2e/helpers/debug-prod-template.mjs frontend/test-my-issue.local.mjs
# Edit the script to customize what you're testing
# You can hardcode tokens in .local files - they won't be committed
# Run it
cd frontend && node test-my-issue.local.mjs
```
### With Environment Variables (Preferred)
```bash
# Get admin token from Secret Manager
export KARAOKE_ADMIN_TOKEN=$(gcloud secrets versions access latest --secret=admin-tokens --project=nomadkaraoke | cut -d',' -f1)
# Run the template directly
cd frontend && node e2e/helpers/debug-prod-template.mjs
# Or with a specific job
DEBUG_JOB_ID=abc123 node e2e/helpers/debug-prod-template.mjs
```
### What the Template Provides
- Playwright browser automation against production
- Auth token injection (via localStorage)
- Console error capture
- HTTP error capture (4xx, 5xx responses)
- Screenshot capability
- Customizable debug steps
### Gitignore Patterns
Files matching these patterns are gitignored in `frontend/`:
- `test-*.local.*` - Ad-hoc test scripts
- `debug-*.local.*` - Debug scripts
**Never commit tokens.** Use env vars or `.local` files only.
### When to Use What
| Scenario | Approach |
|----------|----------|
| Quick one-off debugging | Copy template → `.local` file |
| Reusable production test | Add to `e2e/production/` with env var auth |
| Regression test (mocked) | Add to `e2e/regression/` |
### Example: Debug a Specific Job's Review Page
```javascript
// frontend/test-review-debug.local.mjs
import { chromium } from 'playwright';
const TOKEN = 'your-token-here'; // OK in .local files
const JOB_ID = '26391a79';
async function debug() {
const browser = await chromium.launch({ headless: false }); // See the browser
const page = await browser.newContext().then(c => c.newPage());
await page.addInitScript(t => localStorage.setItem('karaoke_access_token', t), TOKEN);
await page.goto(`https://gen.nomadkaraoke.com/app/jobs/${JOB_ID}/review/`);
// Debug interactively - the browser stays open
await page.pause(); // Opens Playwright inspector
await browser.close();
}
debug();
```
## Running Tests Locally
```bash
+5
View File
@@ -1 +1,6 @@
playwright-report-prod/
# Local debug scripts (may contain tokens - never commit)
# Use this pattern for ad-hoc testing: test-*.local.{js,cjs,mjs,ts}
test-*.local.*
debug-*.local.*
@@ -0,0 +1,172 @@
/**
* Production Debug Script Template
*
* Use this template when you need to quickly test/debug something in production.
* Copy this file and rename it with the .local suffix so it won't be committed:
*
* cp frontend/e2e/helpers/debug-prod-template.mjs frontend/test-my-issue.local.mjs
*
* Then customize and run:
* cd frontend && node test-my-issue.local.mjs
*
* IMPORTANT: Files matching test-*.local.* and debug-*.local.* are gitignored,
* so you can safely put tokens in them for quick debugging.
*
* For proper reusable tests, use the existing E2E infrastructure instead:
* - frontend/e2e/production/ - Production E2E tests with env var auth
* - See docs/TESTING.md for guidance
*
* Environment variables (set these or hardcode in your .local copy):
* KARAOKE_ADMIN_TOKEN - Admin user token (for admin pages)
* KARAOKE_ACCESS_TOKEN - Regular user token (for user pages)
* DEBUG_JOB_ID - Job ID to test (if applicable)
*/
import { chromium } from 'playwright';
// ============================================================================
// CONFIGURATION - Customize these for your debugging session
// ============================================================================
const BASE_URL = 'https://gen.nomadkaraoke.com';
const API_URL = 'https://api.nomadkaraoke.com';
// Get tokens from environment (recommended) or hardcode in your .local copy
const ADMIN_TOKEN = process.env.KARAOKE_ADMIN_TOKEN || process.env.KARAOKE_ACCESS_TOKEN;
const JOB_ID = process.env.DEBUG_JOB_ID || null;
// What page to test - customize this
const TARGET_PAGE = JOB_ID
? `${BASE_URL}/app/jobs/${JOB_ID}`
: `${BASE_URL}/app`;
// ============================================================================
// DEBUG SCRIPT
// ============================================================================
async function debug() {
if (!ADMIN_TOKEN) {
console.error('ERROR: No token found.');
console.error('Set KARAOKE_ADMIN_TOKEN or KARAOKE_ACCESS_TOKEN environment variable,');
console.error('or hardcode a token in your .local copy of this script.');
process.exit(1);
}
console.log('\n=== Production Debug Session ===\n');
console.log(`Target: ${TARGET_PAGE}`);
console.log(`Token: ${ADMIN_TOKEN.substring(0, 8)}...`);
if (JOB_ID) console.log(`Job ID: ${JOB_ID}`);
console.log('');
const browser = await chromium.launch({
headless: true, // Set to false to see the browser
});
const context = await browser.newContext();
const page = await context.newPage();
// Collect errors for summary
const consoleErrors = [];
const httpErrors = [];
// Capture console errors
page.on('console', (msg) => {
const text = msg.text();
const type = msg.type();
if (type === 'error' || text.toLowerCase().includes('error')) {
consoleErrors.push({ type, text: text.substring(0, 300) });
}
});
// Capture HTTP errors (4xx, 5xx)
page.on('response', (response) => {
const status = response.status();
if (status >= 400) {
httpErrors.push({
url: response.url(),
status,
statusText: response.statusText(),
});
}
});
try {
// Set auth token before navigation
await page.addInitScript((token) => {
localStorage.setItem('karaoke_access_token', token);
}, ADMIN_TOKEN);
// Navigate to target page
console.log('1. Navigating to page...');
await page.goto(TARGET_PAGE, { waitUntil: 'networkidle', timeout: 30000 });
console.log(` URL: ${page.url()}`);
// Wait for any async loading
await page.waitForTimeout(3000);
// ========================================================================
// CUSTOMIZE YOUR DEBUG STEPS HERE
// ========================================================================
// Example: Check what's on the page
const bodyText = await page.evaluate(() => document.body.innerText);
console.log('\n2. Page content (first 500 chars):');
console.log(bodyText.substring(0, 500));
console.log('...');
// Example: List all buttons
const buttons = await page.getByRole('button').allTextContents();
console.log('\n3. Buttons on page:', buttons.slice(0, 10).join(', '));
// Example: List all links
const links = await page.evaluate(() =>
Array.from(document.querySelectorAll('a'))
.slice(0, 10)
.map((a) => ({ href: a.href, text: a.innerText.trim().substring(0, 30) }))
);
console.log('\n4. Links on page:', JSON.stringify(links, null, 2));
// Example: Take a screenshot
await page.screenshot({ path: 'debug-screenshot.png', fullPage: true });
console.log('\n5. Screenshot saved to debug-screenshot.png');
// Example: Click something (uncomment and customize)
// const btn = page.getByRole('button', { name: /some button/i });
// if (await btn.isVisible()) {
// await btn.click();
// await page.waitForTimeout(2000);
// console.log('Clicked button, new URL:', page.url());
// }
// ========================================================================
// END CUSTOM STEPS
// ========================================================================
} catch (e) {
console.error('\nERROR:', e.message);
}
// Summary
console.log('\n--- Console Errors ---');
if (consoleErrors.length === 0) {
console.log('None');
} else {
consoleErrors.forEach((err, i) => {
console.log(`${i + 1}. [${err.type}] ${err.text}`);
});
}
console.log('\n--- HTTP Errors ---');
if (httpErrors.length === 0) {
console.log('None');
} else {
httpErrors.forEach((err, i) => {
console.log(`${i + 1}. ${err.status} ${err.statusText}: ${err.url}`);
});
}
await browser.close();
console.log('\n=== Debug session complete ===\n');
}
debug().catch(console.error);