## Summary - E2E happy path test (run #87) failing because landing page beta enrollment was removed - Updated both production E2E tests (`happy-path-real-user.spec.ts`, `full-user-journey.spec.ts`) to use the new AuthDialog magic link signup flow - Updated `E2E-HAPPY-PATH-TEST-SPEC.md` to match ## Changes The beta program was removed from the landing page in #430 but the E2E tests still looked for "Beta Tester Program" text. The new flow: 1. Click "Sign Up Free" → opens AuthDialog 2. Enter email → click "Send Sign-In Link" 3. Wait for magic link email via testmail.app 4. Navigate to verify URL → authenticate → redirect to /app ## Test plan - [x] Local unit tests pass (483/483) - [ ] CI checks pass - [ ] E2E happy path test passes in CI (triggered by `e2e-happy-path` label) <!-- @coderabbitai ignore --> 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
8.5 KiB
E2E Happy Path Test Specification
STATUS: ✅ PASSING - Test completes all 12 steps in ~38 minutes. Last verified 2026-01-02.
IMPORTANT: This document defines the requirements for a comprehensive end-to-end test that validates the entire karaoke generation flow with REAL user interactions only. NO API shortcuts allowed.
Overview
Goal: A single Playwright test that validates the complete karaoke generation journey from first-time user signup through to distribution, using only browser interactions like a real user would.
Environment: Production (gen.nomadkaraoke.com / api.nomadkaraoke.com)
Automation: Runs daily via GitHub Actions and can be triggered on-demand.
Test Requirements
1. New User Registration (testmail.app)
- Navigate to landing page (
gen.nomadkaraoke.com) - Click "Sign Up Free" button on landing page
- Generate a unique test email address (testmail.app namespace)
- Enter the test email in the AuthDialog and click "Send Sign-In Link"
- Wait for magic-link email via testmail.app API
- Extract the magic link from the email
- Click/navigate to the magic link to authenticate
- Verify user is logged in and has credits
Note: When use_test_token=true workflow input is set, steps 1-9 above are skipped and the test uses a pre-authenticated token instead.
2. Create Karaoke Job (UI Only)
- Navigate to the karaoke creation page
- Enter artist name:
piri(test song with cached results) - Enter song title:
dog - Click "Search" or equivalent button
- Wait for search results to appear in the UI
3. Audio Selection & Customization (UI Only)
- Audio selection dialog/panel should appear
- View available audio options in the UI
- Click to select the first/best audio option
- Confirm the selection via UI button
- Enable private mode (check the
#guided-privatecheckbox) — test jobs useNOMADNPprefix to avoid consuming publicNOMADbrand codes and skip GDrive/YouTube uploads - Wait for job to transition to processing state
4. Wait for Lyrics Transcription
- Monitor job status in the UI (not via API polling)
- Wait for job to reach "awaiting lyrics review" state
- This may take 10-15+ minutes for audio download + transcription
5. Lyrics Review (FULL UI Interaction)
CRITICAL: This MUST be done through the UI, not via API calls
- Click to open the lyrics review interface
- Verify lyrics are displayed in the editor
- Interact with the lyrics editor:
- View synced lyrics with timestamps
- Optionally make a small edit to prove UI interaction works
- Preview the video if preview functionality exists
- Click "Approve" / "Submit" / "Complete Review" button in UI
- Wait for job to transition to next state
6. Instrumental Selection (FULL UI Interaction)
CRITICAL: This MUST be done through the UI, not via API calls
- Wait for instrumental options to become available
- Instrumental selection dialog/panel should appear in UI
- View available instrumental options (e.g., "clean", "with backing vocals")
- Click to select the preferred instrumental option
- Confirm selection via UI button
- Wait for job to transition to video rendering state
7. Wait for Video Rendering & Encoding
- Monitor job progress in the UI
- Wait for video rendering to complete
- Wait for final encoding to complete
- Job should reach "complete" status
8. Verify Completion & Downloads
- Verify job shows as "complete" in the UI
- Verify download links are visible and clickable
- Click at least one download link to verify it works
- Verify the downloaded file is valid (non-zero size, correct format)
9. Distribution Verification & Cleanup
CRITICAL: Test MUST clean up after itself to avoid polluting production distribution channels
- Verify job.state_data contains distribution results:
brand_code- The generated brand code (e.g.,NOMADNP-0042for private mode)dropbox_link- Dropbox shared link (if Dropbox enabled)- YouTube and GDrive should NOT be present (private mode skips them)
- CLEANUP - After verification, clean up via
cleanup-distributionendpoint:- Deletes Dropbox folder and recycles brand code
- YouTube/GDrive cleanup skipped for private jobs
- Delete the test job from the system (GCS + Firestore)
Technical Requirements
Timeouts
| Stage | Timeout |
|---|---|
| Page navigation | 30 seconds |
| Email arrival | 120 seconds |
| Audio search | 60 seconds |
| Lyrics transcription | 20 minutes |
| Video rendering | 15 minutes |
| Final encoding | 10 minutes |
| Total test | 60 minutes |
Environment Variables
TESTMAIL_NAMESPACE=xxx # Required - for email testing (testmail.app)
TESTMAIL_API_KEY=xxx # Required - for email testing (testmail.app)
E2E_TEST_TOKEN=xxx # Optional - skip signup, use existing account
# When use_test_token=true, skips email signup and uses pre-authenticated token
Test Song
Use piri - dog because:
- Has cached flacfetch results (faster audio search)
- Known to work reliably
- Short song = faster processing
Screenshots & Artifacts
Capture screenshots at each major milestone:
- Landing page loaded
- Magic link sent
- Email received
- Logged in successfully
- Job created
- Audio selected
- Lyrics review opened
- Lyrics approved
- Instrumental selected
- Job completed
- Download verified
Logging
Log detailed progress at each step with timestamps for debugging.
GitHub Actions Workflow
File: .github/workflows/e2e-happy-path.yml
Triggers
- Daily schedule: Run at 6 AM UTC every day
- Manual dispatch: Can be triggered on-demand with optional parameters
- Label trigger: When
e2e-fulllabel added to PR
Secrets Required
TESTMAIL_API_KEY- For email testing (testmail.app)TESTMAIL_NAMESPACE- testmail.app namespace for unique email addresses
Workflow Steps
- Checkout code
- Setup Node.js
- Install dependencies
- Install Playwright browsers
- Run the e2e happy path test
- Upload artifacts (screenshots, videos, logs)
- Report results
Success Criteria
- All UI interactions complete successfully
- Job reaches "complete" status
- Download link works
- No JavaScript errors in console
- All screenshots captured
What This Test Proves
When this test passes, it proves:
- User signup works - New users can register via magic link
- Authentication works - Email verification and login flow functional
- Job creation works - Can search and create karaoke jobs
- Audio pipeline works - Audio search, selection, and download functional
- Transcription works - Lyrics transcription completes successfully
- Lyrics review UI works - Users can review and approve lyrics
- Instrumental selection UI works - Users can select instrumental versions
- Video rendering works - Full video generation pipeline functional
- Downloads work - Users can download their completed karaoke videos
- End-to-end integration - All systems work together correctly
Anti-Patterns to Avoid
- NO API calls to complete review - Must click buttons in UI
- NO API calls to select instrumental - Must click buttons in UI
- NO pre-authenticated tokens - Must go through full signup
- NO skipping steps - Every user-facing interaction must happen
- NO hardcoded waits - Use proper Playwright waiting strategies
File Location
Test file: frontend/e2e/production/happy-path-real-user.spec.ts
This is separate from the existing full-user-journey.spec.ts which uses API shortcuts.
Iteration Plan
Create initial test with all steps stubbedImplement each section one at a timeRun via GHA workflowDebug failures using artifactsIterate until fully passing✅ Completed 2026-01-02- Add to required CI checks once stable
Key Fixes Applied
During iteration, three issues were fixed:
- STEP 8 status detection: "rendering" status comes BEFORE instrumental selection, not after. Changed to only skip on "encoding".
- Instrumental page navigation: Changed
waitUntil: 'networkidle'to'load'because audio players prevent networkidle. - Download verification: Changed HEAD to GET request with
maxRedirects: 0since download endpoint doesn't support HEAD.
See LESSONS-LEARNED.md for details.
Last updated: 2026-01-02 Author: Claude (via Andrew's request)