feat(frontend): migrate from MailSlurp to testmail.app for E2E testing (#159)

## Summary

- Replace MailSlurp email testing service with testmail.app for E2E
magic link authentication tests
- Simpler API with namespace-based email addresses
(`{namespace}.{tag}@inbox.testmail.app`)
- No new dependencies - uses native `fetch` instead of mailslurp-client

## Changes

- **`frontend/e2e/helpers/email-testing.ts`** - Complete rewrite using
testmail.app JSON API
  - Uses `livequery=true` to wait for emails (no polling needed)
  - Generates unique tags per test run to avoid collisions
  - Same `EmailHelper` interface maintained for minimal test changes
- **`frontend/e2e/production/full-user-journey.spec.ts`** - Update env
var references and messages
- **`frontend/playwright.record.config.ts`** - Update env var comment
- **`frontend/package.json`** - Remove `mailslurp-client` dependency,
bump version to 0.79.0
- **`frontend/.env.local.example`** - New file documenting required env
vars for local testing
- **`docs/DEVELOPMENT.md`** - Add testmail.app env vars to frontend
section

## Environment Variables

| Old | New |
|-----|-----|
| `MAILSLURP_API_KEY` | `TESTMAIL_API_KEY` |
| (none) | `TESTMAIL_NAMESPACE` |

**Note:** GitHub Actions org secrets already configured with new env
vars.

## Testing

- [x] Backend tests pass (`make test` - 57 tests)
- [x] Frontend unit tests pass (139 tests)
- [x] Frontend E2E tests pass (64 tests)
- [x] TypeScript compilation clean for changed files

## Review

- [x] Local CodeRabbit CLI review completed
- [x] Review feedback addressed (removed exposed credentials from plan
file)

@coderabbitai ignore

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Andrew Beveridge ☄️andClaude authored and GitHub committed 2026-01-02 22:24:19 -05:00
1 parent ffa73c3194
commit dfb2ebc184
8 files changed
+268 -105

No files matched your search

@@ -0,0 +1,106 @@
# Plan: Migrate from MailSlurp to testmail.app
**Created:** 2026-01-02
**Branch:** feat/sess-20260102-2109-migrate-mailslurp-testmail
**Status:** Complete
## Overview
Replace MailSlurp email testing service with testmail.app for E2E testing of magic link authentication. testmail.app uses a simpler API with namespace-based email addresses that require no inbox creation/deletion - emails just arrive at `{namespace}.{tag}@inbox.testmail.app`.
## Requirements
- [x] Remove all code referencing MailSlurp
- [x] Remove `mailslurp-client` npm dependency
- [x] Implement testmail.app API integration
- [x] Update environment variable names (`TESTMAIL_API_KEY`, `TESTMAIL_NAMESPACE`)
- [x] Maintain same `EmailHelper` interface for minimal test changes
- [x] Update archive docs to note migration (optional - historical accuracy)
## Technical Approach
### API Differences
| Feature | MailSlurp | testmail.app |
|---------|-----------|--------------|
| Email format | Random `xxx@mailslurp.com` | `{namespace}.{tag}@inbox.testmail.app` |
| Inbox creation | Required (`createInbox()`) | Not needed - use any tag |
| Inbox deletion | Required (`deleteInbox()`) | Not needed |
| Get emails | `waitForLatestEmail(inboxId)` | GET `/api/json?apikey=X&namespace=Y&tag=Z&livequery=true` |
| Auth | API key in constructor | API key as query param |
| Email body | `email.body` | `email.html` or `email.text` |
### Implementation Strategy
1. **Keep same `EmailHelper` interface** - `createInbox()`, `waitForEmail()`, `extractMagicLink()`, `deleteInbox()`
2. **Use testmail.app's live query feature** - `livequery=true` waits for email to arrive (up to timeout)
3. **Generate unique tags** - Use timestamp + random suffix for each test run to avoid collisions
4. **Inbox becomes a simple object** - `{ id: tag, emailAddress: '{namespace}.{tag}@inbox.testmail.app' }`
5. **Delete becomes no-op** - testmail.app emails auto-expire, no cleanup needed
### Environment Variables
| Old | New |
|-----|-----|
| `MAILSLURP_API_KEY` | `TESTMAIL_API_KEY` |
| (none) | `TESTMAIL_NAMESPACE` |
Values: Set via environment variables (see `frontend/.env.local.example`)
## Implementation Steps
1. [x] **Update `email-testing.ts`** - Replace MailSlurp implementation with testmail.app
- Remove `mailslurp-client` import
- Add testmail.app API fetch implementation
- Use native `fetch` (no new dependencies)
- Generate unique tag for each "inbox"
- Implement `livequery=true` for waiting
- Adapt email body extraction (`html`/`text` instead of `body`)
2. [x] **Update `full-user-journey.spec.ts`** - Change references
- Update doc comments (env var name)
- Update error messages
- Update console logs
3. [x] **Update `playwright.record.config.ts`** - Change env var comment
4. [x] **Update `package.json`** - Remove `mailslurp-client` dependency
5. [x] **Run `npm install`** - Update package-lock.json
6. [x] **Create `.env.local.example`** - Document required env vars for local testing
7. [x] **Update archive docs** - Skipped intentionally; historical docs preserved for accuracy
## Files to Create/Modify
| File | Action | Description |
|------|--------|-------------|
| `frontend/e2e/helpers/email-testing.ts` | Modify | Replace MailSlurp with testmail.app |
| `frontend/e2e/production/full-user-journey.spec.ts` | Modify | Update env var refs and messages |
| `frontend/playwright.record.config.ts` | Modify | Update env var comment |
| `frontend/package.json` | Modify | Remove `mailslurp-client` |
| `frontend/package-lock.json` | Modify | Auto-updated by npm install |
| `frontend/.env.local.example` | Create | Document env vars for local testing |
## Testing Strategy
- **Manual testing**: Run production E2E test with testmail.app credentials
```bash
cd frontend
TESTMAIL_API_KEY=your-api-key \
TESTMAIL_NAMESPACE=your-namespace \
npm run test:e2e:prod:headed
```
- **Verify**: Email received, magic link extracted, auth flow completes
- **CI testing**: Env vars already configured in GitHub Actions org secrets
## Open Questions
None - requirements are clear.
## Rollback Plan
1. Revert the commit
2. Run `npm install` to restore mailslurp-client
3. Re-add MAILSLURP_API_KEY to secrets if removed
+5 -1
View File
@@ -151,7 +151,11 @@ ADMIN_TOKENS=token1,token2
```bash
NEXT_PUBLIC_API_URL=http://localhost:8000
KARAOKE_ACCESS_TOKEN=... # For E2E tests
# For E2E tests (see frontend/.env.local.example)
KARAOKE_ACCESS_TOKEN=... # Skip email enrollment
TESTMAIL_API_KEY=... # testmail.app API key
TESTMAIL_NAMESPACE=... # testmail.app namespace
```
## Key Scripts
+19
View File
@@ -0,0 +1,19 @@
# Environment variables for local development and testing
# Copy this file to .env.local and fill in your values
# =============================================================================
# E2E Testing - Email (for beta enrollment flow)
# =============================================================================
# testmail.app credentials for email testing
# Get these from https://testmail.app dashboard
TESTMAIL_API_KEY=your-api-key-here
TESTMAIL_NAMESPACE=your-namespace-here
# =============================================================================
# E2E Testing - Authentication (optional)
# =============================================================================
# Skip email enrollment by providing an existing access token
# Useful for faster local testing when you already have credits
# KARAOKE_ACCESS_TOKEN=your-token-here
+129 -28
View File
@@ -1,13 +1,13 @@
/**
* Email Testing Utilities using MailSlurp
* Email Testing Utilities using testmail.app
*
* MailSlurp provides disposable email inboxes for e2e testing.
* Free tier: 50 emails/month - perfect for CI/CD testing.
* testmail.app provides namespace-based email addresses for e2e testing.
* Emails arrive at: {namespace}.{tag}@inbox.testmail.app
*
* Setup:
* 1. Create account at https://app.mailslurp.com
* 2. Get API key from dashboard
* 3. Set MAILSLURP_API_KEY environment variable
* 1. Create account at https://testmail.app
* 2. Get API key and namespace from dashboard
* 3. Set TESTMAIL_API_KEY and TESTMAIL_NAMESPACE environment variables
*
* Usage in tests:
* const emailHelper = await createEmailHelper();
@@ -17,9 +17,27 @@
* const magicLink = emailHelper.extractMagicLink(email);
*/
import { MailSlurp, InboxDto, Email } from 'mailslurp-client';
const TESTMAIL_API_KEY = process.env.TESTMAIL_API_KEY;
const TESTMAIL_NAMESPACE = process.env.TESTMAIL_NAMESPACE;
const TESTMAIL_API_URL = 'https://api.testmail.app/api/json';
const MAILSLURP_API_KEY = process.env.MAILSLURP_API_KEY;
/**
* Inbox object compatible with existing test code
*/
export interface InboxDto {
id: string;
emailAddress: string;
}
/**
* Email object compatible with existing test code
*/
export interface Email {
subject?: string;
body?: string;
from?: string;
timestamp?: number;
}
export interface EmailHelper {
isAvailable: boolean;
@@ -30,40 +48,122 @@ export interface EmailHelper {
deleteInbox: (inboxId: string) => Promise<void>;
}
/**
* testmail.app API response structure
*/
interface TestmailResponse {
result: string;
message?: string;
count: number;
limit: number;
offset: number;
emails: TestmailEmail[];
}
interface TestmailEmail {
from: string;
subject: string;
html: string;
text: string;
timestamp: number;
tag: string;
}
/**
* Generate a unique tag for this test run
*/
function generateUniqueTag(): string {
const timestamp = Date.now();
const random = Math.random().toString(36).substring(2, 8);
return `test-${timestamp}-${random}`;
}
/**
* Creates an email testing helper.
* Returns a helper with isAvailable=false if MAILSLURP_API_KEY is not set.
* Returns a helper with isAvailable=false if testmail.app env vars are not set.
*/
export async function createEmailHelper(): Promise<EmailHelper> {
if (!MAILSLURP_API_KEY) {
console.warn('MAILSLURP_API_KEY not set - email testing disabled');
if (!TESTMAIL_API_KEY || !TESTMAIL_NAMESPACE) {
console.warn('TESTMAIL_API_KEY or TESTMAIL_NAMESPACE not set - email testing disabled');
return createDisabledHelper();
}
const mailslurp = new MailSlurp({ apiKey: MAILSLURP_API_KEY });
return {
isAvailable: true,
/**
* Create a new disposable inbox for testing
* Create a new "inbox" for testing.
* In testmail.app, this just generates a unique tag - no API call needed.
*/
async createInbox(): Promise<InboxDto> {
const inbox = await mailslurp.createInbox();
console.log(`Created test inbox: ${inbox.emailAddress}`);
return inbox;
const tag = generateUniqueTag();
const emailAddress = `${TESTMAIL_NAMESPACE}.${tag}@inbox.testmail.app`;
console.log(`Created test inbox: ${emailAddress}`);
return { id: tag, emailAddress };
},
/**
* Wait for an email to arrive in the inbox
* @param inboxId - The inbox ID to check
* @param inboxId - The tag to check (returned from createInbox as id)
* @param timeout - Timeout in ms (default: 60000 = 1 minute)
*/
async waitForEmail(inboxId: string, timeout = 60000): Promise<Email> {
console.log(`Waiting for email in inbox ${inboxId}...`);
const email = await mailslurp.waitForLatestEmail(inboxId, timeout);
console.log(`Received email: ${email.subject}`);
return email;
console.log(`Waiting for email with tag ${inboxId}...`);
// testmail.app livequery timeout is in seconds, max ~60s per request
// For longer timeouts, we poll multiple times
const pollInterval = Math.min(timeout, 30000); // 30s max per poll
const startTime = Date.now();
while (Date.now() - startTime < timeout) {
const remainingTime = timeout - (Date.now() - startTime);
const waitTime = Math.min(pollInterval, remainingTime);
const waitSeconds = Math.ceil(waitTime / 1000);
const url = new URL(TESTMAIL_API_URL);
url.searchParams.set('apikey', TESTMAIL_API_KEY!);
url.searchParams.set('namespace', TESTMAIL_NAMESPACE!);
url.searchParams.set('tag', inboxId);
url.searchParams.set('livequery', 'true');
url.searchParams.set('wait', waitSeconds.toString());
try {
const response = await fetch(url.toString());
if (!response.ok) {
throw new Error(`testmail.app API error: ${response.status} ${response.statusText}`);
}
const data: TestmailResponse = await response.json();
if (data.result !== 'success') {
throw new Error(`testmail.app API error: ${data.message || 'Unknown error'}`);
}
if (data.emails && data.emails.length > 0) {
// Return the most recent email
const latestEmail = data.emails.sort((a, b) => b.timestamp - a.timestamp)[0];
console.log(`Received email: ${latestEmail.subject}`);
// Convert to our Email format (use html as body, fallback to text)
return {
subject: latestEmail.subject,
body: latestEmail.html || latestEmail.text,
from: latestEmail.from,
timestamp: latestEmail.timestamp,
};
}
} catch (error) {
// If it's a timeout or network error, continue polling
if (Date.now() - startTime < timeout) {
console.log(`Polling for email... (${Math.round((Date.now() - startTime) / 1000)}s elapsed)`);
continue;
}
throw error;
}
}
throw new Error(`Timed out waiting for email after ${timeout}ms`);
},
/**
@@ -135,21 +235,22 @@ export async function createEmailHelper(): Promise<EmailHelper> {
},
/**
* Delete an inbox after testing
* Delete an inbox after testing.
* In testmail.app, emails auto-expire - this is a no-op.
*/
async deleteInbox(inboxId: string): Promise<void> {
await mailslurp.deleteInbox(inboxId);
console.log(`Deleted inbox ${inboxId}`);
// testmail.app emails auto-expire, no cleanup needed
console.log(`Inbox ${inboxId} cleanup skipped (testmail.app auto-expires)`);
},
};
}
/**
* Creates a disabled helper for when MailSlurp is not configured
* Creates a disabled helper for when testmail.app is not configured
*/
function createDisabledHelper(): EmailHelper {
const notAvailable = () => {
throw new Error('Email testing not available - set MAILSLURP_API_KEY');
throw new Error('Email testing not available - set TESTMAIL_API_KEY and TESTMAIL_NAMESPACE');
};
return {
@@ -166,5 +267,5 @@ function createDisabledHelper(): EmailHelper {
* Check if email testing is available
*/
export function isEmailTestingAvailable(): boolean {
return !!MAILSLURP_API_KEY;
return !!(TESTMAIL_API_KEY && TESTMAIL_NAMESPACE);
}
@@ -25,7 +25,8 @@ import { createEmailHelper, isEmailTestingAvailable } from '../helpers/email-tes
* 10. Verify Outputs - Check download URLs
*
* Environment Variables:
* - MAILSLURP_API_KEY: For email testing (required for full flow)
* - TESTMAIL_API_KEY: For email testing (required for full flow)
* - TESTMAIL_NAMESPACE: Your testmail.app namespace
* - KARAOKE_ACCESS_TOKEN: Skip enrollment and use existing token
*/
@@ -189,11 +190,11 @@ test.describe('Production E2E - Full User Journey', () => {
});
console.log(' Authenticated successfully with existing token');
} else if (isEmailTestingAvailable()) {
console.log(' Starting beta enrollment with MailSlurp');
console.log(' Starting beta enrollment with testmail.app');
const emailHelper = await createEmailHelper();
if (!emailHelper.isAvailable) {
throw new Error('MailSlurp not available');
throw new Error('testmail.app not available');
}
// Create test inbox
@@ -249,7 +250,7 @@ test.describe('Production E2E - Full User Journey', () => {
}
} else {
throw new Error(
'No KARAOKE_ACCESS_TOKEN and no MAILSLURP_API_KEY - cannot authenticate'
'No KARAOKE_ACCESS_TOKEN and no TESTMAIL_API_KEY/TESTMAIL_NAMESPACE - cannot authenticate'
);
}
+2 -69
View File
@@ -1,12 +1,12 @@
{
"name": "karaoke-gen-frontend",
"version": "0.77.0",
"version": "0.78.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "karaoke-gen-frontend",
"version": "0.77.0",
"version": "0.78.0",
"dependencies": {
"@hookform/resolvers": "^3.10.0",
"@radix-ui/react-accordion": "1.2.2",
@@ -75,7 +75,6 @@
"concurrently": "^9.2.1",
"jest": "^29.7.0",
"jest-environment-jsdom": "^29.7.0",
"mailslurp-client": "^16.0.0",
"postcss": "^8.5",
"tailwindcss": "^4.1.9",
"ts-node": "^10.9.2",
@@ -4504,16 +4503,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/cross-fetch": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/cross-fetch/-/cross-fetch-4.1.0.tgz",
"integrity": "sha512-uKm5PU+MHTootlWEY+mZ4vvXoCn4fLQxT9dSc1sXVMSFkINTJVN8cAQROpwcKm8bJ/c7rgZVIBWzH5T78sNZZw==",
"dev": true,
"license": "MIT",
"dependencies": {
"node-fetch": "^2.7.0"
}
},
"node_modules/cross-spawn": {
"version": "7.0.6",
"resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.6.tgz",
@@ -7103,16 +7092,6 @@
"@jridgewell/sourcemap-codec": "^1.5.5"
}
},
"node_modules/mailslurp-client": {
"version": "16.0.0",
"resolved": "https://registry.npmjs.org/mailslurp-client/-/mailslurp-client-16.0.0.tgz",
"integrity": "sha512-4g3Zauq5vcHZ2HAanfLpdcXcsZhZj6ToL/hUY10YS1vHK2DGt9EEO43ms5wzLFHCMFOCsx6So1JMIER/n8rIXw==",
"dev": true,
"license": "MIT",
"dependencies": {
"cross-fetch": "^4.1.0"
}
},
"node_modules/make-dir": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/make-dir/-/make-dir-4.0.0.tgz",
@@ -7355,52 +7334,6 @@
"node": "^10 || ^12 || >=14"
}
},
"node_modules/node-fetch": {
"version": "2.7.0",
"resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.7.0.tgz",
"integrity": "sha512-c4FRfUm/dbcWZ7U+1Wq0AwCyFL+3nt2bEw05wfxSz+DWpWsitgmSgYmy2dQdWyKC1694ELPqMs/YzUSNozLt8A==",
"dev": true,
"license": "MIT",
"dependencies": {
"whatwg-url": "^5.0.0"
},
"engines": {
"node": "4.x || >=6.0.0"
},
"peerDependencies": {
"encoding": "^0.1.0"
},
"peerDependenciesMeta": {
"encoding": {
"optional": true
}
}
},
"node_modules/node-fetch/node_modules/tr46": {
"version": "0.0.3",
"resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz",
"integrity": "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw==",
"dev": true,
"license": "MIT"
},
"node_modules/node-fetch/node_modules/webidl-conversions": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz",
"integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==",
"dev": true,
"license": "BSD-2-Clause"
},
"node_modules/node-fetch/node_modules/whatwg-url": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz",
"integrity": "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==",
"dev": true,
"license": "MIT",
"dependencies": {
"tr46": "~0.0.3",
"webidl-conversions": "^3.0.0"
}
},
"node_modules/node-int64": {
"version": "0.4.0",
"resolved": "https://registry.npmjs.org/node-int64/-/node-int64-0.4.0.tgz",
+1 -2
View File
@@ -1,6 +1,6 @@
{
"name": "karaoke-gen-frontend",
"version": "0.78.0",
"version": "0.79.0",
"private": true,
"scripts": {
"build": "next build",
@@ -93,7 +93,6 @@
"concurrently": "^9.2.1",
"jest": "^29.7.0",
"jest-environment-jsdom": "^29.7.0",
"mailslurp-client": "^16.0.0",
"postcss": "^8.5",
"tailwindcss": "^4.1.9",
"ts-node": "^10.9.2",
+1 -1
View File
@@ -2,7 +2,7 @@ import { defineConfig, devices } from '@playwright/test';
import * as fs from 'fs';
import * as path from 'path';
// Load .env.local file if it exists (for KARAOKE_ACCESS_TOKEN, MAILSLURP_API_KEY)
// Load .env.local file if it exists (for KARAOKE_ACCESS_TOKEN, TESTMAIL_API_KEY, TESTMAIL_NAMESPACE)
const envLocalPath = path.join(__dirname, '.env.local');
if (fs.existsSync(envLocalPath)) {
const envContent = fs.readFileSync(envLocalPath, 'utf-8');