mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
fix(testing): Unify test setup with auto-install and clearer guidance (#262)
## Summary - **`make test` now runs ALL tests** (backend + frontend) and auto-installs dependencies if missing - Added clear "All tests must pass" guidance - agents should never dismiss failures as "pre-existing" - Fixed misplaced test that was always failing (tested wrong frontend) - Fixed logo locator bug in landing-page test ## Changes | File | Change | |------|--------| | `Makefile` | Auto-install deps, unified `make test`, added `test-backend`/`test-frontend` | | `CLAUDE.md` | Simplified to just `make test` | | `docs/DEVELOPMENT.md` | Consolidated testing section | | `docs/TESTING.md` | Updated guidance and quick reference | | `frontend/e2e/regression/landing-page.spec.ts` | Fixed `getByText` → `getByAltText` for logo | | `frontend/e2e/manual/` | Moved `lyrics-review-mobile.spec.ts` here with README | ## Test plan - [x] `make test` passes all 153 tests (76 backend + 77 frontend) - [x] Auto-install works in fresh worktree - [x] Documentation is clear and consistent @coderabbitai ignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
98b4636006
commit
49158f99ac
7 files changed
+133
-65
No files matched your search
@@ -46,13 +46,17 @@
|
||||
|
||||
**Before committing**, run all tests:
|
||||
```bash
|
||||
# Backend (10 min timeout, limit output)
|
||||
make test 2>&1 | tail -n 500
|
||||
|
||||
# Frontend
|
||||
cd frontend && npm run test:all 2>&1 | tail -n 200
|
||||
```
|
||||
|
||||
This single command:
|
||||
- Installs dependencies automatically (backend + frontend)
|
||||
- Runs all backend tests (unit, integration, emulator)
|
||||
- Runs all frontend tests (Jest unit + Playwright E2E)
|
||||
- Takes ~5-10 minutes
|
||||
|
||||
**All tests must pass.** Don't dismiss failures as "pre-existing" - investigate and fix them.
|
||||
|
||||
### Version Bumping
|
||||
- Bump `tool.poetry.version` in `pyproject.toml` for code changes
|
||||
- Skip for docs-only changes
|
||||
|
||||
@@ -1,34 +1,50 @@
|
||||
# Makefile for karaoke-gen project
|
||||
# Run `make help` to see available commands
|
||||
|
||||
.PHONY: help test test-unit test-backend test-integration test-e2e test-all lint clean emulators-start emulators-stop
|
||||
.PHONY: help install install-backend install-frontend test test-unit test-backend test-e2e test-frontend test-all lint clean emulators-start emulators-stop
|
||||
|
||||
# Default target
|
||||
help:
|
||||
@echo "Available commands:"
|
||||
@echo " make test - Run all tests (unit + backend + emulator-based)"
|
||||
@echo " make test-unit - Run unit tests only (karaoke_gen)"
|
||||
@echo " make test-backend - Run backend unit tests only"
|
||||
@echo " make test-e2e - Run E2E tests with emulators (starts/stops automatically)"
|
||||
@echo " make test-all - Alias for 'make test'"
|
||||
@echo " make test - Run ALL tests (backend + frontend, installs deps automatically)"
|
||||
@echo " make test-backend - Run backend tests only (unit + emulator)"
|
||||
@echo " make test-frontend - Run frontend tests only (unit + E2E)"
|
||||
@echo " make test-unit - Run unit tests only (karaoke_gen package)"
|
||||
@echo " make test-e2e - Run emulator tests with auto-start/stop"
|
||||
@echo " make install - Install all dependencies (backend + frontend)"
|
||||
@echo " make lint - Run linter checks"
|
||||
@echo " make emulators-start - Start GCP emulators for local development"
|
||||
@echo " make emulators-stop - Stop GCP emulators"
|
||||
@echo ""
|
||||
@echo "Before committing, run: make test"
|
||||
|
||||
# Install dependencies (only if needed)
|
||||
install-backend:
|
||||
@if [ ! -d "$$(poetry env info --path 2>/dev/null)" ] || ! poetry run python -c "import fastapi" 2>/dev/null; then \
|
||||
echo "=== Installing backend dependencies ==="; \
|
||||
poetry install; \
|
||||
fi
|
||||
|
||||
install-frontend:
|
||||
@if [ ! -d "frontend/node_modules" ]; then \
|
||||
echo "=== Installing frontend dependencies ==="; \
|
||||
cd frontend && npm install; \
|
||||
fi
|
||||
|
||||
install: install-backend install-frontend
|
||||
|
||||
# Run unit tests for karaoke_gen package
|
||||
test-unit:
|
||||
test-unit: install-backend
|
||||
@echo "=== Running karaoke_gen unit tests ==="
|
||||
poetry run pytest tests/unit/ -v --cov=karaoke_gen --cov-report=term-missing --cov-fail-under=69
|
||||
|
||||
# Run backend unit tests
|
||||
test-backend:
|
||||
# Run backend unit tests (excludes emulator tests)
|
||||
test-backend-unit: install-backend
|
||||
@echo "=== Running backend unit tests ==="
|
||||
poetry run pytest backend/tests/ --ignore=backend/tests/emulator -v
|
||||
|
||||
# Run E2E integration tests with emulators
|
||||
test-e2e:
|
||||
test-e2e: install-backend
|
||||
@echo "=== Running E2E integration tests with emulators ==="
|
||||
@./scripts/start-emulators.sh || (echo "Failed to start emulators" && exit 1)
|
||||
@export FIRESTORE_EMULATOR_HOST=127.0.0.1:8080 && \
|
||||
@@ -42,8 +58,18 @@ test-e2e:
|
||||
./scripts/stop-emulators.sh; \
|
||||
exit $$TEST_RESULT
|
||||
|
||||
# Run all tests (use this before committing!)
|
||||
test: test-unit test-backend test-e2e
|
||||
# Run all backend tests (unit + emulator)
|
||||
test-backend: test-unit test-backend-unit test-e2e
|
||||
@echo ""
|
||||
@echo "✅ Backend tests passed!"
|
||||
|
||||
# Run frontend tests (unit + E2E)
|
||||
test-frontend: install-frontend
|
||||
@echo "=== Running frontend tests ==="
|
||||
cd frontend && npm run test:all
|
||||
|
||||
# Run ALL tests (backend + frontend) - use this before committing!
|
||||
test: test-backend test-frontend
|
||||
@echo ""
|
||||
@echo "✅ All tests passed! Ready to commit."
|
||||
|
||||
|
||||
+10
-38
@@ -60,45 +60,21 @@ npm run dev
|
||||
|
||||
## Testing
|
||||
|
||||
### Backend Tests
|
||||
**See [TESTING.md](TESTING.md) for complete testing guidance.**
|
||||
|
||||
Quick reference (run before committing):
|
||||
|
||||
```bash
|
||||
# Run all tests (required before commit)
|
||||
make test 2>&1 | tail -n 500
|
||||
|
||||
# This runs in order:
|
||||
# 1. Unit tests with coverage
|
||||
# 2. Backend unit tests
|
||||
# 3. E2E integration tests with emulators
|
||||
```
|
||||
|
||||
**Important**: Set 10 minute timeout. Tests can take a while with emulators.
|
||||
This runs ALL tests (backend + frontend) and installs dependencies automatically.
|
||||
|
||||
### Frontend Tests
|
||||
For faster iteration, run subsets:
|
||||
- `make test-backend` - Backend only (~2 min)
|
||||
- `make test-frontend` - Frontend only (~3 min)
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm run test:all 2>&1 | tail -n 200
|
||||
|
||||
# Individual commands:
|
||||
npm run test:unit # Jest unit tests
|
||||
npm run test:e2e # Playwright E2E tests
|
||||
```
|
||||
|
||||
### Test Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────┐
|
||||
│ Unit Tests (Mocked) │ Fast, isolated
|
||||
│ ~62 tests, <1s │
|
||||
├─────────────────────────────────┤
|
||||
│ Emulator Integration Tests │ Local Firestore/GCS
|
||||
│ ~11 tests, ~2s │
|
||||
├─────────────────────────────────┤
|
||||
│ E2E Tests │ Full API + Frontend
|
||||
│ Requires deployed backend │
|
||||
└─────────────────────────────────┘
|
||||
```
|
||||
TESTING.md covers: test types, CI requirements, Playwright usage, mocking guidelines, and coverage enforcement.
|
||||
|
||||
## Deployment
|
||||
|
||||
@@ -231,10 +207,6 @@ lsof -i :4443 # GCS
|
||||
./scripts/stop-emulators.sh
|
||||
```
|
||||
|
||||
### Tests Fail with Auth Errors
|
||||
|
||||
Ensure `ADMIN_TOKENS` is set in environment or `.env` file.
|
||||
|
||||
### Docker Issues
|
||||
|
||||
```bash
|
||||
@@ -245,6 +217,6 @@ docker info
|
||||
docker pull fsouza/fake-gcs-server
|
||||
```
|
||||
|
||||
### Coverage Below Target
|
||||
### Test Failures
|
||||
|
||||
Run `./scripts/run-tests.sh --coverage` and check `htmlcov/index.html` to see uncovered lines.
|
||||
See [TESTING.md](TESTING.md#debugging-failed-tests) for debugging test failures, coverage issues, and CI troubleshooting.
|
||||
+42
-10
@@ -2,6 +2,29 @@
|
||||
|
||||
This document defines testing standards, code quality principles, and CI requirements for Nomad Karaoke Gen.
|
||||
|
||||
## Critical: All Tests Must Pass
|
||||
|
||||
**All tests must pass before merging.** If tests fail:
|
||||
|
||||
1. **Fix them** - Don't assume failures are "pre-existing issues"
|
||||
2. **Investigate** - Understand why the test is failing
|
||||
3. **Don't dismiss** - If a test fails, either the code or test needs fixing
|
||||
|
||||
CI blocks merging if tests fail. If you encounter failures locally, fix them as part of your work.
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run ALL tests (backend + frontend) - installs deps automatically
|
||||
make test
|
||||
|
||||
# Run subsets for faster iteration
|
||||
make test-backend # Backend only (~2 min)
|
||||
make test-frontend # Frontend only (~3 min)
|
||||
```
|
||||
|
||||
Dependencies are installed automatically when needed.
|
||||
|
||||
## Core Principles
|
||||
|
||||
### SOLID Principles
|
||||
@@ -96,19 +119,25 @@ make emulators-stop
|
||||
**Locations:**
|
||||
- `frontend/e2e/regression/` - Mocked API tests (CI-safe, fast)
|
||||
- `frontend/e2e/production/` - Real backend tests (comprehensive, slower)
|
||||
- `frontend/e2e/manual/` - Tests requiring manual setup (not in CI)
|
||||
|
||||
**Tool:** Playwright
|
||||
|
||||
**Regression tests** run on every PR:
|
||||
- Mock API responses via fixtures
|
||||
- Test UI behavior in isolation
|
||||
- ~1-2 min total runtime
|
||||
- Playwright auto-starts dev server via `webServer` config
|
||||
- ~3 min total runtime
|
||||
|
||||
**Production tests** run on-demand or daily:
|
||||
- Hit real `https://gen.nomadkaraoke.com` and `https://api.nomadkaraoke.com`
|
||||
- Full user journeys (20-40 min for happy path)
|
||||
- Require authentication tokens
|
||||
|
||||
**Manual tests** are NOT included in CI:
|
||||
- Require separate dev servers (e.g., LyricsTranscriber frontend on port 5173)
|
||||
- See `frontend/e2e/manual/README.md` for setup instructions
|
||||
|
||||
## Playwright Usage
|
||||
|
||||
### Installation
|
||||
@@ -353,10 +382,13 @@ karaoke-gen/
|
||||
│ │ ├── authentication.spec.ts
|
||||
│ │ ├── karaoke-generation.spec.ts
|
||||
│ │ └── ...
|
||||
│ └── production/ # Real backend E2E
|
||||
│ ├── happy-path-real-user.spec.ts
|
||||
│ ├── lyrics-review-only.spec.ts
|
||||
│ └── admin-dashboard.spec.ts
|
||||
│ ├── production/ # Real backend E2E
|
||||
│ │ ├── happy-path-real-user.spec.ts
|
||||
│ │ ├── lyrics-review-only.spec.ts
|
||||
│ │ └── admin-dashboard.spec.ts
|
||||
│ └── manual/ # Manual setup required (NOT in CI)
|
||||
│ ├── README.md # Setup instructions
|
||||
│ └── lyrics-review-mobile.spec.ts # LyricsTranscriber frontend
|
||||
├── jest.config.js
|
||||
├── jest.setup.js
|
||||
├── playwright.config.ts # Default (regression)
|
||||
@@ -489,13 +521,13 @@ When adding a new feature:
|
||||
|
||||
| Task | Command |
|
||||
|------|---------|
|
||||
| All tests | `make test` |
|
||||
| Package unit | `make test-unit` |
|
||||
| **All tests (before commit)** | `make test` |
|
||||
| Backend only | `make test-backend` |
|
||||
| With emulators | `make test-e2e` |
|
||||
| Frontend unit | `cd frontend && npm run test:unit` |
|
||||
| Frontend E2E (regression) | `cd frontend && npm run test:e2e` |
|
||||
| Frontend only | `make test-frontend` |
|
||||
| Package unit | `make test-unit` |
|
||||
| Emulator tests | `make test-e2e` |
|
||||
| Frontend E2E (production) | `cd frontend && npm run test:e2e:prod` |
|
||||
| Coverage HTML | `poetry run pytest tests/unit --cov=karaoke_gen --cov-report=html` |
|
||||
| Install deps | `make install` |
|
||||
| Start emulators | `make emulators-start` |
|
||||
| Stop emulators | `make emulators-stop` |
|
||||
@@ -0,0 +1,33 @@
|
||||
# Manual E2E Tests
|
||||
|
||||
Tests in this directory require **manual setup** before running. They are NOT included in CI or `npm run test:e2e`.
|
||||
|
||||
## Why Manual?
|
||||
|
||||
These tests target components with separate dev servers that aren't auto-started by Playwright's `webServer` config.
|
||||
|
||||
## Tests
|
||||
|
||||
### `lyrics-review-mobile.spec.ts`
|
||||
|
||||
Tests the **LyricsTranscriber frontend** (Vite, port 5173), not the main karaoke-gen frontend (Next.js, port 3000).
|
||||
|
||||
**To run:**
|
||||
|
||||
```bash
|
||||
# Terminal 1: Start LyricsTranscriber frontend dev server
|
||||
cd lyrics_transcriber_temp/lyrics_transcriber/frontend
|
||||
yarn dev # Starts on port 5173
|
||||
|
||||
# Terminal 2: Start LyricsTranscriber backend (for API)
|
||||
# This serves the review API on port 8767
|
||||
poetry run python -c "from lyrics_transcriber.review.server import ReviewServer; ..."
|
||||
|
||||
# Terminal 3: Run the test
|
||||
cd frontend
|
||||
npx playwright test e2e/manual/lyrics-review-mobile.spec.ts
|
||||
```
|
||||
|
||||
## Future
|
||||
|
||||
These tests should be consolidated when LyricsTranscriber is fully integrated into the main karaoke-gen frontend. See CLAUDE.md for project context.
|
||||
File renamed without changes.
@@ -23,7 +23,8 @@ test.describe('Landing Page', () => {
|
||||
// Check hero content
|
||||
await expect(page.locator('h1')).toContainText('Karaoke Video');
|
||||
await expect(page.locator('nav')).toBeVisible();
|
||||
await expect(page.getByRole('navigation').getByText('Nomad Karaoke')).toBeVisible();
|
||||
// Logo is an image with alt text, not visible text
|
||||
await expect(page.getByRole('navigation').getByAltText('Nomad Karaoke')).toBeVisible();
|
||||
});
|
||||
|
||||
test('displays pricing section with 4 packages', async ({ page }) => {
|
||||
|
||||
Reference in new issue
Block a user