Skip to main content
The Soundcheck web app uses Vitest for unit tests and Playwright for end-to-end tests.

Testing stack

Vitest configs

Setup files: vitest.setup.env.ts, vitest.setup.crypto.ts and vitest.setup.ts (web).

Unit tests

Running tests

Test file location

Unit tests are co-located with source files or in __tests__ directories:
Shared lib/ tests live at the repo root (e.g. lib/api/httpClient.test.ts).

Example unit test


E2E tests

Running E2E tests

npm run test:e2e -- --shard=N/M does not use Playwright’s count split. web/scripts/run-local-e2e.mjs rewrites it into a duration-weighted file list from automated-tests/shard-durations.json so long specs do not pile onto one shard. CI writes playwright-report/results.json per shard; refresh the weights with node scripts/update-shard-durations.mjs <results.json...> after adding or reshaping specs, using the artifacts from a push-to-dev (full suite) run. The script keeps a file’s old weight unless every test declared in it ran and passed, so a PR run’s critical-tier subset cannot shrink a heavy file. Third-party services in e2e run against local emulators: the wrapper starts Google Maps, Spotify, and context.dev from tools/emulators and blanks analytics and payment scripts (see tools/emulators/README.md, “Web app”). E2E_WEB_EMULATORS=off uses the real services. Feature flags in e2e come from flags/registry.json: flags whose web surface sets "e2e": true are on, everything else is off (see flags/README.md). Specs read the same state with isE2eFlagOn from automated-tests/utils/featureFlags.ts. Tests tagged @pr-screenshots only capture PR images, so the wrapper skips them unless E2E_PR_SCREENSHOTS=1 or you name the spec; the nightly suite sets it. Auth setup signs in through Clerk’s testing API on /dev (not /sign-in, whose hosted <SignIn /> breaks ticket redemption). It tries clerk.signIn({ emailAddress }) first, then password. The hosted form stays covered by auth.spec.ts. The saved storageState JWT lasts about a minute; specs that run late on a shard should remint with signInFresh from automated-tests/fixtures/freshAuth.ts instead of trusting that snapshot. PR CI sets NEXT_PUBLIC_FF_CREW_AVAILABILITY_OFFERS=false, so do not seed through the venues page — use the venue write helpers in utils/apiEvents.ts. Owner rows have a More options menu (View in org activity) but no Remove Member — pick a row that also has a role combobox. Activity filter tests should wait for the request whose query includes the chosen category; useAuditLogSWR does not keep the previous page.

Test file location

E2E tests are in the automated-tests/ directory:

Example E2E test


Testing requirements for PRs

When to write tests

Test coverage goals

Focus on meaningful coverage:
  • Business logic and calculations
  • API service methods
  • Complex component interactions
  • Error handling paths

PR screenshots

For visual changes, capture screenshots with Playwright:

Capture screenshots

Run screenshot capture

Add to PR description


Best practices

Unit tests

  1. Test behavior, not implementation
  2. Use data-testid sparingly
    • Prefer accessible queries: getByRole, getByText, getByLabelText
    • Use data-testid only when necessary
  3. Mock at a boundary, not our own code. Mock the generated API client (through its generatedApiCore wrapper), third-party SDKs (Clerk, Stripe, Twilio, Sentry, PostHog, the AI gateway), native Expo and React Native modules, logger, config, and the clock. Let our own services, hooks and components run for real, so the test fails when they break.
    • Do not stub our own components and then assert that the stubs rendered. scripts/check-weak-tests.mjs (pre-commit and the Conventions job, both advisory) prints a new one as a MOCK line.
    • In api-events, test SQL and data rules with a *.integration.test.ts against real Postgres rather than a mocked prisma or db/*Crud module.
    • A test that only checks a mock was called with what the test passed in proves nothing. Keep it only when it pins a request shape the types do not.
  4. Reset shared mocks with vi.resetAllMocks() in beforeEach, and set return values inside each test or after the reset. vi.clearAllMocks() keeps queued mockResolvedValueOnce values, which leak into the next test (pitfalls.mdx #22). Give a vi.mock factory its defaults as vi.fn<(...args: any[]) => any>(async () => value), not vi.fn().mockResolvedValue(value). The type argument keeps the loose type of a bare vi.fn(), so later mockResolvedValueOnce calls still type-check. A reset restores the function passed to vi.fn but wipes mockResolvedValue, and a lost default can pass by accident. A lost timezone default, for example, falls back to the machine’s zone and passes everywhere except UTC CI.

E2E tests

  1. Use page objects for reusability
  2. Wait for stable state
  3. Test critical user paths
    • Login/logout flow
    • Creating new records
    • Editing existing records
    • Error scenarios

Debugging tests

Vitest

Playwright


CI integration

Tests run automatically in GitHub Actions: The Smoke tier ("tier": "critical" in automated-tests/coverage-matrix.json) is PR-blocking: A1 sign-in, H1 home, G1 create gig, G2 gigs list → detail, GD1 gig header edit, E1 member accepts offer. Regression (tier: regression) and nice-to-have (tier: nice-to-have) rows run on push-to-dev and nightly only. Cut workflows were removed from the repo per the PostHog keep/cut matrix; see web/automated-tests/README.md. web/scripts/e2e-critical-tier.mjs turns critical rows plus changed spec files into the Playwright --grep. Locally, npm run test:e2e -- --critical --changed-since=origin/dev matches PR CI. Title tags @smoke, @regression, and @nice-to-have document the bucket; CI reads matrix tiers, not tags alone. CI runs npm run test:coverage in web/ for unit tests.

Viewing results

  • Check the Actions tab in GitHub
  • Failed tests include screenshots and traces
  • Coverage reports are generated for PRs