Skip to main content
This page is for agents (Claude Code, Codex, Cursor, Devin, and others) that run a test plan. Read the QA testing guide first. Its test data rules apply to agents without exception.

Rules an agent never breaks

These rules hold even when a test plan step seems to need otherwise. Stop and ask the person who started the run instead.
  • Never publish a draft gig that has invitations. Publishing sends pending invitations by push, text, and email.
  • Never message, @mention, invite, or text a person who is not a test account.
  • Never change a feature flag, an org setting, or a role unless the person who started the run asked for that exact change. One PostHog project serves dev, QA, and production.
  • Never delete data. Archive what you created, and list it in the report so a person can clean it up.
  • Never write to a database through SQL. Read-only SELECT queries on the dev database are fine for checking what was stored.
  • Never print a password, token, or key. Read test credentials with a script that does not echo them.
  • Never act in a customer org. Production test runs stay in the test org the run was given.

Pick the backend

To refresh a row in What’s ready, check every merged PR for each ticket in its Tickets column against each branch:
A row is Yes for an environment only when every PR for its tickets has reached that branch. A mobile change that reached qa is in the next TestFlight build. A web or API change that reached main is live in production.

Drive the API

npm run api calls the API as a seeded persona, so you do not hand-build tokens. It needs npm run worktree:setup -- e2e mcp-ts.
Against dev, pass --env dev and a token. Every write against dev is held until you add --yes, so read the printed request before you confirm it.

Drive the mobile app

Use a local iOS simulator first. It is free and it covers most checks. Follow docs/engineering/mobile/agent-screenshots.mdx in the repository for the setup:
  1. Claim a simulator with node scripts/sim/claims.mjs claim --note "<test plan> run", so no other session uses it.
  2. Start your own Metro with npm run shots:metro -- --port 8085.
  3. Sign in with npm run sim:e2e-login -- --as member for the crew account, or --as user for the owner account. The helper reads the test accounts from .env.e2e without printing them.
  4. Open the screen under test with a deep link, for example soundcheck://events/<eventId> or soundcheck://events/<eventId>/chats/<chatId>.
Checks that need two accounts at once, such as a chat message going from one person to another, need two simulators with one account each. Use Revyl only for checks a simulator cannot prove, such as a store build loading an update. Run only the saved tests that cover the area, not a full pass. Picking the tests and running them are two steps:
  1. npm run revyl:select -- --out <scratchpad>/plan.json <files of the area> picks the saved tests and writes the plan. It runs nothing.
  2. node scripts/revyl-flows/run.mjs --plan <scratchpad>/plan.json --out <scratchpad>/results.json runs each test in the plan and writes the results. Add --dry-run first to see the commands without starting a device.
Checks marked Needs a real device need a person with a phone. List them as not run.

Drive the web app

Use the built-in browser or Playwright against the web URL that matches the backend. Sign in as the persona the check names. Never sign in with a person’s own account.

Report the results

Post one table per run, in the Linear issue or PR the run was for. Use one row per check ID and one of four results:
End every report with what you left in the environment, even when the list is empty.