> ## Documentation Index > Fetch the complete documentation index at: https://docs.soundchecklive.io/llms.txt > Use this file to discover all available pages before exploring further. # Emergency Change and Rollback > How each Soundcheck deployable rolls back today, and the emergency change path through dev, qa and main Use this page when production is broken and the fix cannot wait for the normal promotion. It covers how each deployable ships and how to undo it, then the emergency change path and the review that follows it. The pipelines themselves are described in the [CI/CD overview](https://docs.soundchecklive.io/engineering/ci-cd/overview.md). ## Restore service first Undo what broke before you ship a fix. The fastest undo depends on the deployable: | Deployable | Undo | Deploys from | | - | - | - | | Web (Next.js on Azure App Service) | Revert through `dev` → `qa` → `main` and let the deploy run | `web-ci-cd.yml` | | API (api-events, also hosts `/mcp`) | Move Cloud Run traffic to the previous revision | `api-events-deploy.yml` | | MCP library (`mcp-ts/`) | Roll back the API. The library has no deploy of its own | `mcp-deploy.yml` (CI only) | | CheckAI (Vercel) | Vercel rollback to the previous production deployment | `checkai-deploy.yml` | | gcp-platform-signals (Cloud Run) | Redeploy the previous image digest or move traffic | `gcp-platform-signals-deploy.yml` | | Mobile JavaScript (OTA) | Revert on `main`, or republish an earlier EAS update | `ci-cd.yml` | | Mobile native builds | Nothing in this repo. Installed builds stay installed | `ci-cd.yml` | | Azure Terraform (`infra/`) | Revert the change through the promotion flow | `terraform-apply.yml` | | GCP Terraform (`infra-gcp/`) | Revert the change through the promotion flow | Cloud Build `tf-apply-` | | Database migrations | Forward-fix with a new migration. Exception: the audit cutover has a rehearsed rollback (see [Database migrations](#database-migrations-api-events)) | api-events boot | A rollback does not reverse database migrations (except the audit cutover rollback), data already written, messages already sent, provider calls already made, or builds already installed on devices. Those need a forward fix. External side effects are never undone by a deploy rollback; a compensating action is a new, deliberate change ([async durability patterns](https://docs.soundchecklive.io/engineering/backend/async-durability-patterns)). Production can also change without a merge to `main`. A `workflow_dispatch` run with `environment=prod` can deploy api-events or gcp-platform-signals, or apply Terraform, from any branch. The routes below that use it say so, and [Known gaps](#known-gaps) explains why nothing restricts it. ## Deployables ### Web **How it deploys.** `web-ci-cd.yml` runs its `deploy` job on pushes to `dev`, `qa` and `main` that change one of the workflow's paths (mainly `web/**`), and on `workflow_dispatch` (no inputs). The GitHub environment is the branch name, with `main` mapped to `prod`. The deploy needs `checks` and `unit-tests-merge` to succeed. `e2e-tests` runs only for PRs into `dev` and pushes to `dev`. On pushes to `qa` and `main`, and on dispatch, it is skipped and the deploy still runs. The job builds `web/docker/Dockerfile`, pushes `soundcheck-web:` to Azure Container Registry, and deploys that image to the App Service for the environment with `azure/webapps-deploy`. **Rollback.** * Revert the bad commit on a branch cut from `dev`, open the PR into `dev`, then promote it to `qa` and `main` the same way as any fix. * `gh workflow run web-ci-cd.yml --ref main` redeploys the current head of `main`. Use it to retry a failed deploy. It cannot return to an older commit: the workflow has no commit input, and each deploy overwrites the `:` tag. * Pointing the App Service back at an earlier image is not in any workflow. See [Known gaps](#known-gaps). Dispatch only from `dev`, `qa` or `main`. From any other branch the environment is named after that branch, and the config step falls through to the dev settings. ### API (api-events) and hosted MCP **How it deploys.** `api-events-deploy.yml` deploys on pushes to `dev`, `qa` and `main` that touch its paths (these include `mcp-ts/**`), and on `workflow_dispatch` with an `environment` input (`dev`, `qa` or `prod`). Pull requests run the gates only. On push, the `set-env` job maps `main` to `prod`, `qa` to `qa`, and anything else to `dev`. On dispatch it uses the `environment` input as given, with no check against the branch. The `build-and-deploy` job waits for `checks`, `integration-tests`, `openapi-parity`, `encore-e2e` and `schema-gates` to succeed, then: 1. builds an Encore Docker image tagged `-` and pushes it to `-docker.pkg.dev//-api-events`; 2. runs a blocking OpenAPI breaking-change check against the revision Cloud Run serves now; 3. runs `gcloud run deploy -api-events` with that image; 4. verifies MCP OAuth discovery on the public host, then runs `npm run smoke:deployed` in `mcp-ts/`. The projects are `dev-soundcheck-live`, `qa-soundcheck-live` and `soundcheck-live` (prod), all in `us-central1`. Terraform ignores the image field of the Cloud Run service (`infra-gcp/modules/events/main.tf`), so a later apply does not roll the CI image back. A push that changes Azure `infra/` and api-events migrations together defers this deploy until `terraform-apply.yml` dispatches it, after the apply or, when the plan has no changes, straight away. A push that changes `infra-gcp/` does not defer it: `terraform-apply.yml` does not watch `infra-gcp/`, and nothing dispatches this workflow after a GCP apply. If a new api-events revision depends on a GCP apply, an operator sequences the two by hand. **Rollback, route 1: move traffic.** `gcloud run deploy` sends traffic to the new revision, and no Terraform `traffic` block overrides that. Find the previous revision, then send traffic back to it: ```bash theme={null} gcloud run revisions list --service=prod-api-events --project=soundcheck-live --region=us-central1 gcloud run services update-traffic prod-api-events --project=soundcheck-live --region=us-central1 --to-revisions==100 ``` Before you move traffic, confirm that the previous revision runs against the current schema. Moving traffic does not undo migrations. **Rollback, route 2: redeploy an older commit (prod, emergency only).** This route dispatches to `prod` from a branch with no PR, so it is an emergency change and needs the record described in [Emergency change](#emergency-change). Use it only when route 1 cannot restore service. Push the older commit to a new branch, then dispatch the workflow from that branch: ```bash theme={null} git push origin :refs/heads//rollback- gh workflow run api-events-deploy.yml --ref /rollback- -f environment=prod ``` The dispatch runs the workflow file, and the gates, from the commit you pushed, so an older commit runs its own older workflow. Only the emergency owners dispatch with `environment=prod`, because nothing restricts which branch can (see [Known gaps](#known-gaps)). Before you dispatch, confirm that the older revision runs against the current schema. No gate compares an older revision with the live database, and migrations are never reversed (see [Database migrations](#database-migrations-api-events)). The breaking-change check compares OpenAPI specs only. The dispatched run repeats `checks`, `openapi-parity`, `integration-tests`, `encore-e2e` and `schema-gates`, then `build-and-deploy`. The required `security` check (gitleaks, npm audit and Semgrep) does not run, because they run only on pull requests. `security.yml` also runs weekly and on manual `workflow_dispatch`, but those runs produce the repository-wide report (full-history verified secrets, full Semgrep SARIF, open advisories), not the PR gates, and nothing starts them when you dispatch a deploy. Run it by hand (`gh workflow run security.yml --ref `) and link the run as evidence. Record that gap in the post-deployment review, and land the same commit through a PR afterwards so the checks run on it. The breaking-change step in `build-and-deploy` compares the older spec with the live revision, so an older API that drops fields the live API added can fail it. An intended break is accepted by adding its line to `docs/openapi/breaking-changes-allowed.txt`. The deploy waits behind any running deploy for the same environment, because `api-events-deploy.yml` and `terraform-apply.yml` share the `backend-deploy-` concurrency group. **MCP library.** `mcp-deploy.yml` is CI only: lint, format, typecheck, `tsc` build and coverage. It has no deploy job, no publish step, and no dispatch inputs. The hosted MCP endpoint ships with the API, so roll it back with the API routes above. A revert in `mcp-ts/` goes through the normal flow and ships with the next API deploy. ### CheckAI **How it deploys.** `checkai-deploy.yml` deploys only on pushes to `dev`, and only when the repository variable `CHECKAI_VERCEL_DEPLOY` is `true`. On 2026-10-08 that variable was not set (`gh variable list`), so no workflow deploys CheckAI today and the production alias changes only by hand in Vercel. When the variable is on, the job records the current production deployment, builds, creates a production deployment without moving the alias, promotes it to `https://checkai-omega.vercel.app`, and polls `/eve/v1/health`. If the health check fails after promotion and a previous production deployment was recorded, the job rolls back by itself. The workflow has no `workflow_dispatch` trigger. Once the variable is on, the CheckAI production alias follows `dev`, not `main`. **Rollback.** * While the variable is unset: use Vercel's own rollback for the `checkai` project, with the command below and the previous deployment id from the Vercel dashboard. * Automatic, once the variable is on: the job rolls back to the deployment it recorded if the post-promotion health check fails. * Manual: the job logs `previous production deployment: ` in its "Record the current production deployment" step. Use that id: ```bash theme={null} npx --yes "vercel@54.17.3" rollback --yes --token "$VERCEL_TOKEN" ``` * Or revert the commit on `dev`. The alias moves on the next `dev` deploy. ### gcp-platform-signals **How it deploys.** `gcp-platform-signals-deploy.yml` deploys on pushes to `dev`, `qa` and `main` that touch the service or the release-age scripts, and on `workflow_dispatch` with an `environment` input (`dev`, `qa` or `prod`). A dispatch uses the input as given, with no check against the branch, the same as the API. The job runs the release-age gate and the Rust gates, builds the image tagged `-`, resolves its digest, and deploys that digest to `-gcp-platform-signals` in `us-central1`. Terraform ignores the image field (`infra-gcp/modules/platform-signals/cloud_run.tf`). **Rollback.** Resolve the digest of the previous image tag, then redeploy it. The earlier run's "Tag and push image" step shows the tag. Moving traffic to a previous revision works the same way as for the API. ```bash theme={null} gcloud artifacts docker images describe us-central1-docker.pkg.dev//-gcp-platform-signals/gcp-platform-signals:- --format='get(image_summary.digest)' gcloud run deploy -gcp-platform-signals --project= --region=us-central1 --image=us-central1-docker.pkg.dev//-gcp-platform-signals/gcp-platform-signals@ --quiet ``` A `prod` redeploy from a dispatch is an emergency change, as for the API route 2. ### Mobile app (Expo and EAS) **How it deploys.** `ci-cd.yml` (CI / CD) runs on pushes to `qa` and `main` that touch app code, config or shared inputs. Pushes to `dev` do nothing. The release route step (`scripts/mobile-release-policy.mjs route`) accepts only `qa` and `main`, so a dispatch from any other branch fails at that step. * `qa`: builds a fresh iOS binary with the `qa` profile and submits it to TestFlight. Android is fingerprint-based, and its store submission is manual. * `main`: publishes production OTA updates for both platforms when a compatible production binary exists. A native change blocks the OTA until a new binary is built and installed. * `workflow_dispatch` modes: `smart_build`, `build_submit`, `promote` (needs `build_id`) and `ota_update`. `ota_update` publishes the head of the branch it is dispatched from. Its `update_branch` input accepts only `auto` (the default) or the channel that matches the dispatched branch; any other value fails the route step (`scripts/mobile-release-policy.mjs`). **Rollback.** * JavaScript-only regression: revert the commit through `dev` → `qa` → `main`. The push to `main` publishes a new OTA. * Faster: republish an earlier EAS update group with the EAS CLI (`eas update:republish --group `). Check the flags with `eas update:republish --help` first, because the workflow installs `eas-version: latest` and the flags are not pinned here. * Native regression: a build already installed on a device cannot be pulled back from this repo. Halting a rollout in App Store Connect or Google Play happens outside the repo. See [Known gaps](#known-gaps). The forward fix: 1. Land the fix through `dev` → `qa` → `main`. 2. From `main`, dispatch `ci-cd.yml` with `mode=smart_build` or `mode=build_submit` to build a new production native binary. 3. After approval, submit that exact build. `mode=promote` with `platform` and `build_id` submits an existing build, and its route step accepts `main`. For Android, also pass `android_track=production`; the input defaults to `internal`, which reaches testers only. iOS uploads need a separate App Review step before public release. 4. Never submit the QA-channel binary to the public store, because its embedded QA channel would persist. The steps are in [Production release from main](https://docs.soundchecklive.io/engineering/ci-cd/overview.md#production-release-from-main). ### Azure Terraform (`infra/`) **How it deploys.** `terraform-apply.yml` runs `terraform plan` on pull requests into `dev`, `qa` and `main` that touch `infra/**`. A push to one of those branches applies the saved plan for `infra/environments/`. Dispatch takes `environment` and `run_apply` (default `true`). The apply uses the environment you choose and the Terraform in the branch you dispatch from. Nothing checks that the branch matches the environment, and `prod` has no protection (see [Known gaps](#known-gaps)). A dispatch with `run_apply=false` is a plan-only run from the chosen branch: ```bash theme={null} gh workflow run terraform-apply.yml --ref qa -f environment=qa -f run_apply=false ``` If the pushed commits change api-events schema or migration files (`api-events/src/db/schema.prisma`, `api-events/src/db/migrations/`, `api-events/prisma.config.ts`), the workflow dispatches `api-events-deploy.yml` after the apply, or on a push whose plan has no changes. The trigger is the commit diff, not the plan content. The dispatch runs on `main` for `prod`, `qa` for `qa`, and `dev` for `dev`. **Rollback.** Revert the change and move the revert through the promotion flow. Each merge plans and applies again. Before you merge the revert, read the Plan Summary step of the Terraform Plan job for the add, change and destroy counts. A revert restores configuration. It does not bring back data, secrets or state that a destroy removed. Some Azure changes are one-way. For example, `enable_app_service = false` in `infra/environments/dev/main.tf` and `infra/environments/qa/main.tf` destroys the retired Go App Service on apply, and reverting the setting creates a new App Service from configuration. It does not restore the destroyed one. Treat any destroy in the plan as a forward-fix decision. ### GCP Terraform (`infra-gcp/`) **How it deploys.** `infra-gcp` runs in Cloud Build, not GitHub Actions. Each environment project (`dev-soundcheck-live`, `qa-soundcheck-live`, `soundcheck-live`) has a `tf-plan-` trigger for pull requests, which posts one sticky plan comment per root, and a `tf-apply-` trigger for pushes to `dev`, `qa` and `main`. The apply trigger runs `infra-gcp/terraform-runner/entrypoint.mjs` with `MODE=apply`. The DNS root applies only after a merge to `main`. The apply trigger is disabled until `enable_automatic_applies` is true in that environment's runner root. The runner refuses to apply a plan that deletes any of these: Cloud SQL instances, databases or users; Secret Manager secrets (except the legacy secret); Cloud DNS managed zones; Artifact Registry repositories; projects; networks; subnetworks; or storage buckets. It also refuses to delete a dead-letter topic or subscription unless its address is listed in `infra-gcp/dlq-drain-allow.json` on the last applied commit. **Rollback.** Revert through the normal flow. If the runner refuses a plan during an emergency, do not work around the refusal by deleting the protected resource. Keep the resource and fix the configuration forward. ### Database migrations (api-events) **How they apply.** Migrations live in `api-events/src/db/migrations/`, and the schema is `api-events/src/db/schema.prisma`. On boot, `api-events/src/encore.service.ts` calls `migrateDeployOrThrow()`, which runs `prisma migrate deploy` against the environment's database. The first new revision that boots applies whatever is pending. CI does not run `migrate deploy` against shared environments. **Rollback: forward-fix, with one rehearsed exception.** Nothing in the repo runs `down.sql` files as a generic rollback. One legacy file exists under `api-events/src/db/migrations/20250515223805_add_subscription_tables/`, and no script runs it. For every migration except the audit cutover, fix forward. **Exception: the audit cutover.** Migration `20260825200000_audit_legacy_views` turned seven legacy audit tables into views over `audit_log`. Its header documents a rollback (drop the views, rename the archives back). Do not run that rollback: it drops every audit row written after the cutover from the legacy readers. The corrected procedure is `audit.rollback_cutover()`, installed by migration `20260827190000_audit_cutover_rollback`. It copies the post-cutover rows back into the archives and restores the seven tables in one transaction. Run it through `api-events/scripts/audit/rollback-cutover.mjs`: ```bash theme={null} DATABASE_URL=... node api-events/scripts/audit/rollback-cutover.mjs --dry-run # report per source, write nothing DATABASE_URL=... node api-events/scripts/audit/rollback-cutover.mjs --copy-back-only # copy rows back, keep the views DATABASE_URL=... node api-events/scripts/audit/rollback-cutover.mjs --confirm # copy back and restore the tables (destructive) ``` * Run `--dry-run` first. `--confirm --source ` limits the restore to one source. * The script refuses to run if `audit.rollback_cutover(text)` is not installed, so the target database must already have `20260827190000_audit_cutover_rollback`. * It needs `DATABASE_URL` and a person. It is not a generic down migration. Rows with no legacy shape stay in `audit_log` only, and the `ON DELETE CASCADE` foreign keys dropped by `20260827180000_audit_guard_hardening` are not restored. * After a rollback, do not run `prisma migrate dev`, because the diff proposes dropping the seven restored tables. To go forward again, the script header gives the steps: `prisma migrate resolve --rolled-back` for `20260825200000_audit_legacy_views` and `20260825200100_audit_cutover_straggler_sweep`, re-run `api-events/scripts/audit/backfill.mjs`, then redeploy. For any other migration, fix forward: * Add a new migration that corrects the schema or data. Do not edit an applied migration. * New migration SQL must replay cleanly. Follow `.cursor/rules/sql-migration-idempotence.mdc` and the policy in `api-events/src/db/migrationIdempotence.policy.json`. * Lock-taking DDL follows [migration lock safety](https://docs.soundchecklive.io/engineering/backend/migration-lock-safety.md). If a migration fails, its row stays failed and every later boot fails with `P3009` until someone runs `prisma migrate resolve` by hand. The recovery steps are in [migration lock safety](https://docs.soundchecklive.io/engineering/backend/migration-lock-safety.md). That step needs database access and a person. ## Emergency change ### When it applies Use the emergency path only when one of these is true: * Production is down, or a core customer flow is broken (sign-in, gigs, crew messages, payments). * Production exposes customer data or a secret, or a production dependency is compromised. * The rollback above cannot restore service, and waiting for the normal promotion would extend the outage. If a rollback restores service, do the rollback and take the fix through the normal flow. Declare the emergency with four things: the condition that was met, the incident link, the rollback that was tried first, and the declarer. * With a PR, put them in the PR description. * With no PR (route 2 of the API rollback, or any `environment=prod` dispatch with no PR), write them as a comment on the incident's Linear issue before you dispatch. Add the dispatch run link to that comment afterwards. The declarers are the owners listed in `.github/CODEOWNERS` (today `@steventt07` and `@wskel`). GitHub requires no approval for these changes (see [Known gaps](#known-gaps)), so the declaration is the approval record: it shows that an owner chose the emergency path on purpose. The post-deployment reviewer must be someone other than the author. ### Branch and PR * Name the branch `/`. Create the worktree with `npm run task:new -- `, which cuts `/` from `origin/dev`. The owner is `TASK_OWNER` or your git email. * Do not reuse a branch whose PR has already merged. The Claude Code guard blocks pushes to such a branch. * Open the PR into `dev`. Commits reach `dev`, `qa` and `main` through a PR, with two exceptions. The `expo-version-updater` app bypasses the ruleset for CI version and docs commits (see [Known gaps](#known-gaps)). A `workflow_dispatch` can deploy `prod` from a branch with no PR: route 2 of the API rollback is the emergency exception to "a PR is required", and its declaration above is the record. * Ruleset `Protect default,dev,qa,main` requires a pull request before merging, blocks force pushes and branch deletion, requires conversations to be resolved, and allows only merge commits. * The ruleset requires zero approvals, so the author can merge once the required checks pass. The post-deployment review below is the compensating control. * Do not use the `expo-version-updater` app's token for an emergency change. ### Required checks The ruleset requires the checks below. Each one runs from the PR's own head, so a PR can change the steps of a required job, and a check is only as strong as the review of its definition. [Known gaps](#known-gaps) covers the `.github/` review gap. * **Git hooks.** Pre-commit and pre-push run on your machine, and GitHub does not run them. Do not use `--no-verify`, `core.hooksPath` overrides or any other bypass. Fix the failure instead. The Claude Code guard (`scripts/agent-hooks/guard-bash.mjs`) refuses these bypasses in Claude Code sessions only. * **Security check.** The required `security` check runs on every PR into `dev`, `qa` and `main`, because `security.yml` has no path filter on pull requests. One job runs three scans, and each runs even when an earlier one failed: * gitleaks scans the PR's commits for secrets. * The npm audit fails on new high or critical advisories in lockfiles the PR changes. * Semgrep fails on `ERROR` findings in the code the PR changes. * **Other required checks.** `Enforce Branching Rules`, `Static Checks`, `Mobile - Quality`, the api-events checks, and `components / result`, which covers checkai, mcp-ts, gcp-platform-signals and GCP Terraform. * **Skipped means passing.** A required check whose jobs were skipped by a path filter counts as passing. Confirm that the checks for the packages you changed actually ran. * **Drafts.** A draft PR skips api-events `integration-tests` and `encore-e2e`, and the required `api-events / suites` shows as deferred until the PR is marked ready. Mark it ready before you merge. * **Promotion reuse.** A `dev` → `qa` or `qa` → `main` PR can reuse the `checks`, `unit-tests` and api-events gates from a push run on the same tree, when that run passed. The security checks are not in that list, so they run on promotion PRs as well. * **Terraform plans for `main`-targeted PRs.** The plan job runs under the `prod` environment, which has no protection rules. See [Known gaps](#known-gaps). ### Promotion 1. Merge the fix PR into `dev`. The push deploys to `dev`. 2. Open a PR from `dev` into `qa`. `Enforce Branching Rules` rejects a `qa` PR from any other source branch. The push deploys to `qa`. 3. Open a PR from `qa` into `main`. The push deploys to production. Mobile publishes a production OTA unless the change needs a new native binary. Keep closing keywords out of promotion PR text. `Enforce Branching Rules` rejects closing references to promotion PRs, and a commit or PR that says `fix #N` closes PR `#N` when it lands on `dev`. Write `Follow-up for #123` or `Addresses review feedback on #123` instead. Closing keywords belong on issues that should actually close. The branch rules offer no route that moves one fix from `dev` to `qa`, or from `qa` to `main`, without the rest of that branch. Before you open the promotion PR, check what else `dev` holds, because `dev` → `qa` promotes all of it. ### Post-deployment review Within two business days of the production deploy, someone other than the author reviews the change in production and records the review as evidence. The ruleset requires no approval, and a `prod` dispatch can deploy without a merge to `main`, so this review is the compensating control for changes the author merged and deployed. Record: * the PR and the merge commits on `dev`, `qa` and `main`, or, for a change with no PR, the rollback branch and the commit it points to; * the condition that justified the emergency path, the rollback attempted first, and the declarer; * the deploy run links for each environment the change reached (`api-events-deploy.yml`, `web-ci-cd.yml`, `ci-cd.yml`, `checkai-deploy.yml`, `gcp-platform-signals-deploy.yml`, `terraform-apply.yml`). For a route 2 or other `prod` dispatch, include the dispatch run link and the branch it ran from; * for every action taken outside a GitHub workflow (a Cloud Build `tf-apply-` build, `gcloud run services update-traffic`, a Vercel rollback, `eas update:republish`, a store submission or halted rollout): the provider's build, deployment or update link, the exact revision, deployment, update group or build ID it moved to, who ran it, and the UTC time; * which required checks ran, and any skipped check with its reason (draft, path filter or reuse); * whether migrations or Terraform applied, and whether a plan was refused; * what was checked in production after the change, and by whom; * follow-up issues. Evidence sources are the PR, the deploy run links, and the incident record. The Linear release records from `linear-release.yml` are not evidence yet, because its automatic runs are dry-run (see [Known gaps](#known-gaps)). Until the owners choose a system of record, record the review as a comment on the PR and link it from the incident's Linear issue. For a change with no PR (route 2 or any other `prod` dispatch), record the review as a comment on the incident's Linear issue, next to the declaration and the dispatch run link, and link it from the PR that later lands the same commit. When the review needs repository-wide scan evidence, link the latest weekly `security.yml` run (Mondays at 09:17 UTC). It scans the full history and reports open high and critical advisories and Semgrep findings for the whole repository. ### Routine changes The same gap applies outside emergencies: the ruleset requires no approval, so most ordinary PRs are merged by their author. For those, the review happens on each `qa` → `main` promotion PR instead of per change: * Within two business days of the promotion merge, and at least weekly while promotions happen, someone who authored none of the promoted PRs reviews them. * They leave one comment on the promotion PR that lists every promoted PR merged without another person's approval, states what they checked (the diff, the required checks, the deploy runs), and names anything that needs a follow-up issue. * No emergency declaration or incident issue is needed. A change already covered by an emergency post-deployment review can be listed by link. ## Known gaps These were checked on 2026-10-08 against the GitHub API and the workflow files. They limit what the controls above guarantee. * **Production can be deployed from any branch.** `api-events-deploy.yml` and `gcp-platform-signals-deploy.yml` take `environment` from the dispatch input and do not check that the branch is `main`. `terraform-apply.yml` applies the chosen environment's Terraform from the dispatched branch, and `run_apply` defaults to `true`. The `prod` GitHub environment has no protection rules and no deployment-branch policy. A `prod` deploy or a `prod` Terraform apply therefore does not need a merge to `main`, and [route 2](#api-api-events-and-hosted-mcp) of the API rollback uses this path. A merge to `main` does not gate these runs. The compensating control is the post-deployment review. The fix is a ref guard that allows `environment=prod` only from `main`, or environment protection on `prod` with required reviewers and a branch policy. Owners to decide. * **Terraform plans for `main`-targeted PRs run under `prod`.** For a `pull_request` whose base is `main`, `terraform-apply.yml` sets the `environment` of its `terraform-plan` job to `prod`. The job then logs in to Azure with OIDC (`azure/login@v3`, with no pull-request guard), passes the `AZURE_*` secrets, and has `id-token: write`. The OIDC token names the `prod` environment. Because `prod` has no protection rules and no deployment-branch policy, Terraform code in a branch of this repository, for example an `external` data source, runs in the `prod` environment without a merge to `main`. Whether that reaches production Azure resources depends on the federated credentials, which this repository does not show. Owners to decide: plan PRs under a secret-free environment, add a deployment-branch policy to `prod`, or run PR plans from the trusted base checkout. * **Environments have no approval gate.** The `dev`, `qa` and `prod` GitHub environments have no protection rules: no required reviewers and no wait timer. The [CI/CD overview](https://docs.soundchecklive.io/engineering/ci-cd/overview.md) describes manual approval for prod Azure applies, but the environment does not enforce it. * **No required approvals.** The ruleset requires zero approvals and no code-owner review. The post-deployment review is the only independent check on changes the author merged. * **`.github/` and check definitions are not independently reviewed.** A PR can change the workflow files that define its own required checks. The ruleset does not require code-owner review for `/.github/`, and `.github/CODEOWNERS` only requests review, so those changes are self-approved. Enabling code-owner review for `/.github/` in the ruleset would close this. Until then, the post-deployment review covers it. * **Local hooks are not enforced by the server.** Nothing on GitHub runs the pre-commit or pre-push hooks. The Claude Code guard covers Claude Code sessions only. * **Web checks are not merge-required.** The `checks`, `unit-tests` and `unit-tests-merge` jobs of `web-ci-cd.yml` are not in the ruleset, so a PR that changes `web/` can merge with them red. The web deploy job still needs `checks` and `unit-tests-merge` green on its push run, so that change does not deploy. * **The `expo-version-updater` app bypasses the ruleset.** The ruleset exempts the GitHub App `expo-version-updater` (bypass actor 2751420, `bypass_mode: exempt`), so its pushes reach `dev`, `qa` and `main` without a PR. It makes three kinds of commits: * the mobile version bump, made by the `check-version` action in `ci-cd.yml` (jobs `smart-check-fingerprint`, on `qa` pushes and `smart_build` dispatches, and `build-submit-version-check`, on `build_submit` dispatches). It runs only when a version bump is needed, commits with `git commit --no-verify`, and pushes the branch it checked out; * the OpenAPI spec sync (job `sync-openapi-spec` in `docs-ci.yml`), on pushes to `dev`, `qa` and `main` that change api-events; * the changelog sync (job `changelog-sync` in `docs-ci.yml`), on pushes that change the docs. It is not an emergency-change path, and its commits skip local hooks. Do not use it to land a fix. * **CheckAI deploys by hand.** `CHECKAI_VERCEL_DEPLOY` is unset, so no workflow deploys CheckAI and there is no deploy run to cite as evidence. Once it is set, a CheckAI fix reaches the production alias on the next `dev` push. * **Linear release records are dry-run.** `linear-release.yml` runs after each successful Web, api-events and mobile deploy, but automatic runs set `dry_run` to true, so they only log. The workflow says the rollout stays dry until the first qa deploys are verified in the logs. No Linear release is written for normal deploys yet. * **No documented App Service image rollback.** The web deploy overwrites the `:` tag and records no digest, so the previous Azure image is not identified by any workflow. * **GCP apply state is not visible in the repo.** Each apply trigger stays disabled until its runner root sets `enable_automatic_applies` and that root is applied. Confirm the live trigger state in Cloud Build before you rely on an automatic apply. * **Store rollouts are outside the repo.** App Store Connect and Google Play can halt a rollout, but no workflow in this repository does. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.