Skip to main content

What this wave attached

Migration 20260923130000_audit_trigger_wave_5b calls audit.sync_triggers() — the same function wave 5a used — on four more tables. Combined with wave 5a, audit_log_change now sits on nine tables: Wave 5c (20260923140200_audit_trigger_wave_5c) landed after the parent-resolved organization lookup described in Wave 5c: parent-resolved organization. The sections below that describe the org_col gap are the wave 5b record; the gap is closed now. Every candidate below was reviewed column-by-column against src/db/auditClassification.ts before this migration was written. That review found no classification bugs — every allow/deny/mask/ignore list already partitions its model’s columns exactly (rule (b)), and npx tsx scripts/audit/generateTableConfigSeed.ts --check passes with zero diff against the manifest as it stands today. Nothing here changes auditClassification.ts or adds a new *_audit_table_config_seed migration — the gap this wave found is about attachment readiness, not column classification.

The finding that shaped every deferral: org_col has no join

audit.log_change() resolves organization_id from a literal column on the row image (v_org := v_row ->> v_org_col). There is no join-based fallback for ownership the way scope_sql provides one for scope_id — a table without its own organization_id (or user_id) column simply cannot populate audit_log.organization_id, no matter what orgColumn says in the manifest. That matters because the org activity reader requires it unconditionally. buildOrgActivityClauses() in src/db/orgActivityLog.ts always includes:
scope_type / scope_id are additional filters layered on top of that clause, never a substitute for it. A table with scopeType: 'EVENT' and no orgColumn — several of this wave’s candidates — would still produce AUTO rows that can never be returned by the standard per-org reader. scripts/audit/forecastTriggerWave.ts checks this mechanically for every candidate (the org? / user? columns below), which is what turned this from a hunch into a verified attach/defer input.

Per-table decision

Reproduce this table with npx tsx scripts/audit/forecastTriggerWave.ts from api-events/.
The org column reads direct (a literal organization_id column), parent-N (resolved through an N-hop parent path), or none. The lat us band for parent-N rows adds the measured lookup cost (below) on top of the payload-scaled model.

Attached

  • ledger_items — organization_id resolves directly. Every column is financial evidence (OPEN, denyColumns: []); the wave 5b integration test asserts an INSERT and an UPDATE both produce AUTO rows with redacted_fields: [] — nothing here is silently hidden.
  • transactions — organization_id resolves directly. Denies cleo_contact_id, cleo_funding_source_id, cleo_submit_claim_token (the identity-linkage / bearer-credential split documented in auditClassification.ts’s file header); keeps every Stripe/Cleo reconciliation id.
  • crew_offers — organization_id resolves directly. Money (fee) and access (who is offered a shift) risk, moderate assumed write rate.
  • organization_documents — organization_id resolves directly. SENSITIVE, allow-list model; blob_name denied so an audit row can never hand out a live Azure Blob path. The wave 5b integration test asserts blob_name is absent from before_state/after_state on both INSERT and UPDATE and appears in redacted_fields.

Deferred in wave 5b — no organization_id column (resolved in 5c)

crew_offer_days, crew_offer_recipients, crew_offer_recipient_days, Invitation, EventMember all lack a direct organization_id column. crew_offer_recipients and Invitation at least resolve user_id (userColumn); crew_offer_days and crew_offer_recipient_days resolve neither and would be fully unowned rows. Attaching any of them today would write AUTO rows that are real, correctly redacted, and permanently invisible to the org activity feed. Clearing this required a parent-resolved organization on audit.log_change() — a trigger change, not a wave — which landed with wave 5c (see below). The lookup each of these five needs is a one-hop query, e.g. for crew_offer_recipients:
and for crew_offer_recipient_days (two hops):
crew_offer_recipient_days is additionally the highest assumed write-rate table in the candidate set (per-day response tracking on every recipient), so it needs the prod query below regardless of the ownership fix.

Deferred — Event

Event resolves organization_id directly and could attach today on ownership grounds alone. It is deferred anyway: at 41 columns it is the widest table in the candidate set, and per the Flex-sync clobber-and-correct cycle documented in Event.scheduleSetByUserAt’s schema comment, it is plausibly the highest-churn table in the entire schema — every schedule correction, every human edit, every Flex reconciliation pass touches it. The diff-only UPDATE payload keeps modeled bytes/row moderate (1,616 B, roughly 1.5x transactions), but the model’s write-rate assumption (60 AUTO rows/org/day) is a guess with no prod grounding, and Event’s real number could plausibly be an order of magnitude off in either direction. This is the one table in the batch that should not move on a forecast alone.

Wave 5c: parent-resolved organization

Migration 20260923140000_audit_org_parent_lookup gives the capture floor a declarative, per-table parent path to the organization:
  • Config. audit.table_config gains org_parent_path text[] — flattened (fk_col, parent_table, parent_key) triples, one per hop, child first — and org_parent_col, the org column on the last hop’s table. The manifest field is orgParent in src/db/auditClassification.ts ({ hops: [{ foreignKey, table, key }], orgColumn }), and the seed generator emits both columns (20260923140100_audit_table_config_seed).
  • Compile, don’t store SQL. Unlike scope_sql, the config never carries free-form SQL. audit.org_parent_sql(table, path, org_col) validates every identifier against pg_catalog and compiles one statement, e.g. SELECT (h1.organization_id)::text FROM public.crew_offers h1 WHERE h1.id = $1::text, or a join for two hops.
  • Baked into TG_ARGV. audit.sync_triggers() compiles the path once and passes the first-hop fk column and the compiled lookup as two extra trigger arguments (12 → 14). The insert/update logging path still reads no config and no catalog per row; a 12-argument trigger is still read correctly (no lookup).
  • Ownership order. A direct org_col wins; otherwise the parent lookup; otherwise the transaction-local cascade snapshot; otherwise the existing user_col fallback. audit_log_owner_chk still sees at most one owner.

Fail-closed rules

Migration 20260924061000_audit_cascade_org_snapshot adds this delete snapshot. Migration 20260924170000_audit_indexed_delete_snapshot replaces its growing JSON map with indexed temporary rows. sync_triggers() installs the parent triggers for attached, enabled child paths, including intermediate parents in multi-hop paths. The snapshot holds only child identities and organization IDs in pg_temp.audit_delete_orgs, indexed by table and child ID. Each delete consumes its row; commit drops the temporary table and rollback undoes its writes. Relation ownership is checked before privileged access. It does not copy parent fields into audit payloads or attach logging to otherwise deferred tables. Parent deletion now performs a bounded-by-descendants lookup without repeatedly copying snapshots for earlier parents.

What wave 5c attached, and the redaction review

Column-by-column against the manifest (no classification changed):
  • crew_offer_recipients → crew_offers.organization_id. OPEN; notes is the crew member’s own reply text, already allow-listed; no contact data on the row.
  • crew_offer_days → crew_offers.organization_id. OPEN; per-day fee and call-window overrides are money + schedule evidence. times_changed_at stays ignored.
  • Invitation → Event."organizationId". SENSITIVE allow-list; phoneNumber, email and shortCode (a bearer capability for SMS replies) are redacted. The integration test asserts none of the three values appears anywhere in the AUTO row.
  • EventMember → Event."organizationId". OPEN; special_instructions stays allowed per its manifest note.
Also declared (config only, not attached): crew_offer_recipient_days (two hops via crew_offer_days), availability_request_days, availability_request_recipients and availability_request_recipient_days (two hops via availability_request_recipients). All compile against the live catalog (asserted by auditClassification.integration.test.ts).

Still deferred: a storage decision, not an ownership one

Ownership no longer blocks any of these; each is pending the owner’s storage call. Forecast delta if attached as-is (same model and assumptions as above): For scale: waves 5b + 5c together forecast 28.71 GB/year (200.9 GB over 7 years), so attaching all three deferred tables would roughly 2.5x the floor’s modeled footprint. The two-hop tables also pay the heavier measured lookup (~36 µs vs ~14.5 µs for one hop).

Measured overhead (private database, wave 5c)

Measured on a private local Postgres (the api-db-ephemeral Docker container on a MacBook, not prod hardware), 3,000 writes per case, three runs, trigger enabled vs ALTER TABLE … DISABLE TRIGGER audit_log_change: The parent lookup on its own (the compiled statement through EXECUTE, 20,000 calls × 3): ~14.5 µs for one hop, ~33–39 µs for two hops. It goes through EXECUTE, so it is re-planned per call; that is the bulk of the cost, not the index probe. A parent-resolved table therefore costs about the same as a direct-org table plus that lookup — the one-hop tables measured within the run-to-run noise of crew_offers. The absolute overhead here (~110–135 µs/write) is higher than wave 5a’s documented +50–72 µs; that figure came from different hardware, and this box also measured the untriggered writes slower than CI would. Treat the relative numbers (lookup ≈ +13% of trigger cost for one hop) as the signal. Measured audit_log row sizes (pg_column_size) for the new tables came in under the model: crew_offer_day INSERT ~500 B / UPDATE ~266 B, crew_offer_recipient UPDATE ~266 B, invitation INSERT ~674 B / UPDATE ~290 B — the model’s blended 626–734 B/row is conservative.

The exact prod query for every deferred table

None of these can be run from this environment — no prod access. Each answers “is this table’s write rate low enough, and how wide are its rows really,” over a representative recent window (a week is usually enough to average out day-of-week effects; a full billing cycle is better for anything gig-driven).
That gives ins/upd/del counts since the last stats reset — pair it with pg_stat_reset_time() (or a saved snapshot from a week prior) to turn it into a rate. For actual row width, rather than this doc’s modeled estimate:
(swap the table name and quoting per target). And for the table’s current on-disk footprint, to sanity-check the retention projection independent of the write-rate model entirely:
Re-run scripts/audit/forecastTriggerWave.ts with the real n_tup_ins/n_tup_upd/n_tup_del numbers substituted for the WRITE_RATE_ASSUMPTIONS constants (and the real avg_row_bytes in place of the modeled per-type byte constants, if it diverges meaningfully) before proposing wave 5c.

Modeling notes and assumptions

The forecast script has no access to schema.prisma row statistics or prod pg_stat_user_tables, so every number above is derived, not measured. Stated explicitly so a future wave can correct any of them:
  • Baseline. Wave 5a measured ~1,098 B/row and +50–72 µs/write on a 16-column table (see src/db/migrations/20260825210000_audit_trigger_floor/migration.sql’s header). The script decomposes that into a fixed audit_log row overhead (~384 B — the ~25 non-payload columns: ids, actor fields, timestamps, request metadata) plus a payload component modeled from a synthetic 16-column reference mix (4 uuid, 5 short string, 3 datetime, 2 enum, 2 decimal). Every candidate table’s payload is then computed the same way from its own real columns (via scripts/schema/parsePrismaSchema.ts) and added to that fixed overhead.
  • Per-type byte constants (UUID_COLUMN_BYTES, SHORT_STRING_COLUMN_BYTES, LONG_STRING_COLUMN_BYTES for @db.Text columns, DATETIME_COLUMN_BYTES, DATE_COLUMN_BYTES for @db.Date, DECIMAL_COLUMN_BYTES, ENUM_VALUE_OVERHEAD_BYTES plus each enum’s real average value-name length) are estimates of the JSON-serialized value width to_jsonb writes into before_state/after_state, plus a JSONB key/punctuation overhead per column keyed on the column’s real name length.
  • Redaction-aware payload. Only columns the trigger actually writes count toward payload width: for OPEN tables that is allowColumns + ignoreColumns (deny/mask excluded); for SENSITIVE tables it is allowColumns only (everything else, including ignoreColumns, is redacted). This mirrors audit.log_change()’s v_hidden computation exactly.
  • INSERT/DELETE vs. UPDATE. audit.log_change() writes the full (non-hidden) row on INSERT/DELETE, one-sided, and a diff of only the changed (non-hidden) columns on UPDATE — but BOTH before_state and after_state for those changed columns, i.e. UPDATE pays for two sides of a smaller set. The script assumes an UPDATE changes max(2, round(25% × visible columns)) columns on average — a guess, not a measurement; a table whose typical edit touches many more or fewer fields than that will forecast wrong in that direction.
  • Write mix. Blended bytes/row assumes 5% INSERT / 90% UPDATE / 5% DELETE across all writes, applied uniformly. Real mixes vary by table (organization_documents is INSERT-heavy; crew_offer_recipient_days is almost entirely UPDATE) — this is a simplification stated once rather than guessed per table.
  • Write rate. WRITE_RATE_ASSUMPTIONS in the script states an assumed AUTO-rows-per-org-per-day figure per table — already net of the noise killer (an UPDATE touching only ignoreColumns writes zero rows), since there is no prod gross-write-rate figure to discount from. These are judgment calls grounded in each table’s role (staffing fan-out is high-churn; document uploads are rare), not measurements.
  • Org count. 500 active organizations, platform-wide. Not sourced from anywhere — substitute the real figure and re-run.
  • Retention. 7 years flat (GB/year × 7), not modeling org-count growth over the window. A growing platform will retain more than this number by year 7; a shrinking or flat one will retain about this much.
  • Latency. Scaled linearly off the wave 5a baseline by the ratio of blended bytes/row to the 1,098 B baseline. This is the weakest part of the model — the wave 5a latency figure folds in trigger dispatch overhead that does not scale with payload size at all, so the true latency curve is flatter than this model assumes, especially for the narrower deferred tables.

The kill switch

No deploy needed, same as wave 5a — for any attached table, including the eight from waves 5b and 5c:
Re-arming is explicit and requires listing the table again:

Verifying these waves

Wave 5c ran clean against a private DB (t3_audit): seed --check clean, migration replay (migrationIdempotence.integration.test.ts) and the prisma migrate diff --exit-code drift gate clean, and the new wave 5c parent-resolved organization tests cover each attached table’s org resolution and redaction, the hidden-key skip, the invalid-path and ambiguous-owner refusals, the cascade-delete fallback, and a two-hop compile. Both ran clean against a private DB (t2_audit_wave) before this wave landed: the seed --check reported no drift (no manifest changes were needed), the trigger-attachment tests confirm exactly the wave 5a + 5b set carries audit_log_change, and the new wave 5b redaction tests assert ledger_items captures every column with redacted_fields: [] and organization_documents redacts blob_name on both INSERT and UPDATE.

The endpoint coverage gate is now call-graph level

api-events/src/systems/audit/auditCoverage.test.ts used to credit a route when any module its handler called into (or one import hop further) merely mentioned recordAudit(. That over-reported: one audited operation in a large *Operations.ts module credited every sibling route in the same module. The gate now walks references instead. Starting from the api.raw handler, it follows every named function, class, or constant the code references — same-file top-level declarations, relative import { x } bindings (including re-exports, export *, and ns.member on namespace imports) — and a route counts as instrumented only if that reference graph reaches recordAudit or recordAuditBestEffort in src/systems/audit/recordAudit.ts. Functions passed by reference (the ledger audit hooks that endpoints inject through ledgerPaymentAuditOptions() / ledgerItemAuditOptions()) are followed, but object-method dispatch on a value, class instances created elsewhere, and dynamic import() are not. The walk parses with the raw TypeScript compiler API (no type checker) and only enters modules that can transitively import the audit writer, so the whole gate runs in about a second. Tightening the gate surfaced 46 mutating routes that the module-level rule had been crediting while their own code path writes no semantic row. Two of them write only wave 5a tables (PUT /organizations/:organizationId → Organization, PUT …/members/:userId/onboarding → OrgMember) and are now covered-by-trigger-floor. The other 44 are todo:unfiled.

Event cascade ownership

Invitation and EventMember retain the event organization in the internal audit_organization_id column. Database triggers populate it on child writes and refresh it when an event changes organizations. The audit trigger reads this retained value from the deleted row, because PostgreSQL removes the parent event before firing the child cascade-delete audit triggers. Caller-supplied ownership values are overwritten with the event’s organization. The ownership column is ignored for change detection, so backfills and ownership refreshes do not generate extra audit entries. The migration backfills existing children and can be replayed. Regression tests cover mixed-organization bulk deletes, reparenting, event ownership changes, redaction, and backfill replay.

Migration notes

  • 20260923140000_audit_org_parent_lookup is replay-safe: columns use ADD COLUMN IF NOT EXISTS, and every function (audit.org_parent_sql, audit.audit_log_change, audit.sync_triggers) is CREATE OR REPLACE. sync_triggers() rebuilds each attached relation’s trigger with the resolved parent lookup baked into its arguments, so the trigger reads no config per row.
  • 20260923140200_audit_trigger_wave_5c only calls audit.sync_triggers(), which drops and recreates per relation, so replaying it converges on the same attached set.
  • Kill switch for any attached table, no deploy: UPDATE audit.table_config SET enabled = false WHERE table_name = '<table>'; SELECT * FROM audit.sync_triggers();
Generated seeds preserve an existing enabled = false setting while updating classification and redaction policy. New rows take their enabled default from the manifest. Re-enabling capture remains an explicit operator action. The 20260924203823_preserve_disabled_audit_during_seed migration deliberately sorts before the already committed 20260924203824_audit_table_config_seed. Its temporary trigger prevents that older seed from clearing a kill switch on an environment where the seed is still pending. The new seed uses a preserving upsert, and 20260925034338_remove_audit_seed_guard removes the temporary trigger so manual re-enabling works normally. Applied migration files remain unchanged. If deployment stops between the guard and its cleanup, finish the migrations before re-enabling capture. This cannot recover operator settings already overwritten by an earlier deployment.