PricingObservation row. When there is no evidence, the answer is coverage: "none" with a null range — never a modelled guess.
Owned by api-events/src/systems/pricing-knowledge/.
The model
PricingObservation is one priced fact: a service category, a market, a price band (min_price / max_price as Decimal(12,2)), a pricing unit, when it was observed, and where it came from.
Source types and what they mean
OBSERVED rows are org-private data. They are never returned to a caller as individual rows and never appear in evidence[]. They influence a range only as an aggregate, and only under the k-anonymity rule below.
Provenance rules
- Every row carries
source_typeand, for published/inferred rows, asource_urland anevidence_excerpt. - Rows are de-duplicated on a
fingerprint(source type + origin + category + market + unit + price band + event date), or on(source_record_type, source_record_id, dimension_key)when the row mirrors a Soundcheck record. confidenceis 0–1 and multiplies the row’s weight. Extraction sets it; a missing value is treated as 1.- No row is ever synthesised to fill a gap. An empty bucket stays empty.
K-anonymity: the ≥3 organization rule
Soundcheck booking actuals are only released as aggregates backed by at least 3 distinctorganization_id values in the bucket being priced.
Below that threshold, every OBSERVED row in the bucket is dropped entirely: it contributes nothing to the percentiles, and nothing to min / max either. The response still reports the truth about what was withheld:
observed_included: false with a non-zero observed_count means “we have actuals here but not from enough independent organizations to release them.”
The threshold is echoed to callers as honesty.observed_aggregated_k.
The market ladder
A lookup walks from specific to general and stops at the first rung with any eligible evidence:- market —
{ market, country }, e.g.Chicago, ILinUS. Only when the caller supplied amarket. - region —
{ region, country }, e.g.ILinUS. From the caller’sregion, or the state parsed out of the market key. - country —
{ country }. Always present; defaults toUS.
market_level_used. When no rung had evidence, market_level_used is null, coverage is none, and range is null.
Market keys are normalized by marketKey.ts: "Chicago, Illinois" and "chicago, il" both key to Chicago, IL; state names and USPS codes both resolve; country aliases (usa, united states) resolve to ISO alpha-2.
Weighting and the range
For each eligible row:(min_price + max_price) / 2), and low / typical / high are the weighted, linearly interpolated p25 / p50 / p75 over those midpoints, using the standard cumulative-weight-fraction positions (cumulative_i − w_i/2) / total_weight. min and max are the extremes of the underlying min_price / max_price columns, so the band always contains every row. All money arithmetic runs through decimal.js; only weights and percentile fractions are floats.
coverage is none with 0 eligible rows, thin with fewer than 3, otherwise ok.
evidence[] returns up to 10 PUBLISHED / INFERRED rows, highest weight first, with no organization_id and no record ids.
explanation is a deterministic English sentence built by buildExplanation — no model call:
Range for DJ services in Chicago, IL from 7 published, 2 inferred and 12 aggregated Soundcheck observations across 4 organizations (market level); typical $1,250 per event.
The lookup contract
Route:POST /api/v1/public/pricing/lookup (anonymous, auth: false)
Direct MCP op: v1PublicPricingLookupCreate
Public MCP tool: lookup_market_pricing
Operation: lookupMarketPricingOperation (lookupMarketPricingFromRawOperation for the raw-JSON transports)
Request
The three “echoed” dimensions are part of the contract now so that PRO-421 can start filtering on them without a breaking change.
Response
country must resolve to an ISO 3166-1 alpha-2 code — the aliases (usa, United States, ca, …) still work, but free text such as United States of Amercia is a 400 rather than a bogus country bucket that reports coverage: none. A market key with a second segment is only accepted when that segment resolves to a US state code, unless the country is non-US — a country sent alongside the market applies before that check, so { market: "Toronto, Ontario", country: "CA" } works without repeating the country as a third segment, while { market: "Chicago, ZZ", country: "US" } still fails. A market that states its own country and a country field that disagree are rejected with market country conflicts with country rather than pricing one country’s market against another country’s ladder — Chicago, ZZ is rejected with market must be "City" or "City, ST" rather than quietly becoming the city-only Chicago bucket, while Toronto, Ontario, Canada still reads as locality text. Market and region matching is case-insensitive at the repository, and computeObservationFingerprint hashes the lower-cased market together with the upper-cased region and country — so London, England, GB and London, Ontario, CA stay distinct facts instead of colliding on the bare city, so chicago, il and Chicago, IL read and dedupe as one bucket. City text is never re-cased for display (St. Louis stays St. Louis); only the region code is canonicalized, to IL.
The ≥3-organization privacy gate runs per unit and runs before the unit is chosen, so a unit whose OBSERVED rows the gate empties can never win the selection and hide another unit’s published evidence — it is never named as pricing_unit nor listed in other_units, so the response does not reveal which units a withheld bucket holds. observed_org_count is the real number of organizations behind the selected unit’s observed rows, counted before the gate and never pooled across units, and observed_included is simply that count reaching 3 — a withheld unit therefore reports its true org count with observed_included: false, while a bucket with no selected unit reports 0 and false. When nothing survives the gate the answer is coverage: "none" at the neutral PER_EVENT with other_units: []. A unit named explicitly in pricing_unit stays fixed and reports an honest coverage: "none" when the gate empties it.
A range never mixes units. When pricing_unit is not supplied, the bucket’s dominant unit is chosen — most rows wins, then the greater OBSERVED + PUBLISHED weight, then alphabetical — and only rows of that unit reach range, basis and evidence. PER_EVENT is reported when there is nothing to report. Units present in the bucket that the range does not aggregate are listed in other_units, so a caller can re-ask with an explicit pricing_unit.
Rate limiting
The route has its own per-client-IP token bucket:PUBLIC_PRICING_RATELIMIT_RPS (default 2), PUBLIC_PRICING_RATELIMIT_BURST (default 10), PUBLIC_PRICING_RATELIMIT_TRUSTED_PROXY_COUNT (falls back to PUBLIC_RATELIMIT_TRUSTED_PROXY_COUNT). REST and Direct MCP share the singleton via allowPricingLookup, so each charges once at its own transport boundary. Over the limit returns 429 with Retry-After. The lookup is database-only — no model call — so the limits are looser than the public normalizer’s.
Environment variables
Full inventory: environment variables.
Ingest runbook
Both sweeps are internal HTTP endpoints on api-events. They write through the repository’s idempotent upsert, so re-running them is safe. They are not CLI scripts. Atsx script cannot drive these operations: importing src/db/prisma pulls src/config/config and therefore encore.dev/config, which aborts with ENCORE_RUNTIME_LIB is not set outside encore run. The work has to happen inside the running service.
Both routes are guarded by the same Cloud Scheduler OIDC verifier as /internal/scheduled-jobs/network-density-snapshot/enqueue: an unsigned or wrongly-audienced request gets a bare 401. The caller identity and audience come from scheduler_http_cloud_run_env in infra-gcp/modules/events/scheduled-jobs.tf — SCHEDULER_HTTP_CALLER_EMAIL is google_service_account.scheduler_http.email (a per-environment, random-suffixed account, so read it from Terraform output rather than guessing) and SCHEDULER_HTTP_OIDC_AUDIENCE is local.scheduler_http_oidc_audience, which defaults to soundcheck.api-events/scheduled-jobs/enqueue unless a scheduled-jobs.json entry overrides it.
Triggering a sweep by hand
module.events.domain_name in infra-gcp/environments/dev/main.tf. Impersonation needs roles/iam.serviceAccountTokenCreator on that account.
Soundcheck actuals
POST /internal/pricing-knowledge/ingest-actuals
Both sweeps observe the lock’s abort signal — it reaches
publicIngestOperation and actualsIngestOperation as signal, is checked before every provider call, before each Jev classification (and combined with the evaluator’s own timeout so an in-flight evaluation aborts too), before each batch read, write and the reconciliation delete, and is combined with the request timeout on the context.dev fetch — so a dropped lock session stops the run instead of letting an unlocked sweep keep spending credits. Both internal routes take the shared advisory job lock (pricing-knowledge:ingest-actuals / pricing-knowledge:ingest-public) for the whole sweep, so two overlapping scheduler or manual calls can never both spend credits or both reconcile. The second caller gets 200 with { "outcome": "already_running" } and nothing else — the documented behaviour for private manual calls — and every run logs Scheduled maintenance job finished with job, outcome and duration_ms. A completed run carries outcome: "completed" alongside its summary.
Returns { outcome, scanned, upserted, removed, reconciled, skipped }, where skipped counts each reason (no_market, unknown_category, unsupported_currency, missing_amount, missing_date, not_past). Rows land as OBSERVED with the owning organization_id stamped, which is what the ≥3-org gate counts. Start with {"dry_run":true} and read the counts before writing.
Effective rate. A crew offer is priced the way systems/events/cardPaySummary.ts prices it, not re-derived. An offer is per-day-rated only when it is not hourly and every day carries its own fee — a complete rate card, the same test crewOfferSideEffects.offerIsPerDayRated applies to the payable. Each day then bills its own fee as a PER_DAY row; a partial rate card bills the flat offer fee instead, exactly as the payable does. A recipient counts as engaged when its rollup is ACCEPTED or it holds at least one ACCEPTED day — a multi-day answer can leave the rollup PENDING while days are already claimed and paid. The flat whole-offer row still needs the rollup, because that is the only path that writes a flat payable; the per-day rows follow the accepted days alone. For each engaged recipient the sweep emits what db/crewOffer.ts resolveAcceptedOfferPayable would pay, in its order: zero accepted days on a rate card records nothing at all (the payable is voided, so counting a fee there would invent money that was never paid — skipped as no_accepted_days); a negotiated recipient.fee is the whole-offer amount and lands as one PER_EVENT row (recipient:<id>), never once per day; otherwise each accepted day lands at its own rate (recipient:<id>:day:<dayId>). A recipient on an offer with no rate card takes the flat recipient.fee ?? offer.fee row and needs the offer date. Offer-level rows (day:<dayId>, or '') are the fallback for a FILLED offer with no accepted recipient, and a recipient the sweep cannot date is counted as missing_date without dropping the rows other recipients produced. A negotiated recipient rate therefore wins over the day rate, exactly as the payable does.
Reconciliation. A source that stops qualifying — cancelled, voided, amount cleared, category unmapped — would otherwise leave its derived row behind forever. A full sweep (no since, not a dry run) records every (source_record_type, source_record_id, dimension_key) it wrote and then deletes the OBSERVED rows in scope that it did not write, reporting the count as removed and reconciled: true. Scope is the four swept source record types, narrowed to organization_id when the request named one. Deleting is safe here because these rows are derived — provenance lives on the source tables. A since-scoped sweep or a dry run only sees part of the corpus, so it never deletes and reports reconciled: false. Rows written under an older dimension_key shape are stale by the same rule and are cleared on the next full sweep.
observed_at is derived from the source records that set the amount — the newest updated_at/created_at among them, else the row’s civil day at UTC midnight — never the sweep’s wall clock. The set is exactly the records the amount came from, so an unrelated edit elsewhere on the offer never refreshes it: an accepted per-day row follows its recipient-day row and its CrewOfferDay, a negotiated whole-offer row follows only the recipient, and a recipient row that inherits the flat offer fee follows the recipient and the offer. The sweep is therefore idempotent: re-running it upserts the same observed_at, so an old transaction keeps ageing and the 18-month decay still applies.
Published web evidence
POST /internal/pricing-knowledge/ingest-public
max_pairs exists because the default seed grid is 30 pairs and each pair is a live extraction: one HTTP call has to finish inside the Cloud Run request deadline. When the plan is truncated the response carries remaining_pairs and stopped_reason: "max_pairs" — call again with a narrower markets/categories to cover the rest.
max_credits is reserved before each call, not charged after it: the mode’s known cost (fast 10, ultra 100) must fit inside the remaining budget or the sweep stops with max_credits_reached, and a response that carries no credit metadata is charged the mode cost rather than zero.
The client retries only a 429, which context.dev rejects before execution and therefore never charges; a 502/504 is ambiguous — the work may already have run and been billed — so it fails straight to a ContextDevRequestError rather than paying for a second attempt, and the result carries attempts. The sweep charges the mode cost for any ambiguous failure (transport error, 502, 504), counts it against the ceiling and records it on the failed run row with credits_assumed: true; a definite rejection context.dev never executed (401, 403, 429) is charged nothing. The reserve is also checked before each call: when the last reported credits_remaining minus the mode cost would fall under PRICING_CONTEXTDEV_CREDIT_RESERVE, the sweep stops with credit_reserve_reached rather than making the call that would breach it.
Returns { outcome, pairs, remaining_pairs, credits_consumed, max_credits_applied, stopped_reason, skipped, tasks, runs }. Each entry in runs reports status, credits, parsed, rejected and inserted for one market/category pair. Every pair also records a PricingExtractionRun row with its task, credits consumed and remaining, and the source URLs it read, so spend is auditable. Credits are real money — always start with dry_run and a small max_credits.
A candidate’s source_url is kept only when it matches one of the pages the run actually read (exact match after trimming and dropping a trailing slash or fragment, or the same origin and path). The URL persisted is the run source’s own canonical form — host lower-cased, fragment and trailing slash stripped, promotional parameters (utm_* and the click identifiers gclid, dclid, fbclid, msclkid, mc_cid, mc_eid) dropped and the remaining query sorted — generic parameters such as ref or source are kept because a vendor may use them to select a rate card — not the candidate’s spelling, so the same page hashes to one fingerprint instead of a fresh duplicate on every sweep, even when context.dev itself reports it with a different campaign tag each run. A query that actually selects the page (?page=2) is kept, and the same canonicalization decides whether a candidate matches a run source. An unmatched URL is dropped to null and counted as unverified_source_url, which leaves the observation INFERRED rather than letting a fabricated link become PUBLISHED provenance. The single-run-source fallback still applies when the candidate states no URL at all.
stopped_reason is max_credits_reached, credit_reserve_reached, context_dev_unauthorized, context_dev_rate_limited or max_pairs. skipped: "context_dev_disabled" means PRICING_CONTEXTDEV_ENABLED is off or no API key is set — no run rows are written at all.
A weekly Cloud Scheduler job is a follow-up PR, not built. Today both routes are triggered by hand. When it lands it needs a scheduled-jobs.json entry with scheduler_http_enqueue_path, its Pub/Sub topic, the infra.config.*.json wiring and the infra-gcp variables.tf — not an Encore CronJob — see scheduled maintenance.
The PRO-421 seam
PRO-421 (quote_event) builds a whole-event quote. It must not re-derive prices: it calls lookupMarketPricingOperation once per service category and assembles the quote from the returned ranges, carrying coverage, basis, evidence and explanation through to its own output so the quote stays as honest as the lookup.
That is why event_type, guest_count and duration_hours are already on the request: when PRO-421 needs dimension-filtered ranges, the filtering lands inside lookupMarketPricingOperation and both the HTTP contract and the MCP tool shape stay unchanged.