Tag: bigquery

A dbt on BigQuery starter kit for the parts that usually fail after the demo: raw loads without partition filters, snapshots with weak change detection, and CI that lets expensive SQL promote.

Outcome: Reader can scaffold a dbt and BigQuery project with manifest-backed incremental loads, timestamp-first snapshots, partitioned models, and a dry-run bytes gate before production promotion.

How to use BigQuery primary and foreign key constraints from dbt without confusing optimizer metadata for enforced data integrity.

Outcome: Defined a BigQuery and dbt constraint playbook that keeps optimizer hints, dbt contracts, data tests, compiled SQL review, and INFORMATION_SCHEMA verification in the right order.

A Dataform and BigQuery case study for turning data contracts, release lanes, validation gates, rollback behavior, and cost checks into one governed promotion path.

Outcome: Reduced contract-break risk in the sanitized release pattern by making schema, freshness, cost, and downstream impact checks part of promotion instead of after-the-fact review.

A sanitized GCP platform case study where compliance, analytics delivery, and ML feature access had to be designed as one release path instead of three disconnected workstreams.

Outcome: Reduced governed dataset onboarding from weeks to days in the sanitized pattern while preserving auditability, cost visibility, and promotion rules for analytics and ML use cases.

A practical runbook for scoring changed rows close to the data using Snowflake Streams and Tasks or BigQuery scheduled queries and remote models.

Outcome: Compared Snowflake and BigQuery patterns for scheduled in-warehouse inference, corrected CDC assumptions, and defined monitoring, grants, and deployment checks.