The reality for most organizations is that data lives everywhere. As engineering teams expand and adopt specialized platforms for their workloads, enterprise data naturally disperses across multiple clouds and operational systems. When analytical and AI initiatives require context from across the entire organization,these teams traditionally face a difficult trade-off: build complex extraction pipelines to copy data into a central repository or accept fragmented reporting.
Traditional architectures address this fragmentation with scheduled extraction pipelines that copy remote datasets into a central staging warehouse. This approach creates synchronization lag, adds recurring storage costs, and triggers variable network egress fees. When schemas change upstream, pipelines break, and teams spend hours reconciling metrics across out-of-sync dashboards.
BigQuery and Google Cloud’s borderless Lakehouse offer a streamlined way to solve this challenge with a foundation based on the open Apache Iceberg REST catalog specification. Rather than moving data to the compute engine, you can bring Google Cloud compute and generative AI directly to your data, querying it in place across Databricks, Snowflake, and AWS Glue.
In this post, we walk through how to configure Databricks Unity Catalog for federation into BigQuery (AWS Glue and Snowflake set up in appendix), run a single query joining data across all three external catalogs, and use conversational analytics in BigQuery to explore the consolidated data with natural language.
