اعرف ما لديك من بيانات.اعرف ما تعنيه.

When Data Volume Becomes a Discoverability Problem

At a certain scale, the hardest data problem is not storage or compute — it is discovery. A data engineer joins the team and needs to know which BigQuery table contains the authoritative customer data. There are 47 tables with "customer" in the name across 12 datasets. Some are operational copies. Some are archived. Some are staging tables that should not be used for reporting. There is no documentation. The only way to know which table is correct is to ask the data engineer who's been there longest.

A data catalog solves this. It makes every data asset visible, described, classified, and owned — so that any data consumer can find what they need, understand what it means, and know who to contact when they have questions.

What We Implement

Asset Registration and Metadata

Every data asset in scope — BigQuery tables, datasets, Cloud Storage objects, Pub/Sub topics — registered in Cloud Data Catalog or Dataplex with structured metadata: description, owner, last updated, sensitivity classification, and usage context. Registration can be manual for high-priority assets or automated via catalog export for large environments.

Business Glossary

A business glossary defines the canonical meaning of data terms: what "Customer" means in the context of the data platform (as opposed to the CRM system, where the same term might mean something slightly different). Glossary terms linked to the data assets that contain the relevant data — so that searching for "Monthly Recurring Revenue" returns the exact table and column that contains it.

Data Classification and Policy Tags

Sensitivity classification applied to data assets: public, internal, confidential, restricted. Policy tags configured in BigQuery Data Policy for columns containing PII, financial data, or other regulated information. Automated PII discovery using Cloud DLP scanning to identify sensitive data that was not manually classified.

Data Lineage

Lineage tracking that shows where data came from and where it goes: which source system populated this table, which transformation pipeline processed it, which downstream tables and dashboards depend on it. Dataplex automatic lineage for BigQuery and Dataflow workflows, supplemented with manual lineage documentation for custom pipelines.

Data Stewardship Model

A data catalog without human ownership becomes stale within months. We design the data stewardship model that keeps it current: designated data owners per domain, a curation workflow for new asset registration, a review cadence for existing metadata, and a process for handling data quality issues identified through the catalog.