Lightweight Master Data Management on Microsoft Fabric
Metadata-driven golden records across multiple sources — without a standalone enterprise MDM product.
Organisations can maintain a lightweight, Fabric-native MDM capability inside their existing data platform rather than purchasing full enterprise MDM suites for every entity.
Featured Use Case
Fabric-Native Golden Records
Conform, match, steward, and consolidate multi-source master data using lakehouses, a Metadata Registry, and a Stewardship App built with Microsoft Fabric Apps (preview).
Overview
A practical path to trusted cross-source master data on Microsoft Fabric — without waiting for a full MDM programme.
Most organisations run the same business entities — customers, sites, products — across several operational systems. Each system has its own keys and slightly different attributes. Without a deliberate master data approach, downstream reporting and dimensions inherit duplicates, conflicts, and broken joins.
This use case shows how PODS Analytics implements a lightweight MDM capability on Microsoft Fabric: source-aligned lakehouse data, metadata-driven conform and match stages, a human stewardship gate via Fabric Apps (preview), and consolidate into stable master keys that the Integrated and Analytics layers consume.
The point is not to replace every enterprise MDM product. It is to give Fabric estates a governed, extensible pattern for the entities that matter most — so you can deliver trusted golden records without purchasing a full MDM suite up front.
Project Summary
- Use Case Type
- Lightweight MDM / Golden Record
- Platform
- Microsoft Fabric
- Core Components
- Metadata Registry, Source & Master Data Lakehouses, Orchestrated processing stages, Stewardship App (Fabric Apps preview), Integrated & Analytics layers
- Delivery Style
- Metadata-driven, entity-extensible framework
- Outcome
- Trusted cross-source master records with governed human review
The Challenge
Common master data hurdles that a Fabric-native MDM pattern is designed to address.
Duplicate Records Across Sources
The same customer or entity appears in CRM, ERP, and other systems with conflicting names, emails, and identifiers — and analytics inherits the mess.
No Governed Match Decisions
Auto-merge versus human review is often ad hoc: spreadsheets, one-off scripts, or tribal knowledge that does not survive re-runs or staff changes.
Enterprise MDM Cost and Lock-In
Full MDM platforms are powerful but heavy for a focused Fabric-first delivery. Many organisations need trusted golden records without buying a separate suite for every entity.
Architecture
How source systems, lakehouses, orchestration, and the Stewardship App connect on Microsoft Fabric.
Lightweight MDM Architecture on Microsoft Fabric
Fabric Apps (preview)
Metadata configures each processing stage. Ambiguous matches go through a Stewardship App built with Microsoft Fabric Apps (preview); confirmed decisions flow back before consolidate publishes trusted master keys downstream.
Why a Stewardship App?
Lakehouse SQL endpoints are typically read-only for Delta writes. Microsoft Fabric Apps (preview) provision a writable app database so stewards can confirm or reject matches transactionally. Stewardship bridge jobs move the review queue out and decisions back into the master data domain before consolidate runs.
Why a Metadata Registry?
Entity configuration, source mappings, and match rules drive each processing stage at runtime. That keeps notebook logic generic and entity-extensible — you add configuration rows for a new entity instead of forking one-off pipelines.
The PODS Approach
Seven steps from source landing to trusted golden records in analytics.
Land Source Data
Existing ELT loads source-aligned tables into the Source Lakehouse. MDM never mutates those raw-aligned tables — they remain the system of record for source fidelity.
Configure the Entity
Register the entity, contributing sources, field mappings, and match rules in the Metadata Registry. Processing stages read this configuration at runtime instead of hardcoding mappings.
Conform
Harmonise records from each source into a standard shape: source keys, normalised attributes, and a stable conformed identity per source system record.
Match
Score cross-source pairs with exact and fuzzy rules. High-confidence matches are auto-confirmed; ambiguous pairs are flagged for steward review.
Steward Review
Publish the review queue to a Stewardship App built with Microsoft Fabric Apps (preview). Stewards compare side-by-side records and confirm or reject pairs in a governed UI.
Consolidate
Assign a stable master key, build a unified cross-source view, and select an anchor record using source priority so downstream consumers get one trusted identity.
Publish Downstream
The Integrated Layer exposes golden (anchor) records. Analytics dimensions and facts resolve via master keys so reporting stays consistent across sources.
Stewardship App in Action
Ambiguous matches land in a review queue. Stewards compare side-by-side attributes, then confirm or reject — built with Microsoft Fabric Apps (preview).
Each review card shows a match score, the rules that fired, and canonical attributes from both sources. Optional notes travel with the decision so consolidations stay auditable.
From Source Keys to Master Keys
A thin slice of the key story — not full table schemas. Values below mirror a working Fabric run: Manual and SalesLT sources, scored pairs, then a shared Master Key with an anchor record.
Conform
Each source record gets a stable Conformed Key from source system + source key. Attributes are normalised for matching.
| SourceSystem | SourceKey | ConformedKey | FullName | |
|---|---|---|---|---|
| Manual | 1 | 70331945… | Bike World | — |
| Manual | 11 | a8c41e2f… | Great Bicycle Store | — |
| SalesLT | 29535 | 3f9b0d71… | Bike World Store | bikeworld@example.com |
| SalesLT | 346 | e2c7a91d… | Great Bicycle Store | greatbikes@example.com |
Match
Cross-source pairs are scored. Perfect and near-perfect scores auto-confirm; grey-zone pairs wait for steward review.
| ConformedKey A | ConformedKey B | Source A | Source B | Score | Status |
|---|---|---|---|---|---|
| 70331945… | 3f9b0d71… | Manual | SalesLT | 1.00 | AutoConfirmed |
| a8c41e2f… | e2c7a91d… | Manual | SalesLT | 0.94 | AutoConfirmed |
| 97f8a8ca… | c4d12e90… | Manual | SalesLT | 0.84 | NeedsReview |
Master
Confirmed clusters share one Master Key. The highest-priority source becomes the anchor (golden) record — here SalesLT wins the anchor.
| MasterKey | ConformedKey | SourceSystem | SourceKey | IsAnchor |
|---|---|---|---|---|
| 053e6c9b… | 70331945… | Manual | 1 | false |
| 053e6c9b… | 3f9b0d71… | SalesLT | 29535 | true |
| 7a1b4c2e… | a8c41e2f… | Manual | 11 | false |
| 7a1b4c2e… | e2c7a91d… | SalesLT | 346 | true |
Keys are truncated for display. Ambiguous pairs (for example score 0.84) go to the Stewardship App before consolidate writes the Master Key.
Key Capabilities Delivered
Core capabilities enabled by a lightweight, Fabric-native MDM pattern.
Metadata-Driven Entity Configuration
Entity definitions, source priorities, field mappings, and match rules live in the Metadata Registry — not scattered across notebook code.
Multi-Source Conform and Matching
Exact and fuzzy scoring across sources, with clear separation between auto-confirmed matches and pairs that need human review.
Human-in-the-Loop Stewardship
A Stewardship App on Microsoft Fabric Apps (preview) gives data stewards a compare-and-decide experience for ambiguous matches.
Orchestrated MDM Pipeline
A schedulable MDM Orchestration Pipeline runs conform, match, stewardship bridge jobs, and consolidate in a controlled sequence.
Downstream Dimension and Fact Integration
Integrated and Analytics layers consume master keys so dimensions and facts stay aligned after merges.
Entity Extensibility
Add a new entity (for example Site or Product) primarily through configuration rows — reuse the same processing stages instead of forking code per entity.
Key Considerations
Design choices that keep a lightweight MDM pattern supportable and production-safe.
Lightweight vs Enterprise MDM
This pattern suits focused entities and Fabric-centric estates. Advanced survivorship, deep hierarchy management, and enterprise workflow tooling may still call for a dedicated MDM product later — but you do not need to buy one on day one.
Metadata-First Design
Mappings and rules live in the Metadata Registry. Processing stages load configuration at runtime so onboarding a source or tuning a match rule is a configuration change, not a code rewrite.
Write-Once Match Decisions
Auto and steward decisions must not be overwritten on re-run. Write-once semantics keep audits clean and protect SME judgements across pipeline replays.
Source Priority for Anchor Records
When consolidating, select an anchor (golden) record by source priority. In the first release, the highest-priority source wins the published attributes; field-level survivorship can follow later.
Stewardship Bridge Pattern
Lakehouse SQL endpoints are typically read-only for Delta writes. Fabric Apps provision a writable app database: push the review queue out, capture steward decisions, then bridge decisions back before consolidate runs.
Phased Scope
Start with anchor golden records and basic match rules. Phase later: field-level survivorship, blocking keys for scale, and fine-grained steward permissions per entity.
Testing and Rollback
Design conform and match for idempotent re-runs. Downstream transforms should be able to revert to pre-MDM logic independently while the master data stages continue to evolve.
Outcomes
What organisations gain by implementing lightweight MDM inside their Fabric platform.
Consistent entity keys across analytics
Governed, auditable match decisions
Reusable pattern for additional entities
Fabric-native delivery without separate MDM licensing
Other PODS Use Cases
Additional reusable patterns and frameworks from PODS Analytics.
Metadata-Driven Monitoring & Alerting
Reusable operational visibility and alerting patterns on Microsoft Fabric.
ExploreRide Ops Demo
Interactive streaming operations demo built on PODS platform patterns.
ExplorePlatform Overview
How PODS accelerates Microsoft Fabric delivery with reusable accelerators.
ExploreWant to build lightweight MDM on Fabric?
Talk to PODS Analytics about metadata-driven master data, stewardship with Fabric Apps, or reusable Microsoft Fabric delivery patterns.