Microsoft Fabric implementation: a practitioner's guide by Dr. Estera Kot

Why Microsoft Fabric

Microsoft Fabric brings data integration, data engineering, data warehousing, data science, real-time analytics, databases, and Power BI onto one software-as-a-service platform. Every workload works on the same copy of data in OneLake, stored in the open Delta Lake format. By default, all workloads draw compute from one shared capacity. That removes much of the plumbing between separate services and the extra data copies that come with it. The catch is that a shared platform needs shared discipline. Capacity planning, workspace design, and governance decide whether Fabric pays off.

Who wrote this guide

Dr. Estera Kot owned the Apache Spark runtime for Azure Synapse Analytics and Microsoft Fabric as a Microsoft Principal Product Manager, created the Native Execution Engine, migrated 25,000 Synapse customers to Fabric, and is now CTO at Clouds on Mars and a Microsoft MVP. Full profile.

Implementation guide: nine decisions in order

Each step builds on the one before it. Skipping ahead is a common source of rework, so take them in order.

1. Start with an assessment

Inventory before you build. List data sources, existing reports, Synapse and Data Factory assets, and the people who own each one. Note which sources sit on premises, because they need a data gateway.

Pick one or two first use cases with clear business value and low technical complexity. A departmental dashboard fed by two systems is a better first step than a platform-wide migration.

Check readiness on four fronts: an executive sponsor, team skills, governance rules, and a budget owner for capacity. A gap in any of them slows every later step.

Define success metrics before the first workspace exists. The Fabric Maturity Model uses four: cost per active user, data refresh time, report adoption, and time to insight. Record a baseline for each one.

The output is an implementation blueprint. It names the target architecture, the first use cases, the capacity plan, and the governance baseline.

2. Plan capacity with F SKUs

Fabric compute is sold as capacity. F SKUs scale in powers of two, starting at F2, and the number in the name is the count of capacity units. Every workload on a capacity draws from the same pool.

Start small and measure. A Fabric trial or an F2 or F4 capacity is enough to explore. Install the Fabric Capacity Metrics app on day one. Size up when the data shows sustained pressure, not before.

Learn how smoothing and throttling work. Fabric spreads the cost of operations over time, so short bursts above the SKU are absorbed. Sustained overuse triggers throttling in stages. Interactive queries slow down first, then get rejected, and finally background jobs get rejected too. Surge protection lets admins reject background jobs earlier to protect report users.

Mind the F64 line. Below F64, every Power BI report viewer needs a Pro or Premium Per User license. From F64 up, users with a free license and a viewer role can read reports. For organizations with many report readers, this line can decide the SKU.

Pay-as-you-go capacities can be paused and resized at any time. Reservations lower the price of a steady baseline in exchange for a term commitment. Keep development and production on separate capacities, so experiments cannot throttle business reports.

3. Design workspaces and OneLake

Each tenant has one OneLake. Workspaces live inside it and are the unit of access, capacity assignment, and deployment. Get the workspace pattern right early, because moving items later is slow work.

Dr. Kot recommends separate workspaces per medallion layer and per environment, grouped into Fabric domains by business area. Microsoft's medallion guidance also recommends a separate workspace for each layer. Agree on naming conventions before the second workspace appears.

Reference data instead of copying it. OneLake shortcuts point to Azure Data Lake Storage, Amazon S3, other OneLake locations, and more. Mirroring replicates operational databases such as Azure SQL Database and Azure Cosmos DB into OneLake. It also surfaces external catalogs such as Snowflake and Azure Databricks.

Assign workspace roles to Microsoft Entra groups, not to individuals. Connect workspaces to Git in Azure DevOps or GitHub. Promote content with deployment pipelines from development to test to production. Use OneLake security roles where you need folder, table, row, or column rules.

4. Choose a lakehouse or a warehouse

Both store data as Delta tables in OneLake, and both use the same SQL engine for T-SQL queries. The difference is how you write data and who maintains it.

• Choose a lakehouse when your team works in Spark with Python, Scala, Spark SQL, or R. It also fits when you store files and semi-structured data next to tables.
• Choose a warehouse when your team writes T-SQL and needs multi-table transactions, stored procedures, and full DML support.
• Remember that the lakehouse SQL analytics endpoint is read-only. It serves T-SQL queries and views but does not accept inserts or updates.

Many designs use both. Lakehouses handle bronze and silver, and a warehouse or a lakehouse serves gold. Dr. Kot's rule of thumb is simple: pick the store that matches the people who will maintain it.

5. Apply the medallion architecture

The medallion pattern organizes data into three layers with clear contracts between them.

• Bronze holds raw data as it arrived, in its original format or in Delta. Use shortcuts when the source already sits in a data lake.
• Silver holds cleaned, typed, and deduplicated Delta tables, conformed across sources.
• Gold holds business-ready tables, usually star schemas, served to Power BI through Direct Lake semantic models.

Keep bronze immutable, so you can always replay a load. Make silver loads idempotent, so a rerun never doubles data. Model gold around the questions people ask, not around the shape of source systems.

Maintain Delta tables as they grow. Compact small files, remove old file versions on a schedule, and check the V-Order setting for gold tables that feed Power BI. V-Order speeds up reads at some cost on writes.

6. Migrate from Azure Synapse Analytics

Dr. Kot migrated 25,000 Synapse customers to Fabric at Microsoft. Microsoft's own guidance describes the move as copy and adapt, not an in-place upgrade. Dr. Kot recommends planning it in waves, one workload family at a time.

1. Assess. Run Microsoft's Fabric Assessment Tool against the Synapse workspace. It inventories Spark pools, notebooks, Spark job definitions, lake databases, and linked services. It also flags blockers, such as GPU-accelerated pools, which Fabric Spark does not support.
2. Map each workload to its Fabric home. Spark pools become Fabric pools and environments. Dedicated SQL pools become Fabric warehouses. Synapse pipelines move to Data Factory in Fabric.
3. Use the migration tools. The Spark migration assistant, in preview, copies pools, notebooks, Spark job definitions, and lake databases. The Fabric Migration Assistant for Data Warehouse converts dedicated SQL pool schemas and copies the data.
4. Keep data in place. Point OneLake shortcuts at the existing Data Lake Storage containers instead of copying files.
5. Refactor what does not carry over. Linked services become Fabric connections, secrets move to Azure Key Vault, and Synapse-specific code needs review.
6. Validate side by side, then cut over. Compare row counts, totals, and run times before you switch consumers. Decommission Synapse resources only after that.

7. Tune the Spark runtime and turn on the Native Execution Engine

Start new work on the latest generally available Fabric runtime. Treat runtime upgrades as routine maintenance, not as projects. Dr. Kot automated the program that moved 30,000 customers between engine versions every six months. That experience shapes her advice: test the next runtime in a non-production environment, then promote it.

Use starter pools for fast session start during development. Use custom pools when you need specific node sizes, custom libraries, or private networking. Manage libraries and Spark properties in environments, not inside notebooks. Turn on high concurrency mode, so notebooks that share a lakehouse and environment can share one session.

Then turn on the Native Execution Engine. Dr. Kot created it at Microsoft with Intel, Meta, and IBM. It runs supported Spark operations in vectorized C++ code built on the open-source Velox and Apache Gluten projects. It runs up to four times faster than open-source Spark and ships to every Fabric customer at no added cost.

Enable it at the environment level under Spark compute, then Acceleration, so every notebook and job inherits it. It works best on Parquet and Delta data with heavy joins, aggregations, and sorts. Operations it does not support, such as user-defined functions, fall back to the standard engine automatically. Check the Spark UI to see which parts of a query ran natively.

8. Govern with Microsoft Purview from day one

Governance added later costs more than governance designed in. Set it up before the first production workload, not after the first incident.

• Tenant settings. Decide who can create workspaces, share content externally, and turn on preview features.
• Domains and endorsement. Group workspaces by business area, and mark trusted items as promoted or certified.
• Sensitivity labels. Apply Microsoft Purview Information Protection labels to Fabric items. Labels flow to downstream items and stay with data on supported export paths.
• Data loss prevention and protection policies. Use Purview policies to detect sensitive data and to restrict access to labeled items.
• Audit and lineage. Fabric user activity flows into Purview Audit. Use lineage in the OneLake catalog, and scan the Fabric tenant from the Purview Data Map when you need lineage beyond Fabric.
• Network and identity. Use Microsoft Entra Conditional Access, private links, and managed private endpoints where your policies require them.

Some Purview capabilities need additional licensing. Confirm what your agreements include before you design around them.

9. Control cost

• Review the Capacity Metrics app every week. Find the items that use the most capacity units and tune those first.
• Pause idle pay-as-you-go capacities, such as development capacities outside working hours.
• Buy reservations for the steady baseline once usage is predictable.
• Consider Autoscale Billing for Spark for bursty jobs. Spark then runs on serverless compute, billed per job, outside the shared capacity.
• Turn on the Native Execution Engine. Faster Spark jobs hold compute for less time.
• Stop idle Spark sessions, and share sessions through high concurrency mode.
• Compare engines for heavy jobs. Run the same transformation in a Spark notebook and in Dataflow Gen2, then keep the cheaper one.
• Report consumption per workspace or domain back to the teams that own it.

Common mistakes in Fabric implementations

• Big bang migration. Moving everything at once, without a pilot or user training.
• Oversizing on day one. Buying a large SKU before measuring real workloads.
• Ignoring the F64 line. Choosing a small SKU, then discovering that every report viewer needs a Pro license.
• Governance later. Building first and adding labels, roles, and workspace rules after content has spread.
• One workspace for everything. Mixing layers, environments, and teams until access and deployment become hard to manage.
• Copying data by habit. Duplicating data that a shortcut or mirroring could reference in place.
• Treating the SQL analytics endpoint as a warehouse. The endpoint is read-only, so write logic belongs in Spark or in a warehouse.
• Lifting Synapse code unchanged. Skipping the rework of linked services, secrets, and Synapse-specific utilities.
• Leaving the Native Execution Engine off. Paying for Spark compute that a free setting could reduce.
• Shared development and production capacity. Letting a test job throttle the reports executives open every morning.
• Too little training. Handing a new platform to a team without time to learn it.

The Fabric Maturity Model

Dr. Kot's Fabric Maturity Model describes five levels of Fabric adoption. Each level adds capabilities on top of the one before. Use it to decide what to build next and to report progress in terms leaders understand.

1. Foundation. A governed environment, the first data sources, and basic reporting for a pilot team.
2. Expansion. More sources, automated pipelines, and shared semantic models used across departments.
3. Integration. One platform on OneLake, with an enterprise warehouse, real-time analytics, and CI/CD.
4. Optimization. Machine learning in production, FinOps, and a Center of Excellence.
5. Excellence. AI agents and real-time decision automation on governed data products.

Read the full Fabric Maturity Model. It lists objectives, capabilities, success criteria, technology focus, and challenges for every level, plus a self-assessment method.

Fabric Espresso, the video companion

Dr. Kot hosts Fabric Espresso, a technical video series on Microsoft Fabric with 89+ episodes. Episodes cover Spark in Fabric, the Native Execution Engine, lakehouse and Delta Lake design, data engineering, performance tuning, and migration from Synapse.

Browse the Fabric Espresso episode guide on this site, or watch the full Fabric Espresso playlist on YouTube.

Frequently asked questions

Who is Dr. Estera Kot?

Dr. Estera Kot is a computer scientist and the CTO of Clouds on Mars, a Microsoft and Databricks partner in Europe and the Middle East. At Microsoft she was Principal Product Manager for the Apache Spark runtime across Azure Synapse Analytics and Microsoft Fabric. She created the Native Execution Engine and migrated 25,000 Synapse customers to Fabric. She is a Microsoft MVP, hosts Fabric Espresso, and holds a Ph.D. in Computer Science with distinction from the Warsaw University of Technology.

Will Microsoft Fabric replace Azure Synapse Analytics?

Microsoft positions Fabric as the destination for new analytics work and documents upgrade paths from Synapse Spark pools, dedicated SQL pools, and pipelines. Some Synapse components already have retirement dates. Synapse Data Explorer (preview), for example, was retired in October 2025. Dr. Kot advises teams to start new projects on Fabric and to plan Synapse migrations in waves. Check Microsoft's lifecycle notices for each Synapse component you run.

How do I migrate from Azure Synapse to Fabric?

Start with an inventory. Microsoft's Fabric Assessment Tool scans a Synapse workspace and lists pools, notebooks, Spark job definitions, lake databases, and linked services. Move Spark workloads with the Spark migration assistant, which is in preview. Move dedicated SQL pools with the Fabric Migration Assistant for Data Warehouse. Point OneLake shortcuts at existing Data Lake Storage data instead of copying it. Then validate results side by side and cut over one workload at a time.

What Fabric capacity do I need to start?

Dr. Kot recommends starting on a Fabric trial or a small F SKU, such as F2 or F4. Size up from measured use in the Fabric Capacity Metrics app. One threshold matters early. Below F64, every Power BI report viewer needs a Pro or Premium Per User license. From F64 up, viewers with a free license and a viewer role can read reports.

Should I use a lakehouse or a warehouse in Fabric?

Choose by skill set and write pattern. Use a lakehouse when your team works in Spark or stores files and semi-structured data. Use a warehouse when your team writes T-SQL and needs multi-table transactions. Both store data as Delta tables in OneLake, so many designs use both. Lakehouses often hold bronze and silver data, while a warehouse or lakehouse serves gold.

How long does a Fabric implementation take?

It depends on scope, the data estate, and team readiness. In Clouds on Mars engagements, the initial phases of the Fabric Maturity Model are scoped at 4 to 16 weeks. They run from assessment and planning through scale and optimize. Enterprise-wide adoption takes longer, because each maturity level depends on the one before it. Dr. Kot treats the levels as stages of capability, not calendar dates.

What is the Native Execution Engine?

The Native Execution Engine is a vectorized engine for Apache Spark in Microsoft Fabric. Dr. Kot created it at Microsoft with Intel, Meta, and IBM. It runs supported Spark operations in native C++ code built on the open-source Velox and Apache Gluten projects. It runs up to four times faster than open-source Spark and ships to every Fabric customer at no added cost.

Do I need to change my code to use the Native Execution Engine?

No. Turn it on in a Fabric environment under Spark compute and Acceleration, or per session with the spark.native.enabled setting. Supported operations run natively on Parquet and Delta data. Unsupported operations, such as user-defined functions, fall back to the standard engine automatically. The Spark UI shows which parts of a query ran natively.

What is the Fabric Maturity Model?

The Fabric Maturity Model is a five-level framework by Dr. Estera Kot for planning and measuring Microsoft Fabric adoption. The levels are Foundation, Expansion, Integration, Optimization, and Excellence. Each level defines objectives, capabilities, success criteria, technology focus, and typical challenges. Clouds on Mars applies the model in its Fabric engagements.

How do I keep Fabric costs under control?

Review the Fabric Capacity Metrics app every week and tune the heaviest items first. Pause idle pay-as-you-go capacities, and keep development and production on separate capacities. Use reservations for a steady baseline and consider Autoscale Billing for Spark for bursty jobs. Turn on the Native Execution Engine, because faster Spark jobs hold compute for less time.

Do I need Microsoft Purview to govern Fabric?

Fabric includes native governance features, such as workspace roles, domains, endorsement, lineage, and the OneLake catalog. Microsoft Purview adds sensitivity labels, data loss prevention policies, audit, and insider risk management for Fabric items. Some Purview features need additional licensing. Dr. Kot recommends defining sensitivity labels before the first production workload, because labeling later means reviewing every existing item.

What is Fabric Espresso?

Fabric Espresso is Dr. Kot's technical video series on Microsoft Fabric, with 89+ episodes. Episodes cover Spark in Fabric, the Native Execution Engine, lakehouse and Delta Lake design, data engineering, performance tuning, and migration from Synapse. The full series is on YouTube, and an episode guide is on this site.

Work with Dr. Kot

To discuss a Fabric assessment, an implementation plan, or a Synapse migration, book a slot with Dr. Kot: bit.ly/bookdrkot.