https://ekote.github.io/notes/ Technical notes by Dr. Estera Kot on Apache Spark, Microsoft Fabric, lakehouse architecture, and platform migration, on this site and on the Microsoft Fabric Updates Blog. en-us Tue, 03 Nov 2026 22:07:00 +0100 https://ekote.github.io/notes/fabric-real-time-intelligence-fabric-iq-from-zero/ https://ekote.github.io/notes/fabric-real-time-intelligence-fabric-iq-from-zero/ Dr. Estera Kot Tue, 03 Nov 2026 22:07:00 +0100 Real-Time Intelligence and Fabric IQ from zero, on one invented oud house. The demo writes the winner rule twice, in Activator and in the ontology, and Fabric still has no link between them. https://ekote.github.io/notes/fabric-real-time-intelligence-path-of-one-event/ https://ekote.github.io/notes/fabric-real-time-intelligence-path-of-one-event/ Dr. Estera Kot Tue, 03 Nov 2026 22:06:00 +0100 Five parts, one job each. Follow one tester's vote through Eventstream, Eventhouse, a dashboard, and an Activator rule, and see why the alert waits for 20 testers before 60 counts. https://ekote.github.io/notes/fabric-iq-ontology-explained/ https://ekote.github.io/notes/fabric-iq-ontology-explained/ Dr. Estera Kot Tue, 03 Nov 2026 22:05:00 +0100 An ontology in Microsoft Fabric IQ is the dictionary of the business. One worked example: a rule that a winner has first choice above 60 changes the data agent's answer, with no change to the data. https://ekote.github.io/notes/fabric-data-agent-numbers-from-tables/ https://ekote.github.io/notes/fabric-data-agent-numbers-from-tables/ Dr. Estera Kot Tue, 03 Nov 2026 22:04:00 +0100 A Fabric data agent should write the sentence and the query, never the number. One demo answer taken apart, the guards that keep figures in tables, four build failures, and capacity facts. https://ekote.github.io/notes/fabric-iq-demo-to-production/ https://ekote.github.io/notes/fabric-iq-demo-to-production/ Dr. Estera Kot Tue, 03 Nov 2026 22:03:00 +0100 A Fabric IQ demo proves the mechanics, not quality at scale. Six gaps to close, three ways to build, preview risk, and a production checklist for data agents. https://ekote.github.io/notes/fabric-hands-on-workshop-50-people/ https://ekote.github.io/notes/fabric-hands-on-workshop-50-people/ Dr. Estera Kot Tue, 03 Nov 2026 22:02:00 +0100 A hands-on Fabric session fails at sign-in, not at the demo. Accounts, MFA, a CU budget, and teardown for 50 people, from one evening at Microsoft Qatar. https://ekote.github.io/notes/teaching-microsoft-fabric-from-zero/ https://ekote.github.io/notes/teaching-microsoft-fabric-from-zero/ Dr. Estera Kot Tue, 03 Nov 2026 22:01:00 +0100 Beginners leave at the first word nobody explained. How a two-hour Fabric session was built: five level rules, a minute clock, a glossary in pictures, a paper walkthrough, and a recovery line per demo step. https://ekote.github.io/notes/fabcon-europe-2026-fabric-demos/ https://ekote.github.io/notes/fabcon-europe-2026-fabric-demos/ Dr. Estera Kot Tue, 03 Nov 2026 21:18:00 +0100 What FabCon Europe 2026 changed in Microsoft Fabric, read through 18 recorded demos: which door your data now leaves by, what is GA, and what to check first. https://ekote.github.io/notes/fabric-data-in-microsoft-copilot-setting/ https://ekote.github.io/notes/fabric-data-in-microsoft-copilot-setting/ Dr. Estera Kot Tue, 03 Nov 2026 21:17:00 +0100 A Microsoft 365 admin center setting decides who can ask Copilot about Power BI data, and it ships set to All users. Four gates, three owners, and the Monday check. https://ekote.github.io/notes/semantic-model-ai-setting/ https://ekote.github.io/notes/semantic-model-ai-setting/ Dr. Estera Kot Tue, 03 Nov 2026 21:16:00 +0100 Each semantic model has a switch that opens it to Copilot, data agents, and Microsoft MCP tools for read-only users. It starts on, and models built on it do not inherit it. https://ekote.github.io/notes/fabric-iq-mcp-server/ https://ekote.github.io/notes/fabric-iq-mcp-server/ Dr. Estera Kot Tue, 03 Nov 2026 21:15:00 +0100 Fabric IQ MCP is GA: six read-only tools let a coding agent find a semantic model and run DAX as you. Your agent writes the DAX, and users can grant consent themselves by default. https://ekote.github.io/notes/fabric-core-mcp-server/ https://ekote.github.io/notes/fabric-core-mcp-server/ Dr. Estera Kot Tue, 03 Nov 2026 21:14:00 +0100 Fabric Core MCP is GA, and Learn's tools reference lists 43 tools, 25 of them write, including delete_workspace and OneLake role replacement. Where the safeguards sit, and Learn's warning quoted. https://ekote.github.io/notes/onelake-table-read-api/ https://ekote.github.io/notes/onelake-table-read-api/ Dr. Estera Kot Tue, 03 Nov 2026 21:13:00 +0100 The OneLake Table Read API returns Fabric table rows as Apache Arrow over REST, with OneLake RLS and CLS applied on the server. Preview. The request path, six caller outcomes, and the cost catch. https://ekote.github.io/notes/onelake-unity-catalog-iceberg-rest/ https://ekote.github.io/notes/onelake-unity-catalog-iceberg-rest/ Dr. Estera Kot Tue, 03 Nov 2026 21:12:00 +0100 OneLake serves the same Delta tables through the Iceberg REST Catalog and a Unity Catalog compatible API, with no copy. Why OneLake security blocks instead of filtering there, and which door to use. https://ekote.github.io/notes/onelake-catalog-search-api/ https://ekote.github.io/notes/onelake-catalog-search-api/ Dr. Estera Kot Tue, 03 Nov 2026 21:11:00 +0100 One POST to /v1/catalog/search finds Fabric items across every workspace you can access, and Fabric Core MCP runs the same search. Preview. Two permission gates, and what search exposes. https://ekote.github.io/notes/project-osmos-data-engineering-agent/ https://ekote.github.io/notes/project-osmos-data-engineering-agent/ Dr. Estera Kot Tue, 03 Nov 2026 21:10:00 +0100 The Fabric data engineering agent turns one outcome into notebooks and a validated Delta table on SparkCore, as you. The eight-setting card, the permission wall, and the catches. https://ekote.github.io/notes/fabric-operations-agent-monitor-hub/ https://ekote.github.io/notes/fabric-operations-agent-monitor-hub/ Dr. Estera Kot Tue, 03 Nov 2026 21:09:00 +0100 Select Investigate on a failed pipeline run and the operations agent explains the root cause, read-only and as you. It needs workspace monitoring on before the failure. https://ekote.github.io/notes/fabric-iq-ontology/ https://ekote.github.io/notes/fabric-iq-ontology/ Dr. Estera Kot Tue, 03 Nov 2026 21:08:00 +0100 The ontology agent drafts entities, relationships, and metrics from a semantic model and writes only in Act mode. Rules are context, not enforcement. Old ontology retires Jan 31, 2027. https://ekote.github.io/notes/copilot-real-time-dashboard/ https://ekote.github.io/notes/copilot-real-time-dashboard/ Dr. Estera Kot Tue, 03 Nov 2026 21:07:00 +0100 Copilot turns one sentence into a twelve-tile Real-Time Dashboard on an eventhouse. Preview. What it generates, and the four things to review before you share it. https://ekote.github.io/notes/fabric-data-agent-lakehouse/ https://ekote.github.io/notes/fabric-data-agent-lakehouse/ Dr. Estera Kot Tue, 03 Nov 2026 21:06:00 +0100 The Fabric data agent turns English questions into read-only SQL that runs as you. Grounding, the English-only limit, and how it reaches Microsoft 365 Copilot and Copilot Studio. https://ekote.github.io/notes/onelake-security-roles/ https://ekote.github.io/notes/onelake-security-roles/ Dr. Estera Kot Tue, 03 Nov 2026 21:05:00 +0100 OneLake security roles define RLS and CLS once, and every Fabric engine enforces them, shortcuts included. Who they restrict, and why DefaultReader must change first. https://ekote.github.io/notes/fabric-outbound-access-protection/ https://ekote.github.io/notes/fabric-outbound-access-protection/ Dr. Estera Kot Tue, 03 Nov 2026 21:04:00 +0100 Workspace outbound access protection blocks every outbound connection from a Fabric workspace. What it covers (GA for shortcuts, preview for data agents), what it misses, and what it breaks. https://ekote.github.io/notes/fabric-policies-allow-list/ https://ekote.github.io/notes/fabric-policies-allow-list/ Dr. Estera Kot Tue, 03 Nov 2026 21:03:00 +0100 Fabric policies, in preview, allow an action when a rule matches and block everything else, at tenant or capacity scope. How rules evaluate, and why the first rule changes the default. https://ekote.github.io/notes/onelake-catalog-govern/ https://ekote.github.io/notes/onelake-catalog-govern/ Dr. Estera Kot Tue, 03 Nov 2026 21:02:00 +0100 The OneLake catalog Govern tab shows label coverage, DLP results, and recommended actions, refreshed once a day. What it covers, what it misses, and how to turn its cards into a backlog. https://ekote.github.io/notes/fabric-ai-functions-spark/ https://ekote.github.io/notes/fabric-ai-functions-spark/ Dr. Estera Kot Tue, 03 Nov 2026 21:01:00 +0100 ai.translate, ai.analyze_sentiment, and ai.classify on a Fabric DataFrame column through a built-in endpoint with no API key: runtimes, cost per token, regions, and when to bring your own model. https://ekote.github.io/notes/materialized-lake-views-data-quality/ https://ekote.github.io/notes/materialized-lake-views-data-quality/ Dr. Estera Kot Tue, 03 Nov 2026 21:00:00 +0100 Silver and gold as CREATE MATERIALIZED LAKE VIEW statements with CHECK rules: DROP versus FAIL, lineage, incremental refresh, and a demo where three bad rows never reach gold. https://ekote.github.io/notes/fabric-workspace-identity-landing-zone/ https://ekote.github.io/notes/fabric-workspace-identity-landing-zone/ Dr. Estera Kot Tue, 03 Nov 2026 19:53:04 +0100 Fabric meets an Azure Landing Zone at four crossings: identity, secrets, outbound, and inbound. Where workspace identity works, where the gateway stops it, and the Conditional Access trap. https://ekote.github.io/fabric/maturity-model/ https://ekote.github.io/fabric/maturity-model/ Dr. Estera Kot Mon, 19 Oct 2026 01:37:36 +0200 Five levels from Foundation to Excellence, each with objectives, success criteria, and a self-assessment. https://ekote.github.io/fabric/ https://ekote.github.io/fabric/ Dr. Estera Kot Mon, 19 Oct 2026 01:37:36 +0200 Nine implementation decisions in order, from F SKU capacity and OneLake design to Synapse migration and the Native Execution Engine. Common mistakes and an FAQ close it.