Dr. Estera Kot

From Events to Answers · Part 5 of 7

From demo to production: six gaps and three ways to build

A Fabric IQ demo proves the mechanics, not quality at scale. This is what a real agent over data and documents needs.

By Dr. Estera Kot, Microsoft MVP, CTO at Clouds on Mars, and former Microsoft Principal Product Manager for the Apache Spark runtime in Fabric. Published . Product facts checked against Microsoft Learn on November 3, 2026.

Fabric IQ is the layer in Microsoft Fabric that gives agents business meaning. It combines OneLake data, Power BI semantic models, and the ontology. A Fabric data agent is a chat that turns a plain-language question into SQL, DAX, or KQL. It runs that query with the user's permissions and writes the answer. On October 14, 2026, Dr. Kot showed both in Doha, end to end, for an invented oud fragrance house.

The demo worked. A working demo proves less than it seems. It proves the mechanics: the agent writes the query and the sentence, and a table holds the number. It does not prove quality at scale. The hard part of an agent over data and documents is what the demo skipped: extraction, retrieval, evaluation, feedback, and cost.

In short

  1. The Doha demo was checked against six expected answers from 10 tables and 5 PDF reports. It proved the mechanics. It did not prove quality on thousands of real documents.
  2. Six gaps separate the demo from production: scale, extraction, endpoint, search, feedback, and cost. Each gap has a first test. None of them has a measured number yet.
  3. The same layer can be built three ways: your own build on Azure services, Databricks with Unity Catalog, or Fabric. Choose by where your people work and what you must control, not by the best demo.
  4. The ontology and its use as data agent context are preview on Microsoft Learn. Preview can carry a pilot. It cannot carry a production promise.
The gap ladder from a Fabric IQ demo to production A ladder with six rungs. The demo in Doha stands at the bottom and production at the top. Each rung is one gap. Left of each rung is what the demo did. Right of each rung is what production needs. From the bottom: scale, extraction, endpoint, search, feedback, and cost. Production Quality measured at scale, with feedback and a known cost Demo in Doha, October 14, 2026 10 tables, 5 PDF reports, 6 expected answers, synthetic data What the demo did What production needs 1 Scale 5 PDF reports and 10 tables, 6 expected answers checked by hand Thousands of documents, with accuracy measured at that size 2 Extraction 20 passages written in advance. The live extraction cell failed Tables, formulas, and charts pulled from real PDF files 3 Endpoint Optimizer output stored as a table, returned unchanged A model endpoint with a versioned contract and a named owner 4 Search No embeddings, no vector search. SQL over 20 passages Retrieval tested on real questions, with citations 5 Feedback No user feedback recorded, no monitoring of the agent Feedback captured, a golden set run before every publish 6 Cost No cost per 1,000 documents, none per 1,000 questions Both costs measured on one week of real use The gap ladder from a Fabric IQ demo to production A ladder with six rungs, read from the bottom up. The demo in Doha stands at the bottom and production at the top. Each rung names one gap, what the demo did, and what production needs: scale, extraction, endpoint, search, feedback, and cost. Production Demo in Doha, October 14, 2026 6 Cost Demo: No cost per 1,000 documents, none per 1,000 questions Needs: Both costs measured on one week of real use 5 Feedback Demo: No user feedback recorded, no monitoring of the agent Needs: Feedback captured, a golden set run before every publish 4 Search Demo: No embeddings, no vector search, SQL over 20 passages Needs: Retrieval tested on real questions, with citations 3 Endpoint Demo: Optimizer output stored as a table, returned unchanged Needs: A model endpoint with a versioned contract and an owner 2 Extraction Demo: 20 passages written in advance, live extraction failed Needs: Tables, formulas, and charts pulled from real PDF files 1 Scale Demo: 5 PDF reports, 10 tables, 6 expected answers checked by hand Needs: Thousands of documents, accuracy measured at that size
Figure 1. The gap ladder. Read it from the bottom up. Each rung is one known gap from the session notes. It pairs what the Doha demo did with what production needs.

What the Fabric IQ demo proved, and what it did not

The Doha demo proved three mechanics. A data agent answered one question from a table and a document, and it named both sources. It returned an optimizer result unchanged. One added sentence, the rule for the word winner, changed an answer without code. None of this was proved at scale, on real documents, or under load. Each claim below is tagged as a demo result, a Learn fact, or Dr. Kot's position.

The build was small on purpose. One lakehouse held 10 Delta tables and 5 study reports as PDF, 3 in English and 2 in French. A notebook cell meant to extract the PDF text failed on the day. The agent did not need it. It read 20 passages that the data generator had written into a table in advance.

The ontology plan called for seven entity types and seven relationships. The build ended with six and four, because the ontology item rejected a composite key. Learn now documents keyless entity types, so test that case again. Part 3 explains what an ontology is and why the agent needs one.

The winner rule had a fallback. Version A put it in the ontology. Version B put the same sentence in the agent instructions, in case the preview would not attach the ontology to the agent. On stage, version A ran: the ontology was attached to the agent. A preview feature still shaped the design, because version B had to be ready.

A French question got a French answer. Learn states that the data agent does not currently support languages other than English. One good answer on stage does not make a language supported. Part 4 covers the numbers side of the design: The agent writes the sentence, the table holds the number.

Six gaps between a Fabric IQ demo and production

Six gaps separate a Fabric IQ demo from a production agent: scale, extraction, endpoint, search, feedback, and cost. For each gap, the demo did something smaller than production needs. Each gap has one first measurement. Where a number is unknown, this post writes "to measure" and names the test that produces it.

Gap 1: scale

The demo. Five documents, ten tables, seven trials, and six formulas. The right passage had few rivals.

Production needs. A real corpus has thousands of documents, and many compete for the same question. Learn also caps each data agent answer at 25 rows and 25 columns.

Measure first. Answer accuracy on the full corpus: to measure. The test is a golden question set with expected answers, run against all documents.

Gap 2: extraction

The demo. The passages were prepared in advance, and the live extraction cell failed. Learn states that the data agent does not read PDF files. They must become tables first.

Production needs. Tables, formulas, and charts inside PDF are the hard part. On Azure, the layout model of Azure Document Intelligence in Foundry Tools returns text, tables, figures, and Markdown.

Measure first. Extraction error rate: to measure. A person who knows the documents checks the extracted tables against the source, page by page.

Gap 3: endpoint

The demo. The cost optimizer was a governed table. For formula OB-104, the agent returned the stored row unchanged: a saving of 7.4 percent, method grid-search-v1.

Production needs. A real design calls a model endpoint with a versioned contract and an owner. Learn states that the data agent runs read-only queries and triggers no notebooks or write workflows. Dr. Kot's position: call the endpoint in a pipeline, and land the result in a governed table.

Measure first. Agreement and latency between the endpoint and the table: to measure. Replay the cost questions against the endpoint and compare the outputs row by row.

Gap 4: search

The demo. No embeddings and no vector search. SQL found the right passage because there were only 20.

Production needs. A data agent can use an Azure AI Search index as a source Preview, with full-text, hybrid, or semantic search. Citations appear only when the index has a field such as url, filePath, or path.

Measure first. Retrieval hit rate: to measure. Label the correct passage for each golden question, and count how often retrieval returns it.

Gap 5: feedback

The demo. No user feedback and no monitoring. The plan said to run the six questions by hand on the morning of the session.

Production needs. User feedback, an evaluation on every change, and a trace for each answer. The data agent SDK compares answers with expected answers Preview. Run steps show the query behind each answer.

Measure first. Baseline accuracy before the first user: to measure. Run the golden set on the published version, then on every draft before you publish it.

Gap 6: cost

The demo. Nothing in the demo tells you the cost per 1,000 documents or per 1,000 questions.

Production needs. Learn bills data agent language model work at 100 CU seconds per 1,000 input tokens and 400 per 1,000 output tokens. Its example question, 2,000 tokens in and 500 out, costs about 0.11 CU hours. Query execution is billed separately.

Measure first. Cost per 1,000 questions: to measure. An estimate at Learn's example size is about 111 CU hours, before query execution. The workshop plan estimated 33 CU hours for 50 people asking 6 questions each, and the session notes record no measurement. The test is one week of real use in the Capacity Metrics app. Cost per 1,000 documents: to measure, by indexing a known batch.

What to measure first, in order

Measure upstream before downstream. An extraction error becomes a retrieval miss, then a wrong answer, then a wrong cost per answer. The order below is Dr. Kot's position.

  1. Extraction testRun a batch of real documents through extraction. A person who knows them checks tables, formulas, and charts page by page.
    Produces: extraction error rate and cost per 1,000 documents. Closes gap 2.
  2. Retrieval testLabel the right passage for each golden question. Count how often retrieval returns it.
    Produces: retrieval hit rate. Closes gap 4.
  3. Golden set on the full corpusRun the expected answers on the published version, then on each draft before you publish it.
    Produces: baseline accuracy at scale. Closes gaps 1 and 5.
  4. Permission testAsk the same questions as a Viewer and as a restricted user. Never test as the owner.
    Produces: pass or fail for each source and each role.
  5. Endpoint replayReplay the questions that need the optimizer. Compare the endpoint output with the stored result.
    Produces: agreement and latency. Closes gap 3.
  6. One week of real useOpen the agent to a pilot group. Read AI Query and query execution in the Capacity Metrics app.
    Produces: cost per 1,000 questions and peak load. Closes gap 6.
Figure 2. What to measure first, in order. Each step feeds the next, so start at the top. Dr. Kot's position, not a Microsoft procedure.

Three ways to build a data agent over data and documents

The same layer can be built three ways: your own build on Azure services, Databricks-native with Unity Catalog, or Fabric-native. The session notes compare them on three questions: when each fits, what you operate, and what to measure first. This is a comparison, not a recommendation. Only the Fabric-native path was built for the session. The other two are untested here.

Three ways to build, compared on four questions
Question AYour own build on Azure services BDatabricks-native with Unity Catalog CFabric-native
Fits when Every step needs your own logic, for example formulas and charts inside PDF. Engineering stays on one platform with Unity Catalog permissions. Business users work in Power BI, Excel, and Copilot.
You operate About seven services you build and run. One platform. A Fabric capacity, possibly next to another platform.
Measure first Build time and run cost. Answer quality on documents, and how far you can shape the business meaning. Capacity behavior under load, and which parts are preview.
Built for the session No. Untested here. No. Untested here. Yes: lakehouse, ontology, data agent, and the Real-Time Intelligence items.
Figure 3. Three ways to build the same layer. The first three rows come from the session's architecture notes. The squares count the platforms or services you run. A dashed square is a platform you may already run next to Fabric.

A. Your own build on Azure services

This path fits when every step needs your own logic. Azure Document Intelligence in Foundry Tools extracts layout and tables. Azure AI Search handles retrieval. Microsoft Foundry hosts the model endpoints. Microsoft Entra ID carries identity, and Application Insights holds the traces. The session notes count about seven services you build and run. Measure build time and run cost first.

B. Databricks-native with Unity Catalog

This path fits when engineering lives on one platform. Learn describes Unity Catalog as the governance layer for the data, tools, models, and agents an agent uses. Genie Agents answer natural-language questions over Unity Catalog tables. Databricks AI Search, formerly Vector Search, retrieves documents. MLflow records traces and evaluations. Genie also generates and maintains its own ontology. Measure answer quality on documents first, and how far you can shape that ontology.

C. Fabric-native

This path fits when business users work in Power BI, Excel, and Copilot. You operate a Fabric capacity, often next to another platform. The data agent is GA. The ontology and its use as agent context are Preview. Measure capacity behavior under load first, and list each preview part on your critical path.

Two failure modes from the build matter here. The agent definition file created a phantom schema with placeholder IDs, fixed through the staging API. A new account could query the agent only after its first sign-in to the Fabric portal. Test the deployed item with a real account.

A mixed estate is normal. Copilot Studio, Microsoft Foundry, Microsoft 365 Copilot, and MCP clients can call a published data agent, which stays read-only in Fabric. Dr. Kot's position: path C can be a component inside path A or B, not only an alternative.

What preview means for a Fabric IQ production promise

Microsoft Learn says preview features are not meant for production use. They have no SLA and get best-effort support. They can be limited in function or offered only in some regions. As of November 3, 2026, the ontology, the IQ workload, and the use of an ontology as data agent context are preview. The data agent itself is GA.

The session followed the same line. Its glossary said: you can try it, it can still change, and it is not for production promises. Rule L5 kept the limits to three sentences on stage. Rule L4 moved this comparison into backup for Q&A. Risk P1 was a preview change before the session, handled with a dry run and a recording. A production system has no end date, so that risk never closes.

Learn shows the change in three places. The old ontology experience retires on January 31, 2027, and its copy is a separate item that agents must reconnect to. Foundry stops supporting the Microsoft Fabric tool for data agents on November 6, 2026. Labels drift too. One Learn page calls the operations agent GA, while the tenant setting text still cites preview terms.

IDCapabilityStatus on November 3, 2026Production use
S1Fabric data agentGACan carry a promise, within its limits
S2Ontology item and the IQ workloadPreviewPilot only
S3Ontology as context in a data agentPreviewPilot only
S4Azure AI Search index as a data agent sourcePreviewPilot only
S5Data agent evaluation with the Python SDKPreviewTest pipelines, with a portable golden set
S6Outbound access protection for data agentsPreviewTurn on and test, keep a second control
S7OneLake security rolesGAThe access control to build on
S8Operations agentGA TBCLearn pages disagree, confirm first
S9Data agent in Foundry through the Microsoft Fabric toolPreviewDo not start, support ends November 6, 2026

The status of each FabCon Europe 2026 announcement is in the FabCon Europe 2026 series hub. The operations agent that explains a failed pipeline is in Operations agent in Monitor hub.

Governance a production Fabric data agent needs

A production data agent needs six controls the demo did not show on stage. They are deliberate sharing, source permissions, tested OneLake security, outbound access protection, a residency decision, and capacity planning. Microsoft Learn documents each one. Each one decides who sees which answer, or whether the agent answers at all.

Sharing and the published version

Publishing creates a read-only version of the agent. By default, people you share it with can query only that version, and your drafts stay separate. If you share before you publish, they cannot query at all. Sharing grants no access to sources. Each user needs Read on every source. For an ontology, that means Read on the ontology item and on each bound source.

Permissions on sources and OneLake security

The agent runs every query as the person who asks. OneLake security roles then decide which rows and columns that person sees. These roles restrict only Viewers and users with Read permission. Workspace Admins, Members, and Contributors read all data, whatever the roles say.

The demo did not test this on stage. Of the 20 passages, 8 were marked restricted, and the agent instructions carried one line about them. The test with a second, restricted account stayed a backup beat. Dr. Kot's position: an instruction is a request to a language model, not a control. In the workshop, participants were Contributors on the demo workspace, so security roles would not have restricted them. With synthetic data that is acceptable. With real data it is a defect. Part 6, Fabric for 50 people in one evening, explains that setup.

Outbound access protection for data agents Preview

A data agent makes an outbound request when it adds or queries a source outside its workspace, Azure AI Search included. With protection on, those requests need a data connection rule. Protection does not cover the Microsoft-managed Azure OpenAI connection, and it does not support Azure AI Search yet. Identity and network rules for the wider estate are in Fabric does not live in your VNet.

Region and data residency

A capacity outside the EU data boundary and the US needs two tenant settings. One lets data sent to Azure OpenAI be processed outside the region, and one lets it be stored there. A data agent cannot query a source on a capacity in another region. Answers sent to Copilot Studio, Foundry, Microsoft 365 Copilot, or MCP clients may leave Fabric's compliance boundary.

Capacity planning and throttling

Data agent language model work is a background job, smoothed over 24 hours. Throttling has stages. After 10 minutes of future capacity, new interactive operations wait 20 seconds. At 60 minutes, they are rejected. At 24 hours, all new requests are rejected. A Fabric Copilot capacity can collect data agent usage from capacities smaller than F64, in the home region only.

The session applied this at small scale. The demo ran on an F32 capacity and the playground on a separate F16, so playground Spark jobs could not slow the demo. In production, give the agent its own capacity or its own billing.

Production readiness checklist for a Fabric data agent

Dr. Kot's position. This list is a recommendation, not Microsoft guidance. It is written for regulated industries and for estates that run more than one cloud or data platform.

  1. R1. Label every feature on the critical path as GA or preview, with its Learn link and check date. Give each preview item a fallback and an owner.
  2. R2. Keep preview features out of production promises. Pilot each one on a single workspace you can switch off in one move.
  3. R3. Run the extraction test on real documents before you choose a retrieval design.
  4. R4. Build a golden question set before the first user. Run it on the published version and on every draft.
  5. R5. Put every number behind a query or a governed table. A number with no query behind it is a defect.
  6. R6. Test access as a Viewer and as a restricted user, never as the owner. Contributors bypass OneLake security roles.
  7. R7. Enforce access in the data, not in agent instructions. Treat a rule that lives only in a prompt as missing.
  8. R8. Decide data residency in writing before go-live. Cover processing, storage, observability metadata, and every outside consumer of the agent.
  9. R9. Turn on outbound access protection for workspaces with regulated data. Review Azure AI Search on its own, because protection does not cover it yet.
  10. R10. Size capacity from one week of measured use, not from the demo or from Learn's example.
  11. R11. Keep the agent definition in Git and promote it with deployment pipelines. Test the deployed item, not only the file.
  12. R12. Keep each business rule in one place with one named owner. In the demo, the threshold of 60 lived in Activator and in the ontology. Two copies drift apart.

Rule R12 matters most in a multi-cloud estate. Keep each rule in one governed place, and let every agent read it there. To review a Fabric IQ production plan with Dr. Kot, book a slot. The Microsoft Fabric implementation guide covers the platform work around it.

Key facts at a glance

Facts from Microsoft Learn, checked on November 3, 2026, and from the session notes.

IDFactSource
K1The Fabric data agent is generally available and needs a paid F2 or higher capacity, or P1 or higher with Fabric enabled.Fabric data agent concepts
K2The ontology is preview, and its old experience retires on January 31, 2027.What is ontology (preview)?
K3A data agent uses up to five sources, takes up to 100 example queries per source, and returns at most 25 rows and 25 columns.Fabric data agent concepts
K4The data agent does not read PDF, DOCX, or TXT files directly, and an Azure AI Search index source is preview.Add and configure data sources
K5Data agent language model work costs 100 CU seconds per 1,000 input tokens and 400 per 1,000 output tokens.Data agent in Fabric consumption
K6Throttling delays new interactive operations by 20 seconds after 10 minutes of future use and rejects all new requests after 24 hours.Fabric capacity throttling policy
K7OneLake security roles do not restrict workspace Admins, Members, or Contributors.Data security in OneLake
K8Outbound access protection for data agents does not cover the Azure OpenAI connection or Azure AI Search.Outbound access protection for Data Agent
K9The data agent refreshes ontology context every 10 minutes unless you refresh it by hand.Use Ontology as context
K10The Foundry integration through the Microsoft Fabric tool is not supported from November 6, 2026.Consume Fabric data agent from Microsoft Foundry
K11The Doha demo used 10 tables, 5 PDF reports, and 20 passages prepared in advance, all synthetic.Session notes, October 14, 2026

Frequently asked questions

Is the ontology in Fabric IQ ready for production?

No. Microsoft Learn labels the ontology as preview on November 3, 2026. Preview features are not meant for production use and have no SLA. The old ontology experience retires on January 31, 2027. Use the ontology in a pilot on one workspace, and keep it off the critical path of a production promise.

Can a Fabric data agent read PDF files?

Not directly. Microsoft Learn states that the data agent does not support unstructured files such as PDF, DOCX, or TXT. Extract the text into lakehouse tables first, or connect an Azure AI Search index as a source, which is preview. The quality of the answer depends on how well the documents are extracted.

How much does a Fabric data agent question cost?

It depends on the tokens. Microsoft Learn bills 100 CU seconds per 1,000 input tokens and 400 per 1,000 output tokens. Its example of 2,000 tokens in and 500 out costs about 0.11 CU hours. Query execution is billed separately. Measure one week of real use in the Capacity Metrics app before you size a capacity.

Does a Fabric data agent respect row-level security?

Yes. The data agent runs each query with the identity of the person who asks, and it honors row-level and column-level security. OneLake security roles do not restrict workspace Admins, Members, or Contributors. Test the agent as a Viewer and as a restricted user before go-live, never as the owner.

Fabric, Databricks, or your own Azure services: which should you build on?

Choose by where your people work and what you must control. Build your own on Azure services when every step needs your own logic. Choose Databricks when engineering stays on one platform with Unity Catalog. Choose Fabric when business users work in Power BI, Excel, and Copilot. Measure first, then decide.

What happens when a Fabric capacity is throttled?

Throttling comes in stages. After 10 minutes of future capacity are used, new interactive operations wait 20 seconds. After 60 minutes, new interactive operations are rejected. After 24 hours, all new requests are rejected. Data agent language model work runs in the background, so only the last stage rejects it.

Does the Fabric data agent support languages other than English?

Not officially. Microsoft Learn states that the data agent does not currently support non-English languages and recommends English questions, instructions, and examples. In the Doha demo, a French question got a French answer. One good answer is a result, not support. Test your languages before you promise them.

About the author

Dr. Estera Kot is CTO at Clouds on Mars, a Microsoft and Databricks partner for data and AI in Europe and the Middle East. At Microsoft she was Principal Product Manager for the Apache Spark runtime in Azure Synapse Analytics and Microsoft Fabric, and she created the Native Execution Engine. She is a Microsoft MVP and holds a Ph.D. in Computer Science from the Warsaw University of Technology. She presented this series as the community session "From Events to Answers" at Microsoft Qatar HQ in Lusail, Doha, on October 14, 2026.

Sources

Microsoft Learn, checked on November 3, 2026. Session facts come from Dr. Kot's notes for the Doha session of October 14, 2026.