I’m honored to be hosting T-SQL Tuesday — edition #192.

For those who may not know the history, T-SQL Tuesday started back in December 2009 when Adam Machanic (Blog |LinkedIn) sent out the first invitation for what he called a monthly “blog party.” The idea was simple: pick a topic, publish on the second Tuesday of the month, and share what you’ve learned. No gatekeepers — just community.

Today, Steve Jones (Blog|LinkedIn) organizes the event and maintains the website with links to past topics and posts, preserving more than a decade of shared knowledge.

Everyone is welcome to participate. If you’ve been thinking about writing, this is your invitation.

The topic I’d like us to explore is one that’s a little more personal than technical: What career risks have you taken?

Not just the safe moves. Not the obvious promotions. The real risks — the moments where you weren’t sure how it would turn out.

That’s what I’d love to hear about.

I’ve been in the IT industry for 40 years, and here’s what I’ve learned — not from theory or motivational posters, but from lived experience: if you don’t take risks, your career will stall.

Success doesn’t happen by accident. Nobody “accidentally” ends up in a great job with great pay, and nobody stumbles into owning a successful business. If you ask anyone who has built something meaningful whether they did it without risk, the answer is always the same: “No way.” They took chances. Some worked, some didn’t — but they moved.

Risk-taking was modeled for me early. My dad was a bricklayer on Long Island in his late 20s with a young family — me and my two sisters. Bricklaying is real work: eight hours in 95-degree heat or freezing winters, with a physical toll every single day. After falling off a scaffold and breaking his leg, he decided he wasn’t going to do that for the rest of his life.

So my parents took a huge risk. They moved our entire family from New York to Las Vegas in the mid-70s — when the mob still ran most of the casinos — and my dad became a craps dealer. We knew almost no one there, and it could have gone very differently. But it paid off. He made more money, saved his body from years of physical wear, and built a nearly 40-year career as a dealer and floorman. He loved it.

Years later, in my late 20s, I faced a similar moment. I was working in Las Vegas as a software engineer (what we used to call a programmer), making very little money. Vegas didn’t pay well for IT back then, and we were living paycheck to paycheck with our third child on the way. I didn’t want to struggle indefinitely, so we moved to Houston — sight unseen — without knowing a single person there, for a much higher-paying job.

It was terrifying, but it paid off. That move opened doors and changed the trajectory of my career. When I look back, I’m proud we had the courage to do it. Meanwhile, many coworkers stayed in Vegas. They played it safe, avoided discomfort, and their careers largely stayed where they were.

Over time, I learned something important: you have to get comfortable being uncomfortable. Not every risk involves moving your family across the country, but advancing your career may require changing roles internally, leaving a stable company, switching industries, or even starting something on your own. And sometimes not taking the risk is actually the bigger risk — especially if you’re counting on your career to support your family.

When I face a big decision, I slow it down. I create a pros-and-cons list over several days and talk to people I trust. That process helps remove emotion and brings clarity. I’ve also seen a few truths play out repeatedly: people hate change, we’re addicted to comfort, and the first time you do something uncomfortable it’s hard — but the tenth time is much easier. Growth lives outside your comfort zone.

There are plenty of examples in our industry. Brent Ozar left traditional employment to build his own consulting company. Paul Randal did the same. Those weren’t small bets — they were career-defining risks (I hope those two share their stories as a reply to my post).

So I’ll leave you with this: what career risks have you taken? Did they pay off? What did you learn? Because after four decades in this field, I’m convinced of one thing — change is constant. You can either embrace it and accelerate your career, or resist it and stay on the same rung of the ladder.

T-SQL Tuesday Rules

To participate:

First, write a blog post on the topic:

“What career risks have you taken?”

Be honest. Be reflective. What did you bet on? What scared you? What worked… and what didn’t?

Then:

• Include the phrase “T-SQL Tuesday #192” somewhere in your post.
• Include the T-SQL Tuesday logo in your post.
• Link back to this invitation so others can easily find the event.
• Schedule your post to go live on Tuesday, March 10, 2026 (the second Tuesday of the month).
• Share it on social media (LinkedIn, Twitter, Bluesky) using the hashtag #tsql2sday.

Optional — but appreciated: leave a comment on this post with a link to your contribution so I don’t miss it (I’ll do a recap on March 16).

I’m genuinely looking forward to reading your stories. Career risks are rarely comfortable in the moment… but they’re often the moments that shape everything that follows.

If you’re a Power BI report author who’s just getting into Microsoft Fabric, you’ve probably asked the same question I hear over and over: am I supposed to stop using Power BI Desktop now?

It’s a fair question. Power BI Desktop is a Windows app that has traditionally been the place where report authors do everything: get data, transform it, model it, and build the report. Microsoft even describes that “connect, shape/transform, then load” experience as part of how Power BI Desktop works with Power Query.

Fabric changes the feel of that workflow because Power BI is now also a first-class experience in the browser inside the Fabric portal. And that browser experience isn’t just “view and share” anymore. You can edit semantic models in the service, including using Power Query for import models and building reports directly from that same environment.

This shift also matters a lot for people who simply couldn’t rely on Power BI Desktop before. If you’re on a Mac, using a Chromebook, working from a locked-down corporate machine, or in an environment where installing desktop software requires jumping through weeks of approval hoops, the browser-based Power BI experience is a genuine unlock. For the first time, you can build reports and work with semantic models using nothing more than a modern browser. That alone explains why so many people are revisiting the “Desktop vs Fabric” question now—it’s no longer just about preference, it’s about access.

The goal here is simple: help you decide, when you’re starting a brand-new report, which authoring surface is going to feel smooth… and which one is going to make you mutter “why is that button gray?” (we’ve all been there).

The from-scratch paths

Let’s define “from scratch” the way report authors actually experience it:

You start with nothing, you need to acquire data, shape it, model it, and then build visuals on a canvas. That’s the full loop.

In Desktop, that loop is straightforward because it’s all in one place.

In Fabric (browser), there are two common “from scratch” starting points, and they matter:

One path is “create a new import semantic model in the service using Get data / Transform data, then create a report from it.” Microsoft documents this directly in the semantic model editing experience: you can add new import tables using Get data, shape them with Transform data (Power Query), and then create a report from that model.

The other path is “Create a quick report,” which is a simplified browser experience meant to get you moving fast (often by pasting data or starting from an existing semantic model). The quick report documentation is explicit about what it supports today and what it doesn’t.

Here’s the “picture in your head” diagram I recommend keeping around:

Desktop-first Get Data (many connectors) → Power Query (full) → Model view (full) → Report view (full)

Fabric browser-first (import model route) Create semantic model (Get data) → Power Query online (import-only) → Web model editor (most core modeling) → Web report editor

Fabric browser-first (quick report route) Existing semantic model or paste data (limited) → Autogenerated visuals → Switch to full edit if needed

That last one is great for prototypes, demos, and “I just need to see something.” It is not the same thing as building a durable, repeatable solution. Microsoft even calls out that pasted data can’t be updated later without redoing the Create workflow.

Data import in practice

Power BI Desktop is still the “widest funnel” for data import. Desktop is built around connecting to one or many data sources through Power Query, and it’s where the connector ecosystem (including custom connectors) has been most complete historically.

In the Fabric browser authoring experience, you can absolutely bring in data for import models. But Microsoft lists several connector-related limitations for adding import tables and for enabling query editing/refresh in the web model editing experience. Custom connectors and a specific set of connectors (including OLE DB, R, and Python) aren’t supported for adding import tables in web editing, and models using those connectors also don’t support query editing in the Power Query editor in that web experience.

There’s also a “connection setup reality” that can surprise new Fabric authors: in the web Power Query experience, you can use existing personal cloud connections, but you can’t create new personal cloud connections inside the editor. That setup happens elsewhere.

And if you’re tempted to start with the quick report “Paste data” option, Microsoft is very explicit about the limits: no way to update the pasted data later, a 512 KB paste size cap, and other constraints like an eight-table limit and naming restrictions.

So the author takeaway is pretty clean:

If “from scratch” means “I don’t know what source I’ll need, and I might need some quirky connector or a custom connector,” Desktop is the safer, less-surprising start.

If “from scratch” means “my data is already in an environment Fabric understands well, and I’m building an import semantic model from supported connectors,” browser-first can work.

Data transformation: Desktop vs Fabric’s distributed approach

In Desktop, Power Query is the primary data transformation surface. It’s mature, predictable, and designed for shaping data before loading it into the model.

In Fabric (browser), the Power Query editor exists — but it’s not the only transformation option, and this is where the comparison changes.

Fabric doesn’t narrow transformation. It redistributes it.

Instead of assuming all shaping happens inside Power Query within the report tool, Fabric gives you multiple upstream transformation options:

• Dataflow Gen2 (Power Query at the service level)
• Spark notebooks for large-scale transformation
• T-SQL in the Warehouse
• Lakehouse table transformations
• Pipelines and orchestration
• Python-based or semantic link workflows

In Desktop, Power Query is the workbench.
In Fabric, Power Query is one tool in a broader data platform.

The web Power Query editor is optimized for import models. It does not support Direct Lake or DirectQuery transformations, and dynamic data sources are not supported. You can edit parameters in the service, but you cannot create them there.

That sounds like a limitation — until you realize what Fabric expects.

Desktop assumes:

The report author shapes the data.

Fabric assumes:

The platform shapes the data. The author models and visualizes it.

If your organization is serious about Fabric, Power Query inside the report becomes a last-mile cleanup tool — not the primary ETL engine.

That’s not weaker. It’s architectural.

Data modeling and semantic model editing

For a long time, “modeling” was the easy part of this comparison: Desktop won.

That’s no longer true in a blanket way.

About a year ago, Power BI in Fabric added a feature to allow you to create and edit semantic models in the service: create measures, calculated columns, calculated tables, relationships, properties, and even define row-level security roles.

So yes, the web experience is a real modeling environment now.

But Microsoft also explicitly lists functional gaps between Desktop model view and the service. These include: not being able to change a table’s storage mode, no View as dialog, and other limitations like feature tables and certain data categories.

Another important difference shows up around relationship automation.

In Power BI Desktop, relationship creation can happen in two distinct ways when you load data. First, if you are connecting to a relational source like SQL Server or Azure SQL and that database has actual foreign key constraints defined, Desktop can read that metadata and automatically create relationships in the model during initial load (assuming auto-detect is enabled). In that case, it is not guessing — it is importing real source-defined relationships.

Second, when usable foreign key metadata does not exist — which is common in many warehouses, views, or lake environments — Desktop falls back to heuristics. It will attempt to infer relationships based on matching column names, compatible data types, and cardinality patterns (for example, detecting which side appears unique). This inference behavior is what most people think of as “auto-detect relationships.” It is convenient, but optional, and many experienced modelers disable it to avoid accidental joins.

In the Fabric browser experience, neither of those automatic behaviors occurs. When you import data using Power Query in the service, relationships defined in the underlying source system are not automatically brought into the semantic model. There is also no background auto-detect process that later scans for new relationships based on column names or data patterns. All relationships must be created explicitly by the author.

There’s also a workflow difference worth calling out: semantic model editing in the service uses AutoSave, and Microsoft notes changes are permanent with no option to undo in that experience.

And if you use DAX Query View: Microsoft documents that Desktop-saved DAX queries aren’t visible in the web DAX query view, and queries written in the web are discarded when you close the browser. 

So modeling is now a “depends” conversation:

If your modeling needs stay within the web-supported feature set, the Fabric browser experience can work well.

If you need the full Desktop modeling surface area (and the deeper tooling workflows many modelers rely on), Desktop still feels like the more complete workbench.

Report visualization and the canvas experience

Here’s the part that surprises many people: the report canvas gap is smaller than the data prep and modeling gaps.

Microsoft describes editing view in the Power BI service as the place where you create and edit reports in the browser, similar to Report view in Desktop.

Microsoft also notes that the ribbon is the main part of the report editor that differs between Desktop and the service, and the actions available vary depending on what you select on the canvas.

In practical terms, if you’re building visuals, arranging a layout, formatting, and polishing a report, the browser experience is “close enough” that many authors can be productive quickly.

Custom visuals are not a major dividing line either. Microsoft documentation for importing visuals says you can import visuals from AppSource in both Power BI Desktop and the Power BI service, and you can also import from a file.

The place the browser report editor shines for brand-new authors is the quick report experience: paste data and let Power BI auto-generate visuals, then switch to full edit if you want. Just remember those quick report limitations (especially the “no way to update pasted data later” part).

Side-by-side comparison for building from scratch

The table below summarizes what matters most for report authors building a new report end-to-end.

How to choose without overthinking it

Here’s the bottom line I want you to walk away with:

If your “from scratch” report is going to live or die on data import and transformation flexibility, start in Desktop. You’ll get the widest connector coverage, the most mature Power Query patterns (including creating parameters), and fewer “this feature isn’t supported here” surprises. Desktop is still the most predictable, self-contained transformation workbench when you expect to do most of the shaping inside the report tool itself.

If your “from scratch” report is mostly about building visuals on top of a semantic model that already exists in the Fabric environment, the browser experience can be a perfectly legitimate authoring surface. The web report editor is capable, and web semantic model editing covers a lot of day-to-day modeling tasks now. And if your transformation work is happening upstream — in Dataflow Gen2, Spark, T-SQL, or Lakehouse processes — the browser experience fits naturally into that Fabric-first architecture.

And here’s the most Fabric-era statement I can make without drifting into platform debates:

It’s not really Desktop versus Fabric. It’s Desktop plus Fabric, and you choose the surface based on where your friction will be:

• If your friction is report-level data shaping (cleaning, merging, parameterizing inside the report), Desktop will usually feel smoother.
• If your friction is platform-level data engineering (Spark notebooks, Dataflow Gen2, T-SQL in a Warehouse, Lakehouse pipelines), that work belongs in Fabric — not in Desktop.
• If your friction is purely report-canvas design (building visuals, layout, formatting, iterating quickly), either surface can work — and the browser may be perfectly sufficient.

One last practical bridge worth knowing about: Microsoft documents that for Direct Lake models, the web experience can offer an “Edit in Desktop” path that launches live editing in Power BI Desktop (Windows-only), and Microsoft also documents how Desktop can create and live-edit Direct Lake semantic models via the OneLake catalog. That’s a big deal because it means you can stay in a Fabric-first architecture while still using Desktop as your authoring workbench when you need it.

Other feature differences worth knowing about

Beyond modeling and Power Query, there are a handful of features that still exist in Power BI Desktop (or related desktop tools) that don’t yet have full parity inside the Fabric browser experience. These aren’t deal-breakers for most authors—but they matter depending on your workflow.

Paginated reports are one example. These aren’t created in Power BI Desktop at all—they’re authored using Power BI Report Builder, a separate Windows tool. And there’s no browser-based paginated authoring surface inside Fabric today. If you need pixel-perfect, printable reports (invoices, regulatory forms, operational reports), that’s still a desktop story.

External Tools integration is another. Desktop supports launching tools like DAX Studio and Tabular Editor directly from the ribbon. Those tools are central to advanced modeling, diagnostics, and automation workflows. The Fabric browser experience has no equivalent because those tools rely on local integration.

Custom visuals discovery also feels different. You can use custom visuals in both Desktop and the service, but Desktop provides a richer, more integrated Visuals Marketplace experience for browsing and adding them directly from AppSource.

Q&A and natural language exploration exists in the broader Power BI service experience, but the tuning and authoring surface for it is still more mature in Desktop. If you’ve used Q&A heavily for model validation or exploration, you’ll notice the difference.

Power BI Goals (scorecards) live in the service ecosystem but aren’t part of the browser-based report authoring surface in Fabric. They operate alongside reports—not inside the modeling workflow.

None of these gaps mean the browser experience is weak. They simply reflect that Desktop evolved for deep authoring over many years, while Fabric’s browser surface is optimized for accessibility, shared modeling, and cloud-first workflows.

More info:

What is Power BI

Here’s the mental model that helped me. There are three capability umbrellas (also called IQ layers or architecture labels). Underneath them are build and orchestration tools. And wrapping around all of it is a runtime and governance layer that makes AI safe and scalable. Once you see that structure, the noise quiets down and the pieces start to fit together.

Let’s start with the three IQ layers, because these are not products you buy. They describe what intelligence you are trying to achieve as an organization.

You won’t find “Work IQ” or “Foundry IQ” as standalone SKUs or admin blades. I’m using them here as architectural labels to describe Microsoft’s direction, not official product names. They are the intelligence outcomes Microsoft wants organizations to achieve.

Work IQ

Work IQ is intelligence about how work actually happens inside your company. It models the way people collaborate — their documents, meetings, conversations, relationships, and day-to-day interaction patterns across Microsoft 365. It understands meetings, chats, emails, documents, collaboration signals, overload patterns, and exception handling. It’s powered by Microsoft 365 data and organizational graph signals.

When Work IQ is functioning well, you can answer questions like: Which teams are buried in manual exception handling? Where are email threads exploding? Who is spending ten hours a week chasing status updates?

But it goes a step further.

This intelligence layer becomes the personalization engine for AI agents. Instead of a generic assistant, Work IQ helps agents understand who you work with, what you work on, and how you get things done. It brings user context into the equation — not just data context.

For architects, Work IQ is effectively the user-context layer. It’s the bridge between user behavior and the actions agents take inside the enterprise environment. It connects human workflow signals to intelligent action.

It’s intelligence about human workflow — not just data.

Fabric IQ

Fabric IQ is intelligence grounded in structured enterprise data. It lives in Microsoft Fabric and understands your KPIs, trends, anomalies, lineage, and semantic models. But more importantly, Fabric IQ sits on top of the enterprise data estate and injects business meaning into it through a semantic layer and ontology. It turns raw datasets into entities, relationships, metrics, timelines, and operational context that agents can reason about.

This is the analytical brain.

It doesn’t just see tables. It sees business reality: customers, inventory, regions, exceptions, dependencies, patterns. It recognizes supply chain bottlenecks, revenue shifts, production throughput constraints, and operational performance trends because the data has been modeled with intent.

Fabric IQ is what allows AI to reason over trusted, governed, enterprise-grade data (structured, semi-structured, and unstructured) instead of guessing from disconnected spreadsheets.

From a solution architecture view, Fabric IQ becomes the data-context layer — essential for designing analytics, decision systems, and operational AI pipelines. It’s where analytics maturity meets AI capability.

And for DevOps teams, Fabric IQ introduces a new discipline: semantic models must be versioned, governed, deployed, and monitored the same way we handle code. Because once agents depend on business meaning, that meaning becomes production infrastructure.

In short, Fabric IQ gives AI structured business context — not just raw data.

Foundry IQ

Foundry IQ is intelligence grounded in knowledge and reasoning. This is where Microsoft Foundry comes in. It’s the evolution of what used to be Azure AI Studio, then Azure AI Foundry, and now Microsoft Foundry. Foundry is where models are selected, grounded in enterprise content, evaluated, secured, and managed.

But architecturally, Foundry IQ handles the most difficult part of agent design: knowledge retrieval and grounding.

It connects agents to policy-controlled knowledge bases across Microsoft 365, cloud storage, data platforms, and internal repositories — all through a unified retrieval engine. Agents can perform iterative search, multi-source reasoning, and permission-aware grounding instead of relying on brittle manual retrieval stacks or static document embeddings.

Foundry IQ is what happens when AI understands contracts, policies, procedures, SLAs, regulatory constraints, and unstructured documents — and can reason across them safely. It’s the reasoning layer that makes AI context-aware instead of generic.

In modern architectures, this becomes the knowledge-access layer. It enables traceability, auditing, and reliable decision-making for autonomous workflows. Instead of an agent simply generating an answer, it can show where the information came from, respect access boundaries, and operate within governance constraints.

In short, Foundry IQ gives AI controlled, secure access to enterprise knowledge — and the ability to reason over it responsibly.

How the Three IQ Layers Work Together

If you want a simple picture in your head, imagine a three-layer stack. At the foundation is Fabric IQ, your structured data intelligence (data-context). In the middle is Foundry IQ, your reasoning and knowledge grounding layer (knowledge-context). On top is Work IQ, your understanding of how humans are actually operating day to day (user-context). Together, those layers create the conditions for meaningful enterprise agents.

Let’s make this concrete.

Suppose a company wants to build an AI agent to help manage supply chain delays. This is not a theoretical example. This is the kind of use case I hear from customers constantly.

Fabric IQ detects anomalies in delivery metrics. It sees that certain suppliers are trending late beyond historical norms. It notices that on-time delivery percentages are dipping in specific regions. It correlates delays with upstream bottlenecks. This is data-driven awareness.

Foundry IQ then grounds the agent in supplier contracts, SLAs, penalty clauses, and internal policies. It understands what the agreement actually says about late deliveries. It interprets escalation thresholds. It knows which suppliers have stricter terms and which ones allow flexibility. This is contextual reasoning.

Work IQ observes that operations teams are overloaded handling these exceptions manually. It sees long email chains, recurring “delay review” meetings, and individuals spending hours every week tracking updates from vendors. It identifies patterns of reactive work that are consuming capacity. This is workflow intelligence.

Now you introduce the Agent. The agent combines those three streams of intelligence. It recommends which delays need escalation based on contractual impact. It drafts communications to suppliers referencing the correct SLA language. It suggests internal reprioritization. It surfaces issues before they become crises. It becomes a force multiplier instead of just another dashboard.

Runtime, Governance, and Industrial Scale

But this is where responsible leaders pause and ask the right question. Who monitors this agent? Who ensures it’s accurate? Who ensures it stays within policy? And maybe the most important question: who is accountable when it makes a recommendation that affects the business?

That’s where Agent 365 comes in. Agent 365 is the runtime and governance layer that sits over everything. It monitors agent decisions, tracks accuracy over time, enforces compliance boundaries, and provides deep observability into how agents are behaving in the real world. It gives you visibility into what the agent is doing, why it is doing it, and whether it is staying within the guardrails you defined.

This isn’t just about logging activity. It’s about operational control. It’s about knowing when performance drifts, when policies change, when human override is required. It’s about giving leadership confidence that AI is not operating in the shadows but within a governed framework.

Without this layer, you have experiments — clever, promising, but fragile. With it, you have enterprise-grade systems that can scale responsibly.

And once this supply chain pattern works, Agent Factory allows you to scale it. Agent Factory is about industrializing agent creation. Instead of building one-off AI projects in isolation, you create repeatable blueprints that can extend to finance for invoice exceptions, HR for policy guidance, or operations for maintenance alerts. It’s the production line for enterprise agents.

Most companies today are experimenting with AI in pockets — a pilot here, a proof of concept there. It works, but it doesn’t scale. Agent Factory shifts you from experimentation to manufacturing. You define standard patterns for how agents are grounded in Fabric data, reason through Foundry, and are governed through Agent 365. You build templates, not just individual agents.

So when another department needs an agent, you don’t start from scratch. You extend the blueprint. That’s the difference between isolated AI projects and a true enterprise AI platform — designed once, governed consistently, deployed many times.

Now let’s untangle the product names, because this is where confusion often peaks.

Build and Orchestration Tools

To turn strategy into reality, you need build and orchestration tools. This is where Microsoft Foundry, Copilot Studio, and Copilot Studio Light come into play.

Microsoft Foundry is the engineering backbone for enterprise AI. It’s where foundation models (such as large language models like GPT-4-class systems), embedding models for retrieval and vector search, and task-specific AI systems like classifiers or anomaly detection models are selected, configured, grounded in enterprise data, evaluated for quality and safety, and prepared for production use. If you’re building serious AI solutions — the kind that must meet performance standards, compliance requirements, and security controls — this is where that work happens.

You can absolutely build agents in Microsoft Foundry. But you’re building them at the engineering layer. That means defining prompts, retrieval pipelines, tool calling, evaluation loops, safety filters, deployment endpoints — often in a pro-code environment. Foundry is not just a playground for models. It’s an engineering environment. It supports model evaluation, grounding techniques, prompt flows, fine-tuning scenarios, safety configurations, and deployment management. In other words, it’s where AI moves from experimentation to disciplined engineering.

Copilot Studio sits one layer above that. It’s the orchestration platform. This is where you design agents visually, connect them to business systems, define actions, build workflows, and determine how agents interact with users and applications. Copilot Studio is about behavior. It’s where you decide what the agent can do, what systems it can call, how it responds to certain triggers, and how it escalates when needed.

The clean distinction is this:
Foundry is pro-code, engineering-first agent construction.
Copilot Studio is low-code/no-code, workflow-first agent orchestration.

If Foundry is about building intelligence correctly, Copilot Studio is about putting that intelligence to work inside real business processes.

Copilot Studio Light — formerly known as Agent Builder — is the streamlined, no-code experience embedded directly inside Microsoft 365 Copilot. It allows business users to quickly create internal, knowledge-based agents grounded in SharePoint sites, Teams content, or organizational documents. It’s intentionally simple. It lowers the barrier to entry. But it’s important to understand the distinction: Copilot Studio Light is ideal for quick internal assistants and knowledge helpers. When you need advanced workflows, external system integrations, deeper customization, or structured governance controls, you step up to full Copilot Studio — and potentially integrate with Microsoft Foundry for more advanced model engineering.

When you step back, the architecture becomes clearer. Microsoft Fabric provides structured, governed enterprise data. Microsoft Foundry engineers and evaluates the intelligence layer — and can be used to build highly customized agents directly in a pro-code way. Copilot Studio is where you build and orchestrate business agents using a low-code/no-code approach, including their workflows, actions, and system integrations. Copilot Studio Light enables fast, internal, knowledge-based agents created directly inside Microsoft 365 Copilot. Together, they form a layered system rather than a collection of disconnected tools — pro-code where you need maximum control, and low/no-code where you need speed and broad adoption.

Here’s my slightly opinionated take. Microsoft isn’t just releasing features. It’s assembling an AI operating model for the enterprise. But when terminology evolves quickly — and it has — even experienced leaders can feel like they’re chasing moving targets. The risk is that we focus on memorizing product names instead of designing cohesive systems.

The real question isn’t whether you’ve tried Copilot Studio Light or deployed something in Microsoft Foundry. The real question is whether you are building isolated AI experiments or a layered, governed, repeatable AI system. Are your agents grounded properly? Are they orchestrated thoughtfully? Are they designed with scale in mind?

That’s the shift. Not from one product to another — but from experimentation to architecture.

Start simple. Pick one high-impact use case. Ground it in Fabric. Reason through Foundry. Understand human impact through Work IQ. Deploy responsibly with Copilot Studio. Govern with Agent 365. Then scale with Agent Factory.

I’ve been in this industry long enough to see naming waves come and go. The technology shifts. The labels evolve. But the pattern of success remains consistent. The companies that win don’t chase terminology. They understand architecture. They build deliberately. They scale responsibly.

If you’ve felt overwhelmed by the recent flood of AI terminology, you’re not alone. I was there too. But once you see the layered model underneath, the strategy becomes much less intimidating. And that’s when you can stop decoding names and start building intelligence.

Summing it up

Fabric IQ = Intelligence grounded in your structured business data
(Architectural label — not a SKU.)

Foundry IQ = Intelligence grounded in enterprise knowledge, policies, and reasoning
(Architectural label — not a product name.)

Work IQ = Intelligence about how work actually happens across meetings, messages, and collaboration patterns
(Architectural label describing direction.)

Microsoft Foundry = Where foundation models (LLMs, embedding models, task-specific models) are selected, grounded, evaluated, and engineered — and where highly customized agents can be built in a pro-code environment.

Copilot Studio = Where business agents are built and orchestrated using a low-code/no-code approach — defining workflows, actions, triggers, and system integrations.

Copilot Studio Light = Where quick, internal, knowledge-based agents are created directly inside Microsoft 365 Copilot for fast adoption and simple use cases.

Agent 365 = The runtime layer for monitoring, observability, compliance, and governance across deployed agents.

Agent Factory = The enterprise pattern for scaling and industrializing agent creation across departments.

The simple mental model:

Architectural layers create intelligence.
Engineering and orchestration tools build agents.
Agent 365 governs them.
Agent Factory scales them.

Pro-code where you need control.
Low-code/no-code where you need speed.

This post walks through three practical architectures for using Snowflake data in Fabric OneLake, when each option makes sense, and the key tradeoffs to consider.

The most important decision across all three options is which platform is the system of record and primary writer—most tradeoffs flow directly from that choice.

1) Snowflake-managed Iceberg table + OneLake shortcut

In this pattern, Snowflake reads and writes an Iceberg table stored in external object storage such as ADLS or S3. Microsoft Fabric does not copy the data; instead, Fabric creates a OneLake shortcut to the Iceberg table’s storage location so Fabric engines can query the data in place. Snowflake remains the system of record and the primary writer.

This option is best when you already have Iceberg tables managed by Snowflake and want zero data duplication while still enabling Fabric analytics. The tradeoff is that Fabric access is largely read-oriented, and the experience is less native than using Delta format (how everything is stored in OneLake) in terms of table discovery and management.

From a cost perspective, you pay for Snowflake compute to create and maintain the Iceberg tables and for Fabric compute when querying the data. You do not pay for OneLake storage, ingestion pipelines, or replication. Any additional cost depends on where the external storage lives and whether cross-service or cross-region reads apply.

When to use this option

• You already have Snowflake-managed Iceberg tables in external object storage
• Snowflake must remain the system of record and primary writer
• You want zero data duplication and minimal architectural change
• Fabric is used mainly for read-only analytics or exploration

When to avoid this option

• You want Power BI Direct Lake with a fully native Delta experience, rather than Direct Lake enabled via Iceberg→Delta virtualization
• You expect Fabric to write to, optimize, or manage the tables natively (for example, Delta optimization, compaction, or schema evolution owned by Fabric)
• You want first-class table discovery, governance, and lifecycle management that aligns fully with Fabric-native Delta tables

One-line summary:

Best for quick, low-friction access to existing Snowflake Iceberg data with minimal change, but limited Fabric-native capabilities.

More info: Unify data sources with OneLake shortcuts – Microsoft Fabric | Microsoft Learn

2) Snowflake writes Iceberg tables directly into OneLake

In this model, Snowflake is configured to write Iceberg tables directly into OneLake, creating a single shared physical copy of the data that both Snowflake and Fabric can access. No shortcut and no replication are required, and Fabric works directly against the Iceberg data where it lives.

This is the cleanest long-term architecture for open table interoperability, assuming current Iceberg support in OneLake meets your needs and Snowflake remains the primary writer (or you are comfortable defining shared write semantics).

Cost-wise, you pay for Snowflake compute to write and maintain the Iceberg tables and for Fabric compute to run analytics workloads. You do not pay for duplicate storage, replication, or ingestion jobs. This is often the most cost-efficient option at scale when both platforms need access to the same data.

When to use this option

• You want a single shared physical copy of data across Snowflake and Fabric
• Snowflake remains the primary writer, but Fabric needs deep analytical access
• You are intentionally adopting open table formats for long-term interoperability
• You want to minimize storage duplication and ongoing data movement costs

When to avoid this option

• You need Fabric-native Delta features (such as Direct Lake without relying on Iceberg→Delta virtualization)
• You want Fabric to be the primary write, optimization, and lifecycle management engine
• Your organization is not ready to clearly define write ownership, schema evolution, and conflict resolution between Snowflake and Fabric

One-line summary:

Best long-term architecture for shared analytics using open table formats, with strong cost efficiency and clean data ownership.

More info: Use Snowflake with Iceberg tables in OneLake – Microsoft Fabric | Microsoft Learn, Building an Iceberg Lakehouse with Snowflake and Microsoft OneLake

3) Fabric mirroring from Snowflake to OneLake (Delta format)

With mirroring, Fabric continuously replicates Snowflake tables into OneLake and stores them as Delta tables, delivering the most seamless Fabric-native experience across Lakehouse, Warehouse, Power BI Direct Lake, and AI workloads.

This option is ideal when you want maximum simplicity and performance inside Fabric and are comfortable with Snowflake no longer being the primary analytics engine for those datasets.

From a cost standpoint, you pay for Snowflake compute and cloud services to read source tables and capture changes, and you pay for Fabric capacity to perform replication, optimization, and downstream analytics. You do not pay for OneLake storage for the mirrored data (the mirroring storage cost is free up to a limit based on capacity – for more information, see Cost of mirroring and Microsoft Fabric Pricing), nor do you pay for building or operating ingestion pipelines. While storage is effectively free in OneLake, mirroring is typically the most compute-intensive option.

When to use this option

• You want the most seamless, high-performance Fabric experience
• Power BI Direct Lake, Warehouse, and AI workloads are top priorities
• Fabric will be the primary analytics platform going forward
• You prefer simplicity over shared-write complexity

When to avoid this option

• Snowflake must remain the primary analytics engine
• You want to avoid continuous replication compute costs
• You are standardizing on open formats like Iceberg across platforms

One-line summary:

Best for Fabric-first analytics teams that want maximum performance and simplicity, at the cost of higher compute usage.

More info: Microsoft Fabric Mirrored Databases From Snowflake – Microsoft Fabric | Microsoft Learn, Tutorial: Configure a Microsoft Fabric Mirrored Database From Snowflake – Microsoft Fabric | Microsoft Learn

Summary comparison table

Dimension	Snowflake-managed Iceberg + Shortcut	Snowflake writes Iceberg to OneLake	Fabric Mirroring (Delta)
Primary writer	Snowflake	Snowflake	Fabric
Physical data duplication	None (shared external storage)	None (shared OneLake storage)	Yes (replicated into OneLake)
Format in OneLake	Iceberg	Iceberg	Delta
Fabric experience	Read-oriented, less native	Good, improving	Best / most native
Power BI Direct Lake	Supported via OneLake Iceberg→Delta virtualization*	Supported via OneLake Iceberg→Delta virtualization*	Supported (native Delta)
Ongoing sync	Manual / external	Native (shared)	Automatic (CDC-based)
Separate storage charges	Yes (external storage)	No (included with Fabric capacity)	No (included with Fabric capacity)
Compute cost	Snowflake + Fabric	Snowflake + Fabric	Snowflake + Fabric (highest)
Best for	Minimal change, zero-copy	Long-term open architecture	Fabric-first analytics

*Direct Lake support for Iceberg relies on OneLake’s ability to surface the table as Delta, which may not be guaranteed for all Iceberg configurations or evolution patterns (more info).

Iceberg support in OneLake is evolving quickly, and the gap between Iceberg and Delta experiences in Fabric is narrowing. Over time, Option 2 becomes increasingly attractive as a long-term, open-table architecture—while mirroring remains the fastest path to a fully Fabric-native experience today.

In this post, let’s talk honestly about what it takes to get your enterprise data AI-ready—specifically GenAI-ready. We’ll walk through how this fits into an enterprise data maturity journey (is it Stage 5, or just the “deep end” of Stage 4?), why being AI-ready is about a lot more than just technology, and how focusing on data quality, governance, and architecture can make or break your AI aspirations. We’ll also look at examples of generative AI applications (think chatbots, document summarizers, intelligent search, and copilots) that all share one thing: they demand well-prepared data. And because I’m a fan of practical advice (and biased), I’ll highlight a couple of Azure tools – Microsoft Purview and Microsoft Fabric – that can help you on this journey.

So grab a coffee, and let’s dive in. By the end, you should have a clearer idea of how to start getting your data truly AI-ready.

Note: If you need a refresher on OpenAI and LLMs, check out my blogs Introduction to OpenAI and LLMs, Introduction to OpenAI and LLMs – Part 2 and Introduction to OpenAI and LLMs – Part 3, along with a presentation on that topic that I did for the Toronto Data Professional Community which you can view and download the slides. 

From Stage 4 to Stage 5: Data Maturity Meets AI

Every organization progresses through some kind of data maturity model. You might have seen one of those charts (maybe the one with four stages in my book) that show a company’s evolution from basic reporting all the way to advanced analytics. Typically it goes something like:

• Stage 1 – Reactive analytics: Siloed data, manual reports, and ad-hoc analyses (basically just reacting to past events). Often Excel files are emailed all over (spreadmarts).
• Stage 2 – Informative analytics: A centralized data warehouse and BI provide a single source of truth with standard reports. Focus on stages 1 and 2 is historical reporting. Usually the size and types of data it can handle is limited and solution is not very scalable and data is ingested infrequently (i.e. every night).
• Stage 3 – Predictive analytics: Can handle larger quantity of data, different types, and more frequently (streaming). Data science and machine learning efforts begin, aiming to forecast trends and outcomes, and decisions can be made real-time.
• Stage 4 – Transformative analytics: Can handle any type of data, no matter the size, type, or speed. Advanced analytics (even some AI) is embedded in processes, and a data-driven culture is in full swing.

Now, along comes generative AI. It feels like a leap beyond what we used to consider “advanced analytics.” So where does GenAI fit? It could be a Stage 5 – AI-Driven Enterprise, or simply an evolution of Stage 4 (in reality, it’s a bit of both). If your company is still in Stage 2 or 3, you have foundational work to do before jumping to GenAI – you can’t just skip to the top. But if you’re solidly in Stage 4, getting AI-ready is the next natural step. The key point is that becoming GenAI-ready is a continuation of the data maturity journey, building on everything you’ve achieved so far in BI and analytics, and then going further.

What Does it Mean to Be GenAI-Ready?

Let’s cut through the buzzwords. Being “GenAI-ready” isn’t about having the latest AI tool; it’s about your data being ready to support those tools. It means you’ve set up the plumbing and prepared the ingredients so that an AI system can actually do something useful (and trustworthy) with your data.

How can you tell if your data is AI-ready? Here are a few signs:

• Clean, quality data: It’s accurate, consistent, and up-to-date – no “garbage in” to poison your AI’s outputs.
• Clear context and governance: Data carries business meaning (thanks to good metadata and definitions) and is properly governed, so everyone knows what it represents and how it can be used.
• Managed unstructured data: Documents, emails, and other unstructured sources are stored in accessible formats and tagged with relevant info, ensuring AI models can find and interpret them.
• Flexible architecture: Your data platform can easily integrate new sources and deploy AI models without major rework – it’s built to scale and adapt as AI use cases grow.

If you read that list and feel a bit overwhelmed, you’re not alone! Very few organizations check every box. The idea is to see where you stand and where you need to improve, because any weaknesses in your data will be magnified by AI. Remember, GenAI is not a magic wand – it will amplify whatever you feed it. So feed it well.

Building an AI Solution: As Complex as Building a Data Warehouse (or More)

All those fundamentals – data quality, governance, architecture – underscore one truth: building a robust AI solution is no less complex than building a data warehouse or any major analytics platform (in fact, it can be more).

Why more complex? For starters, GenAI projects often involve diverse data types (text, documents, images, chat logs) and require new infrastructure (like vector databases for embeddings or real-time data pipelines) that you might not have needed in a traditional BI project. Additionally, developing AI is an iterative process of training, fine-tuning, and validating models, which means your data pipelines must be flexible and your team prepared for a lot of experiment-and-learn cycles. In other words, it’s a different style of cooking – you add ingredients, taste, adjust, and repeat – not a simple follow-the-recipe dish.

The takeaway? Approach AI initiatives with the same rigor as your big data projects – if not more. Plan thoroughly, ensure the fundamentals are solid, and involve the right experts (data engineers, data scientists, domain specialists) from day one. It’s a marathon, not a sprint (with a few extra hurdles on the track), but with preparation and teamwork, you’ll reach the finish line.

GenAI Applications in the Wild: Why They Need AI-Ready Data

Let’s ground this in some real-world examples. What do we actually do with GenAI in a business setting, and why does data prep matter so much? Here are a few popular use cases:

• AI Chatbots and Virtual Agents: They need a curated, trusted knowledge base of information; otherwise the bot will either give wrong answers or be embarrassingly clueless.
• Document Summarization and Analysis: The documents must be machine-readable (not just scanned images) and organized, so the AI can find key points and accurately summarize them.
• Intelligent Search: This requires indexing and integrating your data (often via semantic or vector search). Without well-prepared data, the AI won’t retrieve the answers users are looking for.
• AI Copilots for Employees: These AI assistants (for coding, marketing, finance, etc.) rely on internal data. If that data is siloed, outdated, or poorly defined, the copilot’s guidance will be far less useful.

Across all these examples, the pattern is clear: the quality and readiness of data determines whether a GenAI application succeeds or face-plants. Even a state-of-the-art model can’t overcome messy, siloed, or untrustworthy data. I’ve seen brilliant technical prototypes get scrapped because the underlying data pipeline wasn’t sustainable or the outputs couldn’t be trusted by end users. Generative AI can do amazing things, but only if our data house is in order.

Tools for the Journey: Azure Purview and Microsoft Fabric

By now you might be thinking, “Alright, we need to improve a lot of things… where do we even start, and are there tools to help?” The good news is yes – there are platforms that align well with this journey. In the Azure ecosystem, two worth highlighting are Microsoft Purview and Microsoft Fabric:

Microsoft Purview: Purview is Azure’s data governance service. It helps you discover, catalog, and track the lineage of data across your organization. In short, Purview makes it easier to ensure your data is well-defined, trustworthy, and compliant – exactly what you need before unleashing AI on it.

Microsoft Fabric: Fabric is Microsoft’s unified analytics platform, combining data engineering, data warehousing, and data science tools in one place. It’s built with AI in mind – using OneLake to store all your data and integrating seamlessly with Azure AI services. This means you can develop and deploy AI solutions faster, without stitching together a dozen separate systems.

Of course, tools alone won’t magically make your data AI-ready. But leveraging platforms like Purview and Fabric can accelerate the process by reinforcing good practices (governance, single source of truth, scalable architecture) as you embark on your GenAI projects.

Conclusion: Begin Your AI-Ready Data Journey

If you’ve stuck with me this far, you know that getting data GenAI-ready is a journey, not an overnight task. The best way to start is by assessing where you are today. What stage of data maturity are you at, and where are the gaps? Maybe you have lots of data but little governance, or great dashboards but poor data quality. Identifying one or two key areas to improve is a great first step.

Next, consider running a pilot project that leverages GenAI on a small scale. Pick a use case that excites people but is manageable—perhaps an internal Q&A chatbot or an AI-generated report summary. As you execute it, pay attention to what’s blocking you. Are you scrambling to clean data or define metrics? Use those lessons to shore up your data foundations.

Also, remember to celebrate the “boring” work that enables AI – like setting up a data catalog, cleaning datasets, and defining business terms. These may not feel like innovation, but they directly boost your AI projects. And keep the collaboration going: getting data ready for AI isn’t just an IT task, so you need buy-in from leadership and participation from business teams. When everyone sees that better data leads to better AI, it’s much easier to get support for data quality and governance efforts.

If you’ve built a data warehouse or analytics platform before, you already know the playbook: define the goal, get the data in shape, build iteratively, and keep improving. GenAI is simply another chapter in that story.

So, roll up those sleeves and start laying the groundwork. Each improvement in data quality, governance, and architecture moves you closer to the day you can confidently say, “Yes—we’re ready for AI.”

This complexity matters because Microsoft Fabric is the fastest-growing unified data platform in Microsoft’s history, now adopted by over 28,000 customers. Yet for many organizations, the capacity model remains one of the biggest obstacles to adoption. Understanding what’s included versus what’s not, the real impact of reservations, and how Azure quotas affect usage and cost is genuinely hard—and that uncertainty slows down expansion.

Enter Fabric Cost Analysis (FCA) – a free, open-source solution available to everyone on a Microsoft GitHub repository, designed to shine a light on all your Microsoft Fabric costs. FCA was developed by a multidisciplinary team (Cedric Dupui, Manel Omani, Antoine Richet, and led by Romain Casteres) with expertise spanning FinOps, Data, and Go-To-Market, with a clear goal: turn a major adoption barrier into a strategic lever for growth.

Conceived directly from customer questions, FCA answers the things people actually want to know: What are we really paying for? What’s included? Where are the optimization opportunities? It doesn’t just track costs—it builds trust, helps organizations explain spend internally, and ultimately accelerates Fabric adoption.

Built entirely on Fabric itself (talk about eating your own dog food!), FCA demonstrates the platform’s power end-to-end and highlights the use of GenAI through an agentic experience, allowing users to access cost insights in natural language. Deployed at several customers, FCA has been presented to the global community at FabCon in Vienna, the AI Tour in Latin America, and across various community events around the world. The aim is simple: enable everyone to confidently control the cost of the Fabric platform.

Important: Keep in mind that FCA is a solution accelerator, not an official Microsoft product. That means it’s community-supported and carries no formal Microsoft support. It’s been shared freely with the community to help Fabric users, but you use it at your own risk (in practice it’s quite robust, but if something breaks, you’ll be relying on community help). Now, let’s break down what FCA offers and how it works.

Content

In this post, we’ll cover the main aspects of the Fabric Cost Analysis tool: its architecture, the data it uses as inputs, the outputs it provides (reports and a nifty data agent), and how you can set it up and get support. By the end, you should have a clear picture of how FCA can help you monitor and optimize your Fabric costs—and remove friction from Fabric adoption.

Architecture

Figure: High-level architecture of the Fabric Cost Analysis (FCA) solution.

This diagram shows how cost and usage data flows through Microsoft Fabric. Cost data (such as Azure billing info) is ingested via Fabric Pipelines and Notebooks into a Fabric Lakehouse, where it’s stored both in raw form and as optimized Delta Parquet files. This dual storage (raw and curated) allows Power BI’s Direct Lake to access the data directly for analytics, enabling fast and interactive cost reporting.

There’s even a built-in Fabric Data Agent (think of it like an AI assistant for your data) on top of the FCA data model. In short, FCA uses Fabric to monitor Fabric—a self-referential approach that showcases the platform’s capabilities while solving a very real business problem.

FCA Inputs

FCA gathers a variety of data and loads it into your Fabric Lakehouse to analyze costs from multiple angles. The key inputs include:

• Azure cost data in FOCUS format: FCA pulls Azure billing details using the FinOps Open Cost and Usage Specification (FOCUS), providing a normalized, analysis-ready cost model.
• Enriched documentation data: Reference data from Microsoft Learn documentation adds context and definitions, making reports easier to understand.
• Azure Reservations (optional): If you use reservations for Fabric capacities, FCA shows whether they’re actually delivering savings.
• Azure Quotas (optional): Quota data helps you understand usage versus limits—critical for both cost control and capacity planning.

Note: FCA is scoped exclusively to Microsoft Fabric costs. Unrelated Azure services are filtered out, keeping the focus exactly where it belongs.

FCA Outputs

Once the data is collected and processed, FCA delivers two primary outputs: a comprehensive Power BI report and a Fabric Data Agent that allows natural-language exploration of your cost data. Together, they support classic FinOps scenarios—inform, optimize, and operate—while making cost conversations easier across technical and business teams.

Report

Figure: Example of an FCA report page (the “Summary” page).

Each report page focuses on a different aspect of Fabric cost management. The Home page provides a high-level snapshot, clearly separating costs covered by capacity from additional charges—making it much easier to explain spend internally. The Summary page breaks down costs by capacity and region, helping organizations understand how usage is distributed.

Other pages include Capacity Usage (to identify under- or over-utilization), Reservations (to validate savings), Cost Detail (for granular analysis and forecasting), Quotas (to track limits and usage), and a Support page that explains billing nuances and metrics. You can use the reports as-is or extend them using the underlying data model.

Data Agent

Figure: Example of data agent communication in both English and French.

One of FCA’s most powerful features is its Fabric Data Agent integration. This agentic, GenAI-powered experience allows users to ask questions like:

• “Which Fabric capacities cost the most this quarter?”
• “How have my Fabric costs trended over the last six months?”

The Data Agent retrieves answers directly from the FCA semantic model, making cost analysis accessible even to non-technical users. It works in multiple languages and can be accessed via the web or Microsoft Teams. This capability reinforces FCA’s core mission: remove friction, build confidence, and make cost transparency part of everyday decision-making.

Setup

Getting started with FCA is intentionally simple. The solution is delivered through the Microsoft Fabric Toolbox on GitHub and includes an automated, guided deployment process. A one-click deployment sets up pipelines, notebooks, the Lakehouse, datasets, and reports. Check it out here.

As long as you have the appropriate Fabric workspace permissions and an Azure subscription for billing data, deployment typically takes just minutes. Many users start with a limited scope to explore the solution, then expand to full production once comfortable.

Support

Because FCA is a community-driven solution, support happens openly on GitHub. Bugs, questions, and feature requests are handled through GitHub issues, which also serve as the project backlog. The team actively monitors feedback and incorporates improvements over time.

One important reminder: do not open Microsoft support tickets for FCA. Instead, use the GitHub repository, where both the creators and the broader community can help.

Final Thoughts

Fabric Cost Analysis turns one of the most common frustrations with Microsoft Fabric—cost transparency—into a strength. Built by practitioners, shaped by customers, and shared freely with the community, FCA helps organizations understand what they’re paying for, optimize usage, and confidently scale Fabric.

If you’re running Microsoft Fabric capacities, FCA is well worth deploying. It demystifies billing, highlights optimization opportunities, and helps teams move from hesitation to expansion—exactly what a modern FinOps-driven platform needs.

Check out the demo presentation available on YouTube : Fabric Cost Analysis.

Happy cost optimizing.

Microsoft SQL Server 2025, with built-in AI and developer-first enhancements, is now generally available. The platform enables customers to securely use data they already have and work in the familiar T-SQL language. It provides:

• A way to access AI models of choice, hosted locally or in the cloud, and to securely use data to best fit business needs.
• Simplified data processing with native JSON support, built-in REST APIs and change event streaming.
• Near real-time analytics by replicating SQL Server data to Microsoft OneLake with database mirroring in Microsoft Fabric.
• Increased workload performance, uptime and concurrency for SQL Server apps with enhanced query optimization, optimized locking and improved failover reliability.
• Improved credential management and fewer potential vulnerabilities with Microsoft Entra ID for authentication through Microsoft Azure Arc.
• GitHub Copilot integration in Visual Studio Code and SQL Server Management Studio 22 for better productivity.
• A new Microsoft Python driver for SQL Server (mssql-python) for a fast and developer-friendly experience in Windows, macOS and Linux.

(more info) (Bob Ward post)

Microsoft Azure DocumentDB now generally available
Microsoft Azure DocumentDB, the first managed service built on the open-source DocumentDB standard, is generally available. Now governed by the Linux Foundation, DocumentDB delivers an open and community-driven MongoDB-compatible engine with multicloud flexibility, running consistently across Azure, other clouds and on premises. This gives organizations freedom from proprietary lock-in and the ability to standardize on open source while operating at a global scale. Azure DocumentDB was previously known as Azure Cosmos DB for MongoDB (vCore). Note that this new Azure DocumentDB is not the same product as the original DocumentDB that became Cosmos DB—it’s a newly branded, separate vCore-based MongoDB service. (more info)

Azure HorizonDB, a new PostgreSQL database, in private preview
Microsoft Azure HorizonDB, a new PostgreSQL cloud database service for building or modernizing mission-critical apps, is now in private preview. Integrated with Microsoft Foundry, Microsoft Fabric, Visual Studio Code and more, Azure HorizonDB streamlines development with the following features:

• Transactions and vector search up to three times faster than open-source PostgreSQL, based upon internal benchmarking.
• Scale-out compute to 15 replicas with 192 vCores each.
• Auto-scaling storage up to 128 TB.
• Advanced DiskANN vector indexing for AI workloads and native semantic operators.
• AI-readiness with pre-provisioned models.

Organizations can right-size consumption to their workloads’ needs and save capacity for future requirements by independently scaling compute and storage scale. Modern authentication with Microsoft Entra ID and security features like Microsoft Defender and private endpoints support enterprise-grade protection. It is optimized for performance, scale, and AI apps and designed to compete directly with AWS Aurora and GCP AlloyDB. (more info)

Microsoft Fabric databases, now generally available, bring together SQL database and Cosmos DB in a new, unified software-as-a-service (SaaS) experience for organizations to manage, analyze and activate their data. Fabric databases provide instant provisioning, autonomous architecture, enterprise-grade security and native AI integration — including support for vector data and retrieval-augmented generation (RAG) patterns — to help teams build intelligent, real-time apps. (more info)

Announcing Microsoft Fabric IQ: The Semantic Intelligence Platform
Microsoft Fabric IQ is the new semantic intelligence layer that elevates Fabric from a unified data platform to a unified intelligence platform. It turns your unified data estate, already consolidated in OneLake, into a live, structured, connected model of how your business operates. It bridges the gap between where your data lives and how your teams and AI reason, decide, and act.

Fabric IQ combines five integrated capabilities into one semantic intelligence system:

• Ontology (preview): shared model of business entities, relationships, rules, and objectives. This was just released at Ignite.
• Semantic Model: trusted BI definitions, now extended beyond analytics into operations and AI. Semantic models have been available for a long time.
• Graph model (preview): native graph engine for multi-hop reasoning and system-wide insights. This was released a couple of months ago.
• Data Agent (preview): virtual analysts that answer business questions using structured business meaning. This has been available for about a year.
• Operations Agent (preview): autonomous agents that reason, learn, and act in real time to advance outcomes. This has been available for about a year.

So to clarify: Fabric IQ is a new “umbrella” name for five features, only one of which is brand new – Ontology.

More about Ontology:

An ontology in Microsoft Fabric is a shared, machine-understandable vocabulary that defines the core business concepts that exist within an organization—things like customers, products, orders, and assets, rather than just the raw tables that store data. It provides a business-level semantic layer that standardizes terminology across domains, ensuring all teams and tools refer to the same entity names, properties, and relationships. This eliminates inconsistencies that commonly arise when different teams model the same concepts in different ways.

Once created, the ontology is bound to actual data sources in Fabric, including lakehouse tables, event streams, and semantic models. This binding process maps columns to properties, links identifiers to relationships, and transforms raw table rows into typed entity instances enriched with consistent semantics, metadata, provenance, and timestamps. The ontology becomes the layer through which physical data is interpreted, giving Fabric a unified view of business meaning across disparate systems.

When Graph in Microsoft Fabric is enabled, the ontology is materialized as a graph where each entity instance becomes a node and each relationship becomes an edge. This graph representation allows visual exploration (e.g. browsing relationships, lineage, dependencies), graph-style queries or algorithms (e.g. pathfinding, impact analysis) and also semantic queries: you ask in terms of business concepts (not tables) — e.g. “Find all shipments that are delayed and associated with high-risk routes” — and Fabric handles the underlying joins, filtering, data reconciliation. This enables a far richer and more intuitive way to navigate enterprise data.

Using an ontology brings multiple benefits over traditional table-centric models. It enforces cross-domain consistency and governance by providing a single semantic standard for the entire organization. It simplifies integration by allowing data from different sources—batch, real-time, structured, or semantic—to be unified under the same conceptual model. Its semantic richness enables deeper context and expressiveness, supporting advanced analytics and reasoning that are difficult or impossible with flat tables.

Finally, because the ontology abstracts business meaning away from physical schemas, teams gain flexibility and agility. Changes to underlying tables or new data sources can often be accommodated simply by rebinding them to the ontology, without rewriting analytics or business logic. Combined with Fabric’s graph and AI capabilities, the ontology becomes a powerful foundation for agents, copilots, and intelligent applications that interact with data in terms of real business concepts.

To create an ontology in Microsoft Fabric, you begin by defining the ontology itself—either manually or by generating it from an existing semantic model. When generated, each table is converted into an entity type, its columns become properties, and any existing relationships are turned into ontology relationships. After the initial creation, the next step is to refine and rename these entity types so they reflect business-friendly concepts rather than raw technical table names, such as transforming “dimproduct” into “Product” or “factsales” into “SaleEvent.”

With the conceptual model established, you then bind your actual data sources to the ontology. This involves mapping tables or event streams to the appropriate properties and relationships, defining identifiers and keys, and handling differences between time-series and static data. If desired, you can then enable Graph support, which turns the ontology into a first-class graph structure. This unlocks graph traversal, lineage insights, dependency analysis, and more advanced graph operations.

Finally, once the ontology is bound and optionally graph-enabled, you can query your data through the business-level semantic layer rather than through raw tables. Queries are expressed using business concepts—such as “Show all Orders for Customer X between date Y and date Z”—and Fabric handles the necessary joins, filters, and reconciliations. If natural-language querying is enabled, users can interact with the data even more intuitively, relying on the ontology to interpret their intent.

In short, Fabric ontology is a business-level semantic layer you create over your data that standardizes meaning across the organization and makes it far easier for users—and AI agents—to ask questions and get answers in business terms rather than table structures. It unifies data from disparate sources, represents it as a connected graph when enabled, and provides a consistent, intelligent foundation for analytics, exploration, and automation. It differs from a sematic model in that a semantic model describes how your data is stored, while an ontology describes what your business actually is.

Fabric OneLake and Databricks integration announcements: Mirroring data into OneLake – already generally available; By the end of 2025, Azure Databricks will enable native reading from OneLake through Unity Catalog in preview, allowing users to seamlessly access data stored in OneLake without duplication or complex pipelines; Looking ahead, Azure Databricks will support writing and storing data directly in OneLake, without any additional storage resources to manage. 

Preview of OneLake shortcuts to SharePoint and OneDrive, allowing you to bring your unstructured, productivity data into OneLake without copying files or building custom ETL flows. 

Fabric capacity overage/expanding surge protection – To help you gain control over the jobs running on your Fabric capacities, Microsoft is expanding surge protection and introducing a new tool called Fabric capacity overage—both of which will be released into preview in Q1 2026—and adding Fabric capacity events in the Real-Time hub. First, surge protection will now let you set limits on specific workspace activity to protect your capacities from unexpected surges from non-critical workspaces.  Also to be released is Fabric capacity overage which admins can turn on for specific capacities, allowing them to automatically pay for excess consumption and avoid throttling whenever high-traffic periods occur. Rather than over-provisioning for rare spikes, you can right-size your capacity for typical usage and enable overage only when needed. Admins can even set a 24-hour limit so you don’t break your budget, and the feature can be toggled on or off in seconds. These tools are designed to work together to help you prevent over-use and maintain smooth, uninterrupted operations even during peak demand.

Data Clustering in Fabric Data Warehouse (Preview) – Fabric Data Warehouse introduces data clustering capabilities to optimize query performance and reduce storage costs through intelligent data organization

IDENTITY columns (Preview) in Fabric Data Warehouse, a long-awaited feature that simplifies surrogate key generation during data ingestion. IDENTITY columns automatically produce unique values for each new row, eliminating the need for manual key assignments and eliminating the risk of key duplication and key integrity issues. (more info)

Fabric Warehouse Snapshots have GA’d. Create read-only views of your warehouse at a specific point in time. (more info)

Mirroring for SQL Server in Fabric for all in-market versions of SQL Server from SQL Server 2016 to SQL Server 2025 is Generally Available. (more info)

Fabric Capacity Events in Real-Time Hub (Preview) – Real-Time Hub now streams Fabric capacity events, enabling proactive monitoring and management of compute resources and workload performance.

Cosmos DB in Microsoft Fabric is now Generally Available – Cosmos DB integrates natively with Microsoft Fabric, enabling seamless NoSQL workloads alongside analytics and AI within a unified data platform.

What’s New for Fabric Data Agents at Ignite 2025 – Fabric Data Agents gain enhanced reasoning capabilities and improved AI interoperability, enabling more sophisticated data analysis and automated insights.

ReadWrite access controls within lakehouse (preview) is now supported for items via OneLake security. This enhancement gives data owners the ability to grant precise write permissions to users—without requiring elevated workspace roles like Admin or Member. With ReadWrite access, workspace viewers or users with only Read access can now write data to specific tables and folders in a lakehouse, while remaining restricted from creating or managing Fabric items. (more info)

Some name change history: Azure AI Studio was rebranded as Azure AI Foundry at Microsoft Ignite 2024, introducing a unified platform for building and managing AI applications. In Ignite 2025, it was again renamed, this time to Microsoft Foundry, along with the announcement of many new features in Foundry (workflow, direct integration with m365).

More info:

Azure at Microsoft Ignite 2025: All the intelligent cloud news explained

Reflections from Microsoft Ignite 2025 – Podcast

Microsoft Databases and Microsoft Fabric: Your unified and AI-powered data estate

Fabric November 2025 Feature Summary

Foundational Books (Start Here)

I highly recommend starting with the classics from Ralph Kimball. Though they predate modern cloud platforms, their core concepts remain essential (I talk about all his books here).

They also sell all three books as a bundle on Amazon. I used these same books when I was a DBA building my first warehouse.

Shameless plug: My own book is a continuation of what has transpired since the Kimball era. Order here.

Learning Modern Architectures

Kimball’s books don’t cover data lakes or lakehouses (they weren’t around yet), so I recommend the following:

• Building Medallion Architectures – A great book that introduces the medallion pattern (bronze/silver/gold layers) for lakehouse design.
• Learning paths on Fabric if using it to build your data warehouse:
• Implement a data warehouse with Microsoft Fabric – A guided Microsoft Learn path.
• Get started with Microsoft Fabric – Helpful if Fabric is brand new to you – you may first want to start with the overview modules.
• Microsoft Fabric Data Warehouse End-to-End Tutorial – A practical walkthrough.
• Books on Fabric if using it to build your data warehouse:
• Fundamentals of Microsoft Fabric: Designing End-to-End Analytics Solutions
• Learn Microsoft Fabric: A Practical Guide to Performing Data Analytics in the Era of AI

YouTube Videos to Check Out

My own videos:

• Deciphering Data Architectures: Modern Data Warehouse, Data Fabric, Data Lakehouse, Data Mesh
• Data Architectures and Microsoft Fabric
• Microsoft Fabric: Lakehouse vs Warehouse

Other recommended videos:

• Creating Your First Data Warehouse in Microsoft Fabric (by Guy in a Cube – they have many great Fabric videos)
• Data Warehouse vs Data Lake vs Data Lakehouse
• What’s the Best Big Data Architecture for You?

Blogs to Follow

• James Serra’s Blog (another shamless plug)
• Navigating Modern Data Architecture: DW, Lakehouse, and Lakebase Explained
• Bill Inmon
• Ralph Kimball
• Piethein Strengholt

Workshops Worth Taking

My own workshops:

• Deciphering Data Architectures (via eLearningCurve)

Others I recommend:

• DAB0126: Data Architecture Bootcamp: Practical Exercises in Architecture Design – Dataversity (Jan 20-22, 2026)
• Data Warehousing 101 – EWSolutions
• Data Architecture Essentials: Building a Data Foundation for Enterprise Analytics – TDWI

Final Thoughts

If you’re a DBA transitioning into data architecture, you don’t need to reinvent the wheel. Lean on the enduring frameworks from Kimball, combine them with modern cloud-based architecture strategies, and pace yourself through trusted books, tutorials, and workshops.

Have go-to resources that helped you build your first data warehouse? Share them with me in the comments below or via my email jamesserra3@gmail.com—I’m always on the lookout for great new content.

Microsoft Purview is a comprehensive solution for managing, protecting, and governing data across an organization. But it’s important to understand that “Purview” is actually an umbrella brand that includes three main areas of functionality:

1. Data Governance (Azure Purview) – focuses on discovering, classifying, and managing data across on-premises, multi-cloud, and SaaS environments.
2. Data Security (M365 Purview) – covers data loss prevention, insider risk management, information protection, and adaptive protection. This set of capabilities evolved from what was previously known as Microsoft Information Protection (MIP).
3. Data Compliance (M365 Purview) – focuses on compliance manager, eDiscovery and audit, communication compliance, data lifecycle management, and records management. These capabilities came from what was formerly called Microsoft Information Governance.

This blog will focus only on the first area—data governance—often referred to as Azure Purview. It’s the part of Microsoft Purview that gives organizations visibility into their data landscape: where data lives, how it’s used, who owns it, and whether it can be trusted.

Data governance is the foundation for any data-driven organization. It ensures that everyone—from analysts to executives—can discover and use reliable, secure, and well-understood data. Below are the six major business benefits of Microsoft’s data governance capabilities in Azure Purview.

1. Centralized Data Catalog and Discovery

(Preventing “Reinventing the Wheel”)

Imagine if every department in your organization could see, at a glance, what data already exists and what it means. That’s exactly what Azure Purview’s data catalog enables.

A data catalog is like an intelligent inventory of all your data assets—databases, reports, files, dashboards, and even SaaS data sources. Azure Purview automatically scans your environments and builds a central catalog containing metadata (the “data about the data”) such as file names, owners, schemas, and data types.

This catalog serves as a single source of truth across the enterprise. Users can search for data using familiar business terms, browse datasets, and immediately see what’s available instead of starting from scratch. For example, instead of recreating a customer sales dataset, an analyst can search “sales revenue” and find the certified, approved dataset that already exists.

Each data asset in the catalog also includes data lineage—a visual representation of where data originates, how it moves through systems, and how it’s transformed along the way. If a report shows unexpected numbers, you can trace them back through every step to the original source system.

Beyond visibility, the catalog encourages collaboration and reuse. Data stewards can add notes, business definitions, and quality tags, making it easier for others to understand and trust the data. Certified datasets can be flagged so users know they’re safe and accurate to use.

The result: no more duplicated work or disconnected knowledge. Teams can focus on insights instead of spending hours searching for or recreating data.

2. Sensitive Data Identification and Classification

(Improving Security and Building Trust)

Every organization holds sensitive data—personal details, financial records, health information—and managing that data responsibly is critical for both compliance and reputation. Azure Purview helps by automatically discovering and classifying sensitive data across your entire environment.

As Purview scans your data estate, it uses AI-powered rules and predefined patterns to detect sensitive information such as credit card numbers, Social Security numbers, or email addresses. Once detected, it applies classification labels—metadata tags that categorize data (see Classifications and sensitivity labels in Microsoft Purview).

This automatic classification provides visibility into where sensitive data resides and how it’s used. You can quickly identify which databases or files contain personal data and assess whether they’re being stored or shared appropriately.

From a business perspective, this capability is essential for compliance with regulations such as GDPR, HIPAA, and CCPA. You can demonstrate that your organization knows where its sensitive data is, who owns it, and what safeguards are in place.

And while data governance (Azure Purview) focuses on identifying and cataloging sensitive information, data security (M365 Purview) takes it a step further—enforcing rules to prevent data loss, manage insider risk, and apply encryption policies. Together, these two sides of Purview ensure that data is both understood and protected.

It’s important to note that while Azure Purview can identify and classify sensitive data, it doesn’t actually secure that data—it focuses on metadata, not the content itself. To protect the underlying information, you would use M365 Purview for items like Word documents or coordinate with, for example, an Oracle DBA to secure an Oracle database.

In short, Azure Purview strengthens trust in your data by making the invisible visible. You can’t protect what you don’t know you have—and with Purview, you finally do.

3. Governed, Self-Service Data Access

(Centralized and Compliant Access to Data)

Even with well-cataloged data, employees often face a major roadblock: getting access. Azure Purview streamlines this challenge with governed self-service access.

When a user discovers a dataset in the catalog, they can request access directly through the Purview portal. This request is automatically routed to the dataset’s owner or steward, who can approve or deny it with one click. The entire process is logged for transparency and compliance.

This approach replaces the inefficient back-and-forth of emails or IT tickets with a centralized, automated workflow. Users get faster access to the data they need for insights, while data owners maintain control and oversight.

Note that Azure Purview can only automatically grant read access in limited cases—currently, primarily for certain Azure data sources such as Azure SQL Database. For most other systems, a data administrator or DBA must manually grant access outside of Purview (for example, within Oracle or SAP), and then return to Purview to mark the request as approved.

Governed access ensures that sensitive data is shared responsibly. Access requests can be tied to specific policies—for example, only Finance department members can access financial results. Purview enforces these rules automatically, maintaining compliance without slowing productivity.

In many organizations, this becomes a cultural shift: teams start to share more data because they can do so safely and traceably. The catalog becomes not just a repository, but a trusted marketplace for data assets, where users can browse, request, and use data confidently.

The business benefit is twofold: decision-making speeds up, and risk goes down. People can access what they need, when they need it, without bypassing governance controls.

4. Data Quality and Health Monitoring

(Ensuring Reliable, Actionable, and Governed Data)

High-quality data is the foundation of trustworthy analytics and sound decision-making. Azure Purview now includes powerful data quality and health monitoring capabilities that help organizations continuously measure, improve, and govern the quality of their data.

Data Quality

With Purview’s new data quality model, organizations can identify and fix data quality issues using a no-code/low-code approach. Business users, data stewards, or domain owners can define rules at different levels—business domains, data products, or individual data assets—without writing complex code.

Purview also provides a growing library of out-of-the-box (OOB) rules that check for common problems such as duplicate rows, empty fields, and missing or non-unique values. Copilot can even suggest new rules automatically, helping teams establish data standards faster.

Once rules are configured, the data quality model evaluates your data against those rules and generates data quality scores at the asset, product, or domain level. These scores give you a quick snapshot of how your data measures up to the business rules you’ve defined.

Data profiling and data quality scans further enhance visibility:

• Data profiling provides quick insights from a small sample set to spot potential issues early.
• Data quality scans perform in-depth analysis across full data sets to detect inconsistencies and anomalies.

When problems are detected—such as a sudden drop in a quality score or unusual data patterns—Purview can generate data quality actions that highlight what needs attention. These actions can be assigned to specific people to resolve issues using tools like Azure Data Factory. Once the fix is complete, the issue can be marked as resolved, maintaining a full audit trail.

You can also configure data quality alerts that notify users when certain conditions are met—for example, when the data quality score for the Sales domain drops below 50%. Alerts appear within Purview and can also be sent by email, ensuring that data stewards can take quick corrective action.

Finally, Purview allows you to define data access policies at the business domain, data product, or glossary-term level. Any time a glossary term is applied to a data product, all its associated policies—such as access limits, required approvals, or permissions for data copies—are automatically enforced. This unifies governance and quality under one framework.

Health Controls

Purview’s health controls track your organization’s overall progress toward complete data governance. These controls measure how well your data environment aligns with governance standards and provide a governance health score.

Examples of health controls include:

• Metadata completeness – Are key fields documented?
• Cataloging – Are all major data sources registered?
• Classification – What percentage of assets are classified?
• Access entitlement – Who can access what data, and is it governed?
• Data quality – How does the data score against your rules?

Data officers can configure thresholds that define red, yellow, or green indicators for each metric. For instance, you might set a target that 80% of data assets must be classified or that 90% should be mapped to data products for discoverability.

Health Actions

Whenever Purview detects gaps or misalignments—such as unclassified assets or unmapped data products—it automatically creates health actions. These actions appear in a new Action Center that aggregates governance-related tasks by role, data product, or business domain.

Each action includes recommendations for how to fix the issue and can be assigned to an owner for resolution. Clicking an action provides direct guidance on how to bring the data asset or domain back into compliance. As teams complete these actions, the organization’s overall governance posture improves.

This approach turns governance into an interactive, team-based process. Instead of static reports, Purview provides a living dashboard of your data’s health—showing where to focus next and tracking progress over time. By cleaning up outstanding actions and maintaining high-quality data, your organization strengthens both trust and agility in its data-driven decision-making.

Note that data quality and health monitoring were one of the new features added last year (see Microsoft Purview new data governance features).

5. Business Glossary

(Speaking a Common Data Language)

Many data problems start not with numbers, but with words. Departments often use different terms for the same concept, leading to confusion and misaligned reporting.

Azure Purview solves this with a Business Glossary—a central library of business terms and definitions. This glossary creates a shared vocabulary that connects business language to technical data assets.

For instance, the term “Active Customer” might be defined as “a customer with at least one purchase in the last 12 months.” That definition is stored in Purview and linked to the specific datasets and reports that use it. Everyone—from Finance to Marketing—can see and use the same definition.

This not only ensures consistent understanding but also prevents data disputes. When executives review dashboards or metrics, they know that terms like “Revenue” or “Churn Rate” mean the same thing across the organization.

Glossary terms can also be linked to governance policies. For example, a glossary term like “Personal Data” can automatically trigger stricter access controls or encryption requirements whenever it’s applied to a dataset.

The result: clearer communication, fewer misunderstandings, and stronger alignment between business and IT.

6. Business Domains and Data Products

(Organizing Data for Business Context and Discoverability)

A major advancement in Azure Purview’s data governance framework is the introduction of business domains and data products—two concepts that bring structure, business meaning, and reusability to your data catalog. These features help organizations align their data estate with how the business actually operates.

Business Domains

A business domain is a framework for organizing data around a common business purpose or capability, such as Sales, Finance, or Marketing. Think of it as a mini catalog inside your main data catalog—a logical boundary that aligns your data assets with your organizational structure.

Business domains make it easier to manage business concepts, assign ownership, and define governance rules. They differ from the technical domains used in the Data Map (known as collections), which group assets by project, technology, or ownership. However, business domains can be mapped to these collections, so assets tied to a business domain are automatically linked to the corresponding technical assets beneath it.

Within each business domain, you can:

1. Create and manage business domains to organize and curate your catalog.
2. Assign owners and stewards responsible for data governance within that domain.
3. Relate business domains to the underlying data collections in the Data Map.
4. Create glossary terms for key business concepts—using Copilot to suggest relevant terms automatically.
5. Monitor the health of your domains, taking timely actions to keep them well-governed.
6. Define business objectives and key results (OKRs)—such as increasing sales by 10% or reducing support cases by 3%—and track progress directly within Purview.
7. Define critical data elements (CDEs), which logically group key pieces of information (for example, mapping “CustID” in one table and “CID” in another under a single “Customer ID” concept).

Business domains provide a way to connect the technical world of data assets with the business world of strategy and operations. By aligning data governance structures with business functions, organizations can make data more understandable, more accessible, and more relevant.

Data Products

A data product represents a curated group of data assets packaged together for a specific business use case or purpose. Data products are assigned to business domains and act as logical business concepts that make data easier to find and use.

Instead of users hunting across dozens of individual tables or files, a data product bundles them all into one logical unit. For example, a “Global Sales Revenue for 2023CY” data product could include tables, files, and Power BI reports related to sales performance. When users request access to that data product, they automatically get access (after approval) to all the associated assets—no more requesting permissions for 15 separate tables.

Data products streamline governance and improve efficiency:

• Organization: Each data product belongs to one business domain but can be discovered across multiple domains.
• Discoverability: Data products are searchable using natural language. For instance, you can type, “Show me daily retail sales data for the past six months,” and Purview will surface relevant products.
• Context: Descriptions within data products include use cases, examples, and instructions for analysis.
• Ownership: Each data product has an owner—often a data scientist, analyst, or steward—responsible for maintaining its accuracy and usefulness.
• Governance: Access requests, approvals, and policies apply at the data product level, ensuring consistency and compliance.

Data products also tie back to glossary terms and governance policies. For instance, if a glossary term labeled “Customer Data” includes specific access policies, applying that term to a data product automatically enforces those policies.

The hierarchy in Azure Purview now looks like this:
Business Domains → Data Products → Data Assets.

An example might be:
Sales (Business Domain) → Global Sales Revenue for 2023CY (Data Product) → Global Sales for 2023 Power BI Report (Data Asset).

This model gives users a clean, intuitive way to explore data. Instead of sifting through thousands of assets, they can browse or search by business domain, open a data product, and find everything they need in one place. On the Data Product Search page, users can explore data products, view their details, and track data access requests using the “My Data Access” tab—all from a single, business-centric interface.

Why Business Domains and Data Products Matter

Together, business domains and data products transform Azure Purview from a technical catalog into a business-aligned data marketplace.

• Business domains give your data structure and purpose.
• Data products make it consumable and actionable.

This approach empowers teams to focus on business outcomes—like growing revenue or improving customer satisfaction—while ensuring the data behind those goals is well-organized, governed, and easy to find.

It’s the next step in making data governance not just a compliance exercise, but a true business enabler.

Note that business domains and data products were one of the new features added last year (see Microsoft Purview new data governance features).

Conclusion

Microsoft Purview is far more than a simple data catalog—it’s a unified ecosystem for governing, securing, and understanding data across your entire organization. While M365 Purview focuses on data security (protecting sensitive information and preventing data loss) and data compliance (managing records, retention, and regulatory requirements), Azure Purview is all about data governance—helping you discover, understand, organize, and trust your data.

The data catalog prevents duplication and drives data discovery.
The classification engine identifies sensitive data, improving transparency and compliance readiness.
The governed access framework streamlines how users request and receive permission to use data while maintaining control.
The new data quality and health monitoring capabilities ensure that your data is accurate, complete, and reliable—empowering your teams to make decisions with confidence.
The business glossary aligns everyone around a consistent vocabulary, breaking down communication barriers between business and technical teams.
And the introduction of business domains and data products brings everything together—organizing data into meaningful business contexts and packaging it into reusable, governed assets that anyone in the organization can find and use.

Together, these components create a living, breathing governance framework that turns data chaos into clarity. Azure Purview provides not just visibility into your data, but also accountability, structure, and business meaning—all built on a foundation of automation and AI assistance.

In short:

• Azure Purview helps you govern and understand your data.
• M365 Purview helps you protect and comply with your data.

By combining these two sides of Purview, organizations can finally achieve what most only talk about: a complete, end-to-end data governance and protection strategy—one that’s modern, scalable, and aligned to how the business actually operates.

• There is now mirroring support for Oracle and Google BigQuery (preview) in Fabric. See video and more info on BigQuery mirroring and Oracle mirroring
• Fabric is now extending Fabric data agents to support all mirrored databases (preview)
• Now in public preview is a new OneLake shortcut transformations to automatically convert JSON and Parquet files to Delta tables. More info
• OneLake security is now in full preview, and also available is a new tab in the OneLake catalog called Secure, where you can manage the security and permissions for all your data items (see video). More info
• Graph in Fabric (preview) is designed to enable organizations to visualize and query relationships that drive business outcomes. Built upon the proven architecture principles of LinkedIn’s graph technology, graph in Fabric can help you reveal connections across customers, partners, and supply chains (see video)
• Maps in Fabric (preview) can help you bring geospatial context to your agents and operations by transforming enormous volumes of location-based data into interactive, real-time visualizations that drive location-aware decisions and enhance business awareness (see video). More info
• Released the Fabric Extensibility Toolkit into preview—an evolution of the Microsoft Fabric Workload Development Kit but newly designed to help any developer bring their data apps to Fabric for their own organizations along with a simplified architecture and additional automation to drastically streamline development. Developers can now simply build their own Fabric items, and everything else like distribution, user interface, and security is taken care of for you. More info
• Introduced the preview of Fabric MCP, a developer-focused Model Context Protocol that enables AI-assisted code generation and item authoring in Microsoft Fabric. Designed for agent-powered development and automation, it streamlines how you build using Fabric’s public APIs with built-in templates and best-practice instructions. It also integrates with tools like Microsoft Visual Studio Code and GitHub Codespaces and is fully open and extensible
• Added horizontal tabs for open items, support for multiple active workspaces, and a new object explorer—all designed to make multitasking in Fabric smoother, faster, and more intuitive. More info
• General availability of an end-to-end migration experience natively built into Fabric, enabling Azure Synapse Analytics (data warehouse) customers to transition seamlessly to Microsoft Fabric. The migration experience allows you to migrate both metadata and data from Synapse Analytics and comes with an intelligent assessment, guided support, and AI-powered assistance to minimize the migration effort. More info
• You can now add, view and manage multiple lakehouses in a single unified view
• Mirrored database support for Fabric Data Agent: Enables users to directly connect mirrored database artifacts including Azure Cosmos DBs, Azure SQL, Oracle, Snowflake, Databricks, and other databases using open mirroring. Including this integration allows users to leverage Data Agent’s Natural Language to SQL (NL2SQL) capability, so users can ask questions in plain English, and receive LLM- powered insights across their data estate
• CI/CD support in Fabric Data Agent: Now supports CI/CD, ALM flow, and Git integration, enhancing management, version control, and collaboration for Data Agent artifacts. These features promote reliable, scalable, and auditable development practices by enabling systematic management of changes, dedicated workspaces for development stages, and broad data source support. Git integration tracks all modifications, supports branching for independent experimentation, and enables controlled merging, improving teamwork and allowing quick reversion if issues arise. More info
• Discover which query examples influenced the Data Agent response: When the Data Agent answers a question, it reviews the example queries you’ve provided and uses them to guide its reasoning. With the latest update, creators can view exactly which example queries were used during a run-step to help shape the agent’s response
• Download Diagnostics in Data Agent: Download a diagnostics file for any run step in the Data Agent chat canvas—giving you clear visibility into how the agent processed your question behind the scenes. The file includes details like which tools were used, how the question was interpreted, the intermediate reasoning steps, and any errors or fallback logic that occurred. More info
• MERGE Transact-SQL (Preview): This command blends INSERT, UPDATE, and DELETE operations all into a single statement based on your specified conditions between two tables, improving readability and providing a uniform standard for transformations across your ETL jobs. More info
• Anomaly detection in Real-time Intelligence (Preview): Quickly spot unusual patterns or behaviors in streaming data with no coding required. You select an Eventhouse, choose ID fields to monitor, and the system automatically tests models to recommend the best fit for your data. Detected anomalies appear directly in the interface, and you can experiment with models before publishing results to the Real-Time Hub. With instant alerts via Teams or email, this no-code tool empowers both analysts and business users to act fast on real-time insights

More info:

FabCon Vienna: Build data-rich agents on an enterprise-ready foundation

Fabric September 2025 Feature Summary

Fabcon Vienna 2025 – Key announcements

FabCon Europe 2025 Recap

I’ve had a number of customers asking about the new AI features in SQL Server 2025, so I wanted to write a quick post summarizing what’s available today. In reality, there are two major capabilities worth highlighting: Copilot in SQL Server Management Studio (SSMS) and support for vector data types with semantic search.

Keep in mind that SQL Server 2025 is still in preview, with Release Candidate (RC0) recently announced. You can find the full list of updates in the official documentation: What’s new in SQL Server 2025 Preview.

Copilot in SQL Server Management Studio (SSMS)

One of the most exciting new AI features in SQL Server 2025 is the introduction of Copilot in SSMS (requires SSMS 21). This capability allows you to interact with your data using natural language instead of writing T-SQL from scratch. With Copilot, DBAs and developers can quickly perform text and data analytics without needing to move data elsewhere.

Copilot in SSMS lets you:

• Ask natural-language questions about your database and environment
• Get help writing and optimizing T-SQL
• Explore and analyze data without knowing SQL in detail

For example:
Type “List all orders from the last 30 days for New York customers” and Copilot will return both the T-SQL query:

SELECT * 
FROM Orders 
WHERE OrderDate > DATEADD(DAY, -30, GETDATE()) 
  AND City = 'New York';

and the corresponding results. This makes it easy to learn T-SQL, validate your intent, and accelerate development.

Privacy, Security, and Responsible AI

Copilot in SSMS is built on Azure OpenAI resources, which you provision in your own subscription. Importantly, it does not retain prompts, responses, or system metadata, and your data is never used to train or retrain AI models. It follows Microsoft’s Responsible AI practices for Azure OpenAI models, with more details available in the Transparency Note for Copilot in SQL Server Management Studio and Data, privacy, and security for Azure OpenAI Service.

Supported Databases and Permissions

Copilot works across:

• SQL Server
• Azure SQL Database
• Azure SQL Managed Instance
• SQL Database in Microsoft Fabric

It respects your database permissions. If your login doesn’t have access to a table, Copilot won’t be able to run queries against it. For example, if you don’t have permission to query Sales.Orders, a request to select from that table will fail.

Beyond Query Writing

Copilot isn’t just for writing queries—it’s also useful for database and environment exploration. You can ask questions like:

• “What version of SQL Server is this instance running?”
• “Which columns store email addresses?”
• “How will changing compatibility mode affect query performance?”
• “What queries have executed most frequently in the last two hours?”
• “What’s the difference between a full and log backup?”

It can even help with database development tasks, such as creating tables, indexes, or generating sample data.

Getting Started

To use Copilot in SSMS, you’ll need (see instructions):

1. An Azure OpenAI endpoint and deployment in your subscription (see create the necessary Azure OpenAI resources). Note that the future SSMS 22 will use your Github Copilot account instead of an Azure OpenAI endpoint – see What’s next for Copilot in SSMS.
2. The latest version of SQL Server Management Studio with Copilot enabled.

Once configured, you’ll be able to start writing Language-Assisted Queries (LAQ)—natural language instructions that Copilot translates into valid T-SQL, returning both the code and results (like charts or tables).

More info on using Copilot in SSMS can be found here.

Vector Data Types with Semantic Search

The second major AI capability in SQL Server 2025 is support for vector data types and functions, enabling true semantic search directly inside the database. With this feature, you can store AI-generated embeddings as vectors, compare them using functions like VECTOR_SEARCH(), and generate embeddings inline with AI_GENERATE_EMBEDDINGS().

This unlocks use cases such as:

• Find products similar to this one
• Show me documents related to this query
• Surface support tickets about login issues

Instead of matching just keywords, SQL Server can now search by meaning.

How Semantic Search Works

1. Text → Embeddings
   Text (like descriptions, reviews, or documents) is converted into embeddings — high-dimensional numeric vectors created by an AI model such as Azure OpenAI.
2. Store in a VECTOR Column
   SQL Server introduces a new VECTOR column type to persist embeddings.
3. Similarity Search with VECTOR_SEARCH()
   Queries compare embeddings based on semantic closeness, not just exact words.

Example: Finding Similar Products

-- Step 1: Create OpenAI model
CREATE EXTERNAL MODEL MyOpenAI_Embedding_Model
WITH (
      LOCATION = 'https://my-azure-openai-endpoint.openai.azure.com/openai/deployments/text-embedding-ada-002/embeddings?api-version=2023-05-15',
      API_FORMAT = 'Azure OpenAI',
      MODEL_TYPE = EMBEDDINGS,
      MODEL = 'text-embedding-ada-002'
);

-- Step 2: Create table with a VECTOR column
CREATE TABLE Products (
    ProductID INT PRIMARY KEY,
    Name NVARCHAR(100),
    Description NVARCHAR(MAX),
    DescriptionEmbedding VECTOR(1536)  -- new in SQL Server 2025, stores AI-generated vector
);

-- Step 3: Create vector index (DiskANN)
CREATE VECTOR INDEX vec_idx_Products_DescriptionEmbedding
ON Products (DescriptionEmbedding)
WITH (
    METRIC = 'cosine',      -- can also use 'dot' or 'euclidean'
    TYPE = 'diskann',       -- algorithm type (only DiskANN supported in preview)
    MAXDOP = 4              -- optional, controls degree of parallelism
)
ON [PRIMARY];               -- or your preferred filegroup

-- Step 4: Insert data with embeddings
INSERT INTO Products (ProductID, Name, Description, DescriptionEmbedding)
VALUES (
    1,
    'Wireless Earbuds',
    'Bluetooth earbuds with noise cancellation and 24-hour battery life.',
    AI_GENERATE_EMBEDDINGS('Bluetooth earbuds with noise cancellation and 24-hour battery life.' USE MODEL = MyOpenAI_Embedding_Model)
);

-- Step 5: Run a semantic search
DECLARE @queryVector VECTOR(1536) = 
    AI_GENERATE_EMBEDDINGS('Noise-cancelling wireless headphones' USE MODEL = MyOpenAI_Embedding_Model);

SELECT TOP 5 p.ProductID, p.Name, p.Description, v.distance AS Similarity
FROM VECTOR_SEARCH(
    TABLE = Products AS p,
    COLUMN = DescriptionEmbedding,
    SIMILAR_TO = @queryVector,
    TOP_N = 5,
    METRIC = 'cosine'
) AS v
ORDER BY v.distance;

This returns the top 5 products whose descriptions are semantically closest to “noise-cancelling wireless headphones” — even if those exact words never appear.

Why It Matters

Traditional Full-Text Search (FTS) matches words, stems, or phrases. That fails when:

• Synonyms are used (earbuds vs. headphones)
• Users type casual language (can’t log in vs. authentication error)
• Context or intent matters more than literal words

With vector search, SQL Server can now match based on meaning.

Example:

DECLARE @queryVector VECTOR(1536) = 
    AI_GENERATE_EMBEDDINGS('I can’t log into the portal');

SELECT TOP 5 TicketID, Subject, Body, Distance
FROM VECTOR_SEARCH(
    TABLE = SupportTickets,
    COLUMN = DescriptionEmbedding,
    SIMILAR_TO = @queryVector,
    TOP_N = 5,
    METRIC = 'cosine'
ORDER BY Distance;

This surfaces tickets about login problems — even if the phrase “log into the portal” never appears.

SQL Server as a Vector Database

SQL Server 2025 brings capabilities typically found in specialized vector databases like Pinecone, Weaviate, or Milvus, — as well as in search services like Azure Cognitive Search (now called Azure AI Search), but with the advantage of being integrated into the same platform enterprises already use for structured data.

That means you can:

• Avoid managing a separate vector database
• Add semantic search to catalogs, support tickets, or documents
• Build Retrieval-Augmented Generation (RAG) systems with T-SQL and Azure OpenAI

And since Copilot in SSMS works with vector search, you can simply type “Show me electronics that block outside sound” and SQL Server will translate that into a semantic search using vector comparison.

The bottom line: SQL Server 2025 turns your database into a vector-powered AI engine, enabling smarter, more relevant search and analytics without leaving T-SQL.

SQL Server 2025’s vector-powered AI engine can certainly enhance a simple SELECT statement by making it return more relevant text results. But its real value shines as part of a larger Generative AI (GenAI) architecture, where the database retrieves contextually relevant text for a user’s question and sends both to a large language model (LLM). This approach, known as Retrieval-Augmented Generation (RAG), improves accuracy by grounding the LLM’s response in your own data. I covered this pattern in more detail in my post Introduction to OpenAI and LLMs.

Think of Azure IoT Operations as the control center for your digital operations at the edge. It sits directly alongside your machines and sensors—on the factory floor, in warehouses, or across distributed sites—and ensures that data from these environments flows smoothly into the cloud. Built on modern, scalable infrastructure, it connects operational systems with business applications, making it easier to unlock insights, improve efficiency, and respond quickly to changes in your environment.

The real impact comes from bringing together Information Technology (IT) and Operational Technology (OT). For years, IT has handled business data and applications while OT has run the physical equipment. Azure IoT Operations unifies the two. Imagine a factory where equipment performance data is captured in real time and immediately analyzed to predict failures before they happen. You can then use Microsoft Fabric to build real-time dashboards with visualizations that show the health of each asset and trigger alerts when anomalies are detected. These dashboards can be displayed right on the shop floor, giving operators the information they need to take immediate action and prevent costly downtime. The result is lower maintenance costs, higher productivity, and more consistent product quality. By securely connecting these systems to the cloud, companies can also optimize energy usage, reduce waste, and make faster, data-driven decisions. In short, Azure IoT Operations helps organizations modernize operations, cut costs, and gain the agility needed to compete in today’s digital-first economy.

Here is an Azure IoT Operations architecture overview:

At its core, Azure IoT Operations is a set of data services that run on Azure Arc–enabled edge Kubernetes clusters, where you can deploy Azure IoT Operations by using the Azure portal or the Azure CLI.. These services work together to make it easier to capture, process, and route data from assets on the shop floor to applications in the cloud.

• MQTT broker – An edge-native broker that enables event-driven architectures, allowing devices and applications to exchange messages reliably in real time.
• Connector for OPC UA – Handles the complexity of OPC UA communication with servers and devices, making it easier to integrate industrial equipment into the data pipeline.
• Data flows – Provide data transformation and contextualization capabilities. They let you enrich and reshape raw messages and then route them to a variety of destinations, including cloud endpoints.
• Operations experience – A web-based UI that gives operational technology (OT) teams a unified view to manage assets, devices, and data flows. IT administrators can also use Azure Arc site manager (preview) to group IoT Operations instances by physical location, making it easier for OT users to find and manage deployments.

Together, these services create a consistent, secure, and flexible environment for IT and OT teams to collaborate, ensuring that edge data is not only captured but made meaningful and actionable.

One of the biggest challenges in IoT is balancing the needs of IT and OT. IT teams want control, governance, and security. OT teams—those who actually run the machines on the shop floor or in the field—want simple tools that let them keep operations running without opening tickets for IT every time a change is needed. This is exactly where the Azure IoT Operations experience comes in.

The experience is a web-based interface that provides a unified view into your IoT Operations environment. Instead of sifting through Kubernetes manifests or JSON configs, OT users can log into a simple portal and manage the things that matter to them: assets, data flows, and system health.

Through this experience, you can:

• Manage assets by defining machines, controllers, and sensors so they are represented in a way that makes sense for operations staff.
• Configure data flows that determine how telemetry moves from devices, through the MQTT broker, into transformations, and eventually to cloud endpoints like Event Hubs or Fabric.
• Apply transformations directly at the edge to contextualize data before sending it on—reducing both latency and bandwidth costs.
• Monitor status and health with visibility into system throughput, errors, and connectivity, all in one place.

Here is a sample of the overview page in the Azure IoT Operations experience:

The real value is that it lets OT teams operate independently, while IT still has the governance and security controls they need in the background. IT administrators use Azure Arc to deploy and secure the underlying infrastructure, and they can group multiple IoT Operations instances by physical location using the Azure Arc site manager (preview). But once everything is in place, OT staff can work in the experience every day without needing deep technical expertise.

Here is a sample of building a data flow in the Azure IoT Operations experience:

To connect to the cloud from Azure IoT Operations, you can use the following data flow destination endpoints:

• Azure Event Grid and other cloud-based MQTT brokers
• Azure Event Hubs or Kafka
• Azure Data Lake Storage
• Microsoft Fabric OneLake
• Azure Data Explorer

Microsoft always supports three generally available (GA) versions of Azure IoT Operations at any one time: the latest version, and the two previous minor versions.

Currently, there are only two minor versions available. Azure support is currently available for the following versions:

• 1.2.x (latest preview version)
• 1.1.x (latest GA version)
• 1.0.x (previous minor GA version)

Make sure to check out the supported environments and supported regions.

Note that Azure IoT Operations is different from a product I talked about in a previous blog post called Azure IoT Central. When deciding between Azure IoT Operations and Azure IoT Central, it helps to understand that while both deal with connecting and managing IoT devices, they serve very different purposes. Azure IoT Operations is best thought of as a set of building blocks for creating industrial-strength IoT solutions, particularly in environments that require hybrid architectures and edge processing. It gives you full control over protocols like OPC UA, Modbus, and MQTT, and allows you to design the data flows, monitoring, and integration points exactly as your business needs. This flexibility makes it ideal for complex industrial scenarios where low latency and tight IT/OT integration are essential.

Azure IoT Central, on the other hand, is a fully managed SaaS offering. Instead of building from the ground up, you get a ready-made platform with device provisioning, templates, monitoring, and dashboards already included. It’s designed for speed and simplicity, letting you connect devices and gain insights with very little setup. The tradeoff is that you sacrifice some deep customization and architectural control. For most businesses that need a predictable, subscription-based model and quick time to value, IoT Central is a great fit.

The choice ultimately comes down to how much control and flexibility you need versus how quickly you want to deploy. If you are building a large-scale, industrial IoT platform that requires fine-grained control and edge-heavy workloads, Azure IoT Operations is the right tool. If your priority is getting started fast with minimal overhead, Azure IoT Central will meet your needs. Both approaches are valid—it’s about matching the tool to the problem you’re solving.

More info:

Accelerate industrial transformation with Azure IoT operations (Ignite video)

Real time data ingestion with Azure IoT Operations and Microsoft Fabric CWB virtual meet up (video)

Azure IoT Operations: Empowering the Future of Connectivity and Automation

Uncovering Azure IoT Operations: A New Era of Industrial IoT

Here are the supported AI transformations:

• Text Summary: Generates concise summaries from long-form text.
• Translate: Translates text between supported languages.
• Sentiment: Labels text sentiment as positive, negative, or neutral.
• PII detection: Finds and redacts personally identifiable information (names, phone numbers, emails).
• Name recognition: Extracts named entities such as people, organizations, or locations.

It’s very easy to setup: While in Fabric, go to a Lakehouse and under the Files section, select + New  Shortcut and create a connection that points to a folder of .txt files in ADLS, Blob Storage, S3, Dataverse, GCS, OneDrive (preview), SharePoint Folder (preview) or OneLake in another Fabric workspace. Only .txt files are supported now, but soon JSON and Parquet will be supported (UPDATE 10/21/25: JSON and Parquet support is now available, with Excel in the future). Then in the wizard, select the folder and from Transform, select one of the five AI transformation options listed:

Once you complete creating the shortcut, it will take a few minutes for the initial transform to finish. Each table and folder selected will appear as a unique shortcut in the Files section, under which the transformation will show up in Delta format. The columns in the transformed Delta file/table depend on the AI transform you chose. For example, if you selected Text Summary, the columns would be “text” and “TextSummary”:

If you selected Translate, the resulting columns look like:

If you selected Sentiment, the resulting columns look like:

If you selected PII detection, it does not create a Delta table. Instead, a .txt file is created with the PII info replaced with asterisks:

If you selected Name recognition, the resulting columns look like:

The engine checks the source folder every two minutes. New, modified, or deleted files are reflected in the Delta table. Use the resulting table in reports, notebooks, or downstream pipelines (if using it for reports, you would next need to use the Load to Tables option on the resulting folder where the Delta table is located and choose the Parquet file type to copy it to the Tables section). The status of the transformation can be observed under Manage shortcut:

Since a shortcut points to a folder, and you can only select only one transform format for each folder, you will want to create a folder for each type of transform you will do, and put only the files for that particular transform in it. For example, create a folder called TranslateFiles and put all the files to translate text between languages in that folder, then create a shortcut that uses the Translate transform format. Create another folder called SentimentFiles and put all files to determine the sentiment labels in that folder, then create a shortcut that uses the Sentiment transform format. So if you use all the transform formats, you will wind up with five folders and five shortcuts.

Here are some ideas for using this new feature:

• Customer‑feedback sentiment dashboard: Point a shortcut at yesterday’s review exports, choose Sentiment, and publish a Power BI report that surfaces net sentiment by product, region and time period.
• Multilingual support intelligence: Route global support logs through Translate (to English) followed by optional Text Summary (do this by chaining two shortcuts together), giving managers a single, prioritized list of issues across every market.
• Privacy‑first LLM fine‑tuning: Apply PII detection to call center transcripts to create a compliant dataset that is immediately ready for model training.
• Real‑time market signals: Stream a newswire feed through Name recognition to tag companies, people and locations, enrich your knowledge graph and trigger alerts the moment a relevant story breaks.

This new feature allows you to import unstructured data and semi-structured data into Fabric while transforming it on the fly, all without writing any code or building any pipelines!

Also note there is another feature if you need to create shortcuts to csv files to import into a Delta Lake table.

More info:

Accelerating Insights from Unstructured Text with AI Powered OneLake Shortcut Transformations

AI-powered transforms in OneLake shortcut transformations

What is Azure AI Language?

Why Present?

Why should you even think about presenting if it makes you nervous? Well, speaking in public has a bunch of benefits. I found that preparing a talk really forced me to learn my subject inside-out. I also built my personal brand and network – people started remembering me when I spoke on something I was passionate about. Presenting helped me become a better communicator overall (including at non-technical events like my speech at my daughter’s wedding), and it’s been critical to my career success. Plus, there’s something really fun about teaching others and seeing their eyes light up when they understand what I am presenting.

• Passion: If there’s a topic you care about, why not share it? I always focus on areas that excite me, because that energy comes through in my talk.
• Learning: I learned early on that “teaching is learning twice.” By presenting, I forced myself to really understand the material.
• Career Boost: Presenting led to new opportunities for me. (Honestly, I went from an unknown to speaking at big conferences, and my income nearly tripled over a few years.) It can happen to you too with some persistence.

Finding Topics and Venues

New presenters often ask: Where do I even present? The good news is, there are many places to start small and work your way up.

• Start Local: I began with a quick “lunch and learn” at my office and my local user group/Meetup. These are low-pressure ways to share something with peers.
• Virtual Meetups: Online chapters and community streams are great too. Even during the pandemic, I did plenty of talks from my home office.
• Community Events: Look for events like SQLSaturday, local tech conferences, or hackathons. They often have speaker calls or welcome new speakers in “lightning” or “new speaker” tracks.
• Larger Conferences: Eventually I submitted abstracts to bigger conferences. They often encourage new speakers, so don’t be shy about trying.

Keep a list of every idea you have for a talk. I use OneNote and even just a simple text file. It doesn’t have to be unique or groundbreaking – presenters often reuse and update topics. For each idea, jot down a working title and a few bullet points on what you’d cover. I even keep a running list of talks I’ve given (title, event, date) on my blog, and upload my slides online after each talk. Seeing that list grow over time was really motivating for me.

Writing Your Abstract

Once you pick a topic and find a venue, you’ll usually need to write an abstract (a short description of your session) that you submit to the conference and hopefully get chosen. Here are a few things I do to improve my chances of getting chosen:

• Hit a Pain Point or Hot Topic: Think about what problems people have right now. For example, if everyone at work is talking about a new tool or feature, that’s a good hook.
• Know Your Audience: Will it be beginners, mid-level, or experts? Tailor your abstract so the reader knows it’s just for them.
• Get Feedback: I often ask a colleague or a mentor to review my abstract. A fresh set of eyes can catch stuff I missed.

I try to write the abstract before making slides. It forces me to outline the key points. In fact, I often create a simple outline or “story arc” for the talk first: intro, main points, conclusion. This makes it easier when I actually build the slides.

Building the Presentation

When I sit down to create slides, I follow a few personal rules to keep them effective and engaging:

• One Idea Per Slide: I try to put a single main idea or image on each slide. This keeps the audience focused. I’ve learned that people only retain maybe 20-30% of what you say, so I try not to overload slides with text.
• Less Text, More Graphics: I try to avoid many words on slides. Instead, I use diagrams, charts, or photos that illustrate the point. A good picture can say more than a paragraph of text.
• Tell Stories: Whenever possible, I add a short story or real-world example from my own experience. Stories are memorable and they keep people awake!
• Outline First, Then Slides: I actually create a bullet-point outline of my talk before bothering with slide design. Once the outline looks solid, I flesh out each bullet into a slide (or two), making sure each has a clear purpose.
• Rehearse with Tools: I use the PowerPoint Speaker Coach or Teams Speaker Coach or other apps (even a simple timer app) to practice my pacing and get feedback. This helps avoid surprises, like running out of time or having too much content. After you do many presentations, you will likely develop an innate sense of how long your deck will take to present.

Above all, I imagine I’m explaining the topic to a smart friend who’s curious but does not know the topic well. This mindset keeps the talk balanced between informative and entertaining.

Before I Step Up (Preparation)

I’m a big believer in not leaving anything to chance before a talk. Here’s my pre-talk checklist:

• Time It: I wear a watch or use my phone’s timer, so I can keep track of time without looking at the clock. I also ask the session chair or room monitor for a 10-minute warning. Many of the best clickers (like my Logitech R800) even has a built-in timer or vibration alert.
• Backup Plan: I always bring a copy of my slides on a USB stick and have them on the cloud. If my laptop fails or I can’t get on the Internet, I can quickly pull my slides up. I also shut off any screen savers or auto-updates on my device to avoid interruptions.
• Pointer/Clicker: A wireless clicker with a laser pointer is a lifesaver. It frees me to walk around and not be glued to the keyboard.
• ZoomIt (or equivalent): For technical talks, I often need to highlight parts of the screen. I use ZoomIt so I can draw on the screen during a demo.
• Health & Dress: I have a small bottle of water onstage in case my mouth goes dry. I also make sure to wear something at least as professional as the audience – often just a step above what people expect.
• Arrive Early: At least 15 minutes early, if possible. This gives me time to test the projector/mic, make sure my slides look good on the big screen, and to calm my nerves.

For virtual talks, I do extra prep too: I use an upgraded webcam and a powerful mic, and even join from a second device to see what the audience will see. I mute notifications and put my laptop in “Do Not Disturb” mode so I don’t get pop-up distractions. Little things like that help me appear professional and keep the focus on my content.

When You’re Up There

Once I’m in front of the audience (in-person or on camera), I follow a few “on stage” tips I’ve learned:

• Eye Contact: I try to look at different people around the room for a few seconds each. This makes it feel more conversational.
• Energy and Passion: I consciously add energy to my voice. I vary my tone and volume a bit – louder on key points, and I pause for effect. I smile and show that I’m enjoying the topic. I also try to add some humor along the way.
• Avoid Fillers: I watch out for filler words (“um”, “ah”, “you know”, “like”). Whenever I feel one coming, I just pause instead. A short silence is much better than a string of “ums”.
• Tell Personal Stories: If there’s a good story or memory related to the slide, I share it. This keeps things human and relatable.
• Engage the Audience: I like to ask a couple of quick, simple questions to the group. It wakes people up and lets me gauge interest. If someone asks a question, I repeat it for everyone and then answer.
• Watch the Clock: I keep an eye on my progress. If I see we’re running late, I might skip or speed up less essential parts.
• Stay Humble: I try to come off as someone friendly, not a know-it-all. If I don’t know an answer to a question, I’ll admit that and say I’ll look into it.

Throughout the talk, I use my voice and hands to emphasize points. Walking around is fine, but I’ll usually pause and stand still when making a key statement. That helps people absorb it.

After the Talk

I’ve found the most rewarding part often comes after the talk—when people come up to share their thoughts or ask questions.

• Q&A: I love hanging around afterward to chat. I put my email on the last slide, and I tell people they can email me if they think of questions later.
• Feedback: If there’s a survey or feedback form, I encourage attendees to fill it out honestly. I read every comment later.
• Reflect: I jot down notes right after the session about what went well and what didn’t.
• Share Slides: I usually mention at the start or end where they can find my slides.

And of course, celebrate a little. Every presentation I do, big or small, feels like a milestone. I give myself a quick mental high-five for facing the fear and getting through it.

Final Thoughts

If you’ve stuck with me this far, here’s my main message: Just do it. Seriously, sign up for a talk (even a tiny one) and go for it. The path to becoming a great presenter does start with that very first talk. You’ll learn more in that single presentation than weeks of reading about it. And you’ll have fun along the way.

So, schedule your first presentation soon. Write the abstract, make the slides, practice a bit, and get it on the calendar. Trust me, the anxiety you feel now will fade after a few minutes into the talk, and you’ll end up feeling proud and energized afterward. The first step is the hardest, but once you take it, you’ll never look back.

Go ahead – take that risk. You’ve got this!

Everything announced at Build can be found in the Microsoft Build 2025 Book of News.

Here you go:

Cosmos DB in Fabric (preview): First there was SQL Database in Fabric, and now Cosmos DB. You can now store semi-structured NoSQL data in Cosmos DB in Fabric, alongside your relational data in SQL databases, enabling a unified data platform for your applications. This further positions Fabric as a complete data platform to handle all your organizational needs, from operational to analytics and BI. Check out this video to see it in action. You can try Cosmos DB in Fabric today or learn more by reading the Cosmos DB in Fabric blog.

Digital twin builder in Fabric (preview): A new capability designed to help organizations bridge their physical and digital worlds to create an AI ready foundation for their operations. It simplifies the creation and management of data-driven digital twins at scale, enabling users to understand system-wide connections and create unparalleled insights and operational efficiency. More info. Here is a video demo.

Microsoft 365 Copilot Tuning (available via Early Access Program): This is huge! Offers customers a new way to unlock the value of fine-tuning for organizations without the cost and complexity of other solutions.  Now, makers can use the low-code tooling in Microsoft Copilot Studio to take advantage of highly automated fine-tuning “recipes” that can use your enterprise data to train models to assist with domain-specific tasks.  Once training is finished, agents, built in Agent Builder with just a few clicks, can then tap into these fine-tuned, task-specific models and easily integrate them into the Microsoft 365 apps. More info and video.

Copilot Studio multi-agent orchestration: Also huge! Enables agents to exchange data, collaborate on tasks, and divide their work based on each agent’s expertise. For example, multiple agents can collaborate across HR, IT, and marketing to help onboard a new employee. See this Video and more info.

Warehouse Snapshots in Microsoft Fabric (preview): A new capability to provide a stable, read-only view of your data warehouse at a specific point in time. With Warehouse Snapshots, you can confidently support analytics, reporting, and historical analysis without worrying about the volatility of live data updates. More info.

Materialized Lake Views in Fabric (preview soon): Allows you to build declarative data pipelines using SQL, complete with built-in data quality rules and automatic monitoring of data transformations. In essence, an MLV is a persisted, continuously updated view of your data that simplifies how you implement multi-stage Lakehouse processing, commonly referred to as medallion architecture. More info.

Mirroring for SQL Server in Microsoft Fabric (preview): For all in-market versions of SQL Server from SQL Server 2016 to SQL Server 2025. More info.

Microsoft Fabric SKU estimator: Predict your capacity needs by entering details about how your team plans to use Fabric. Use it.

Fabric roadmap tool: Brings the upcoming features all together in one place, with a cleaner interface, real-time updates, and direct integration with the internal planning tool used by the Fabric team. See it.

Shortcut transformations (preview): Introduces the ability to transform data as it’s shortcut into Fabric including changing the data format into Delta tables or applying AI transformations to unstructured data—such as summarizing text, translating content, or classifying documents. More info.

Fabric data agent integration with Microsoft Copilot Studio (preview): In Fabric, you can create data agents—virtual business analysts which enable you to engage in natural language conversations about your data in OneLake. Building on this capability, Microsoft is bringing these data agents into Copilot Studio. Once you connect your Fabric data agent to your custom agent in Copilot Studio, it can be deployed across Microsoft Teams and Microsoft 365 Copilot to reason over complex datasets, get insights directly from data in OneLake (respecting data access permissions), and take action. Your agents can even automate tasks like sending emails or triggering workflows, making it easier for users to interact with enterprise data and make data-driven actions in context.  Check out the video.

Fabric data agent with Copilot in Power BI (preview soon): Previously, you may have access to multiple resources, but finding the right data to answer specific questions can be challenging. Copilot was limited to the right pane of a single report, allowing questions only about that open report. This new standalone Copilot in Power BI addresses this by streamlining the process. This full-screen Copilot experience, easily accessible from the left navigation of Fabric, can help you find and ask questions about any data you have access to. It serves two purposes. First, you can ask Copilot to find Power BI reports, semantic models, apps (coming soon), and Fabric data agents that you have access to. It will match across many different factors to quickly find the most relevant items (if you already know which resource to use, you can manually add it to the Copilot session and interact with it directly for more relevant results). Second, it allows you to ask natural language questions and receive accurate, relevant answers from your available Fabric resources. For example, using Copilot’s summary capabilities allows you to quickly identify the most interesting data within your report. Often, reports can become quite complex, and you can easily spend 30 minutes to a couple of hours combing through all the details. However, Copilot can help you sift through the report, giving you easy to digest overviews of your data. Just ask Copilot to ‘Tell me about trends in sales’ or more specific topics, like ‘What should I know about bike sales in Washington?’. You may also have a business question that cannot be answered by the existing report. Typically, what you’d have to do in these cases is work with an analyst to get a new visual added to the report, which can take time, delaying your ability to make a data driven decision. This can be especially cumbersome if it’s not something you’d want to track long term. Also, with the standalone Copilot experience, you can easily ask questions on a semantic model, allowing you to quickly find answers to ad hoc questions. Finally, you can also ask questions against your Fabric data agents.  Check out the video and more info and more info.

Prepare your data for AI: Power BI is introducing new capabilities to help you get your data ready for natural language experiences with Copilot. The first step to get the best results from Power BI Copilot is always to have a semantic model that follows best practices. However, models often require more assistance to be fully prepared for the optimal interactions with AI. Now available in Power BI Desktop is tooling features to help you prepare your data for AI (coming soon for the Power BI service). These features allow you to provide more context about your model, help guide Copilot to the right data in the model and help increase the quality of Copilot output. More info.

Dataflow Gen2 parameterization (preview): Leveraging query parameters while authoring Dataflows Gen2 has been possible for a long time, however, it was not possible to override the parameter values when refreshing the dataflow. The ability to pass values from a pipeline into a Dataflow parameter for refresh has been one of the top ideas in the Fabric ideas portal since Dataflow Gen2 was released.

SQL Server 2025 (public preview): Microsoft announced the public preview of SQL Server 2025 at Build, positioning it as the AI-ready enterprise database. This release brings native AI integration into the SQL engine, allowing developers to build intelligent applications using familiar T-SQL syntax. Key features include built-in vector search for semantic queries, enhanced support for JSON and regular expressions, and compatibility with popular AI frameworks like LangChain and Semantic Kernel. It also introduces an open-source Python driver and GitHub Copilot integration within SQL Server Management Studio (SSMS) 21. Security and performance improvements include support for Microsoft Entra managed identities, optimized locking, and intelligent query processing. SQL Server 2025 also connects seamlessly with Microsoft Fabric for real-time analytics and supports hybrid deployments via Azure Arc. More info.

More info:

Powering the next AI frontier with Microsoft Fabric and the Azure data portfolio

Get to insights faster with SaaS databases and “chat with your data”

Introducing Microsoft 365 Copilot Tuning, multi-agent orchestration, and more from Microsoft Build 2025

Fabric May 2025 Feature Summary

Modern Data Warehouse (MDW)
A Modern Data Warehouse architecture combines a data lake for storing raw, unstructured, and semi-structured data, with a relational data warehouse for serving structured and curated data to business users. This hybrid architecture offers the best of both worlds: the flexibility and scalability of a data lake with the governance, performance, and usability of a traditional data warehouse​. (more info). How the MDW architecture looks like at a high level:

Data Fabric
Data Fabric is an evolved form of the Modern Data Warehouse, enriched with more technologies to support real-time processing, metadata catalogs, data virtualization, APIs, and governance tools. It creates a unified architecture that allows users to seamlessly access and manage distributed data across various platforms and formats, enhancing scalability, automation, and security​. (more info). In purple are the data fabric features:

Data Lakehouse
A Data Lakehouse attempts to merge the capabilities of data lakes and relational data warehouses into one platform, typically by using a transactional storage layer like Delta Lake, Apache Iceberg, or Apache Hudi on top of a data lake. It allows both raw data storage and structured querying in a single repository, enabling cost-efficient analytics and simplified architecture without a separate relational data warehouse​. (more info). This is how Microsoft Fabric operates – with data from lakehouses and warehouses all stored in a delta lake called OneLake – there is no relational storage. See Microsoft Fabric reference architecture. In the diagram below Delta Lake is added and the RDW is removed. Note that the model step is still included as you want something like a semantic model as the interface to the data instead of a list of folders and files.

Data Mesh
Unlike the Modern Data Warehouse, Data Fabric, and Data Lakehouse—which all rely on a centralized architecture where operational data is ingested into a central system—Data Mesh decentralizes data ownership, allowing each domain to retain its own operational data and take responsibility for building and managing its own analytical data. Rather than depending on a central IT team, domains treat data as a product and create their own analytics pipelines. Importantly, Data Mesh is a conceptual framework, not a technology, and each domain can choose to implement its analytical architecture using a Modern Data Warehouse, Data Fabric, or Data Lakehouse—whichever best fits their specific needs​. (more info)

This diagram shows on the left picture the centralized approaches used for the first three architectures, where operational data is copied to a central location owned by IT, who then creates the analytical data. In a data mesh, data is kept within several domains within a company, such as manufacturing, sales, and suppliers. Each domain has its own mini-IT team that takes its operational data, cleans it, and makes it available as analytical data that it owns, using its own compute and storage infrastructure.  This results in a decentralized architecture where data, people, and infrastructure are scaled out – the more domains you have, the more people and infrastructure you get. 

Here is a very high-level use case for each architecture (in ascending cost and complexity):

Modern Data Warehouse
Ideal for organizations handling relatively small volumes of data (typically <1TB), particularly those already familiar with relational data warehouses (RDWs). If your dataset is very small, you may even skip implementing a data lake. This architecture excels at structured reporting and business intelligence scenarios, offering well-established design patterns and a low barrier to adoption. However, its scalability for AI and real-time use cases is limited.

• Best for: Traditional BI, reporting, and small-scale analytics
• Advantages: Easier adoption, well-known patterns, minimal learning curve
• Considerations: Limited scalability for AI; less suitable for large, diverse datasets

Data Fabric
Suited for companies that must integrate and analyze a wide variety of data sources differing in size, speed, and format. It’s also appropriate when modernizing a legacy environment where a full rewrite (e.g., of many stored procedures) would be cost-prohibitive. Data Fabric supports real-time access, federated queries, and AI-driven use cases via a semantic layer. However, it demands strong data governance and integration discipline.

• Best for: Real-time integration, federated access, and complex data landscapes
• Advantages: Unified access layer, real-time support, AI-ready semantic modeling
• Considerations: Complex to implement; requires robust governance framework

Data Lakehouse
Best used as a flexible and cost-efficient solution for combining raw data storage and structured analytics in a single platform. It supports both AI and traditional BI, with transactional layers like Delta Lake, Apache Iceberg, or Apache Hudi providing additional functionality. A good rule of thumb: “Use it until you can’t.” When performance or governance needs outgrow the Lakehouse, offload specific datasets to an RDW as needed.

• Best for: Unified analytics platforms, scalable AI workloads, and mixed data types
• Advantages: Strong AI support, balance of structure and flexibility, lower cost
• Considerations: Moderate organizational change; governance needs to be layered in

Data Mesh
Designed for very large, domain-oriented enterprises experiencing major pain points with scalability and central IT bottlenecks. Data Mesh decentralizes control, giving each domain responsibility for managing its own data pipelines and analytics. Each domain can implement its analytical architecture using a Modern Data Warehouse, Data Fabric, or Data Lakehouse—whichever suits their needs. This approach fosters cross-domain AI scalability but requires a high degree of organizational maturity and a cultural shift toward treating data as a product.

• Best for: Large enterprises with mature data practices and strong domain ownership
• Advantages: Scales AI across domains, reduces bottlenecks, promotes autonomy
• Considerations: Long implementation timelines, requires cultural and process change

Most companies won’t adopt just one architecture. Instead, they’ll blend elements of several, depending on use cases, legacy systems, team capabilities, and AI goals. The key is aligning architecture with both your technical constraints and your organizational maturity.

Here is a table from my book that compares all the architectures, along with RDW and data lake:

I have done a video describing and comparing all four architectures that you can view here. If you want to learn more about these architectures and the concepts behind them, then check out my book.

Also, I will be discussing this topic on May 14th at 1:00pm ET for a webinar on “Navigating the Human Elements to Modernize your Data for AI Transformation” with myself and Christopher Samulski from Argano. Learn how to build a well-organized, informed, and empowered data team. Plus, we’re raffling off 5 copies of my book to attendees! Register here: https://bit.ly/4cyoM46.

• With OneLake security (preview), you define access once, and Fabric enforces it consistently across all engines (Power BI, Spark notebook, SQL analytics endpoint, Excel, Onelake file explorer, API calls). Data owners can create security roles, grant precise permissions, and control access at the row and column level—for example, restricting Personally Identifiable Information (PII) while keeping other data available. This security propagates automatically, ensuring that whether users query via SQL or build Power BI reports, they only see what they’re authorized to access. OneLake security replaces the existing OneLake data access roles preview feature. OneLake security will be available in public preview within a few months. In the meantime, if you are interested in trying OneLake security and providing feedback, please visit this early access sign-up page.  More info
• Copilot and AI capabilities will be enabled for all paid SKUs in Fabric, making these tools accessible to everyone within the coming weeks. With this latest update, customers on F2 and above can use Copilot and AI capabilities, such as Fabric data agents, to streamline workflows, generate insights, and drive impactful decisions. More info
• Now available is a preview of a migration experience, called Migration Assistant, that is natively built into the Fabric UI, enabling Azure Synapse Analytics (data warehouse) customers to transition seamlessly to Microsoft Fabric. With a built-in, intelligent assessment, guided support, and AI-powered assistance, this experience simplifies migration of code and data while helping customers unlock Fabric’s unified data foundation, AI-driven analytics, and enhanced performance—without the complexity of traditional migrations.  More info
• Organizations can now use Azure AI Foundry to connect customized, conversational agents, created in Fabric. AI developers can now use Azure AI Agent Service to securely ground AI agent outputs with enterprise knowledge in Fabric data agents (formerly known as AI skills), so that responses are accurate, relevant, and contextually aware. By combining Fabric’s sophisticated data analysis over enterprise data with Azure AI Foundry’s cutting-edge GenAI technology, businesses can create custom conversational AI agents leveraging domain expertise. . More info
• The Variable library (preview) is a new item type in Microsoft Fabric that allows users to define and manage variables at the workspace level, so they could soon be used across various workspace items, such as data pipelines (already available!), notebooks, Shortcut for lakehouse and more. More info
• The preview of User Data Functions introduces a way for developers to implement and reuse custom business logic in Fabric data science and data engineering workflows, streamlining development and improving efficiency. More info
• The preview of Command Line Interface (CLI) in Fabric introduces a new terminal that allows users and admins to execute commands across Fabric using interactive prompts or scripts, enabling a seamless, code-first experience without relying on clicks.  More info
• The general availability of Tags, which allows users to optimally describe items they own, and help enhance organization and discoverability of data in Fabric. More info
• Several enhancements to Dataflow Gen2 including the general availability of Incremental Refresh, and the preview to save a Dataflow Gen1 as a Dataflow Gen2. 
• Database Mirroring has been enhanced with key new enterprise capabilities, including a new source (Azure Database for PostgreSQL flexible server) and support for connecting to data sources over the On-premises and Virtual Network Data Gateways. 
• The preview of key orchestration enhancements is now available, enabling the creation of metadata-driven pipelines that orchestrate Dataflow Gen2 (CI/CD) parameterized invocation from Data Pipelines.
• Read more about all the Data Factory enhancements in the What’s new with Fabric Data Factory.
• Concerning real-time intelligence (RTI), announced was the preview of new eventstream connectors which allows users to bring in data from additional non-Microsoft sources, including Weather, Solace PubSub+, ADX Table Streamify, MQTT v5, Event Grid Namespaces, and Confluent with Schema Registry.  More info
• The preview of Autoscale Billing for Spark helps optimize Spark job costs by offloading Data Engineering workloads to a serverless billing mode. Capacity admins can set a max capacity units (CUs) limit in capacity settings, ensuring Spark jobs use dedicated CUs instead of shared Fabric Capacity. 
• A new Copilot experience in Fabric notebooks that improves productivity with in-cell interactions, better code generation, and seamless Fabric integration. 
• The preview of AI functions provides powerful capabilities to apply LLM-powered transformations, such as summarization, classification, and text generation to your OneLake data—all with a single line of code.
• A modern get-data experience with OneLake catalog integration in Microsoft Excel (in Office Insiders Fast) enables users to explore the OneLake catalog directly from Excel, expanding accessibility beyond the existing Microsoft Teams integration.
• The preview of Direct Lake semantic models in Power BI desktop, which allows users to build Power BI semantic models for lightning-fast reports that query data directly from OneLake without scheduling refreshes and without data duplication. This feature will also enable users to add in tables from multiple Fabric artifacts in the same Direct Lake semantic model for ultimate reusability of OneLake data.
• Coming soon, the preview of Microsoft Purview for Copilot in Power BI. The integration will enable discovery of data risks such as sensitive data in user prompts and responses, protect sensitive data with Insider Risk Management to identify and investigate risky AI usage, and govern AI usage with audit, eDiscovery, retention policies, and non-compliant usage detection. 
• SQL audit logs are now in preview in Microsoft Fabric Data Warehouse! Audit Logs provide a detailed record of warehouse activity, capturing essential information such as when events occur, which triggered them, and the T-SQL statement behind the event. This feature is crucial for security and compliance, helping organizations monitor access patterns, detect anomalies, and meet regulatory requirements.
• Many new features coming to help with performance for data warehouse: result-set caching, self-managed performance for scale, intelligent workload management, custom SQL pools, data clustering, warehouse snapshots, proactive statistics refresh, and incremental statistics refresh.
• New features coming to shortcuts and mirroring: Cross-network connectivity, table discovery, manage shortcut sessions, key value support, and mirror sources behind a firewall.

More info:

FabCon 2025: Fueling tomorrow’s AI with new agentic capabilities and security innovations in Fabric 

Microsoft Fabric Community Conference Recap – #FabCon 2025 Highlights! (video)

Fabric March 2025 Feature Summary

New innovations in Microsoft Purview for protected, AI-ready data

Here is a diagram of the components used with Real-Time Intelligence (RTI) in Fabric, followed by a description of each:

Eventstream: Capturing and Processing Data in Motion

Eventstream is the foundational component for real-time data ingestion in Microsoft Fabric. It allows users to collect and process data from multiple sources, such as IoT devices, applications, and external event hubs. Eventstreams are event listeners and wait for messages to be sent to them (data is pushed). This is very unlike pipelines, notebooks, and other traditional data processing tools which pull the data from their sources. Eventstream provides built-in connectors, transformation capabilities, and seamless integration with other Fabric components to ensure that data is formatted and routed efficiently. It supports ingestion from various real-time sources such as Azure Event Hubs, Apache Kafka, and Azure IoT Hub (listed on the left in the above diagram and described here). Additionally, it provides low-latency data streaming while enabling data transformation and enrichment before storage or analysis, such as aggregate, filter, and join (full list of operations listed here). Eventstream supports sending data to the destinations listed on the bottom right of the above diagram and described here. Eventstream has the functionality of Azure Eventhub and Azure Stream Analytics. It is a no-code environment that has an authoring canvas that looks like this:

Eventhouse: Storing Real-Time Data Efficiently

Once real-time data is ingested, it needs a place to be stored and accessed quickly. Eventhouse is a high-performance, scalable storage solution designed specifically for event-driven data. It provides structured storage optimized for real-time analytics, ensuring that data remains accessible for downstream processing. Eventhouse is optimized for large-scale event storage and retrieval, supports time-series and event-based analytics, and integrates seamlessly with other Fabric components for querying and reporting. It can handle up to millions of events per hour. Eventhouse has the functionality of Azure Data Explorer and contains KQL databases, which allow you to query billions of rows in just a few seconds. You can think of Eventhouse as simply managing a group of KQL databases. An Eventhouse looks like:

And a KQL database looks like:

Activator: Triggering Automated Actions

Real-time Intelligence is not just about monitoring data—it’s about taking action based on insights. Activator enables automated workflows by triggering actions based on data patterns, thresholds, or anomalies detected in the event stream. It provides configurable event-driven triggers, integrates with Power Automate, Azure Functions, and other automation tools, and supports business logic and rules-based processing to automate responses efficiently. In short, it is a rule-based engine that detects conditions in event streams and triggers responses: Eventstream captures data in motion, and Activator monitors the data for conditions that warrant a response, and when a condition is met, an alert can notify users, while an action can trigger an automated workflow to address the issue. You can create an alert such that when the number of bikes falls below five, an action is taken that sends a person a Teams message:

KQL Queryset: Analyzing Real-Time Data

Kusto Query Language (KQL) is a powerful query language designed for fast and efficient data exploration. KQL Queryset allows users to run real-time queries against event data stored in Eventhouse, enabling deep insights and pattern detection. It facilitates high-speed querying for real-time event data, supports aggregations, pattern matching, and anomaly detection, and enables data filtering and transformation for dashboards and reports. In addition to KQL, users can also leverage T-SQL for querying structured data in real time, making it easier for those familiar with SQL-based analytics to perform real-time analysis within Microsoft Fabric. The query workspace looks like this:

Real-Time Dashboard: Visualizing Insights Instantly

Real-time intelligence is only valuable if decision-makers can interpret the data quickly. Microsoft Fabric enables the creation of Real-Time Dashboards, providing live visualizations of streaming data without relying on external tools (Real-Time Dashboards do not use Power BI). These dashboards help organizations monitor KPIs, detect anomalies, and make data-driven decisions in real time. They offer live data visualizations with minimal latency, customizable dashboards with interactive filtering, and integration with event-driven alerts and automated actions. If you are from the Power BI world, think of Real-Time Dashboards like DirectQuery, but without the need to load data into a semantic model. It is the equivalent of Azure Data Explorer dashboards. Here is what a Real-Time Dashboard looks like:

Real-Time Hub: The Centralized Control Panel

The Real-Time Hub serves as the central interface for managing all Real-Time Intelligence components in Microsoft Fabric. It allows users to configure, monitor, and optimize real-time data processing, ensuring smooth operation across various components. By consolidating management, monitoring, and alerting capabilities into one place, the Real-Time Hub enhances visibility and control over event-driven data, helping organizations make faster, more informed decisions. The Real-Time Hub looks like:

Power BI and its role in Real-Time Intelligence

Power BI is a comprehensive business intelligence platform that enables users to create reports and dashboards using a variety of data sources. While Power BI can display real-time data, it is not exclusively designed for real-time analytics. Instead, it supports multiple data connectivity modes, including batch processing, live querying, and streaming data ingestion.

A Real-Time Dashboard in Microsoft Fabric using RTI is specifically built for monitoring live data streams, ensuring near-instant updates as new information arrives. These dashboards are optimized for low-latency event tracking and are commonly used for operational monitoring, alerting, and tracking key performance indicators (KPIs) that change rapidly.

In contrast, creating a real-time dashboard in Power BI involves using streaming datasets or DirectQuery mode. Streaming datasets allow for continuous data updates, but they often lack the ability to perform advanced aggregations and transformations compared to RTI components in Microsoft Fabric. DirectQuery mode enables real-time data querying from sources like Eventhouse, but it can introduce some latency depending on query complexity and source performance.

The main differences between a real-time dashboard in RTI and Power BI include data latency, processing methods, and visualization capabilities. While Power BI is excellent for historical analysis, trend discovery, and batch-reporting, a real-time dashboard in RTI is tailored for immediate event-driven insights with minimal delay. By leveraging both, organizations can achieve a balanced approach to real-time intelligence and long-term analytics. An example Power BI report using Eventhouse data:

Summary

In short, here is how all the components work:

1. Eventstream captures and processes live data from various sources.
2. Eventhouse stores the ingested data for quick access and analysis.
3. Activator triggers automated responses based on pre-defined rules.
4. KQL Queryset enables fast querying and analysis of real-time data.
5. Real-Time Dashboards provide instant visibility into key metrics.
6. Real-Time Hub acts as the centralized interface for managing all these components.
7. Power BI generates historical reports from sematic models built from the stored data in Eventhouse.

You can create a logical copy of KQL database data located in an eventhouse by turning on OneLake availability, and all the KQL database data is made available in OneLake. Turning on OneLake availability means that you can query the data in your KQL database because it exists in OneLake (in Delta Lake format) using other Fabric engines such as Direct Lake mode in Power BI, Warehouse, Lakehouse, Notebooks, and more. The OneLake availability is used via Lakehouse shortcuts: go into a Lakehouse and create a shortcut to Microsoft OneLake, and you will see tables under the KQL DB if you have enabled “OneLake availability“.  If not, no tables will show up. More info at Eventhouse OneLake Availability – Microsoft Fabric | Microsoft Learn.

Not only can RTI be used to generate reports/dashboards/queries on new data coming from real-time streaming sources such as Internet of Things (IoT) devices, but it can also be used to replace the operational reporting on source systems to take some of the load off of those systems. For example, you can have an inventory database in Azure SQL Database that an application is using to get up-to-the-second Power BI reports that contain inventory counts. With RTI, the data in the Azure SQL Database can be copied into an eventhouse using eventstream via change data capture (CDC) within milliseconds of it being created, and the Power BI reports can instead be run against the eventhouse to get those inventory counts, reducing the compute load on the application system. For more info on this, check out Operational Reporting with Microsoft Fabric Real-Time Intelligence.

Make sure to check out the Real-Time Intelligence documentation and also what is new and planned. An excellent RTI tutorial where you’ll learn how to set up and use the main features of Real-Time Intelligence using a sample set of data can be found here.

More info:

Microsoft Fabric RTI: Eventhouse and Real-Time Dashboards

Azure SQL is a cloud-based suite of database services designed to offer flexibility, scalability, and ease of management. It comprises three main products, each catering to different deployment needs and compatibility requirements. Within each of the products are various deployment options, service tiers, and compute tiers:

Below I expand on the options from the above diagram:

Azure SQL Products

1. SQL Server on a Virtual Machine (VM)

For organizations needing full control over their SQL Server environment, running SQL Server on an Azure VM is a great option. This Infrastructure-as-a-Service (IaaS) offering provides complete access to the operating system and database engine, enabling users to configure and manage SQL Server as they would on-premises. It is best suited for:

• Lift-and-shift migrations with minimal changes.
• Applications requiring full SQL Server features.
• Custom configurations and third-party integrations.
• See Provision SQL Server on Azure VM and VM size.

2. Azure SQL Database

Azure SQL Database is a fully managed, Platform-as-a-Service (PaaS) database solution that automates maintenance tasks like patching, backups, and scaling. It is ideal for modern cloud applications and is available in two deployment options:

Single Database

• A dedicated, isolated database with predictable performance.
• Best for applications that require resource guarantees at the database level.
• Supports service tiers: General Purpose, Business Critical, and Hyperscale.
• Can use the serverless compute tier (only in the General Purpose and Hyperscale tiers), allowing dynamic scaling of resources based on workload demand.
• See What is a single database in Azure SQL Database?

Elastic Pool

• Multiple databases share a set of resources, optimizing cost efficiency.
• Best for SaaS applications with variable workloads across multiple databases.
• Supports service tiers: General Purpose, Business Critical, and Hyperscale.
• Does not support the serverless compute tier.
• See Elastic pools help you manage and scale multiple databases in Azure SQL Database.

3. Azure SQL Managed Instance

Azure SQL Managed Instance is an instance-scoped deployment option that provides near 100% compatibility with SQL Server while delivering full PaaS benefits. This makes it ideal for organizations looking to modernize their database infrastructure with minimal friction. Key benefits include:

• Native support for SQL Server features like cross-database queries, linked servers, and SQL Agent.
• Built-in high availability, automated maintenance, and security.
• Supports service tiers: General Purpose and Business Critical.
• Does not support the serverless compute tier.
• See What is Azure SQL Managed Instance?

Service Tiers in Azure SQL

Azure SQL Database and Managed Instance offer three service tiers to cater to different workload needs:

1. General Purpose

• Balanced performance and cost-effective for most applications.
• Available in both Azure SQL Database (single and elastic pool) and Azure SQL Managed Instance.
• Supports the serverless compute tier (only for single database deployment).
• See vCore purchasing model – Azure SQL Database.

2. Business Critical

• Designed for applications requiring high transaction rates and low-latency I/O performance.
• Includes built-in high availability with multiple replicas.
• Available in Azure SQL Database (single and elastic pool) and Azure SQL Managed Instance.
• Does not support the serverless compute tier.
• See vCore purchasing model – Azure SQL Database.

3. Hyperscale

• Optimized for extremely large databases, supporting up to 128 TB.
• Provides rapid scaling of compute and storage independently.
• Available for Azure SQL Database (single database and elastic pool deployments).
• Supports the serverless compute tier for single database deployment.
• See Hyperscale service tier

Serverless Compute Tier

The serverless compute tier is a cost-effective option that allows automatic scaling of compute resources based on demand. It is available in the General Purpose and Hyperscale service tiers for Azure SQL Database (single database deployment).

Benefits of Serverless Compute Tier:

• Autoscaling of CPU and memory based on workload fluctuations.
• Automatic pausing of the database during inactivity, reducing costs.
• Best suited for intermittent, unpredictable workloads that do not require continuous availability.
• See Serverless compute tier for Azure SQL Database.

Choosing the right Azure SQL option

When selecting an Azure SQL offering, consider factors like workload requirements, compatibility, cost, and management overhead. Below is a quick reference guide:

Conclusion

Azure SQL offers a diverse set of solutions to accommodate different business and technical requirements. Whether you need full control with SQL Server on a VM, a fully managed single database, a cost-effective elastic pool, or near full SQL Server compatibility with Managed Instance, Azure SQL provides the flexibility to optimize cost, performance, and scalability. Understanding these options and service tiers will help you make an informed decision tailored to your specific workload needs.

This diagram may help with choosing the right option for your use case:

HeyGen – Speak to an avatar live. Check out their demos where you can interact with avatars such as a therapist, fitness coach, and doctor. They keep adding new ones. Amazing stuff!

OpenAI Sora – Create video from text. My favorite is Historical footage of California during the gold rush.

GPT Store – Discover and create custom versions of ChatGPT that combine instructions, extra knowledge, and any combination of skills. My favorites are Books, Movies, and Therapist/Psychologist.

ChatGPT advanced voice mode – Have spoken interactive conversations with ChatGPT, where you can screen share and also share live video. I had a long conversation with ChatGPT while driving alone to help keep me from getting sleepy – we discussed the best Yankee teams of all time. The ChatGPT voice is animated so it’s like talking to real person (choose from nine lifelike output voices for ChatGPT, each with its own distinct tone and character). Screen share can be used for things like help guiding you through settings on your computer in order to help troubleshoot a problem, while live video can be used in ways such as helping you recognize objects or tell you the color of a shirt (great for colorblind people like me).

ChatGPT DALL-E 3 – Create images from text. This used to be a separate product but is now built into ChatGPT. I recently asked it to draw me a picture of a crawfish blowing out a birthday cake for a crawfish boil I did for someone’s birthday and sent that picture with the birthday invitations.

Azure AI Speech Studio – Create a custom text to speech avatar using the Azure AI Custom Neural Voice and the Custom Avatar Self-Service capabilities in Azure AI Speech Studio. You can use the video and voice of anyone you wish. I hope to one day create an avatar that looks and sounds like me and then talk to it. Freaky!

Clonos – Create your own virtual avatar from existing video and sound, and make it say anything. See examples of Clonos in the sports world that is very funny at memerunngergpt on Instagram, where they modify video from sports figures to say hilarious things in their own voice (warning: foul language).

Microsoft Copilot – An AI chatbot that is now in many Microsoft products. Check out a few of my favorites: Copilot in Teams, Copilot in Microsoft Fabric, Copilot on Windows, Copilot in Word, Microsoft 365 Copilot Chat.

If you have not used ChatGPT, you need to do so immediately! One way to use it that you may not be aware of is via roleplay, where OpenAI takes on a persona. This is a great way to learn about a subject matter. For example, I am reading about the crusades, so I prompted OpenAI: “Pretend you are a knight from the first crusade and fought from the very beginning to the very end of the crusade. I will ask you questions, and I want you to answer in the first person. Draw upon historical knowledge and accounts of the first crusade to immerse yourself in the mindset, beliefs, and experiences of such a knight, responding to me as though you were truly from that era”. Then I was able to ask questions directly to a “knight” and get all sorts of great info. Other amazing persona’s you can ask it to emulate are such things as “pretend you are SQL Server and I’m using SSMS” and “pretend you are the game Zork” (for you old-timers out there like me).

I also used ChatGPT on my iPhone and told it to pretend it was Santa Clause and to talk to my 6-year old grandson. I turned on voice mode, which has a seasonal Santa voice, and my grandson had a long and animated conversation with Santa (ChatGPT)!

To see some of the capabilities of OpenAI and ChatGPT, check out OpenAI YouTube that has demo videos. My favorites are Interview Prep with GPT-4o, Live demo of GPT-4o’s vision capabilities, and Interview roleplay with GPT-4o voice and vision.

A helpful tip on asking questions with ChatGPT: you will get wrong answers sometimes, especially if your question (“prompt”) does not have much information. If you get a wrong answer, tell ChatGPT via a follow-up prompt that the answer is wrong and tell it what the correct answer is. If it then responds with the correct answer, prompt it with “change my original prompt to prevent the incorrect answer if I were to ask the question again”. You will then get an improved prompt that you can use to ask the question again, or to build upon it.

More info:

AI Companions Will Change Our Lives