Microsoft Unveils Fabric, New End-To-End Data, Analytics Platform

Microsoft on Tuesday unveiled Microsoft Fabric, a brand-new end-to-end data and analytics platform centred on Microsoft's OneLake data lake which can also import data from Amazon S3 and (soon) Google Cloud Platform.

It includes integration tools, a Spark-based data engineering platform, a real-time analytics platform, and, thanks to the recently upgraded Power BI, an intuitive visualisation and AI-based analytics tool.

Additionally, there is a new no-code developer experience that enables users to monitor their data in real-time and initiate actions and notifications depending on it.

These capabilities are all closely connected, and Microsoft will undoubtedly also include its AI Copilot into Fabric.

“Over the last five to 10 years, there has been a pretty massive level of innovation — which is great and that’s awesome because there’s lots of new technologies out there — but it’s also caused a lot of fragmentation of the modern data stack,” Arun Ulag, Microsoft’s corporate VP for Azure Data, told me. 

“There’s literally hundreds — if not thousands — of products and open source technologies and solutions that customers have to make sense of.”

He also noted that a lot of the data and analytics products tend to keep their data in silos. 

“When I talk to customers, one of the messages I hear consistently is that they’re tired of paying this integration tax,” he said.

Microsoft therefore considered how it might provide a single experience around the fundamental data analytics tasks (data integration, engineering, warehousing, data science, real-time analytics, and business intelligence).

The team opted to concentrate on a single data lake and a unified computational architecture to achieve this.

“There’s a unified compute infrastructure; there’s a unified data lake.

"There’s a unified product experience for all your data professionals, so that they can really collaborate deeply. [There’s] unified governance so that IT can manage this and create sources of truth that everybody can trust, and really a unified platform that both IT and business share — and the unified business model.

"There’s literally just one thing to buy, and it allows customers to save a lot of costs, which, especially in today’s environment, is really important,” said Ulag.

Ulag added that over the course of the previous year, he personally demonstrated Fabric to 100 of the Fortune 500, and that many businesses are enthusiastic about it since it substantially simplifies their data architecture without tying them to a particular cloud vendor.

This is due in part to the fact that the team chose to construct the main data lake around the column-oriented, open-source Apache Parquet file format for storing and retrieving data.

Microsoft has made the decision to base Fabric's design on a multi-cloud strategy, with built-in support for data on Google Storage and Amazon S3 (coming shortly).

The business also made the decision to streamline the pricing structure, which is centred on a single Fabric computing unit and emphasises the compute infrastructure.

Cost is, of course, a key factor in today's workplace IT purchases, and this sometimes necessitates consolidating providers.

Most companies nowadays use a variety of suppliers to piece together their data and analytics platforms.

That presents an integration problem and entails extra expense.

“This wastage is something that Fabric addresses because it creates a unified compute model,” explained Ulag.

“Overnight, it might do a lot of data engineering and data science, maybe data integration.

"In the morning, the same compute flows to maybe BI and SQL as people walked into the office.

"Because all compute is virtualized, all compute is serverless in Fabric, it really allows you to reuse the capacity that you purchased. That is attractive for [enterprises].”

Microsoft says that having a single, integrated system makes it simpler to control data access and governance (using Microsoft Purview).

The service will ensure that the documents created with this data will inherit the same confidentiality label and rules associated with it (and, based on these rules, automatically encrypt these files so that even if they leak outside of the company, nobody would be able to access them).

For example, if an employee with the right access rights wants to analyse highly confidential employee salary data and export that to Excel or Power BI.

Despite being a highly connected system, there are many moving elements in this situation.

An integration service called Data Factory offers more than 150 pre-built interfaces.

Additionally, Microsoft uses a variety of its Synapse-branded data solutions to give data scientists the data engineering and data science toolset they need to, for instance, create AI models.

The new no-code Data Activator service will enable users to automatically initiate certain activities based on the incoming real-time data, while Power BI will sit at the other end of this spectrum and enable business analysts and other users to derive insights from all of this data.

Since this is Microsoft in 2023, it goes without saying that there will be a Copilot in Microsoft Fabric that will make it simpler for users to create code, develop machine learning models, and more.

But since this Copilot isn't yet accessible, we can't yet say how beneficial it will be.

The public preview of Fabric is currently open. By Microsoft's norms, this is a fairly unique step because anybody may sample the service without even having to enter their credit card information.

All Power BI tenants will be able to use Fabric as of July 1.