How Microsoft Fabric aims to beat Amazon and Google in the cloud war

Microsoft unveiled a brand new cloud knowledge and analytics platform two weeks in the past that analysts say give it an edge over its most important rivals, Amazon and Google, in the fiercely aggressive cloud market. 

The platform, referred to as Microsoft Fabric, is a complete suite of instruments that permits enterprise clients to retailer, handle and analyze the knowledge that drives their most vital purposes. It additionally integrates merchandise that cater to all of an organization’s knowledge customers, from engineers who deal with the technical points of knowledge processing to analysts who need to derive insights and make choices from the knowledge. (See our reporting on the announcement). 

Microsoft Fabric, which is at present in public preview mode and shall be up to date with extra options in the coming months, shocked many {industry} consultants who weren’t briefed by the firm beforehand. Some reserved full judgment till they will see it work in observe. But they praised the platform as a big development that might assist Microsoft “leapfrog” Amazon and different cloud suppliers, resembling Google — not less than when it comes to serving giant enterprise firms. Fabric will even strain different tech suppliers like Snowflake and Databricks, an in depth companion of Microsoft, analysts mentioned. 

“With all these capabilities coming together,” mentioned Noel Yuhanna, an analyst at Forrester, “Microsoft definitely has a slight advantage over the other hyperscalers at the moment.”

Even earlier than the announcement Microsoft had already turn into a frontrunner in knowledge and analytics software program, in accordance to Gartner, a analysis agency. But with Microsoft Fabric, analysts mentioned, the firm has taken its choices to a brand new degree of integration and ease of use that could possibly be laborious for its opponents to match anytime quickly.

While Fabric provides Microsoft a dominant providing, the secret is now in execution, analysts mentioned. Amazon’s AWS cloud service nonetheless enjoys a transparent lead over Microsoft’s Azure in total income, and will most likely proceed to accomplish that for a while. But in the space of enterprise analytics and knowledge, Microsoft’s cloud choices now lead in phrases of breadth of capabilities. “But the ability to execute is often defined by sales. So that number is yet to be proven,” mentioned Hyoun Park, analyst at Amalgam Insights.

Fabric’s secret sauce: OneLake

So what makes Microsoft Fabric stand out? According to analysts, it’s the manner Microsoft has simplified and unified its knowledge structure with a single knowledge lake, referred to as OneLake, that may retailer and enable entry to every kind of knowledge from totally different sources and purposes. 

This strategy, they mentioned, will supply vital advantages to clients in phrases of price financial savings, transparency, flexibility, governance and knowledge high quality. OneLake is designed to be the central repository for not solely the knowledge generated by Microsoft’s personal software program companies, however for knowledge from exterior sources, resembling third-party purposes. It additionally gives a constant expertise and interface for customers, no matter the sort or format of the knowledge. This might sound like an apparent concept, nevertheless it has been elusive for many cloud suppliers, together with Microsoft, Amazon and Google. 

Over the years, these tech giants have acquired or developed dozens of software program instruments for varied knowledge and analytics duties, resembling enterprise intelligence, knowledge science, machine studying and real-time streaming. But they’ve largely bolted collectively these instruments in a piecemeal trend, with out making a coherent and seamless platform.

As a end result, clients have to take care of a posh and fragmented panorama of instruments and databases, every with its personal provisioning, pricing and pooling of knowledge. This creates frustration and inefficiency for patrons, who’ve to spend extra time and cash on managing their knowledge infrastructure. It additionally imposes an “integration tax” on clients, who’re charged individually for every service’s compute and storage sources.

Microsoft Fabric guarantees to eradicate this Frankenstein-like complexity by providing true integration — together with just one copy of knowledge, and one expertise and one interface. “Part of the innovation here is that Microsoft is providing all of these capabilities by themselves as an integrated package,” mentioned Amalgam’s Park. “And as simple as that sounds, it’s not something that the majority of data and analytic vendors are able to provide.”

Jason Medd, an analyst at Gartner, concurs. He mentioned that Gartner’s surveys of chief knowledge officers have proven that solely about 30% mentioned they get worth from their knowledge and analytics instruments. By integrating its instruments and decreasing its costs, Microsoft is addressing these ache factors, Medd mentioned.

How OneLake knowledge lake works

How does Microsoft obtain this simplicity and unification with OneLake? The secret is that OneLake shops a single copy all the knowledge from Microsoft’s varied companies in a typical format, referred to as Apache Parquet. This is an open-source file format that’s extensively used in the {industry} and that organizes knowledge by columns.

This makes it simpler and quicker to question and analyze knowledge. Whenever clients add or replace any knowledge to their techniques, Fabric robotically saves it in OneLake in the Parquet format, no matter its unique format. This signifies that clients can entry and question their knowledge from OneLake immediately, with out having to undergo a number of sources or companies.

For instance, if a buyer desires to use Microsoft’s enterprise intelligence instrument, Power BI, to analyze knowledge from Microsoft’s knowledge warehouse, Synapse, they don’t have to ship a question to Synapse. Power BI merely retrieves the knowledge from OneLake. This reduces the variety of queries throughout companies and lowers the price for patrons, who’re charged for a single storage and knowledge bucket, as a substitute of a number of ones.

How OneLake pulls in knowledge from exterior sources

OneLake’s simplicity and unification additionally prolong to knowledge from outdoors Microsoft’s ecosystem. This is the place the technical particulars matter: OneLake shops its knowledge tables in an open-source format referred to as Delta Lake, which creates a single layer of metadata that converts uncooked knowledge from varied sources, resembling CSV or JSON information, into a typical format that may be analyzed by any compute engine in the {industry}. 

“Microsoft has done the right thing here,” mentioned Tony Baer, an analyst at DBInsights, of its embrace of open supply. 

He mentioned that the competitors amongst distributors shouldn’t be about file codecs, however about reaching a regular of accuracy and consistency, generally known as ACID, for databases. And Fabric’s integration, by means of open codecs, is a step in that path. Microsoft makes it simple for patrons to rework knowledge from third-party companies with its Data Factory, which affords greater than 150 pre-built connectors. 

Microsoft can also be engaged on methods to automate the transformation course of, as a substitute of counting on the conventional and time-consuming methodology of extract, rework and load (ETL). 

Microsoft Fabric additionally helps multicloud eventualities, one thing that Amazon has been gradual to do. With a characteristic referred to as “Shortcuts,” OneLake can virtualize knowledge storage in Amazon’s S3 and Google’s storage (coming quickly). 

“Now that you’re going to a single open format that’s shared, all these engines can work natively with the data as opposed to getting fragmented,” mentioned Arun Ulagaratchagan, Microsoft’s company vp of Azure Data, in an interview with VentureBeat. He mentioned Microsoft is the first main cloud vendor “that is moving away from completely protected formats to completely open formats.”

Ulagaratchagan mentioned that he had talked with 100 of the Fortune 500 firms over the previous few years, and they have been most excited by Fabric’s promise of decrease price, ease of use and lack of lock-in.

Fabric’s integration work took years

Microsoft’s Fabric announcement might have appeared sudden, nevertheless it was the results of not less than 4 years of labor by the firm to break down silos and combine its knowledge companies. This additionally concerned overcoming inner politics and turf wars amongst totally different executives. 

One of the milestones was Synapse, which mixed a number of companies, resembling knowledge lake and knowledge warehouse, right into a single hub. But Fabric is the final integration, bringing collectively Synapse, Power BI and different knowledge companies as a single software-as-a-service (SaaS) providing.

“I think it’s leapfrogging,” mentioned Andrew Brust, an {industry} guide who runs BlueBadge Insights, referring to Microsoft’s transfer with Fabric. “The functionality is comprehensive and cohesive, and that hasn’t been possible before.”

Brust acknowledged that he’s biased. He mentioned Microsoft is a consumer of his, and that he’s a Microsoft Data Platform MVP, which made him a part of a small group of consultants, clients and companions who have been privy to Fabric earlier than its announcement. Brust additionally mentioned that Microsoft’s providing of Fabric as an SaaS, somewhat than a platform-as-a-service (PaaS), was vital. It signifies that knowledge engineers shouldn’t have to take care of provisioning items of compute, which simplifies their work. He mentioned Amazon and Google nonetheless have a whole lot of work to do in this space.

Quality of knowledge is the key to successful the enterprise cloud race

Analysts additionally emphasised that the most important competitors amongst cloud suppliers is about the high quality of knowledge, which is what allows clients to get higher insights and make higher choices. 

Noel Yuhanna, an analyst at Forrester, mentioned he talks to three or 4 enterprise clients day-after-day who complain that transferring to the cloud didn’t remedy their issues with knowledge high quality. “We get compute, we get storage, we get Kubernetes,” Yuhanna mentioned, summarizing the view of most enterprise executives. “That’s cool. But did we really modernize the system?” He mentioned that’s why system integrators, resembling BearingPoint, Capgemini, Infosys and Wipro, have to date made the most cash from offering insights from the cloud. They have consultants who write up reviews on what they discover from the knowledge. 

That’s additionally why Microsoft is pushing ahead with Fabric. By connecting knowledge sources collectively, Fabric improves the consistency and trustworthiness of knowledge, Yuhanna mentioned. “The biggest challenge with data replication is that data is all over the place, and you can’t get consistent data anymore … Fabric really gives you that consistency of data.”

By offering one place to go, it’s like offering a single window to look by means of knowledge administration: “Security, governance, integration, discovery, that’s exactly what this is about,” he mentioned. 

If clients need to apply safety guidelines to their knowledge, they will do a lot of this at the OneLake degree. And all of the Fabric purposes downstream that entry the knowledge could have to comply with these guidelines, Microsoft mentioned in its announcement. For instance, if clients have delicate wage info in Power BI that they solely desire a sure crew to entry, they will arrange guidelines that guarantee this. And information will carry the identical guidelines, wherever they’re exported — even carrying the identical encryption if despatched outdoors of Microsoft’s Fabric.

Microsoft catches up with the ‘lakehouse’ development

One of the areas the place Microsoft has lagged behind a few of its opponents is the so-called “lakehouse,” which mixes two applied sciences: a knowledge lake to retailer an organization’s knowledge, and a knowledge warehouse to analyze it.

The lakehouse has turn into common due to the rise of apps like synthetic intelligence, which require large quantities of knowledge and evaluation. And one firm in specific, Databricks, has been a pioneer in making a safe, open lakehouse that many analysts think about industry-leading. Databricks, in spite of everything, created the Delta Lake protocol.

Another firm, Snowflake, has additionally supplied a effectively built-in lakehouse product. Microsoft’s choices in this space, underneath its Synapse model, have reportedly not carried out as effectively, and Microsoft has compensated for this by forming an in depth partnership with Databricks, providing its assist on its Azure cloud. So it’s no shock that Microsoft’s Fabric adopted the Delta Lake protocol as effectively. All clients who use Databricks will proceed to be pleased utilizing Microsoft’s Fabric.

But Fabric’s integration additionally narrows the hole with Databricks and Snowflake, and aims to surpass them, analysts mentioned. Fabric extends the open format pioneered by Databricks to the remainder of Microsoft’s knowledge stack, which is extra complete. While Microsoft’s Ulagaratchagan says Microsoft is pleased to give clients selection by working with platforms like Databricks, he additionally makes clear that Microsoft’s Synapse intends to lead the lakehouse market: “We literally intend to be the best in breed and best of suite,” he mentioned. 

Microsoft’s single expertise, and transfer to SaaS providing, helps Fabric’s Synapse leap forward in some key points, analysts mentioned. Databricks stays a PaaS providing, which signifies that knowledge engineers nonetheless have to do extra work and specify issues like the variety of nodes they need to run processing jobs.

Microsoft Fabric combines its power in enterprise intelligence (Power BI) with knowledge science, and provides different capabilities, resembling sample detection and workflows (Data Activator), and that’s “a big deal,” mentioned Amalgam’s Park. He mentioned that bridging BI to AI continues to be a problem for the enterprise. Microsoft is offering a package deal that solves this to a higher extent than any of its opponents.”

The energy of generative AI has but to be realized

Finally, Microsoft mentioned it’s utilizing its new generative AI know-how, acquired from its funding in OpenAI, to improve its Copilot instrument. Copilot helps customers carry out duties, resembling studying and summarizing knowledge reviews. With OpenAI’s know-how, Copilot can now enable builders and analysts to use pure language to ask questions of knowledge, and to obtain solutions in pure language as effectively. Here, Microsoft’s Ulagaratchagan says that whereas it will enhance productiveness, the full impression of making use of generative AI throughout the Fabric choices will take a while to be seen.

After all, Fabric is the first time clients have skilled an end-to-end integration of their knowledge, and they’ve but to discover what generative AI can do.

“You can think about it not just accelerating a step in the customer’s journey with generative AI, but the entire journey, so that’s an opportunity that customers haven’t found yet,” Ulagaratchagan mentioned. “It’s critically important that we learn from actual customer usage and get the experience right.”

VentureBeat’s mission is to be a digital city sq. for technical decision-makers to achieve information about transformative enterprise know-how and transact. Discover our Briefings.