Many of us had no idea what to make of Fabric. To me, it seemed as if Microsoft were doing a rebranding of sorts. They changed the name of Azure Synapse Analytics, also called a Dedicated SQL Pool, and previously known as Azure SQL Data Warehouse. Microsoft excels (ha!) at renaming products every 18 months, keeping customers guessing if anyone is leading product marketing.

Microsoft Fabric also came with this thing called OneLake, a place for all your company data. Folks with an eye on data security, privacy, and governance thought the idea of OneLake was madness. The idea of combining all your company data into one big bucket seemed like a lot of administrative overhead. But OneLake also offers a way to separate storage and compute, allowing for greater scalability. This is a must-have when you are competing with companies like Databricks and Snowflake, and other cloud service providers such as AWS and Google.

After Some Thought…

After the dust had settled and time passed, the launch and concept of Fabric started to make more sense. For the past 15+ years, Microsoft has been building the individual pieces of Fabric. Here’s a handful of features and services Fabric contains:

• Data Warehouse/Lakehouse – the storing of large volumes of structured and unstructured data in OneLake, which separates storage and compute
• Real-time analytics – the ability to stream data into OneLake, or pull data from external sources such as SnowFlake
• Data Engineering – the ability to extract, load, and transform data including the use of notebooks
• Data Science – leverage machine learning to gain insights from your data
• PowerBI – create interactive reports and dashboards

Many of these services were built to support traditional data storage, retrieval, and analytical processing. This type of data processing focuses on data at rest, as opposed to streaming event data. This is not to say you couldn’t use these services for streaming, you could try if you wanted. After all, the building blocks for real-time analytics go back to SQL Server 2008, with the release of StreamInsight, a fancy way to build pipelines for refreshing dashboards with up to date data.

Streaming event data is where the real data race is taking place today. According to the IDC, by 2025 nearly 30% of data will need real-time processing. This is the market Microsoft, among others, is targeting, which is roughly 54 ZB in size.

So, it seems the more data collected, the more likely it is used for real-time processing. Therefore, if you are a cloud company, it is rather important to your bottom line to find a way to make it easy for your customers to store their data in your cloud. The next best thing, of course, is making it easy for your customers to use your tools and services to work with data stored elsewhere. This is part of the brilliance of Fabric, as it allows ease of access to real time data you are already using in places like Databricks, Confluent, and Snowflake.

The Bundle

Now, if you are Microsoft, with a handful of data services ready to meet the needs of a growing market, you have some choices to make. You could continue to do what you have done for 15+ years and keep selling individual products and services and hope you earn some of the market going forward. Or you could bundle the products and services, unifying them into one platform, and make it easy for users to ingest, transform, analyze, and report on their data.

Well, if you want to gain market share, bundling makes the most sense. And Microsoft is uniquely positioned to pull this off for two reasons. First, they have a comprehensive data platform which is second to none. Sure, you can point to other companies who might do one of those services better, but there is no company on Earth, or in the Cloud, which offers a complete end-to-end data platform like Fabric.

Second, bundling software is something Microsoft has a history of doing, and doing it quite well in some cases. People reading this post in 2024 may not be old enough to recall a time when you purchased individual software products like Excel and Word. But I do recall the time before Microsoft Office existed. Bundling everything into Fabric allows users to work with their data anywhere and, most importantly to Microsoft’s bottom line, the result is more data flowing to Azure servers.

I am not here to tell you everything is perfect with Fabric. In the past year I have seen a handful of negative comments about Fabric, most of them nitpicking about things like brand names, data type support, and file formats. There is always going to be a person upset about how Widget X isn’t the Most Perfect Thing For Them at This Moment and They Need to Tell the World. I think most people believe when a product is released, even if it is marked as “Preview”, it should be able to meet the demands of every possible user. It is just not practical.

Summary

Microsoft Fabric was announced at Build this year to be GA, which also makes users believe it should meet the demands of every possible user. The fastest way for Microsoft to grab as much market share as possible is to focus on the customer experience and remove those barriers. You can find roadmap details here, giving you an idea about the effort going on behind the scenes with Fabric today. For example, for everyone who has raised issues with security and governance, you can see the list of what has shipped and what is planned here.

It is clear Microsoft is investing in Fabric, much like they invested in Office 30+ years ago. If there is one thing Microsoft knows how to do, it is creating value for shareholders:

Since the announcement of Fabric last May, Microsoft is up over 25%. I am not going to say the increase is the direct result of Fabric. What I am saying is Microsoft might have an idea about what they are doing, and why.

Microsoft Fabric is the new Office – it is a bundle of data products, meant to boost productivity for data professionals and dominate the data analytics landscape. Much in the same way Office dominates the business world.

The post Microsoft Fabric is the New Office appeared first on Thomas LaRock.

Well, in Azure ML Studio, exporting to a CSV file should be simple, but is not, unless you already know what you are doing and where to look. I was reminded of this recently, and decided to write a quick post in case a person new to ML Studio was wondering how to export data to a CSV file.

When you are working inside the ML Studio designer, it is likely you will want to export data or outputs from time to time. If you are starting from a blank template, the designer does not make it easy for you to know what module you need (similar to my last post on finding sample data). Would be great if CoPilot was available!

Now, if you are similar to 99% of data professionals in the world, you will navigate to the section named Data Input and Output, because that’s what you are trying to do, export data from the designer. It even says in the description “Writes a dataset to…”, very clear what will happen.

So, using the imdb sample data, we add a module to select all columns, then attach the module to the Export Data model. So easy!

When you attach you need to configure some details for the module. Again, so easy!

We save our configuration options and submit the job to run. When the job is complete, we navigate to view the dataset.

Uh-oh, I was expecting a different set of options here. Viewing the log and various outputs does not reveal any CSV file either. Maybe I need to choose the select columns module:

Ah, that’s better.

Except it isn’t. Instead of showing me the location of the expected CSV file, what I find is this:

I can preview the data from the select columns module, but there isn’t a way to access the CSV file I was expecting. I suspect this export module is really meant to pass data between pipelines or services. But the purpose and description of the export module is not clear, and a novice user would be unhappy to head down this path only to be disappointed and frustrated.

What we really want to use here is the Convert to CSV module:

Viewing the results will display this:

Which has what we are looking for, a download button:

Selecting Download will either default to your browser settings, or you can do a Save As.

As I wrote at the beginning of this post, exporting to a CSV file from within Azure ML Studio is easy to do, if you already know what you are doing. If you are new to Azure ML Studio, you may find yourself frustrated if you expect the Export Data module to produce a CSV file. You will want to use the Convert to CSV module instead.

The post Export to CSV in Azure ML Studio appeared first on Thomas LaRock.

Recently I was working inside of Azure ML Studio and wanted to browse the sample datasets provided. Except I could not find them. I *knew* they existed, having used them previously, but could not remember if that was in the original ML Studio (classic) or not.

After some trial and error, I found them and decided to write this post in case anyone else is wondering where to find the sample datasets. You’re welcome, future Tom!

First, you need to login to Azure ML Studio: https://ml.azure.com/. Once logged in, you will create a workspace. Once the workspace is ready, open it and you will see a splash screen with a lot of interesting widgets, but alas no sample datasets to select.

To locate the sample datasets you must create a Pipeline. You create a Pipeline either through the designer or the Pipeline menu on the left of the workspace screen, as selecting Pipeline | New Pipeline opens the Designer.

Once inside the Designer, create a Pipeline either by selecting the pre-defined samples or by selecting the upper-left tile:

Now you are in the Authoring screen, and here is where you will find the sample data. However, your default portal experience could have the left-hand menu collapsed. You can expand the menu by clicking on the two brackets (WTH is this really called, a vertical chevron? No idea.) This was not intuitive for me, it took me a bit of time to understand I needed to click on this to view a menu.

Once opened, you’ll find sample data as well as some other goodies.

Expand the Sample data option and view the full list of datasets.

I don’t know how often the sample data is refreshed, and the answer is “likely never”. So, if you are looking for up to date census data, or iMDB movie data, you should consider a different source than the sample datasets provided through Azure ML Studio.

The post Azure ML Studio Sample Data appeared first on Thomas LaRock.

Receiving the MVP award during my unforced sabbatical this summer was a bright spot, no question. It reinforced the belief I have in myself – my contributions have value. Microsoft puts this front and center on the award by stating (emphasis mine):

“We recognize and value your exceptional contributions to technical communities worldwide.”

I recall the aftermath of my first award, when I was told I was the “least technical SQL Server MVP ever awarded”. Talk about feeling you have no value! And that was certainly the feeling I had two months ago.

It’s amazing how something as simple as being recognized by your peers can go so far in making a person feel valued. We should all strive to go out of our way daily to help another human feel valued.

There are plenty of people in the world who are recognized as experts in the Microsoft Data Platform. I’d like to think I am one of them. I also happen to be fortunate enough to know Microsoft recognizes me as one as well.

But MVPs advocate for Microsoft because we want to, not because we want an award. After all these years I’m still crazy for Microsoft, and I am happy to help promote the best data platform on the planet.

For my fellow MVPs renewed this year, I offer this suggestion – say thank you. Then say it again. Email the person on the product team who made the widget you enjoy using over and over and tell them how much you appreciate their effort. Email your MVP lead(s) and thank them for all their hard work as well.

A little kindness goes a long way. You never know how much reaching out could mean to that person at that moment.

The post Microsoft Data Platform MVP – Fifteen Years appeared first on Thomas LaRock.

You can pre-order here: https://amzn.to/3fQr7hz

I thoroughly enjoyed this project, and I want to thank Apress and Jonathan Gennick for giving me the opportunity to update the previous edition. It felt good to be writing again, something I have not been doing enough of lately. And many thanks to Enrico van de Laar (@evdlaar) for giving me amazing content to start with.

The book is an effort to help explain how, why, and when wait events happen. Of course, I also want to show how to solve issues when they arise. Specific wait events are broken down into parts: definition, remediation, and an example. There are plenty of code examples, allowing the reader to duplicate the scenarios to help understand the wait events better.

It is my understanding we will have a GitHub repository for the sample code. This will make it easy for a reader to access the code for their use. I am hoping to keep the repo up to date and expand upon the example as I look towards the next version.

Pro SQL Server 2022 Wait Statistics at Live 360!

I will be presenting material from the book at SQL Server Live! this November where I have the following sessions, panel discussion, and workshop:

• Fast Focus: SQL Server Data Types and Performance
• Locking, Blocking, and Deadlocks
• Performance Tuning SQL Server using Wait Statistics
• SQL Server Live! Panel Discussion: Azure Cloud Migration Discussion
• Workshop: Introduction to Azure Data Platform for Data Professionals

The workshop is a full day training session delivered with Karen Lopez (@DataChick), and you can register for Live 360 here: Live 360 Orlando 2022 – Choose Registration

I am hopeful to have copies of Pro SQL Server 2022 Wait Statistics at SQL Server Live!. At the time of this post, I do not know of a publish date. Amazon shows the book as pre-order right now.

The post Pro SQL Server 2022 Wait Statistics Book appeared first on Thomas LaRock.

This past week I received another care package from Satya Nadella. Inside was my Microsoft Data Platform MVP award for 2021-2022. I am happy, honored, and humbled to receive the Microsoft Data Platform MVP award for the thirteenth straight year. I still recall my first MVP award and how it got caught in the company spam folder. Good times.

I am not able to explain why I am considered an MVP and others are not. I have no idea what it takes to be an MVP. And neither does anyone else. Well, maybe Microsoft does since they are the ones that bestow the award on others. But there doesn’t seem to be any magical formula to determine if someone is an MVP or not.

I do my best to help others. I value people and relationships over money. And I play around with many Microsoft data tools and applications, and blog about the things I find interesting. Sometimes those blog posts are close to fanboi level, other times they are not. But I do my best to remember that there are people over in Redmond that work hard on delivering quality. Sometimes they miss, and I do my best to help them stay on target. Maybe that’s why they keep me around.

Looking back to 2009 and my first award, I recognize the activities I was doing 13 years ago which earned my my first MVP award are not the same activities I am doing today. And I think that is to be expected, as I’m not the same person today as I was then. I have a different role, different responsibilities, and different priorities. I’ve branched out into the world of data security and privacy as well as data science.

I now spend time writing for multiple online publications instead of my here on my personal blog. Often those articles are not about SQL Server, but are almost always about data. I remain an advocate for Microsoft technologies, and continue to do my best to influence others to see the value in the products and services coming out of Redmond.

This past year was a bit different, of course, as COVID affected different parts of the world in different ways, at different times. As such, the Microsoft MVP program made the decision to auto renew MVPs without evaluating our current activity. This year’s award is the equivalent of Free Parking in Monopoly, Sure, you’re still in the game, but you aren’t doing much, things are happening around you, and everything is OK.

Summary

I’m going to enjoy this ride while it lasts. I’m also going to do my part to make certain that the ride lasts as long as possible for everyone. Here’s a #ProTip for those of us renewed this year:

Say thank you. Then say it again. Be grateful for what we have. Email the person that made the widget that you enjoy using over and over and tell them how much you appreciate their effort. Email your MVP lead and thank them for all their hard work as well.

MVPs do advocate for Microsoft because we want to, not because we want an award. After all these years I’m still crazy for Microsoft, and I am happy to help promote the best data platform on the planet.

The post Microsoft Data Platform MVP – A Baker’s Dozen appeared first on Thomas LaRock.

This past week I received another care package from Satya Nadella. Inside was my Microsoft Data Platform MVP award for 2020-2021. I am happy, honored, and humbled to receive the Microsoft Data Platform MVP award for the twelfth straight year. I still recall my first MVP award and how it got caught in the company spam folder. Good times.

I am not able to explain why I am considered an MVP and others are not. I have no idea what it takes to be an MVP. And neither does anyone else. Well, maybe Microsoft does since they are the ones that bestow the award on others. But there doesn’t seem to be any magical formula to determine if someone is an MVP or not.

I do my best to help others. I value people and relationships over money. And I play around with many Microsoft data tools and applications, and blog about the things I find interesting. Sometimes those blog posts are close to fanboi level, other times they are not. But I do my best to remember that there are people over in Redmond that work hard on delivering quality. Sometimes they miss, and I do my best to help them stay on target. Maybe that’s why they keep me around.

Here’s what I do know about the current Microsoft Data Platform MVPs:

There is no magic number. In my time as an MVP I’ve heard talk about how the number of MVPs is limited by Microsoft budgets. When we’ve asked about numbers, we’ve been told that the right number of MVPs is to find and recognize everyone that is worthy of being an MVP. I do not know anyone cut from the MVP program as a result of a budget decision. If a person meets the bar set by Microsoft, they earn the award.

MVPs have a great relationship with Microsoft employees, the MVP lead(s), and the folks on the product team. Being able to have respectful conversations as you discuss products, features, and strategy is important. It’s also how Microsoft measures influence. Being able to work with disparate teams, provide valued input, and influence the way others think are important skills for anyone, and especially true for MVPs. We are never shy about providing lovingly critical feedback. We don’t take advantage of our NDA status to write “breaking news” blog posts, either.

MVPs give more than they receive. They find opportunities to help others and look to put the needs of others ahead of their own. MVPs serve as leaders and spend their time answering questions on forums and on Twitter (#sqlhelp). They spend time writing blog posts about weird things they have found, hoping that the information may prove useful to someone else. An MVP will encourage others to share knowledge, and not make a habit of mocking others who lack knowledge. We participate whenever and wherever we can. In a world of “everyone get’s a trophy”, the MVP award is the greatest participation trophy on Earth.

MVPs are thirsty for more. We are constantly looking to improve and expand our skills. It is rare to find an MVP that doesn’t want to learn something new or dismisses new technology altogether. Sure, such MVPs do exist, but not for very long. Microsoft is a company that needs to move forward, and they need people that can help piece together solutions using multiple products in new ways. And the solutions and products aren’t just Microsoft ones, they can be 3rd party or open source. As technology changes, you need new skills. You either keep pace or fall behind.

In addition to the MVP program, Microsoft has other ways to get product feedback. The Microsoft SQL Server Facebook page has 348,000 likes and the Microsoft page has almost 14 million likes. The Microsoft SQL Server Twitter account has over 226,000 followers. That’s a lot of reach. (Warrants mentioning – do not mistake reach for influence. Reach is having a drawer full of business cards with phone numbers to call. Influence is when people call you back. The MVP program favors influence over reach, IMO).

My point is the Data Platform team can get product feedback in a variety of ways. Microsoft also has telemetry on product usage through the customer feedback program that you opt-in (hopefully) when installing products. I’m amused when someone says “NO ONE USES PRODUCT X!” and I later find Microsoft has data to suggest otherwise.

Microsoft can deliver their message to, and get feedback from, their industry influencers in a variety of ways. MVPs take part in webinars (called Product Group Interactions, or PGIs) on a regular basis, there are regional MVP events, and my MVP lead pings me weekly on events.

Summary

I’m going to enjoy this ride while it lasts. I’m also going to do my part to make certain that the ride lasts as long as possible for everyone. Here’s a #ProTip for those of us renewed this year:

Say thank you. Then say it again. Be grateful for what we have. Email the person that made the widget that you enjoy using over and over and tell them how much you appreciate their effort. Email your MVP lead and thank them for all their hard work as well.

MVPs do these things because we want to, not because we want an award. After all these years I’m still crazy for Microsoft, and helping to promote the best data platform on the planet.

Perhaps that’s the difference.

The post Microsoft Data Platform MVP – Still Crazy After All 12 Years appeared first on Thomas LaRock.