Last October I wrote a review of the Gigaom SQL Transactional Processing Price-Performance test. That post references the original data warehouse test also published by Gigaom. I believe both Gigaom reports were funded by Microsoft. I found the first report on data warehousing to be of good quality. The SQL transaction report had some weak areas (IMO) which I detail in this post.
This latest report, well, I didn’t bother promoting it, or writing a review. I felt this report was incomplete and suggested to Microsoft they have at least one more round of revisions. They did not agree.
Turns out AWS had similar concerns. Earlier this week I found this tweet from Corey Quinn (@QuinnyPig), linking to a blog post from AWS titled “Fact-checking GigaOm’s Microsoft-sponsored benchmark claims“.
With all love and respect to @azure: Are you folks out of your goddamned minds? https://t.co/A7qos0XZgC
Go read that and then come back.— Corey Quinn (@QuinnyPig) January 30, 2020
You can read for yourself the response from AWS, it details many of the same concerns I had with the report.
But one comment from AWS stood out to me, it was “Publishing misleading benchmarks is just one more old-guard tactic by Microsoft”.
OK, I’ve got some opinions to share.
Grow Up
I’m not going to defend Microsoft or the quality of the report. Microsoft may not be perfect, but right now AWS looks more “old-guard” than Microsoft does. I watch Andy Jassy spread misinformation from the re:Invent keynote. Jassy did similar tricks at the last keynote, and it was awful to watch. There’s a lot to love about AWS, but the idea that they don’t use similar poor marketing tactics as other companies is laughable.
As for these AWS and Azure benchmark reports, I find them to be a fairly useless disk-measuring contest. The cloud technology changes fast. These reports are out of date just after the ink dries. I do not believe there is a company today making a choice for being “all-in” on AWS or Azure and basing their decision on such marketing propaganda. “Oh, look, this article says they are the fastest, let’s use them!”
Look, anyone can build a contrived edge case that will show their hardware outperforms someone else. Watching AWS and Azure bicker over these reports is like listening to two rich kids argue during lunch who has the nicest sports car.
No one cares, Richie Rich.
Summary
As a user of cloud services what I want is a reliable, secure, stable, and affordable solution. That’s it. I expect you to be updating your hardware and configurations to make things better. I expect you are making your services easier to consume, administer, and monitor.
We don’t need these AWS and Azure benchmark reports to see whose disk is bigger. We need guidance on what servers to select, how to configure my workload, how to monitor, and adjust as necessary.
Give us more of that content, please.
Focus on building great services, creating happy customers, and less on poking holes in each other. </rant>
