That’s why last year our company (SolarWinds) launched a free tool called SQL Plan Warnings. Often mining the plan cache involves custom scripts and forcing you to work with text output only. We wanted to make things easier by providing a graphical interface. A GUI will allow for the user to have basic application functionality. Things like connecting to more than one instance at a time, or filtering results with a few clicks.

Let me give a quick tour of SQL Plan Warnings.

Connect to an instance

The first thing noteworthy here is how SQL Plan Warnings supports connecting to a variety of flavors of SQL Server. There’s the Earthed version, Azure QL Database, Azure SQL Database Manage Instance, and Amazon RDS for SQL Server as shown here:

From there you fill in your connection details. The login you choose will need either the VIEW SERVER STATE or SELECT permission for the following DMVs: dm_exec_query_stats, dm_exec_sql_text, and dm_exec_text_query_plan. I’ve provided links to the Microsoft docs for each, so you can review the permissions defined there.

Being able to easily connect to instance of SQL Server, no matter where they are located, is a must-have these days.

SQL Plan Warnings Settings

After you connect to your instance, SQL Plan Warnings will return the top 100 plans, with a default sort by CPU time. However, it is possible after connecting you may see no results. This is likely due to the default settings for SQL Plan Warnings. You get to the settings by clicking on the gear icon in the upper-right corner. Here is what the default settings look like:

If you are not seeing any results, change the default settings and refresh plan analysis. For me, I simply made the change to filter by executions, with 1 as the minimum. This returns a lot of noise, so you need to discover what makes the most sense for your particular instance.

Please note these default settings apply to each connected instance. Think of these settings as the highest level filter for all your connected sources. It may be possible you spend time adjusting these settings frequently, depending on the instance, the workload, and your query tuning goals.

Reviewing the SQL Plan Warnings Results

After plan analysis is complete, you will see a list of warnings found. It should look like this:

Note that a plan can have multiple warnings. So this list could be generated by one or more plans found.

From here we are able to filter on a specific warning type with a simple click. This allows us to narrow our focus. Perhaps today we want to focus on Key and RID lookups. We select that filter, then open the plan:

From here we can zoom and scroll, and view the node that has the lookup warning:

If we select the node a properties dialogue that opens to the right. We also see other warnings are included in this plan, if we want or need to investigate those at this time. We also have the ability to download the plan, if desired.

Summary

The SQL Plan Warnings tool is easy to use and allows for you to be proactive in optimizing your environment. The use of a GUI allows for quick filtering at the plan cache level as well as plan warnings themselves. This allows you to focus on the plan warnings with the most impact.

One thing to note is the size of the plan cache you choose to analyze. Instances with larger plan cache sizes (1GB or greater) may require a larger number of plans to parse for warnings.

You can download the SQL Plan Warnings tool here.

The post SQL Plan Warnings appeared first on Thomas LaRock.

This latest report, well, I didn’t bother promoting it, or writing a review. I felt this report was incomplete and suggested to Microsoft they have at least one more round of revisions. They did not agree.

Turns out AWS had similar concerns. Earlier this week I found this tweet from Corey Quinn (@QuinnyPig), linking to a blog post from AWS titled “Fact-checking GigaOm’s Microsoft-sponsored benchmark claims“.

With all love and respect to @azure: Are you folks out of your goddamned minds? https://t.co/A7qos0XZgC

Go read that and then come back.— Corey Quinn (@QuinnyPig) January 30, 2020

You can read for yourself the response from AWS, it details many of the same concerns I had with the report.

But one comment from AWS stood out to me, it was “Publishing misleading benchmarks is just one more old-guard tactic by Microsoft”.

OK, I’ve got some opinions to share.

Grow Up

I’m not going to defend Microsoft or the quality of the report. Microsoft may not be perfect, but right now AWS looks more “old-guard” than Microsoft does. I watch Andy Jassy spread misinformation from the re:Invent keynote. Jassy did similar tricks at the last keynote, and it was awful to watch. There’s a lot to love about AWS, but the idea that they don’t use similar poor marketing tactics as other companies is laughable.

As for these AWS and Azure benchmark reports, I find them to be a fairly useless disk-measuring contest. The cloud technology changes fast. These reports are out of date just after the ink dries. I do not believe there is a company today making a choice for being “all-in” on AWS or Azure and basing their decision on such marketing propaganda. “Oh, look, this article says they are the fastest, let’s use them!”

Look, anyone can build a contrived edge case that will show their hardware outperforms someone else. Watching AWS and Azure bicker over these reports is like listening to two rich kids argue during lunch who has the nicest sports car.

No one cares, Richie Rich.

Summary

As a user of cloud services what I want is a reliable, secure, stable, and affordable solution. That’s it. I expect you to be updating your hardware and configurations to make things better. I expect you are making your services easier to consume, administer, and monitor.

We don’t need these AWS and Azure benchmark reports to see whose disk is bigger. We need guidance on what servers to select, how to configure my workload, how to monitor, and adjust as necessary.

Give us more of that content, please.

Focus on building great services, creating happy customers, and less on poking holes in each other. </rant>

The post Why AWS and Azure Benchmarks Don’t Matter to Me appeared first on Thomas LaRock.

I can’t begin to tell you how excited I am for the opportunity to work with the VividCortex team.

Well, maybe I can begin to tell you. Let’s review two data points.

In 2013, SolarWinds purchased Confio Software, makers of Ignite (now known as Database Performance Analyzer, or DPA) for $103 million. That’s where my SolarWinds story begins, as I was included with the Confio purchase. I was with Confio since 2010, working as a sales engineer, customer support, product development, and corporate marketing. We made Ignite into the best of breed monitoring solution loved by DBAs globally.

The second data point is from last month, when SolarWinds bought VividCortex for $117.5 million. One thing I want to make clear – SolarWinds just doubled down on our investment in database performance monitoring.

Anyone suggesting anything otherwise is spreading misinformation.

Through all my conversations with members of both product teams one theme was clear. SolarWinds is committed to providing customers with the tools necessary to achieve success in their careers. We want happy customers. We know customer success is our success.

Another point made clear is the VividCortex product will complement, not replace DPA. We are expanding our database performance monitoring portfolio in a meaningful way. Sure, there is some overlap with MySQL, as both tools offer support for that platform. But the tools have some key differences in functionality. Currently, VividCortex is a SaaS monitoring solution for popular open-source platforms. DPA provides both monitoring and query performance insights for traditional relational database management systems and is not yet available as a SaaS solution.

This is why we view VividCortex as a product to enhance what SolarWinds already offers for database performance monitoring. We’re stronger today than yesterday. And poised to grow stronger.

This is an exciting time to be in the database performance monitoring space, with 80% of workloads still Earthed. If you want to know about our efforts regarding database performance monitoring products, just AMA.

I can’t wait to get started on helping build next-gen database performance monitoring tools. That’s what VividCortex represents, the future for database performance monitoring. And it is why this acquisition is so full of goodness. Expect more content in the coming weeks from me regarding our efforts behind the scenes with both VividCortex and DPA.

The post Thoughts on the VividCortex Acquisition appeared first on Thomas LaRock.

I found this latest effort to be a good start, but it fell short of the effort the authors put forth in their previous benchmark for data warehousing.

Before I go any further, I want to thank the authors for putting together their results. I recognize that these are humans, working hard, putting forth their best efforts at being fair and thorough. Comparing cloud services is not an easy task. I found this latest effort to be good, but not great. If they were students of mine I would grade this latest paper from them a solid B-.

Let’s break it down.

The Good Stuff

First, the good stuff. I love how they drove everything towards a formula, price/performance, where performance is tracked in transactions per second. The downside to price/performance is that not every workload is focused on transactions per second. Still, I’d like to see this formula adopted as a standard way of comparing services.

In the past I’ve focused only on total price as shown by the online pricing calculators. This is because (1) you aren’t supposed to publish benchmarks without permission from the company (Microsoft, AWS) and (2) I can’t bankroll this level of test AND maintain my scotch and bacon addictions. By using price/performance you level the playing field somewhat. A service may cost more, but if it runs your query in half the time, the cost may be worth it.

I also liked the choice of using TPC-E as their test, I believe that to be a fair way to compare how the services will handle a workload. And I liked how they explained the difficulties in comparing services and the associated hardware. That’s something I’ve written about previously. Many times, really.

It is frustrating to compare the data services being offered between Azure and AWS. Part of me thinks this is done on purpose by both companies in an effort to win our favor without giving away more information than is necessary. This is a common practice, and I’m not bashing either company for doing what has been done for centuries. I’m here to help others figure out how to make the right choice for their needs. At the end of the day, I believe both Amazon and Microsoft want the same thing: happy customers.

But it is not in their best interest to make it easy for anyone to compare costs. This is how utilities operate. Make no mistake, AWS and Azure are the new electric company.

Now, for the items that I didn’t like as much. I’ll capture the quote from the article and explain my concern.

The Not As Good Stuff

“There are no exact matches in processors or memory.” – This is a bit of nitpicking, but I took issue here with the use of the word “or”. As someone who charges (and receives) top dollar for performing technical reviews of books, it bugged me. The authors are correct in saying that it is hard to find exact matches. However, I can certainly find a match for vCPU, but not for memory. Azure publishes memory as weird increments, starting at 10.2 GB while AWS shows traditional increments of 8, 16, etc. So, yeah, it’s a nitpick. But it was this exact item what caught my eye and made me dig deeper to fact check everything. Warrants mentioning.

“Thus, R4 seemed a suitable instance class to use for AWS RDS.” – The authors explain why they chose R4 (memory optimized instance) versus the M4 (general purpose). I have no issue with this except that neither M5 or R5 was considered. This study just came out, why were those instances not considered? And since the authors went out of their way to tell us what AWS says about the R4, let me tell you what AWS also says about the R5:

“R5 instances deliver 5% additional memory per vCPU than R4 and the largest size provides 768 GiB of memory. In addition, R5 instances deliver a 10% price per GiB improvement and a ~20% increased CPU performance over R4.”

I can’t think of any reason why the authors chose R4 here. But let’s move past this, because now is time for the hard part: finding a suitable match for Azure SQL Database.

“On the Azure side, we expect customers to gravitate towards SQL Database Business Critical (BC) offerings…” – Well, Azure doesn’t offer a memory optimized version of SQL Database, so I guess using BC is fine. But the question I have now is why not consider using Managed Instance? In the data warehouse benchmark study they tried a variety of sizes against the workload. This study focused ONLY on one size machine. This is part of the reason they got a B-, they weren’t thorough enough for my liking. I’d send them back and tell them to run more tests against different sized machines and include Managed Instance. At the very least they could have made an effort to simply use general purpose, it would have been closer to an apples-to-apples comparison.

“Therefore, we chose the BC_Gen5_80 instance, which has more CPUs than R4.16xlarge, but less memory at 408 GB.” – Yes, finding an exact match is difficult. Here’s a breakdown of what they chose:

But this image shows AWS at 64,000 provisioned IOPS, and further in the study they say they tested against 32,000 provisioned IOPS. So, which is it? I’ve no idea. Neither do you. But I do know that provisioning 32,000 IOPS added about $6k to the monthly bill.

“…the monthly cost of Microsoft Azure comes to $40,043.71. The monthly cost for AWS comes to $65,239.43.” – Verified, I can get the same prices using the AWS and Azure calculators. But the small detail that is glossed over here is single versus multi-zone. The AWS calculator is clear, if you deploy multi-zone, the price doubles. The Azure calculator doesn’t have this option, it only exists when you create your SQL Database. I’d be shocked to find out that deploying multi-zone in Azure didn’t bump the price as well. But the chart above clearly states “in a single availability zone”. So, which is it?

I’ve no idea. Neither do you.

Summary

Some quick math tells me that if we drop the multi-zone from AWS RDS the price/performance result comes in at $1,269.85, slightly cheaper than the $1,410.04 for SQL Database. And this is why I like price/performance as a metric. A database service may have a slightly higher price, but offers greater throughput.

This was the exact conclusion from the data warehouse study, too. The cost for Azure SQL Data Warehouse was just a tad more than AWS Redshift, but the performance with Azure was better. I wanted to see a similar conclusion in this study.

Instead, we have a report with a handful of inaccuracies. Perhaps in an effort to rush to publish ahead of Ignite, they simply used a wrong graph, or missed doing one final round of edits. When you are doing this work it is easy to have such things fall through the cracks.

I’d love to see this study republished without these errors. I’d also love for AWS and Azure to find a way to make it easier to compare costs and services.

REFERENCES:

Azure vs. AWS Data Services Comparison
Updated Data Services Comparison: AWS vs. Azure
Azure vs AWS Analytics and Big Data Services Comparison
Updated Analytics and Big Data Comparison: AWS vs. Azure
Azure SQL Data Warehouse Costs vs AWS Redshift
Azure pricing calculator
AWS pricing calculator
Amazon EC2 Instance Types
Sizes for Windows virtual machines in Azure
Azure SQL Database pricing
Data Warehouse in the Cloud Benchmark
SQL Transactional Processing Price-Performance Testing

The post Reviewing the GigaOM SQL Transactional Processing Price-Performance Testing appeared first on Thomas LaRock.

The 2018 TrustWave Global Security Report listed SQL Injection as the second most common technique for web attacks, trailing only cross-site scripting (XSS) attacks. This is a 38% increase from the previous year. That same report also shows SQL Injection ranked fifth on a list of vulnerabilities that can be identified through simple penetration testing.

You may look at the increase and think “whoa, attacks are increasing”. But I believe that what we are seeing is a rising awareness in security. No longer the stepchild, security is a first-class citizen in application design and deployment today. As companies focus on security, they deploy tools and systems to help identify exploits, leading to more reporting of attacks.

SQL Injection is preventable. That’s the purpose of this post today, to help you understand what SQL Injection is, how to identify when it is happening, and how to prevent it from being an issue.

SQL Injection Explained

SQL injection is the method where an adversary appends a SQL statement to the input field inside a web page or application, thereby sending their own custom request to a database. That request could be to read data, or download the entire database, or even delete all data completely.

The most common example for SQL injection attacks are found inside username and password input boxes on a web page. This login design is standard for allowing users to access a website. Unfortunately, many websites do not take precautions to block SQL injection on these input fields, leading to SQL injection attacks.

Let’s look at a sample website built for the fictional Contoso Clinic. The source code for this can be found at https://github.com/Microsoft/azure-sql-security-sample.

On the Patients page you will find an input field at the top, next to a ‘Search’ button, and next to that a hyperlink for ‘SQLi Hints’.

Clicking on the SQLi Hints link will display some sample text to put into the search field.

I’m going to take the first statement and put it into the search field. Here is the result:

This is a common attack vector, as the adversary can use this method to determine what version of SQL Server is running. This is also a nice reminder to not allow your website to return such error details to the end user. More on that later.

Let’s talk a bit about how SQL injection works under the covers.

How SQL Injection works

The vulnerability in my sample website is the result of this piece of code:

return View(db.Patients.SqlQuery
("SELECT * FROM dbo.Patients
WHERE [FirstName] LIKE '%" + search + "%'
OR [LastName] LIKE '%" + search + "%'
OR [StreetAddress] LIKE '%" + search + "%'
OR [City] LIKE '%" + search + "%'
OR [State] LIKE '%" + search + "%'").ToList());

This is a common piece of code used by many websites. It is building a dynamic SQL statement based upon the input fields on the page. If I were to search the Patients page for ‘Rock’, the SQL statement sent to the database would then become:

SELECT * FROM dbo.Patients
WHERE [FirstName] LIKE '%Rock%'
OR [LastName] LIKE '%Rock%'
OR [StreetAddress] LIKE '%Rock%'
OR [City] LIKE '%Rock%'
OR [State] LIKE '%Rock%'

In the list of SQLi hints on that page you will notice that each example starts with a single quote, followed by a SQL statement, and at the end is a comment block (the two dashes). For the example I chose above, the resulting statement is as follows:

SELECT * FROM dbo.Patients
WHERE [FirstName] LIKE '%' OR CAST(@@version as int) = 1 --%'
OR [LastName] LIKE '%' OR CAST(@@version as int) = 1 --%'
OR [StreetAddress] LIKE '%' OR CAST(@@version as int) = 1 --%'
OR [City] LIKE '%' OR CAST(@@version as int) = 1 --%'
OR [State] LIKE '%' OR CAST(@@version as int) = 1 --%'

This results in the conversion error shown above. This also means that I can do interesting searches to return information about the database. Or I could do malicious things, like drop tables.

Chance are you have code like this, somewhere, right now. Let’s look at how to find out what your current code looks like.

SQL Injection Discovery

Discovering SQL injection is not trivial. You must examine your code to determine if it is vulnerable. You must also know if someone is actively trying SQL injection attacks against your website. Trying to roll your own solution can take considerable time and effort.

There are two tools I can recommend you use to help discover SQL injection.

Test Websites with sqlmap

One method is to use sqlmap, an open-source penetration testing project that will test websites for SQL injection vulnerabilities. This is a great way to uncover vulnerabilities in your code. However, sqlmap will not tell you if someone is actively using SQL injection against your website. You will need to use something else for alerts.

Azure Threat Detection

If you are using Azure SQL Database, then you have the option to enable Azure Threat Detection. This feature will discover code vulnerabilities as well as alert you to attacks. It also checks for anomalous client login, data exfiltration, and if a harmful application is trying to access your database.

(For fairness, I should mention that AWS WAF allows for SQL injection detection, but their process is a bit more manual that Azure).

If you try to roll your own discovery, you will want to focus on finding queries that have caused errors. Syntax errors, missing objects, permission errors, and UNION ALL errors are the most common. You can find a list of the common SQL Server error message numbers here.

It warrants mentioning that not all SQL injection attacks are discoverable. But when it comes to security, you will never eliminate all risk, you take steps to lower your risk. SQL injection discovery is one way to lower your risk.

SQL Injection Protection

Detection of SQL Injection vulnerabilities and attacks are only part of the solution. In an ideal world, your application code would not allow for SQL Injection. Here’s a handful of ways you can lower your risk of SQL injection attacks.

Parameterize Your Queries

Also known as ‘prepared statements’, this is a good way to prevent SQL injection attacks against the database. For SQL Server, prepared statements are typically done using the sp_executesql() system stored procedure.

Prepared statements should not allow an attacker to change the nature of the SQL statement by injecting additional code into the input field. I said “should”, because it is possible to write prepared statements in a way that would still be vulnerable to SQL injection. You must (1) know what you are doing and (2) learn to sanitize your inputs.

Traditionally, one argument against the use of prepared statements centers on performance. It is possible that a prepared statement may not perform as well as the original dynamic SQL statement. However, if you are reading this and believe performance is more important than security, you should reconsider your career in IT before someone does that for you.

Use Stored Procedures

Another method available are stored procedures. Stored procedures offer additional layers of security that prepared statements may not allow. While prepared statements require permissions on the underlying tables, stored procedures can execute against objects without the user having similar direct access.

Like prepared statements, stored procedures are not exempt from SQL injection. It is quite possible you could put vulnerable code into a stored procedure. You must take care to compose your stored procedures properly, making use of parameters. You should also consider validating the input parameters being passed to the procedure, either on the client side or in the procedure itself.

Use EXECUTE AS

You could use a security method such as EXECUTE AS to switch the context of the user as you make a request to the database. As mentioned above, stored procedures somewhat act in this manner by default. But EXECUTE AS can be used directly for requests such as prepared statements or ad-hoc queries.

Remove Extended Stored Procedures

Disabling the use of extended stored procedures is a good way to limit your risk with SQL injection. Not because you won’t be vulnerable, but because you limit the surface area for the attacker. By disabling these system procedures you limit a common way that an attacker can get details about your database system.

Sanitize Error Messages

You should never reveal error messages to the end user. Trap all errors and redirect to a log for review later. The less error information you bubble up, the better.

Use Firewalls

Whitelisting of IP addresses is a good way to limit activity from anomalous users. Use of VPNs and VNETs to segment traffic can also reduce your risk.

Summary

The #hardtruth here is that every database is susceptible to SQL injection attacks. No one platform is more at risk than any other. The weak link here is the code being written on top of the database. Most code development does not emphasize security enough, leaving themselves open to attacks.

When you combine poor database security techniques along with poor code, you get the recipe for SQL Injection.

REFERENCES

2018 TrustWave Global Security Report
Contoso Clinic Demo Application
sqlmap: Automatic SQL injection and database takeover tool
Azure SQL Database threat detection
Working with SQL Injection Match Conditions
How to Detect SQL Injection Attacks
sp_executesql (Transact-SQL)
EXECUTE AS (Transact-SQL)
Server Configuration Options (SQL Server)

The post SQL Injection Protection appeared first on Thomas LaRock.

It is my hope that this post will be a starting guide for you when you need to research these analytic and big data services. I have included relevant links for each service, along with some commentary, in the text of this post below. I’ve done my best to align the services, but there is some overlap between offerings. (Click image to embiggen)

I’m not going to do a feature comparison here because these systems evolve so quickly I’d spend all day updating the info. Instead, you get links to the documentation for everything and you can do your own comparisons as needed. I will make an effort to update the page as frequently as I am able.

Data Warehouse

Azure offerings: SQL Data Warehouse

AWS offerings: Redshift

It feels like these two services have been around forever. That’s because, in internet years, they have. Redshift goes back to 2012, and SQL DW goes back to 2009. That’s a lot of time for both Azure and AWS to learn about data warehousing as a service.

Data Processing

Azure offerings: HDInsight

AWS offerings: Elastic MapReduce

Both services are built upon Hadoop, and both are built to hook into other platforms such as Spark, Storm, and Kafka.

Data Orchestration

Azure offerings: Data Factory, Data Catalog

AWS offerings: Data Pipeline, AWS Glue

These are true enterprise-class ETL services, complete with the ability to build a data catalog. Once you try these services you will never BCP data again.

Data Analytics

Azure offerings: Stream Analytics, Data Lake, Databricks

AWS offerings: Lake Formation, Kinesis Analytics, Elastic MapReduce

I didn’t list Event Hubs here for Azure, but if you want to stream data you are likely going to need that service as well. And Kinesis is broken down into specific streams, too. (In other words, “Analytics” is an umbrella term, and is one of the most difficult things to compare between Azure and AWS).

Data Visualization

Azure offerings: PowerBI

AWS offerings: QuickSight

I saw some demos of QuickSight while at AWS re:Invent last fall, and it looks promising. It also looks to be slightly behind PowerBI at this point. Of course, we all know most people are still using Tableau, but that is a post for a different day.

Search

Azure offerings: Elasticsearch, Azure Search

AWS offerings: Elastisearch, CloudSearch

Elastisearch for both is just a hook to the Elastisearch open source platform. For Azure, you have to get that from their marketplace (that’s what I link to because I can’t find it anywhere else). One of the biggest differences I know between the services is the number of languages supported. AWS CloudSearch claims to support 34, and Azure Search claims to support 56.

Machine Learning

Azure offerings: Machine Learning Studio, Machine Learning Service

AWS offerings: SageMaker, DeepLens

DeepLens is a piece of hardware, but I wanted to call it out because you will hear it mentioned. When you use DeepLens you use a handful of AWS services such as SageMaker, Lambda, and S3 storage. I enjoyed using Azure Machine Learning Studio during my data science and big data certifications. But the same thing is true, you use associated services. This make price comparisons difficult.

Data Discovery

Azure offerings: Data Catalog, Data Lake Analytics

AWS offerings: Athena

Imagine a library without a card catalog and you need to find one book. That’s what your data looks like right now. I know you won’t believe this, but not all data is tracked or classified in any meaningful way. That’s why services like Athena and Data Catalog exist.

Pricing

Azure Pricing calculator: https://azure.microsoft.com/en-us/pricing/calculator/

AWS Pricing Calculator: https://calculator.aws/

Same as the previous post, you will find it difficult to do an apples-to-apples comparison between services. Your best bet is to start at the pricing pages for each and work your way from there.

Summary

I hope you find this page (and this one) useful for referencing the many analytic and big data service offerings from both Microsoft Azure and Amazon Web Services. I will do my best to update this page as necessary, and offer more details and use cases as I am able.

The post Updated Analytics and Big Data Comparison: AWS vs. Azure appeared first on Thomas LaRock.

Since both Microsoft Azure and Amazon Web Services offer many data services, I thought it worth the time to create a graphic to help everyone understand the services a bit more. Essentially, I wanted to build a cheat sheet for any data services comparison (click to embiggen):

You might notice that there is no Data Warehouse category. That category is located in the Analytics and Big Data comparison chart which I will share in a future post.

It is my hope that this post will be a starting guide for you when you need to research cloud data services. I’m not going to do a feature comparison here because these systems evolve so quickly I’d spend all day updating the info. Instead, you get links to the documentation for everything and you can do your own comparisons as needed. I hope to have future posts that help break down features and costs, but for now let’s keep it simple.

Relational

Azure offerings: SQL Database, Database for MySQL, Database for PostgreSQL, Database for MariaDB

AWS offerings: RDS, Aurora

RDS is an umbrella term, as it is six engines in total, and it includes Amazon Aurora, MySQL, MariaDB, Oracle, Microsoft SQL Server, and PostgreSQL. I’ve listed Aurora as a distinct offering because it is the high-end service dedicated to MySQL and PostgreSQL. Since Azure also offers those distinct services it made sense to break Aurora out from RDS. (Or, to put it another way, if I didn’t call out Aurora here you’d finish this post and say ‘what about Aurora’, and now you don’t have to ask that question.)

NoSQL – Key/Value

Azure offerings: Cosmos DB, Table Storage

AWS offerings: DynamoDB, SimpleDB

Cosmos DB is the major NoSQL player for Azure, as it does everything (key/value, document, graph) except relational. DynamoDB is a workhorse for AWS. SimpleDB is still around, but there are rumors it will be going away. This might be due to the fact that you cannot create a SimpleDB service using the AWS Console. So, short story, look for this category to be just Cosmos DB and DynamoDB in the future.

NoSQL – Document

Azure offerings: Cosmos DB

AWS offerings: DocumentDB

Azure used to offer DocumentDB, but that platform was sunset when Cosmos DB came alive. AWS recently launched DocumentDB with MongoDB compatibility in what some people see as a major blow to open source.

NoSQL – Graph

Azure offerings: Cosmos DB

AWS offerings: Neptune

As of May 2019, Neptune is in Preview, so the documentation is likely to change in the coming weeks months years (well, that’s my assumption, because Neptune has been in Preview since November 2018.) Cosmos DB uses a Gremlin API for graph purposes.

In-Memory

Azure offerings: Cache for Redis

AWS offerings: ElastiCache

Both of these services are built upon Redis, so the real question here is if you want to use Redis-as-a-service from a 3rd party provider as opposed to just using it Redis itself.

Time Series

Azure offerings: Time Series Insights

AWS offerings: Timestream

If you are in need of a time series database for your IoT collections, then both Azure and AWS have a service to offer. Azure Time Series Insights was launched in early 2017, and AWS announced Timestream in late 2018. In other words, the world of data services is moving fast, and the two major cloud providers are able to roll out services to meet growing demand.

Ledger

Azure offerings: [Sad Trombone]

AWS offerings: Quantum ledger Database

Setting aside the silliness of using the buzzword ‘Quantum’ in the name of this product, AWS does have a ledger database service available. As of May 2019, Azure does not offer a similar service.

Pricing

Azure Pricing calculator: https://azure.microsoft.com/en-us/pricing/calculator/

AWS Pricing Calculator: https://calculator.aws

I like using pricing as a way to start any initial comparison between data services. These calculators will help you focus on the important details. Not just costs, but how the technology works. For example, Azure SQL Database focuses on the concept of a DTU, which has no meaning in AWS. Using the calculators forces you to learn the differences between the two systems. It’s a great starting point.

That being said, trying to compare the data services offered by AWS and Azure can be frustrating. Part of me thinks this is done on purpose by both companies in an effort to win our favor without giving away more information than is necessary. This is a common practice, and I’m not bashing either company for doing what has been done for centuries. I’m here to help others figure out how to make the right choice for their needs. At the end of the day, I believe both Amazon and Microsoft want the same thing: happy customers.

By starting at the pricing pages I can then dive into the specific costs, and use that as a first level comparison between the services. If you start by looking at resource limits and maximums you will spend a lot of time trying to compare apples to oranges. Just focus on costs, those resources, throughput, and DR. That should be a good start to help you determine the cost, benefit, and risk of each service.

Summary

I hope you find this page useful for referencing the many data service offerings from both Microsoft Azure and Amazon Web Services. I will do my best to update this page as necessary, and offer more details and use cases as I am able.

The post Updated Data Services Comparison: AWS vs. Azure appeared first on Thomas LaRock.