24 Feb Patching SQL Azure
When the Cloud breaks, who fixes it? Why, the people who are responsible for maintaining it, right? And that person would not be you, so why do you care? And how do you patch a cloud, anyway? What do you do when the silver lining rips apart? Oh, nevermind.
So then, when does it get patched? If you are using SQL Azure, do you get notified in advance that a hotfix or service pack is being applied? Ideally the answer is “yes”, but I’m not certain if that is true.
Let’s say you are an ISV. You have built up a business and are currently using SQL Azure as the back end for what is considered a non-critical application. It is an application, however minimal, that does face outward. That is, external customers use the application periodically. Now let’s suppose that on a Wednesday you get an email from Microsoft that a critical patch needs to be applied to SQL Azure, and it will be deployed on Friday evening.
How do you feel now? Better, or worse?
Where I sit, these are the actions you have available:
Just sit there and do nothing. Don’t worry about a thing, because you don’t have to. SQL Azure is the new Water Works, it is a utility and it will always be on, and your stuff will always work.
Backup Your Data
Take a backup of your data. Unfortunately, the only way to do that right now is to build a custom SSIS package that will use BCP to pull the data from the cloud and down to your datacenter, where I am certain you have servers available for this anyway. I mean, it’s not like you spun up a business using only SQL Azure in order to save money, right? You did buy hardware, just in case, right? (UPDATED: This post is 6 years old, there are many more ways to get backups of your data).
By the way, unless you are also using Windows Azure, then that SSIS/BCP process is also going to cost you money, because in addition to paying for the SQL Azure database itself, you also pay for the downstream bandwidth (unless you keep your data in the same Azure data center) as well as the extra storage. Either way, you are going to pay money, so you get to pick which way you want to pay. I suppose that’s a good thing.
So, the patch forces you to do this backup, which could cost you money. Will SQL Azure and Microsoft reimburse you for this? After all, if it wasn’t for their patch, then you wouldn’t need to be doing this particular backup (but we know you have backups happening anyway because you are a wise administrator). What if they decide to patch SQL Azure three times a week? Could be a nice revenue stream.
Have Microsoft apply the patch and then you test your application. Wait a minute, this is just the same as ‘do nothing’, except you can have people think you are doing something because you can arrange for testing to happen after the patch is applied. But what happens if the patch breaks your application? Do you think Microsoft will roll back the patch, just for you? I wonder at what threshold of customer dissatisfaction with a patch will Microsoft decide to roll back a patch? My guess is ‘never’, especially if it is a critical patch, but I suppose it is possible.
Still, the thought that my application could break, with little to no warning, causing my customers to be upset, and forcing me to build a patch in a very short amount of time is slightly unsettling. It certainly makes me want to think twice about what can and can not be put into the cloud.
It also makes me wonder if there is a chance that we could get a SQL Azure test sandbox, where Microsoft rolls out patches first and allows for their customers to test changes. Or if it would be possible for me to apply the changes locally in order to test for myself.
I am sure I am being paranoid, but I am a DBA. As Buck Woody once told me “When DBA’s go to a movie we are looking for the fire exits because we always expect disaster.” With SQL Azure still being so new there are a handful of details that need to be hashed out by all parties. I believe patching the cloud to be one of those things that needs to be thoroughly discussed before anyone starts getting too serious about cloud computing.