What does a DBA do all day?
It is clear to me that no one, except for database administrators, has any idea:
What do DBAs do
— SwiftOnSecurity (@SwiftOnSecurity) April 21, 2016
I’ve read many articles over the years that help people understand tasks involved for the DBA role. I’ve written a few myself, including a book I have mentioned once or twice before here.
Today I want to help everyone understand what a DBA does all day long. I have put together a summary of the tasks that I find are common for the DBA role. This is a partial list of the items that came to my mind right away. No, not every DBA will be doing all these tasks but chances are they will do one or more:
Recovery – If you can’t recover data, you can’t keep your job. This is the number one task for any DBA.
Backups – Having backups makes your ability to recover a bit easier.
Performance – Performance tuning and optimization of queries that we didn’t write, against databases we didn’t design. DBAs get paid for performance but we keep our jobs with recovery. (see above)
Standards – Working with other teams to agree upon a set of database standards for your shop.
Risk – Assessing risk, working with auditors, outlining security and access control.
Installation – Installing database software on servers.
Configuration – Configuration of database servers.
Monitoring – Monitoring database servers for performance, including maintenance for things like indexes, corruption, etc.
Capacity – Helping to plan for future growth.
Troubleshooting – Being able to respond to issues and locate the root cause quickly.
HA/DR – Help architect an effective business continuity plan.
ETL – Integration and migration of data between systems.
Development – Writing stored procedures, designing tables.
That’s quite a list! It looks like one of those horrible job postings I’ve ranted about before. But the list does help frame the DBA role for others to understand.
Vertical vs. Horizontal Role
The above list helps others to understand why the DBA role is as a vertical role. These roles have a focus on the immediate tasks completed by a single person on a daily/weekly/monthly basis. This is why you hear the phrase “the best DBA is never seen nor heard”. If a DBA is doing their job well then the number of issues are minimal. And so they don’t need to leave their cube except for nourishment and some limited human contact.
To have less people questioning what we do all day long we need to transform the DBA role into a horizontal role. A horizontal role is one that thinks about, and includes, other teams. With the DBA role so focused on data, and data the most critical asset any company owns, it makes sense for the DBA to work across teams and not alone. DBAs must make certain that data is being treated right as it flows in, around, and out.
Automation is Key
With so many tasks to manage, and only so many hours in the day, DBAs turn to automation to get the job done. Sure, it would be great to hire additional staff to offload the work. But headcount is harder to come by than a few PowerShell scripts. Automation is key to transforming the DBA role from vertical to horizontal.
You know who does automation well these days? Cloud providers, that’s who. Check out this list of services from Azure:
Recovery – Point-in-time restore
Backups – Automated backups
Performance – Query Performance Insight
Standards – Managed services
Risk – Threat detection
Installation – Did I mention the managed services yet?
Configuration – OK, now I know I have mentioned managed services
Monitoring – Database advisor
Capacity – Seriously folks, its a managed service
Troubleshooting – Application Performance Insight
HA/DR – Disaster Recovery Service
ETL – Hybrid integration
Development – Entity Framework
You can find similar tools deployed by Amazon AWS. That’s right, the top two cloud providers are automating away the core DBA tasks. This is happening, right in front of our eyes, whether you want to believe in Cloud or not.
The days of tuning queries and rebuilding indexes is ending, one page at a time.
What Will a DBA Do All Day?
So if all the common core tasks are being automated away, what will the future DBA be doing?
The answer is simple: It’s all about the data. It’s always been about the data.
The future of the DBA is in building solutions, not tables and indexes. It’s in understanding how data is being used, not in how data is stored. The future is analyzing data, not in how it is administered.
The future view must also be a horizontal view, one that applies the logic and analytical skills that DBAs already possess, across teams.
Because the DBA that works in a silo, in a vertical role, with a only a vertical view, won’t be around much longer.
No arguments with anything you’ve said. But I run into this same question every day. The problem is one of perception. Business Guys do NOT understand RPO, RTO, HA vs DR, why indexes must be tuned, and why 15 different backups are needed. DBAs don’t speak the “language” of business.
Instead, Business Guys just see the DBA as an impediment to getting value from data. DBAs are the guys who tell my crack development staff that they won’t release some code because the table isn’t modeled correctly, or the guy that insists on performance code reviews…or the guy who insists on DRI everywhere in a EDW when my source systems should be handling that…or whatever else.
This is the Number One reason, no, it’s the only reason, anecdotally, why Business Guys want to migrate to PaaS cloud db offerings. “I need less Ops Guys to manage my stuff, I don’t know what those guys do anyway.” I have yet to hear a Business Guy mention cloud economics or rent-a-license…it’s always, “Let’s get more stuff out of the handles of my Ops Guys.”
Telling Business Guys all of those “things” that DBAs do, that you doc well above, which I get, just leads more Business Guys down the road of solutions that *supposedly* don’t require a DBA. Think about it, when MongoDB came out what were the Top 2 reasons on their marketing slides for business people? “You don’t need a DBA and with eventual consistency you don’t need backups.” We all know that’s bunk, but that is the perception. Or how about hadoop? “You don’t need an expensive SAN, hadoop works best with cheap local storage…and no DBA”. Note here that SAN Guys have the same (mis)perceptions as the DBA.
I think the solution is not to mention all of those things you listed above, rather to have a generic answer of, “we enable your business to get better insights from its data today.” Then stop being so grumpy and ornery.
Just my 2 cents
Well, I don’t see the grumpy and ornery part disappearing anytime soon, so maybe we can focus on getting that generic message out.
I do see your point about the desire to move to PaaS as a way to remove roadblocks. But it is also a way to reduce TCO. And that can mean headcount, but I think it gives DBAs the opportunity to shift into different roles. What I tried to say above is that we are the ones best suited to slide into roles that involve the collection, curation, and analysis of data better than most.
Thanks for the comment, I do appreciate your sharing your thoughts. Would love to sit down with you to discuss further if we ever have the chance.
I would also submit that software patching should be included in the list, as a regular required maintenance activity.
Perhaps “Monitoring” should be “Monitoring & Maintenance”, because the point of monitoring is to _DO_ something about it if required.
Agreed, I should have included patching, and even upgrades and migrations. And I like how you reminded us that the point of monitoring is to take action when needed. Thanks!
And to Dave Wentzel’s point, software patching would be another reason to move to PaaS and get rid of those DBA folks.
But I would suggest that even if you have adopted PaaS solutions, and maybe not needing to do stuff like patching, there’s still a certain amount of attention that needs to be paid to the thing. Maybe not the same kind of attention, in the same amounts. But still isn’t (or shouldn’t be) zero attention.
No, not zero attention, but less. Far less, really.
I’m a “full stack” developer. I’ve never quite understood the point of Entity Framework. I guess if you aren’t familiar with databases it makes sense. But SQL is the higher order language and really good at what it does. I would think people would be moving closer to SQL over time rather than C#. Granted some abstractions are easier in C#, or even better, F#. But SQL does what it does well, when you learn it.
But having know all that information is a bit overwhelming.
Yeah I get what you mean. I started off doing pure SQL development, and moved into C#. I tried to learn EF and things like it but just found it bulky and imprecise. I’m sure to a certain extent, tools that make things easier for those who learned them from scratch just find them pointless because it’s a new thing to learn which is less powerful than the skill you already know.
I think EF serves two purposes. First, it allows for there to be some uniformity in how code is constructed. Second, it reduces the learning curve for anyone unfamiliar with SQL or databases in general. This is both good and bad, of course. At the end of the day I see EF as more good than bad, but only if you have someone driving the project that know how to use EF in an efficient manner (one that optimizes ease of code creation versus query performance).
The way I build out my APIs I just create a bunch of Table-valued functions, wrap a Stored proc around them so I can throw errors for permissions and then I have a thin layer of ASP.NET that calls these procedures (ideally I would use FSharp.Data.SqlClient – but .NET Core isn’t prime time to let me use it yet and my employer wants me deploying the apps in Linux). I might aggregate some of these calls together but now that I can create JSON in the database that isn’t always necessary either. With tools like PostgREST I would think there would be less and less of a reason to even be developing the back end API, just do everything in SQL.
This is all unit testable with tSQLt. The database seems to have everything I need, especially with SSDT for version controlling and updating.
It will be interesting so see where the industry ends up in 10 years. Maybe I’m just headed in the wrong direction and the industry is heading another way.
I think you’re right that the cloud reduces some of the traditional DBA work. This should free DBAs up to alter their roles in the way you mention.
The DBA role is changing and the cloud is a big influence on this, but I think some of the automation mentioned doesn’t remove the need for all of these tasks.
A few thoughts:
RecoveryBackups – For me this is the biggest improvement that the cloud brings for databases but what if my DR doesn’t end in the cloud or in one partitcular cloud provider?
Performance – This is the biggie, we get some good insight on Azure but this doesn’t tell us how to rewrite a procedure to improve it’s performance. Poorly performing code is going to have a recurring cost in the cloud so DBAs DB developers will be even more important here.
Risk – Threat detection looks like a good tool but DBA types will still be required to manage object permissions.
Capacity – While capacity is less of a problem, the amount of space used and the amount of space we expect to use in the future is very important as it will drive the cost up.
DR – see RecoveryBackups
Nice post!
Yep, I’d agree that for the systems that remain Earthed, you will need someone with those unique skills. But I see the cloud providers building the tools necessary to perform a lot of those tasks, making it easier for companies to want to migrate.
One thing we do where I work is that the DBA is also responsible for windows updates on the boxes that SQL run on. Plus the DBA is responsible for non-DBA systems like the IIS servers. So I agree with Tim Plas about the maintenance part being added on.
And for those folks who have had to do data migrations (SQL 2008 to 2012 for example), the DBA is (at least where I work) the one who gets to do a lot of the testing and take a lot of the blame if it doesn’t go as smoothly as it is supposed to.
I think the DBA often gets pegged as the scapegoat too.
DBA = Default Blame Acceptor
I see the maintenance operations disappearing in time for those that are using any cloud provider.
Firstly thanks for the article.
In respect (SQL) PaaS this has pretty well summarised something I have been thinking for some time. Even cloud IaaS deployments today incorporate a number of automation elements, and lets not ignore SaaS solutions that a business consumes (where database is never even mentioned). I also think even if staying with SQL on-prem then virtualisation vendors will also incorporate automation in some way.
Given the growth of PaaS and its capability to automate “standard” DBA tasks the role of DBA is undeniably changing. When I can deploy an async DR copy in 3 regions with a button click, or have my database auto-tune indexes, or scale ten-fold in 60 sec, etc etc then this is a risk to the traditional DBA role/skills.
Its unlikely you’ll see cloud storage prices go up anytime soon – and also just because something isn’t automated today doesn’t mean it wont be tomorrow!
I agree that today there is still a need for deep SQL knowledge (tuning SP, code, ++), modelling, business education etc – but there is less requirement for “lights-on” maintenance in lieu for more “value-add” things that a business needs in the (likely) highly competitive environment in which it operates.
This really equates to working closer with building business opportunity though embedding within development teams, building DW/reporting solutions, bringing in exploratory data science (ML, R), experimenting with big data (if applicable), etc. These are the modern skills that DBA’s need to consider and although not a natural extension, they aren’t that far as to be insurmountable.
Will DBA’s be here in 5-10 years? Yes I think so, but there will be less DBA roles going around and those surviving roles will undoubtedly have a different role description.
Thanks for the comment!
I don’t see any risk because even with the growth of PAAS , the CEO won’t be having time to click that that button which deploys 3 databases in 3 regions. The role will just from a database administrator of SQL Server to an administrator of the whole stack ,but thats what we have been doing in the physical world right?
Not all DBAs are deploying servers in todays Earthed infrastructure. But in the future I expect that a DBA will be tasked with the deployment of database servers, having them spin up and down as needed, etc. The future will involve a lot of scripting until the tools get caught up with the tasks.
My DBA spends time fixing up the database after the developers f’it up.
Did I work for you at some point?
The DBA cant really do a DataScoience Job. Because he miss the deep understanding what all These data stand for. He see IDs,texts,numbers and columns and rows. Bit he dont know which Business case and which part of the Business produce a specific combination of data.
That depends on what you mean by data science. If you are talking about applying formulas and building models, then maybe not. But if you are talking about the collection and curation of data to feed those models, then yeah, they have a role.
Well I think it depends on which DBA are you? a production DBA , development DBA or a BI/DW DBA. If you are a production DBA , you tend lean towards hardware , orchestration and virtualisation not analyzing data as you say.
Also it depends on the size of company you are working for , for example if you work in a small shop with few developers , your daily duties may lean towards development and working closely with the developers to create a solution.
Agreed, those are all factors, thanks for the comments!