No one can predict the future.

And yet it seems as if we all tend to worry about what the future will hold for us. As data professionals we are bombarded every day with new concepts and buzzwords like ‘cloud’, ‘big data’, ‘PaaS’, and ‘Snooki’. For many server and database admins these terms represent an unknown future, a future that they cannot picture themselves having a role in.

I am here to help you make sense of it all. I am going to lay out the foundation for you. Instead of fearing the change that is coming, I am going to outline for you the trends I have been observing for years. These are the trends that you need to recognize and use to your advantage.

These are the five trends that will affect your future as a data professional.

1. You Will Have More Stuff To Manage

How many of you reading this blog post are administering fewer systems today than you were three or five years ago? When I ask that question during my presentations I likely get one or two hands raised out of two hundred. Quite simply everyone is doing more these days. And here’s a dirty little secret that often doesn’t get mentioned: you have likely also started managing an additional database platform during that time (registration required to download the PDF). This is why we have so many ‘accidental DBAs’; admins who were a developer, or a power user, or a generalist that were assigned the responsibility of administering a new platform such as MySQL.

We are rarely asked to do less at work, only more. And the list keeps growing.

2. You Will Eventually Run Out of Hardware To Throw at the Problem

As the list of systems being managed keeps growing there will come a point where you decide to simply throw hardware at the problem. This is especially true when it is a cheaper option than spending time and resources trying to optimize existing code. But even throwing hardware around has its limitations when you talk about things like Big Data. Unless you work for the NSA, then the idea that your company is going to import over 100TB of data daily for research analysis is going to end in disaster. You may be able to throw enough hardware at the problem to get the project started, but within six months you will likely find yourself in need of another hardware upgrade and in the middle of a meeting being asked to explain why your company needs to spend even more money that originally projected.

3. You Will Learn To Love the Cloud

This is where you will start to look to the cloud in one form or another. You can find many flavors of cumulonimbus these days: PaaS, IaaS, SaaS. And it is likely that a combination of all three is what it will take for you to (1) reduce the number of systems you have to administer daily and (2) understand how much you really didn’t want to spend each and every weekend performing hardware upgrades. The cloud is here, it is not going away, and if you continue to dismiss it as something not real or practical you are going to find yourself left far, far behind. That may work for you if you are looking to retire in the next few years but for the rest of us that need to work another 30-40 years we are putting our heads in the clouds now.

4. Your Job Will Disappear

The other night my daughter told me matter-of-factly that “…one day I will have a job that nobody has ever even heard of before.” I told her “yes, yes you will.”

The job you have today is going to disappear. Whatever you call yourself right now, that title will change in the next five years. And there is a really good chance that your next job title doesn’t even exist yet. I’ve never heard of ‘Data Integrator’, ‘Global Systems Enterprise Architect’, or  ’Chief Data Officer’ (AKA ‘Big Data Daddy’), but I can certainly imagine each of those jobs being fairly common in the near future.

Start thinking of yourself as a data professional. It is a generic enough title that allows you to slide in and out of whatever job role is in vogue at the time. If you are determined to stay in your current role as a database administrator/developers/whatever, then you will find you have fewer opportunities in the future.

5. You Will Start Talking Like a Lawyer, Possibly Matlock

With each passing year it becomes more apparent that security and compliance are topics relegated not just to the members of an internal audit team. You will need to develop an understanding of compliance or be doomed to fail an audit. You should probably hire a lawyer or become one. Bottom line: start understanding more about what your company needs to satisfy audit requirements so that as your data shifts you can help architect a model that keeps everyone out of the hoosegow.

There you go, the five trends that are going to do more to influence your career as a data professional than anything else. Each one also has something else embedded inside of it: learning. It is more important than ever before to be constantly learning and adapting. Gone are the days when you could show up at work and fulfill the same role for twenty years. These days you need to be learning all the time, embracing new ideas, and trying new things in order to keep your skills current and in demand.

It’s the night before Valentine’s Day and I know what you are thinking: “What would Tim Berners-Lee be doing right about now?”

I’ll tell you what he is doing. He is linking data. After all, that is what drove him to create HTML over 20 years ago. No, he didn’t invent the web. He wasn’t the first to dream up the idea of linking together documents and information between computers and systems.

But what he does do is to see the potential in things. And for the past few years he has been seeing the potential in connecting open data sources from around the web.

His ultimate goal is simple enough: to connect all data being created, everywhere, in real-time.

Piece of cake.

Big data is not new. It is not a passing fancy. Linked data  is, quite simply, the Egg McMuffin of future opportunities.

[This was first posted to Technorati on February 6th, 2012. You can read the original here.]

I keep hearing the term “Big Data”. My old-school IT instincts always makes me think of inefficiently storing two-letter state abbreviation codes inside of an nvarchar(50) field because that’s the default data type you get when creating a database table from inside Visual Studio. But it turns out that Big Data means something much more than that.

Big Data means big profits. It means predictive analysis. It means a shift in how we process data. It means that as database professionals our skills are going to be needed for years to come.

It means everything.

Check out this article from Popular Science magazine that discusses how the city of Santa Cruz is using data in order to better prevent crime from happening (yes, almost like “Minority Report”). This is not your ordinary “business intelligence”, this is the next stage of development. This is “Business Analytics“, and people everywhere are going to want to have trusted experts that can help them reliably mine their data.

Want to know when and where the next crime will be committed? How about how the spread of a disease from city to city? Or finding a way to help traffic flow? Don’t believe that Big Data can help us predict such things? Then check out this book: Linked: How Everything Is Connected to Everything Else and What It Means. Or take a few minutes to learn about the “3 Vs” of Big Data (Volume, Velocity, Variety) from Edd Dumbill and how more companies are using these concepts to gain a competitive edge.

Turns out the concepts behind Big Data is not so new. Researchers have been trying to find ways to connect data for decades. Even the idea for hyperlinks goes back to 1945, a full ten years before Tim Berners-Lee was even born.

Think about all this for a moment. We have been trying to find ways to link data together for over 60 years. Big Data is not new. What is new are the shiny tools we are being handed each year to mine the data. And why are companies investing in building us shiny new tools? Because they know the market for business analytics, and Big Data, is quite vast.

And there will also be a market for folks who know how to use the tools. Start investing in some training now, it will pay off for you later. Those folks who shun such concepts as Big Data, or the “cloud”, are going to be left far behind. And at the rate changes happen these days, you may find yourself left behind sooner than you think.

It’s the end of the year and already I am seeing the usual blog posts that either reflect on the year that was, or are pondering the year to come. I thought about writing another post covering all of the things that didn’t happen to me this past year but since I’ve already done that before I figured I needed to do something different. So I decided on this post instead. You’re welcome.

Here are the 12 things you should stop doing in 2012.

1. Using antiquated technology

The line in the sand for me would be the ages of my children. If your software was written before they were born, I don’t want to be using it. But that measuring stick won’t work forever, so a better one is to make certain you are only at most one full version behind. If you are still running SQL2000 or Oracle9i, you are more than one version behind, and you should really stop. It’s time to move on. Seriously.

2. Responding immediately to every email

Stop doing that. Trust me on this, your response to that email can wait. Don’t believe me? Try an experiment. For one month do not reply immediately to your emails. At the end of the month add up the number of emails you received, and the number of emails that required an immediate response. I’m willing to bet that the number is quite low, much lower than you realize. And once you realize just how few emails require an immediate reply, you’ll never look at email in the same way again.

3. Over-thinking everything

Some things in your life require analysis and thought, no question. But you don’t need to spend 30 minutes ordering coffee at Starbucks. Lots of things in your life are like that coffee, you should be able to just make a decision and move on. But for whatever reason we tend to analyze lots of decisions in our lives that don’t really need any extra analysis.

4. Doing unproductive work

Start tracking your daily tasks and see which ones are the most productive. Chances are you have a few tasks that don’t align with your highest level of goals. If you want to become the world’s best widget-maker then you need to focus on tasks that help you achieve that goal, and spend less time on tasks that don’t help you get there.

5. Pretending that work-life balance exists

It doesn’t. Sorry to be the bearer of bad news, but the idea of work-life balance is a myth perpetuated by HR departments that are trying to recruit you. This is especially true the higher up the corporate ladder you find yourself. And yes, that means if you are self-employed, you are essentially your own CEO, and you don’t just work at home but you also live at work. So stop pretending that any such work-life balance exists, it doesn’t. What you really have is work-life management, and we all know how hard it is to find a good manager these days.

6. Multitasking every possible moment of the day

Would it hurt you to focus on the task at hand for once? You don’t always have to be doing three things at once. Like texting and driving, for example. Remember I told you how that email can wait? Yeah, that is especially true if you are driving. Try living in the moment a few times a day and you may be surprised to see the quality of your output rising as a result of your increased focus on one specific task.

7. Trying to keep up with the Kardashians

…or watching Jersey Shore, or those shows about “real” housewives. If we all just stop watching, there is a chance the shows may disappear altogether. That’s the world I’d like to leave for my children: a world free from crappy reality TV, and reality TV “stars”.

8. Stop arguing with people on the internet

It’s just not worth it, even when they are wrong, and even when they attack you. The internet is a big place, and it is full of opinions. Let people express their opinion. It’s OK. If you really feel the need to inject yourself into a conversation, don’t do it. You won’t win. You can’t win. It’s the internet, and every time you try to get the last word in about something, or try to demean someone because you feel you have been wronged, all it does is make you look a little more foolish than if you had written nothing at all.

9. Resisting the Cloud

Look people, stop making excuses for why you can’t move to the Cloud. It’s here, and people are using it already. That means they are learning lessons about Cloud technology. They are learning what works best for them, and they are ahead of you by doing so. How much longer do you want to wait at the starting line while everyone passes you by? Get in the game now. Find something that you can deploy to the Cloud and put the learning curve behind you, and do it quickly. Otherwise you are going to find your skills are out of date, and quickly.

10. Comparing yourself to others

You’ll never be as good as that other person. It’s true. You will only ever become second-best, because you will always feel they are better than you anyway. It’s a losing effort, so don’t bother. Instead, find out what one or two things you like most about them and see if you can adapt those things to fit your personality or lifestyle. At the end of the day you have to be yourself, but it’s OK if you get some influence and inspiration from others. Just don’t try to be an exact replica because that’s just creepy.

11. Thinking that you need to go it alone

I see this a lot. In fact, I lived it. I thought I was all alone in my role as a DBA. But I wasn’t, because I had a network of colleagues out there willing to help. You don’t need to do anything alone in life, you can always find someone willing to help. Even misery loves company. And whether you are a DBA or the CEO of your company, you don’t have to work in a vacuum, there is a network of your peers willing to lend you a hand. It’s OK to ask for help, to admit that you may have been able to do something better. If your end goal for life is to have “I did it myself” on your tombstone then you should expect a lot of lonely Holiday dinners.

12. Reading blog posts titled “12 Things in 2012″

They’re a waste of your time. Go kiss your children goodnight and read them a story instead.

I figure that eliminating the above items will save me a lot of time in 2012. I have no idea what I will do with the extra time yet, but I’m sure I will think of something before the Mayans attack in late December.

Had a quick chat with Buck today regarding a recent bill (for $1.38 USD) for my Azure subscription.

If you are using Azure for testing (like me), and you want to minimize your chances of being billed, then just drop the user databases you have created. You don’t need to drop your Azure subscription, so please do not think that is the case.

Now, if you want to be extra safe, you can also go ahead and drop the SQL Azure database server, but that step is not necessary at this time as you do not get billed for a server that is only running the master database. You only get billed for user databases that exist.

And as always, the web moves fast. Things change. Don’t blow by the agreement you’re signing up for – it’s a new model, so understand what Microsoft is saying. More here: http://msdn.microsoft.com/en-us/library/ee621788.aspx

Thanks Buck!

I decided to drive over to Waltham tonight to listen to Brent Ozar (blog | twitter) talk out of his ass about virtualization. The place was packed and @BrentO put on a good show, as expected. He gave us the four hour slide deck but condensed it down to about 60 minutes.

I chatted with Brent a bit before the talk about some of the issues I had come across with virtualization but it was during the talk that I was struck by a sense of déjà vu. Brent was talking about various ways that certain configurations could affect a host which will then affect all the guests. In my mind I stopped to ponder where I had come across that situation before.

It was with a SAN.

See, when you build yourself a SAN, the idea is to have a crapload (that’s the technical term) of disks carved out into logical units (LUNs). Your servers end up getting assigned to the LUNs which are presented to the server as a disk. This makes it easy for your SAN admin to expand disks quickly when more storage is needed. It also makes it easy for your SAN admin to choke every last bit of performance from your database server, especially if they treat it like it is a file server. But that’s a post for a different day. The point here is this: your stuff is shared.

So, what does that mean? It means that if you have a performance issue on one server, which can cause issues with your SAN, then you could see the effects manifested on other servers. You could roll into work one morning and see that there were problems on 20 servers and not think it was related in any way. Worse yet, you could see a problem on just one server and the culprit was a different server sharing the same spindles as yours.

But with virtualization this problem goes away, right? WRONG! In fact, you not only still have the SAN issue but now you have a possible issue with guests and hosts. And we can throw vMotion into the discussion just because we haven’t had enough things to possibly go wrong. There are so many possible reasons for you to see a performance degradation on your guest that it will really make the forensics challenging.

For those folks that like checklists, you should add in items for checking to see if servers are sharing the same SAN storage, if guests are tied to the same hosts, and if your guest was migrated during the night. It will make your troubleshooting easier. When you go virtual your servers are more co-dependent on each other than ever before. Awareness of this fact will be crucial in how quickly you can isolate problems.

After the talk was done I had Brent call my cell phone to verify that his ringtone would come up (“Bad Romance”) and it did.

I heart technology.

or…not…

Well played, Microsoft

I wanted to cancel my account before I got hit up for “hidden charges” or something ridonkulous like that and could not find any way to cancel my subscription short of calling someone on the phone which could be my least favorite thing to do.

From what I could tell, I should be allowed to “opt-out” of an automatic renewal, except I don’t appear to have that option. I am wondering if I am simply not getting a proper user experience because I am part of some trial program, but it is rather annoying to not be able to simply click a button and say “stop charging me at the end of this month”.

As of right now I owe $1.06 for usage charges over the past month. I would be fine with that except for the fact that I was under the impression I was not going to be charged anything at all during my trial period. I find it hard to believe that Microsoft needs my $1.06 that badly but apparently they do. And they also need it again next month, so that may be why they won’t let me cancel my account so easily.

I suppose I could just drop the server I am using, essentially ending any chance of them charging me for SQL Azure altogether, but something tells me they would find a way to keep billing my credit card. So instead I will just open a new credit card, change my billing information to use the new card, and then just cancel the card.

That should take care of them trying to charge me for a trial service they don’t want me to cancel easily. And yeah, I understand that opening the card and closing it is a lot more effort than just calling someone at Azure support, but I am annoyed to the point that I would go out of my way to circumvent the “process” they have configured.

Since I have been railing on SQL Azure lately, I thought I could share with you some of the videos you can watch for free over at MSDN. In case you were not aware, there is a LOT of free content over at MSDN and I am trying to utilize it more and more these days. Right now I am using it as a way to understand more about SQL Azure and I thought I could share three videos that you may also find useful.

I recently presented a session on SQL Azure (you can download the slide deck here). During the session we talked a lot about some of the little things I have found by simply using SQL Azure. The need for a clustered index on every table was a surprise to me when I first found out about it, and it seemed surprising to everyone in the room as well. One person mentioned how it is a common practice to drop indexes before loading data to a table, for example, so they would need to rethink the steps involved in pushing data to the cloud.

I still think the biggest hurdle will be how Microsoft approaches the application of service packs and hotfixes to SQL Azure. One person at my session suggested that they would assume that Microsoft would have done thorough testing of the patch and also that with such limited features available right now that the testing would be able to be streamlined. I tended to agree but decided to point out something I found with their documentation:

Click to see full image

So, they went out of their way to link to four DMVs that they DO NOT support, and the one DMV they do support in this category is not even a hyperlink? Why, exactly, would you hyperlink to things I cannot use? With such a huge usability FAIL, why is it that I would trust them to do thorough testing of any patch when they cannot even handle the usability of their documentation?

I want to love SQL Azure, I really do. I intend to find more and more ways to utilize it whenever possible. But when I see things like above it really makes me question how some of the plumbing is really being handled. I strongly believe that if you take care of the little things then you will inevitably also take care of the big things. And if you can’t do the little things right then you might as well “use the button”.

Use the button, please.

In the past four weeks I have come across a lot of issues with the documentation being borderline unusable. When I was trying to push data to the cloud a few weeks back I talked to Buck Woody (blog | twitter) about a handful of options to which he replied “Why don’t you just use Sync?”

“Well,” I replied, “because I don’t see that mentioned anywhere as an option. Don’t your departments talk to each other? Why is this option not in the documentation?”

I know this is really v1.0 of a product, and that things will get better in time. I truly hope that is the case and we are not going to be subject to having things half-done tossed over the fence at us to use. But the more time I spend with SQL Azure the less comfortable I am that they are taking care of the little things.

And I consider documentation to be more than just a little thing.

Tomorrow I will be presenting at the first ever Hartford SQL Server Users Group. The title of the talk is Cloud Computing with SQL Azure, but only because I couldn’t pull together my first choice which was “Dora the Cloud Explorer”. Sorry, but I ran out of time on that because I’ve been a little busy with the upcoming 24 Hours of PASS event. By the way, I am going to adopt Brent Ozar’s moniker of 24iHop for the event. In fact, if I can find an iHop near me that has wireless, I may just UStream myself from there during the event.

The talk tomorrow will start around 1-ish. Our meeting is schedule to go from 12-4PM EDT and I expect that we will start broadcasting around 1PM but don’t get upset if we are a few minutes behind because Rob and I need to spend some time taking care of business and I doubt anyone really wants to watch that. Oh, before I forget, here is the LiveMeeting link for tomorrow:

https://www.livemeeting.com/cc/microsoft/join?id=TJB64W&role=attend&pw=4BMMRD

We have over fifty people registered for the event tomorrow, which would be an incredible number for Rob and I to have for our first event. We are really hoping we can continue to meet here at least once a quarter. Oh, how about a calendar invite? Yeah, that could be helpful. I cannot save .ics files to my current web host but what I can do is configure an event in Google Calendar and share it with you. Just click on the friendly button:

I mentioned that our meeting is scheduled from 12-4PM EDT, and I will start talking at 1PM EDT or so, but don’t expect a three hour talk on SQL Azure. My talk will only last about an hour, including questions. So, if you are planning on watching during your lunch hour, that’s fine, you don’t have to worry about missing the last two hours of the presentation because you won’t miss anything except fifty people eating cold pizza and milling about before heading home.

I hope to see you there, or online, tomorrow!