Welcome to the world of artificial intelligence (AI) and machine learning (ML), where the struggle is real.

At some point in your career, you have experienced a failed project. If not, don’t worry, you will. Projects fail for all sorts of reasons. Unclear objectives. Unrealistic expectations. Poor planning. Lack of resources. Scope creep. Just to name a few of the more common reasons.

When it comes to projects with AI/ML at the core, all those same reasons apply, plus a few new ones. AI/ML is perhaps the most important piece of general-purpose technology today, which means we are bombarded with AI/ML solutions to solve random or ill-defined problems in much the same way we are bombarded by blockchain solutions for tracking fruit trucks or visiting the dentist.

The overhype of AI/ML has left people skeptical regarding the promises made through project proposals. Even if you manage to get a project funded, the initial results produced by your model may be difficult to explain, leading to apprehension about deploying solutions which cannot be understood. Nobody wants to blindly follow the decisions and predictions produced by machine learning models no one understands.

It is clear the business world needs a way to build, deploy, and maintain AI/ML models in a consistent manner, with a higher rate of success than failure, and completed on time and within budget.

bizML

Thankfully, there exists a modern approach to AI/ML projects. It is called bizML, and it is the core subject inside the new book by Dr. Eric Siegel – The AI Playbook.

For any project, not just AI/ML projects, to succeed there must be a rigorous and systematic approach for real-world deployments. Every successful project has similar characteristics – measurable goals, stakeholder involvement, risk management, resource allocation, fighting scope creep, effective communication, and monitoring project progress before, during, and after deployment.

The AI Playbook breaks this down into digestible sections for anyone with business experience to understand. It outlines bizML as a six-step process for guiding AI/ML projects from conception to deployment: define, measure, act, learn, iterate, and deploy. Using stories from familiar companies such as UPS, FICO, and various dot-coms, Dr. Siegel leans on his experience to help the reader understand how and why even the best ideas often fail.

I don’t want to give away the surprise ending, so I will just say the real secret behind bizML is starting with the end state in mind. Many projects fail due to stakeholders not aligned with the reality of deployment versus expectations. bizML attempts to remove this roadblock by getting everyone aligned with what the end state will look like, and then build towards the agreed upon state.

I read through the book in less than a couple of days, absorbing the material as fast as possible. The use of personal stories was easier to read as opposed to a purely technical book focusing on code and examples. I cannot emphasize enough how this book is not a technical manual, but a business guide for business professionals, executives, managers, consultants, and anyone else wanting to learn how to capitalize on AI/ML tech and collaborate with data professionals.

Summary

As AI/ML solutions continue to gain traction in the market, this book provides the right framework (bizML) for successful AI/ML deployments at the right time. Anyone, or any company, looking to deploy (or has deployed) AI/ML projects should buy copies of this book for all stakeholders.

I’m putting this onto my bookshelf and 15/10 would recommend.

The post Book Review: The AI Playbook appeared first on Thomas LaRock.

I was a developer, working for a small software company outside of Boston. Our product was a warehouse management system, built with PowerBuilder on top of Oracle. We had a handful of large customers helping to keep the lights on, but a few went dark at the start of the year, casualties of the dot-com bust. 

We were soon a casualty as well, forced to sell to a competitor at the start of the year. And despite their promises and assurances about not making changes, the layoffs started within the first few months. 

They hit about a dozen folks the first day, including me. I was stunned.

I still remember the phone call from an executive I spoke infrequently with. “Can you come meet with me in the small conference room we never use for anything?” Yeah, sure. I walked across the office, knowing what was coming.

I’ve been cut from a team before, but I was just not prepared for this. And no one could explain to me why I was chosen as opposed to someone else, there didn’t seem to be any reason. It just…was.

And it really pissed me off. 

I did everything I was assigned. Traveled when other developers refused. I asked to take on challenging assignments. Noticing we had no DBA, I asked to attend Oracle certification classes. It seemed any and all efforts were ignored, or discounted.

And thus my first real corporate lesson: Nobody cares about your effort, they only care about results.

The remaining employees huddled in the large recreation room to hear about the layoffs. My friends looked around and noticed I wasn’t there. That’s because I was at my desk, putting my stuff into a box.

Packed up and walking out the door I said goodbye to a few people, and piss off to a few others. 

I put the box into my car and I started driving. I’m thinking to myself “Surely someone must need a PowerBuilder developer, right?” I drove up to Lexington, where Suzanne was working. She had stepped out to get a coffee. While she was walking back she saw me walking towards her. She didn’t hesitate to stop and yell across the street “what happened?”

She knew.

I told her what happened.

We were scared.

I vowed to never allow this to happen to me again. Never would I be the one to get cut, not in this manner, and not treated so poorly in the process. But I needed to find a job, and fast, since living in West Newton was not cheap.

My only marketable skill was PowerBuilder. Fortunately, it was still in enough demand, and it wasn’t long before my phone rang.

Not from job offers, no. It was recruiters calling me with the hope and promise of finding a job. I had worked with a handful of recruiters in the past, but I was still young and inexperienced in how to play the recruiting game. Or, to be more to the point, how they play games with kids like me.

And I was still scared. 

So I listened to what they told me to do. One recruiter took my resume, rewrote it, put c++ at the very top (because I knew how to spell it, apparently) and sent me off on interviews where I would sit with someone for 5 minutes and they would say “why are you here if you haven’t done any c++ programming before?”

Good question. Talk to the recruiter, I guess.

Another recruiter scheduled me for an interview where I had to sit for an analytical reasoning test, because apparently my MS in Mathematics wasn’t a good enough indicator of my analytical reasoning ability. Shockingly, I scored 50 out of 50, was told no one ever scored perfect before, I didn’t get the job, and never got those 45 minutes of my life back. 

I had no offers, and no prospects.

I was low.

And then, out of nowhere, a recruiter contacted me about a job down in Hartford.

Hartford!

Hartford is 90 miles away from West Newton, but mostly a reverse commute. I could drive to Hartford in about 75-80 minutes as it was all highway driving. Folks commuting into Boston from Nashua, NH along Route 3 take longer than that!

I decided to go for the interview for two reasons. First, the scared thing. Fear is a great motivator. If I had to drive 90 miles each way, I would. Second, I knew getting hired by an investment firm in Hartford is job security. People there had careers, you know? It didn’t matter if you had skills or not, you just needed to get past your 90 days. Once you are in, you are in for as long as you want. 

I interviewed with a wonderful human being named Craig. He needed PowerBuilder help. I needed a job. He asked about the drive and I told him it would not be a problem. He mentioned the company offered some relocation assistance, if I wanted to move. I thanked him, we shook hands, and I drove home.

When I got home, I had an offer. Well, offers. One was from Craig. The other was from a software company in East Boston who specialized in point-of-sale systems for cruise lines. East Boston is only 11 miles away from West Newton. However, it would take me an hour to get to work during rush hour combined with the Big Dig. They also expected employees to work at least ten hours a day, but in reality they were only paying me for eight. I really liked their company, and the idea of being “forced” to work on a cruise ship a few weeks each year. 

Suddenly, the fear was back.

I couldn’t choose East Boston because I’d be doing the same PowerBuilder job, with nothing to grow into. I needed a job, sure.

But I wanted a career. 

And I had vowed to never get cut from the team again.

So I chose Hartford.

And my life was forever altered.

Within a few months 9/11 happened. A few months later, Suzanne and I bought our first home, in Worcester, in order to cut down on my commute (reduced to 60 miles, but not by much time because I was a bit further from the highways). We had our first child there, got pregnant again (the best part about babies is making them) and decided to move once more. This time to be closer to both sets of grandparents. Right around the time of the move, “it” happened.

“It” was the 1.5 DBAs at my company quit within a week or so of each other. One was a contractor who simply gave his 30 day notice. The other was a part-timer who was offered the full time job and he said “yes” then changed his mind a week later. This left no one except the guy who had previously executed the following commands successfully:

BACKUP

and

RESTORE

No bootcamps. All it took for me to be handed this opportunity was the fact I knew how to do a backup, a restore, and change passwords.

So, the greatest manager in the world asked Craig (one of the few truly good people on this Earth) if he could “borrow” me, and Craig knew that the best thing for me (not for HIM, but for ME) was to become a DBA. So Craig agreed, we set a transition period over the next six months or so and that was that.

I was now the DBA.

There was a lot to do, a lot to learn, a lot to accomplish. In time I found and joined PASS. I connected with others. I helped take my company from what was essentially a wild west show into a more stable environment. Now, I wasn’t always nice (I think my communication skills have improved in time), but I was always thinking about the company, and whatever actions I took needed to benefit everyone and not just the one or two people looking to take a shortcut.

As my skills grew, so did my thirst for new challenges. Eventually, I hit the limits of my role. There was nothing more for me to grow into. And what had once seemed an amazing opportunity and role now seemed like a prison sentence.

I could still remember how I was so excited and so proud to have a job with a company in Hartford. It was not just a job, but a career. And here I was, nine years later, burned out and tired. Worse yet, the people around me were tired of having me around. It seemed as if I would say ‘black’, three people would say ‘white’ just on principal alone. I felt my skills were called into question every day, or the environment I spent six years building would be called into question, if for no other reason because I was the voice on the other end of the phone.

There was nowhere else to go.

On a whim, I decided to drive into Boston for a SQL Saturday event. While there I was walking past the vendor tables and I heard the phrase “I bet you’d look good wrapped in bacon”. It was probably the best pickup line anyone could use on me. I handed him a bacon gumball. He tried one, got sick, and still wanted to talk to me. I felt bad about making him ill with the gumball so I did something I had not done in over five years:

I attended a dedicated vendor session.

They talked about their product. They promised me it would monitor my SQL and Sybase instances, something I asked other vendors to give me for years. I agreed to give their product a trial, if for no other reason because they did some heavy name dropping and they told me they read my blog. 

I installed the trial, let it run for a week, and started to fall in love. About a month later they came to visit and walk through my data with me. I liked it even more. We had lunch together. The next month we all met up again at a SQL Saturday. It was me, David Waugh, and Matt Larson (CEO of Confio Software). We agreed to meet and have dinner prior to the SQL Saturday dinner. It was, essentially, my job interview. There was no exchange of resumes, just some conversation over dinner. No talk of salary, just talk about our children. No talk about managers, just talk about the role they felt I could help with.

And there was no job opening. We were creating one. Right there. In a hole in the wall in New York City, on a Friday night in the Spring. 

I was a long way from that walk across the office. 

I still had fear. Joining a startup was a risk. Matt looked at me and said “Tom, lots of things can happen. But I’m fairly certain in a stack of resumes, yours would float to the top”. 

Suddenly, I felt like I had a place to go. A place to grow. New challenges. The opportunity to build something, together. 

I took the job(s). I was a sales engineer, customer support, marketing, product support, and maybe a few other roles. You know how everyone wears a lot of hats when you are small.

I clashed with others, wanted to quit, and was almost fired. Fortunately, Confio had their version of Craig. His name was Don, and Don believed I was a value-add even if others did not. Don saved me, just as Craig saved me before. 

I held on. I survived.

Not long after, we were purchased by SolarWinds, where I am today.

This post is really long and doesn’t have a point. If you’ve read this far I thank you and feel I owe you something of value. Something like “focus on the things you can change” or some type of sage advice. 

I’ve got nothing. Well, maybe something.

Always be learning.

Twenty years ago I had no job. Today I have a career. I went from software developer to database administrator to sales engineer/customer support/product support to technical product marketing. I’ve sought roles where I have the opportunity to grow and be presented with new challenges. These past few years I’ve been immersing myself in data science.

If I needed to find work tomorrow, I believe I could. The same should be true for you, too.

Don’t settle for what you already know. Don’t let your opinions block your judgement when it comes to tools, products, or technology. Be open to new possibilities. Be humble to the idea that you don’t know everything, and you can always be learning something new.

If you can do that, then you’ll always have a place to go, a place where you will add value.

That’s it, that’s the post.

The post Twenty Years appeared first on Thomas LaRock.

And honestly, it’s as if you have learned nothing in that time.

Remote meetings have a different dynamic that in-person meetings. I was hopeful the past year you would all come to understand this and adjust your habits accordingly. 

Well, I’m not afraid to admit I was wrong. None of you have learned a damn thing.

You treat remote meetings as if they are in person. You jump at the chance to speak, just as a half dozen other people are speaking. 

I am here today to make some suggestions. 

First, a meeting should have an agenda. Just a couple of bullet points at the very least so everyone has an opportunity to prepare for the discussion. The agenda also allows for the meeting organizer to keep everyone on topic.

Second, the organizer should use the mute button with extreme prejudice. With a dozen or more people in a meeting there is zero reason to allow everyone to be unmuted by default.

Lastly, and most important, when you want to say something, raise your damn hand. 

Here is the most important feature in Microsoft Teams you aren’t using: 

It’s not that hard to do, folks. It is a courtesy to everyone else, kinda like wearing a mask. When you have something to say, show respect for everyone on the call and raise your hand. Allow the organizer to call upon you to speak.

This isn’t rocket surgery folks. It’s basic manners.

And if you think this post is probably about you, you’re right.

The post The Most Important Feature in Microsoft Teams You Aren’t Using appeared first on Thomas LaRock.

Frequent readers of this blog may recall last year I earned a certificate in Data Science through the Microsoft Academy and edX. I followed up with a certificate in Big Data earlier this year. Today I am happy to tell you that I am now certified in Artificial Intelligence, giving me a Trifecta of Microsoft Academy certifications.

That makes three certificates in the past year. I’m not certain if I will look to do more certifications right now. I think I would rather use my free time to apply these new skills, and get some practical experience.

Microsoft Professional Program for Artificial Intelligence track

Here’s a list of courses needed to obtain the certificate:

Introduction to Artificial Intelligence (AI)
Introduction to Python for Data Science
Essential Math for Machine Learning: Python Edition
Ethics and Law in Data and Analytics
Data Science Research Methods: Python Edition
Principles of Machine Learning: Python Edition
Deep Learning Explained
Reinforcement Learning Explained
Natural Language Processing (NLP)
Microsoft Professional Capstone : Artificial Intelligence

The capstone project was frustrating, maddening at times, and one of the most rewarding projects I have ever completed. The project involved taking 20,000 images with labels, using them to build a predictive model, and then running that model against 20,000 test images and outputting the predicted label for each. A passing score was to have your model be about .9933 accurate. In other words, out of 20k images, you are allowed 134 incorrect. My score was .9997, which means my model had six incorrect. Yes, it still bothers me that I missed those six.

Some of the Lessons Learned

Compared to the first two certifications, there was a lot of new material this time around. That meant a lot of new learning, as well as the reinforcing of concepts from earlier courses. Here’s a summary list of the important items:

– The most time consuming part of any AI project is at the start, with data curation, consumption, and preparation taking up to 95% of your total time.

– There is no “right” number when it comes to things like epochs, batch sizes, layers, and percentage of data divided into train and test sets. You will build many models and adjust these parameters, looking to improve your model each time.

– But sometimes, the answer is really as simple as “Up the Epochs“.

– It’s easy to get Jupyter notebooks running locally, on Windows or MacOS, allowing a lot of functionality providing you have the right packages installed.

– Oh, and it’s easy to have 37 different versions of Python installed. When you are done with this course, you will be a master at debugging issues with Python and packages such as Keras, TensorFlow, and OpenCV2.

– VSCode is a great tool at editing and running Python scripts, too. I spent a lot of time in VSCode, but ultimately got my project done using local Jupyter notebooks.

– Before I forget, a quick shout out to Azure Notebooks, because they are useful for getting started with notebooks. They weren’t powerful enough for my final project needs, though, but I could not complain about the price (F-R-E-E).

– I used to ask why someone would use R versus Python. Well, now I know. R is a great tool, but Python seems to offer a lot more functionality and has a larger user base, which means you are going to find data science and AI examples written in Python more often than R. If I had to recommend one of them to learn, it would be Python.

– While your model is building you have time on your hands. It’s helpful to have some additional projects to get done, like building a firepit or maybe enjoying some good Scotch next to the fire in your new firepit.

Summary

I’ve written and talked on the subject of the changing role of DBAs and data professionals before, so I’m not going to do that again here today. If you aren’t on board with how this tech is evolving, then we are riding different trains. There’s nothing wrong with that. You do you, and I’ll do me.

But I love this stuff. And this stuff is the future. And the skills you develop in these programs are going to help make your job easier over time. If you don’t think you have time to do a course right now, that’s fine. But take another look at the list of classes available, and see if there might be one you are interested in learning more about.

Start there, with something that looks interesting, maybe even fun to try.

For me, I’m heading over to Kaggle, to try my hand at some of the projects and competitions they have. I want to keep my skills sharp, and they have the data to help.

The post Achievement Unlocked: Certified in Artificial Intelligence appeared first on Thomas LaRock.

After completing the Microsoft Professional Program for Data Science last year I made up my mind to dive into the Big Data certification next. The end of the year snuck up on me though. Time doesn’t stop. You either make time to get something done, or you don’t.

When 2018 started I listed the Big Data certification on my goals for the first half of the year. I made certain to set aside time to complete the work. The track is ten courses long, but two of them were completed as part of the Data Science program. That left eight courses to get done. I knew the final capstone project was likely in April. So, I budgeted time to get seven courses done before the end of March.

I finished in mid-March, allowing me time to review the previous capstone project. That extra work paid off, allowing me to complete the April capstone project once it was live.

Nobody goes to school to become a data janitor

But this certificate might be the closest thing we have to a Data Janitor school. Check out the course list required to earn the Big Data certificate:

Microsoft Professional Orientation: Big Data
Analyzing and Visualizing Data with Power BI
Introduction to NoSQL Data Solutions
Querying Data with Transact-SQL
Delivering a Data Warehouse in the Cloud
Processing Big Data with Azure Data Lake Analytics
Processing Real-Time Data Streams in Azure
Orchestrating Big Data with Azure Data Factory
Developing Big Data Solutions with Azure Machine Learning
Microsoft Professional Capstone: Big Data

To get started, go to https://academy.microsoft.com and create an account. From there, link your Microsoft Academy account to your edX account. edX is where you take the courses, and Microsoft Academy issues the certificates once edX reports your scores. This process can take a few days for your scores to appear, so be aware there could be some latency.

One thing I want to stress is the quality of the edX courses taught by Graeme Malcom. The way he structures his courses resonates with my learning style. If you decide to go through these programs, I would urge you to select his courses every time you have the chance.

The course content is a great mix of Azure technologies. But it is not Azure only, as there are options to use platforms such as Spark, Hadoop, Jupyter Notebooks, Python, and Node.js, to name a few. It was through this content that I started thinking a bit more about our Global Data Estate, resulting in my recent posts on Azure and AWS data technologies.

The capstone project incorporated a little bit of everything covered in the course. Unlike the Data Science program, we did not need to submit a final report or go through a peer review process. I liked that aspect of the previous capstone, as it added to the overall educational experience.

There is a dearth of people in this world that can analyze data

Data science and analytics is a growth field. Data administration is not. You should consider hitching your career path to something on the rise. Data cleansing is now over 95.4% of any data science project.

The role of data administration is being automated away. Our days of tuning indexes are ending one page at a time. As a data professional the pivot you must consider right now is taking on a larger role in data analytics projects.

Find a way to get some experience with something other than resetting passwords.

Stop spending your lunch hour arguing with others about the nuances of a temp table versus a table variable.

Take the time to enhance your current skills. Otherwise, you may find yourself replaced by a handful of Powershell scripts. In the cloud.

Summary

Nobody ever brags about having small data.

The Big Data program is a wonderful way for data professionals and administrators to augment their current skills. There is a benefit to yourself and others when you reach out to learn new data technologies. I’m not saying to quit your day job just yet. What I am saying is that you should notice as Microsoft makes shifts. BY the way, there is now an Artificial Intelligence certificate.

That’s where you’ll find me next.

Because data isn’t just sexy, it’s addictive.

The post I’m Certified in Big Data appeared first on Thomas LaRock.

“What do you think made him so good?”

“He was very quick, and he was also very generous.”

That line. That sentence fragment. It hit me. Hard. Right in the brain.

And it came from an unlikely source. Season four, episode three of Comedians in Cars Getting Coffee, with Robert Klein. Go to 1:57 in and watch for yourself:

If you watch the series you will hear that same phrase used over and over again. Jerry and his guests use it to describe other comedians…no, other humans that were influential to others. They just happened to work in comedy. But the key ingredient for success is their generosity. It’s what made Johnny Carson so great, and it’s why Jimmy Fallon is so good as well. And it *may* explain why Conan O’Brien struggled in his first attempt at hosting, too. But I digress.

I don’t see enough of this generosity inside of IT departments. I see more of a “what’s in it for me” attitude. And, since I work in marketing, I see many examples of where people treat others as prospects, and not as friends or colleagues.

For me, being generous means you don’t expect anything in return. If you are expecting kindness in return then you are doing business, not kindness.

But there are exceptions in the world of data professionals. Robert Davis was one of them. He passed away early this week. And with him, our #sqlfamily lost a very generous soul.

Robert showed us all that generosity need not be of a monetary value. Sometimes a person’s time and/or skill is the most precious thing they can give. And Robert did just that. Here are a few examples:

– Robert’s contributions to the #sqlhelp hashtag
– His efforts on Spiceworks forums
– All his MSDN forum activity
– His SQL Server Mirroring book
– The private Microsoft MVP and MCM email distribution lists

I’m certain that I’ve missed many more examples of his reach. The point I want to make is that he was generous with his time and it influenced thousands of other data professionals.

And he did that without ever expecting anything in return.

That’s being generous. And that’s why there is an outpouring of admiration and respect for Robert.

If you are so motivated to do so, you can make a donation to his wife, Chrissy, at their GoFundMe page.

The world needs more people like Robert.

Thank you, Robert, for all your generosity, to me and everyone else, for so many years.

RIP, my friend.

The post The Generous DBA appeared first on Thomas LaRock.

The list is specific to Netflix, but if you want you could make your own summary and apply it to your company. Here are the ones I want to call attention to the most:

Act in the best interests of your company. This seems like a no-brainer, but I’ve learned that common sense isn’t so common. So I’m here to remind you to act in the best interests of your company.

Beware the local optimum, or consider how your actions integrate with the needs of everyone else. Just because something is perfect for your silo does not make it perfect for everyone else.

There is no secret ingredient or recipe. Successful teams, or organizations, understand that every decision comes down to cost, benefit, and risk. You will build better solutions, as a team, over time, by understanding how the weighting of variables is different from person to person.

Gather your data, and then make a decision. Don’t be afraid to make a choice. If it is wrong, that’s OK. Learn and move on.

Don’t be afraid to ask questions. I don’t mean in a child-like way, I mean ask meaningful questions. One of my favorite techniques here is the Five-Whys, where you ask “Why?” five times in order to drill into the root cause of an issue. If questions are asked in a constructive and meaningful way, everyone benefits.

Standardize, but only when it makes sense. Many companies will spend far too much time trying to implement standards when they should be implementing a process.

The users own the platform. Here’s something I wish I had learned years ago as a production DBA. I viewed the databases as “mine”, but they weren’t. My role should have been one of caretaker, not gatekeeper. If I had taken the view of the users as the owners, it would have removed a lot of friction.

Let your users have visibility into their systems. There can be reluctance on the part of IT admins to let users see how the servers are performing. The thought is that if the manager sees a red ‘X’ near their server they will be calling you every five minutes. But the reality is that the users *need* this visibility. You want them to have this visibility, trust me. It will also reduce friction.

You can’t have it all. “Fast, cheap, and easy, pick any two” is the common phrase here. Sometimes you have to give up one thing to get another.

The first 80% of anything is easy. It’s that last part, the last mile, that will often be the hardest to get done.

Set expectations. If you are not doing this already, you should start immediately.

Tell instead of ask. Another item I wish I had learned years ago. If you ask for approval from every person and team, nothing will get done. In my case, the project was decommissioning SQL 2000 servers. I made the mistake of asking when I should have been telling.

Make it a shared decision. Involve everyone in the process, but the decision itself should be done by only a few, or one. The point here is that the facts and the decision would not be a surprise.

Trust others. This falls into the “treat others the way you want to be treated”. If you want to be trusted to do the right thing, you need to trust that others will do the right thing. I find what works best for me here is to assume good intentions on behalf of others.

Summary

I think this video should be required watching for any data professional or IT admin out there. On Day 0 of your career, you should watch this first thing in the morning. It will help your career, and your company. If everyone were to watch this then, in time, we could make the world of data a better place.

The post 20 Ways to Get The Most Out of Your Data Platform appeared first on Thomas LaRock.