What Humpty Dumpty Can Teach Us About DevOps

I’m convinced Humpty Dumpty is a story of DevOps gone wrong.

What Humpty Dumpty Can Teach Us About DevOps

Emily Dowdle September 7, 2016

I’m convinced Humpty Dumpty is a story of DevOps gone wrong.

Humpty Dumpty sat on a wall,
Humpty Dumpty had a great fall.
All the king’s horses and all the king’s men
Couldn’t put Humpty together again.

You see, Humpty is a deploy. He was all right in QA but collapsed after being deployed to production. Now, the site’s down and your boss is losing his shit. IT is saying the code is broken. The developers are saying it’s a server issue.

While everyone’s busy arguing, Humpty is bleeding out. Customers are complaining. The pressure is building.

Sounds familiar, huh?

It’s a familiar scene. A deploy goes awry and everyone’s up in arms, defending themselves and blaming each other.

Confusion + Chaos

All the king’s horses and all the king’s men
Couldn’t put Humpty together again.

First, who asks a horse to operate on an egg? Hoofs can’t hold scalpels. Second, either the king’s men are inept or they’re not communicating. Two kids with some Elmer’s could have done the job.

Meme
I don’t know about you, but I always think best when I have four people looking over my shoulder telling me I’ve made another typo.

At first glance, it would seem that the team is well equipped to handle any situation. After all, you’ve hired talented engineers and operations people.

But it doesn’t always work out that way. It always devolves into a blame game. You know the scene.

Developers often say…

  • “Must be a server issue.”
  • “It worked in staging.”
  • “A configuration must have changed.”

Operations people often say…

  • “Has to be a code change.”
  • “What was just deployed?”
  • “Not my problem.”

And there we have it. All the king’s horses and all the king’s men aren’t working together.

“Dev”.concat(“Ops”)

Most companies split their development and operations teams. Which means there can be some animosity. It could be described as friction, attitude, or a general inability to tolerate each other without eye rolls and audible sighs.

Change is hard. I hate it as much as you. But I think pursuing a DevOps culture is worth the struggle.

By now you’ve likely heard something about DevOps. It’s the latest craze.

For those of you who haven’t, here’s a quick history:

The term was coined by Patrick Debois and Andrew Clay Shafer in 2008. Hilariously, Andrew had planned to speak about DevOps at a conference, but received a negative response and decided to bail on his own session. Only one guy showed up: Patrick. They connected later and DevOps was born.

Paul Hammond and John Allspaw flamed the #devops conversation further when they gave their talk, 10+ Deploys per Day: Dev and Ops Cooperation at Flickr. It’s an oldie, but a goodie. And very much worth your time. Just play it during family time tonight. Your kids are gonna love it. Promise.

At its essence, DevOps describes a company culture where developers and operations people work together.

Traditional Thinking

Before we discuss DevOps shambhala, it’s important to recognize a key difference between development and operations: they have different priorities.

Developers are measured by the number of features they release. No manager has ever opened up an IDE to review your impressive test suite or was awed by your glorious method naming (I appreciate it, though. So you have that going for you).

Operations people, on the other hand, are measured on an entirely different part of the business: site uptime. And you better believe building a reputation of site reliability is no easy feat.

Developers release new features through a deploy. But deploys are the most frequent cause of downtime.
No wonder there’s some friction.

highlander.jpg

How to Implement DevOps

What we need is operations people who think like developers and developers who think like operations people. It’s simple. But I didn’t say it was going to be easy.

Operations People: Empower Developers

Trust your team. Team is the key word there. You’re on the same side. If a developer says the code works, take their word for it. They don’t want to make your life a living hell just for shits and giggles. They honestly believe the code works.

Create consistent platforms. In addition to improving user experience, it’s easier for development and operations to support when platforms are seamlessly integrated and compatible.

Share your code. Keep your configuration tools on GitHub with the rest of your company’s source code. Developers can only help solve server issues if they know where to find your code.

Simplify deploys. Pushing code to production should not, in and of itself, be a production. Remove unnecessary steps to reduce the risk of error.

Developers: Don’t Be Jerks

Include operations in planning. Talk through new features with operations before you write a single line of code. Share why the feature is critical to the business and be open to feedback.

Be Agile. Make small changes. Deploy. Repeat. If your feature requires you to change 30% of your app’s spaghetti code, break the feature into smaller pieces. Not sure if your feature is too big? Apply the same rule you use for method naming. If the method needs an “and” it’s doing too much. Small deploys make it MUCH easier to determine what went wrong in case of failure. (It’s no surprise Agile inspired DevOps.)

Yes, and… One of the rules of improv is that participants can say “yes, and…” but not “yes, but…” Give this a try next time you’re in a meeting. The results may surprise you. The word switch will make everyone feel heard and validated.

Have some humility. Your ops team are the people who get woken up in the middle of the night because you released buggy code. When you make a mistake, take responsibility. Be human. Buy ’em a cup of coffee and say you’re sorry.

Nighttime Reading List

Like to read? Here are a few books I recommend.

The Lean Startup by Eric Ries
Web Operations by John Allspaw
The Pheonix Project by Gene Kim, Kevin Behr and George Spafford
The Goal by Dr. Eliyahu M. Goldratt
Implementing Lean Software Development by Mary and Tom Poppendiek
Continuous Delivery by Jez Humble and David Farley

No, I haven’t read every book on this list. I don’t think people read nearly as many books as they say they do. Or I watch too much Netflix. Don’t judge me.


Freemantle