Habit #5: Failure is not an option

Although there are many ways to do DevOps right, there are several ways DevOps is commonly done wrong. Often, these bad “habits” are merely relics of the old way of doing things. In this blog series, we’re unpacking seven of those scenarios so companies can learn how to do DevOps right. This week, we’re examining why the mantra ‘failure is not an option’ often holds teams back.   

Often, mindsets do not update as fast as technology. That’s really unfortunate because it causes teams to miss out on some of the benefits technological advances offer.  

Believing that failure is not an option is a mindset that resulted from previous technological limitations but remains prominent. With traditional development cycles, software was put into production following a release calendar, which did not offer many chances to get things right or to correct errors. Once the software was sent to production, two or three months had passed since the code was created. The details were no longer fresh in developers’ minds. Doing releases was a daunting task where mistakes were costly. Under those circumstances, it’s understandable teams became extremely risk-averse. They wanted to be certain of everything before launch. 

That strong risk aversion impacted company culture. People were hesitant to speak up if they encountered a problem or got stuck. The traditional work environment was very hierarchical with managers laser-focused on Key Performance Indicators (KPIs) and other metrics. When bugs were discovered the mentality was more: “Who’s fault is it?” rather than: “How can we fix it?” Obviously, the latter is more productive.  

Responding In Real-time  

Thankfully, modern technology eliminates a lot of the costly and time-consuming processes that caused people to become risk averse. Mistakes can be fixed almost instantly so the stakes are much lower.  

Now, it is possible to separate deployment and release. A new version of a product can be put into production, which is a deployment, but releasing the features in that new product doesn’t have to happen at the same time. Feature toggles, or other products with the same function, can be implemented so that the new version still behaves like the previous version. Before anything else is done with the product, the team can revalidate whether or not it actually runs. Once its functionality is confirmed, the next step is to say, “OK, let’s toggle the switch and start using the new features.” 

The ability to limit how many users see the new features further mitigates risk. For example, with a webshop, the new version could be limited to 1% of users while the rest of the users still see the old version. That way, if problems are discovered only a small percentage of users are impacted and the whole site will not crash, which would lead to a loss in revenue. Releases can be gradual. Bugs can be fixed in real-time. The worries of “big bang” releases really are a thing of the past.  

Shared Responsibility  

A more equal work environment helps mindsets start to shift. Instead of a manager making all the final decisions, teams decide together. Silos are less likely to form. Rather than seeing their focus as one specific thing, teams are more likely to focus on the big picture. When the mindset is “We’re in this together,” the idea of failure is less frightening.  

The DevOps way of working also makes work more transparent. People work together closely and are in constant communication whereas traditionally, developers would be given a task by their manager and then work on it alone at their desk. While in constant contact, it is difficult to hide issues that arise. A more collaborative work environment positively impacts workflow and overall company culture.  

Ultimately, seeing failure as bad or negative stifles progress. Failure is an important part of learning. With today’s technology, mistakes can be corrected almost instantly. Accepting failure as part of the process actually leads to better outcomes.  

Take the next step towards doing DevOps right – get in touch! 

We want to help companies optimize the DevOps way of working. This blog series is just the start of what we can offer! Get in touch with us to learn more about how we can help your team do DevOps right.  

Until then, tune in next time to find out why "our superheroes always fix production issues really fast" is the sixth habit of a highly in-effective DevOps team.