Technical debt. Bringing features live faster by taking shortcuts in the source-code. As a result, future features are harder to build because the source-code got more complex.

The same applies to testing. A lack of automated tests slows a team down and makes it more complicated to release software. Especially, when you’re just getting started with continuous delivery, there is few test-automation, if any at all. Fixing it all in one go is too much work. So, how do you deal with that?

Test-debt

Technical debt eventually slows a team down. So does the lack of automated acceptance tests. Not having an automated acceptance test for a new feature creates manual work, and a higher risk to break things in production.

When transitioning to frequent, automated deployments, test-automation is what you need. It speeds up the delivery of the software without the risk of quality loss.

Installing software on a site isn’t frequently done in a waterfall project. Automating the installation- and the test process probably doesn’t pay off.

A higher release-frequency doesn’t change the number of things you want to validate before going live. Without any test-automation, the testing process will take just as long as it always did. Testing will remain the bottleneck in the process. So the choice is to either hire more testers or to automate the validations they do. You’ve got some test-debt to pay.

Plan it?

It would seem obvious to create a user story, plan it, and get started building some test-automation. Take a sprint or two and be done with it. But that only changes the code-base. And a code-base is a reflection of a mindset. Change the mindset and you’ll change the codebase. If you don’t, your code will change back.

Automating tests is a learning experience. Just like coding is. And every code-base is different. Every domain is different. It takes time to figure out what to test and how to test it. It takes a few tries to get it right.

In other words: Change the process and the people, not the code. The code is a side-effect. Change your mindset to have automated acceptance tests for every user story from now on. Coverage will follow.

“Software Development is a learning process, working code is a side effect.” — Alberto Brandolini

Keep delivering value increments

I’ve rarely met a product owner who was willing to plan the type of user-stories that are purely technical, with only a single stakeholder: The development-team.

Think about it. Imagine you’re hiring a plumber to fix the kitchen sync and you’re asking him to estimate the amount of work. The sync is flooding, and you’ve got to mop every hour. Imagine the plumber wants to take a week to set up all sorts of tooling, first. Because he’s sure he’ll need them in the future. I’d hire another plumber.

Some product owners are willing to spend some time on technical improvements. But stakeholders often feel they’ve got more urgent things to spend development time on. But the product owner shouldn’t care how things are tested, he should care about what is tested. Choosing the appropriate way to test is up to the team. So, by automating a few tests every story, the team will keep delivering business value and the test-suite will grow organically.

Start from the outside in

Test automation should speed the process up, not slow it down. Yes, you need unit-tests. They are important. But are they the first thing you need? Their scope is small. Oftentimes it’s hard to tell whether or not the software is compliant with all of the business rules of the processes that have been automated, judging from the unit-tests.

1.) Automate acceptance tests first

Testing is done to discover and describe the behavior of the software. It’s done to make it possible to add features or to refactor. Testing is the act of collecting information to support the decision to go live or not.

Writing a bunch of tests that make sure that some class throws ArgumentNullExceptions will not support that decision. Tests are useful if they provide information, not data.

Automate processes to speed processes and to prevent human error. Manual acceptance testing could be one of those processes. An acceptance test, implemented on the API level, allows the team to refactor. It provides information about the behavior of the application and it’s one less manual process to perform.

2.) Use test-automation to reproduce and to debug production issues

Production issues occur when things happen that weren’t anticipated during the development process. Some strange combination of mutations or input might have caused the system to break.

Reproducing production issues is often done by inserting data into a system and doing an assertion on the outcome. Usually, the tester provides user input and inserts data into a database or an external system to reproduce the issue.

Bugs are (forgotten) acceptance-criteria that aren’t met. Every bug is a test-case. It’s the missing part of your regression set. Automating it completes the test-suite, makes it easier to reproduce and to fix, and it prevents it from ever happening again.

Gain trust

Don’t underestimate how hard it is to build a test-suite that is trust-worthy. Don’t assume your first automated tests are. Getting it right is a learning experience. Keep performing manual tests for a while and fix tests that give false positives, until you trust them enough.

Refactor

Some say test-driven development is a software design method, not a test-method. It’s a good process to get a cleaner code-base. It’s a good way to get a proper amount of getting a solid base layer of Fowlers testing-pyramid.

Codebases of applications that weren’t designed to support automated tests can be hard to test. Testability is an architectural aspect of an application. The first holistic acceptance tests, that are representational enough, are too slow in many cases. Use acceptance tests and unit tests to safely refactor the code-base to be more testable.

Summary

Code-bases of products that are annually released, probably don’t have any automated tests. But you’ll need them if you want to deliver continuously, and often.

Changing people and processes changes the code-base. Change the codebase without changing the people, and it will change back.

Automate acceptance tests first. They provide the team and stakeholders enough information to decide to go live or not. Don’t blindly trust test-automation. Keep testing manually too. Repair tests that provide false positives.

Start doing test-driven development to get a proper amount of unit tests. At first, the tests might be slow. That’s obvious if the software architecture doesn’t support test automation. Don’t worry about that. As soon as the tests are trustworthy enough, you can safely refactor that.