The Test Pyramid Part 7: In what order should functional test automation be implemented?

This post is part of a series on functional testing using the test pyramid.

For new code

New code has unit tests added as the code is created. If you are good, you write them before the code is created (TDD), but even if you don’t use TDD, get them done and merged into the master branch at the same time as the code. If you put it off it won’t happen — or it will be bad.

If you are doing good API-first design on new code (and you should), you’ll want to build the API contracts and tests for them before you fill in the actual functionality behind the interfaces.

After creating the unit tests and API contracts, add one or more good end-to-end tests. These tests act as your BVT test and help keep the overall quality up while you develop. The end-to-end tests have great value, but they don’t prevent teams from breaking each other. End-to-end tests are also expensive to maintain and tend to find issues late, so don’t start with more than 10 of these tests. Instead, switch to component and integration tests next after you have a few end-to-end ones in place.

Component (or new API) tests are derived from the requirements in initiatives, epics and stories that are used to define the component features. The epics and stories have acceptance criteria that become tests, so when the work completes, it should have automated acceptance tests to prove it does what the requirements say. We found that stories tended to be mostly testable with unit or integration tests, while epics needed component or end-to-end tests.

For legacy (untested) code

Legacy code doesn’t have much unit testing, by definition, so add unit tests to new or changed code. Don’t go out of your way to add tests to areas just to increase test coverage. You’ll have to refactor the legacy code to make it testable and you want to do that only when you have a reason to change the code anyway. Changing working code just to add unit tests is high risk, low benefit.

Most of the legacy projects I’ve dealt with already had some end-to-end automation. In fact, most had more than they needed, because other types of testing were hard to add to a monolithic architecture. If you don’t have any end-to-end tests, start by adding a few, but stop before you hit 10 of them. Instead, work to separate off pieces of the project behind APIs (use extract interface) as you need to work on those areas anyway and implement API tests for the new interfaces.

Copyright © 2019, All Rights Reserved by Bill Hodghead, shared under creative commons license 4.0