The 3 Golden Rules of Value-Driven Test Suites

Value-driven test suites offer a way optimize the maintenance effort by focusing on the test cases that provide most value for you. This means defining those tests that matter most, but also regularly reducing your tests. But how to do this in practice? I’ve pinned down the answer to this question to 3 golden rules of value-driven test suites in this article:

1Have the best data available

The basis of value-driven test suites obviously is knowing the value of your each of your test cases. I’ve written a dedicated article on that (How to evaluate the value of your test suite), but in the end, the #1 rule here is having data available to judge the value. This data can come from metrics such as code coverage, mutation scores, the execution time of your tests, or can even combine several of these metrics (The 3 ultimate steps to define your perfect regression test suite). You should however choose wisely when defining which metrics you want to rely on, as this might depend on the requirements of your particular coding setting and skills. The resulting data should give you a very clean insight into the effectiveness of your test cases for your daily work, and enable you to focus on the most effective test cases, and sort out the rest over time, in order to actually have this value-driven test suite.

2Define only the best

When it comes to defining unit tests, you should only regard yourself with those who actually bring the most value to your daily programming skills. This avoids maintenance effort for unnecessary test cases right from the beginning (well, the definition). In order to choose the best test cases to create for a given part of your software, you can apply methods such as equivalence class partitioning and boundary value analysis (described in detail here: 2 methods that help you save 60 % of your effort in Unit Testing).

Once you have defined these best test cases, you should also take care of writing them down in code that is as understandable as possible, in order to maximize the value of this code (what’s the value of a test case where you need to think about what it actually does before you can start debugging your software?). Luckily, there are clear clean code best practices on how do deal with this as well (read my personal summary here: Clean Unit-Testing – die Kunst, wartbare Unit-Tests zu schreiben).

So the better your test code, the easier it is for you to find the defect causing a bug, thus the higher the quality of your test suite!

3Choose Wisely when it comes to running unit tests

The more tests your test suite contains, the more difficult it is to maintain them, and the longer it will take. The former (time for maintenance) will leave you with the frustration of having to update many tests after every change to your software – usually leading to a state where you simply start ignoring your tests at all. The latter (execution time of your test suite) will lead to a state where you don’t want to execute your test suite too often, because waiting for the result for several minutes every time simply takes too much of your precious time. So you should in general aim for running as few test cases as possible. Obviously, minimizing the number of test cases is also not the goal (that would mean not executing any test case at all), as each test case should ideally (particularly if you have followed rule #2), because each test should contribute to some extent to the assessment of your software quality. In the end, you are faced with a tradeoff here – execute as few tests as possible, to keep your effort low, but as many as necessary to have meaningful insights into your software quality. In practice, this means that throughout your project, you should regularly assess your test suite and check which test cases should be deleted because they do not provide enough value any more. If you have followed rule #1, you should also have the right data available to do this judgment.

These are my 3 golden rules for value-driven test suites. How did you make use of them in your project to optimize your test suite? Do you already have value-driven tests? Do you want to add another rule to this list? Let me know in the comments section below!

Image by vectorjuice on Freepik