14 Jan
While most software projects fail to implement unit testing in a way that actually provides value (time benefit outweighs the time spent for creating and maintaining unit tests), some success stories still exist. Throughout the various projects I’ve worked in in the past, I’ve discovered some patterns that arise in projects and indicate whether unit testing will work or not.
In this article, I summarize these 5 signs that your project is doing unit testing wrong, together with actionable steps to make unit testing a success in your organization.
1. You have no unit tests at all.
I guess it’s quite obvious why this is bad practice in unit testing. Well, actually, writing no unit tests at all is in my opinion still better than doing unit testing wrong (I’ve elaborated on that in another article if you are interested). But still, you want to start having a good feeling when you commit something, because you can rely on your unit tests telling you that you have done everything correctly.
2. You don’t want to touch your unit tests.
This is a very good indicator that you are doing unit testing wrong (which is, as already mentioned, even worse than not having unit tests at all). Try to use this checklist for improving the quality and maintainability of your test cases now!
3. You start ignoring failing test cases.
Why is that the case? As soon as you cannot trust whether your test failure means an actual bug detection, or you just did not update the test in a longer time, executing it loses its purpose at all. Re-running outdated unit tests does not make any sense, because they do not give any information about the correctness of the implemented software. This is why you want to minimize the effort to actually update your test cases (keep them as clean as possible). To do so, you can use the following checklist to assess and improve the test cases in your project.
4. Someone else does not know what’s going on in your unit tests without telling them.
The meaning of a test case should be crystal clear from reading (i) its class name, and (ii) its method name. Additionally, (iii) you should immediately understand test code from reading it. If this is not the case, you cannot really collaboratively work on unit tests within your project team. And most importantly, if your explanation is required for these things, chances are pretty high that you cannot give this explanation in a few months. Which leads to what? Exactly – a crappy collection of garbage code that no-one understands (or wants to touch – see point 2 in this article) or wants to execute (see point 3 of this article) any more.
You can see some examples for bad/good practices in creating unit tests (written in c#) in this article. To help you implement a clear project-wide strategy, part 6 of this ebook series helps you to “structure your unit test project” correctly.
5. Different results for different test runs, although the system under tests did not change.
Try executing your test suite. Remember which test cases failed. And now try executing the test suite, again. Did the results change? Did some test cases pass in this second round, that failed in the first? Or the other way around?. If yes, you (obviously) have test cases that do not yield constant results. This can be a serious problem for unit tests (e.g. how can you reproduce a failing test case?). Check out this article for examples describing (i) problems that arise with this non-determinism and (ii) how to change test cases to make them deterministic.
What is your opinion on these 5 signs?
Did you experience some other signs for bad unit testing in your software projects?
Tell me about it in the comments section below!
written by Daniel Lehner
Daniel Lehner is a PhD student at the Johannes Keppler University in Linz, Austria. His research focus is on traceability of software information between different stages in the development cycle. During his master studies in Business Informatics at the Vienna University of Technology, Daniel worked as a consultant in the area of software quality assurance.
Information
Daniel Lehner