D E V M A T E

Loading

Illustration good vs. bad unit tests

In this post, I describe the difference between good and bad unit tests based on some examples created in C# using the NUnit unit testing framework.
For these examples, test cases are created based on the following code unit (download example code).

Example Class Person - static function makeChild
 

Good Unit Tests

In general, the following aspects characterize good unit tests.
  • Test only one Unit, that is decoupled from the rest of the system (e.g. data stores).
  • Mock/Stub dependent Units to enable decoupling
  • Use Patterns to allow readability
  • Are regularly reviewed and updated to the current state of implementation
  • Are repeatable
 
The following examples showcase how these aspects are applied in practice, using the NUnit testing framework for C#.
The examples use the Naming Pattern “MethodName_StateUnderTest_ExpectedBehavior” [https://medium.com/@stefanovskyi/unit-test-naming-conventions-dd9208eadbea], and the Code Pattern AAA (Arrange-Act-Assert) [explained here]. The individual blocks of the AAA are each marked by comments.
 
good test example code 1
 
  • Based on its name, it can be derived that this tests checks that a child is created by the method createChild, if father and mother are fertile
  • In the Arrange-Section of the Code, required objects are instantiated and mocked
  • In the Act-Section of the code, a single call to the tested code unit (method) is performed.
  • In the Assert-Section of the code, the expected condition is checked (an actual object is returned by the method)
good test example code 2
 
  • Based on its name, it can be derived that this tests checks that a child created by the method makeChild() has age of 0
  • In the Arrange-Section of the Code, required objects are instantiated and mocked
  • In the Act-Section of the code, a single call to the tested code unit (method) is performed.
  • In the Assert-Section of the code, the expected condition is checked (an actual object is returned by the method)
good test example code 3
 
  • Based on its name, it can be derived that this tests checks that an exception is thrown by the method createChild if father is not fertile
  • In the Arrange-Section of the Code, required objects are instantiated and mocked
  • In the Act-Section of the code, a single call to the expected method is performed.
  • In the Assert-Section of the code, the expected condition is checked (an actual object is returned by the method)
 

Bad Unit Tests

  • Have several sources of failure
  • Have Dependencies to other code units
  • Are created Ad-Hoc, without any patterns in mind
  • Are written in Spaghetti-Code (no structure or pattern observable)
  • Are not updated when requirements or implementation of the tested system changes
  • Yield different results when being executed several times (non-determinism)
The above examples are used to showcase examples of such bad unit tests, again using the NUnit testing framework for C#.
 
bad test example code 1
 
What’s the issue here?
  • If this test fails, there is no information on what caused this failure.
  • Does actualChild have the wrong age? Does actualChild have the wrong father? Does actualChild have the wrong mother? Is the implementation of getAge/getFather/getMother wrong? You have to dig into the individual Assert Statements, eventually different code fragments (implementation of createChild(), implementation of getAge(), etc.) to find the source of a failing test case, which is not the intention of a Unit Test! 
  • If actualChild equals null, an Exception is thrown by the test case implementation (not by the tested code unit), which also makes it makes it more difficult to find the reason for a failing test case
  • The test outcome is also implicitly dependent on methods called by the createChild-method (isFertile, in particular). The result of createChild depends on the logic implemented in isFertile.
  • Although the Arrange-Act-Assert pattern is followed, it is difficult to observe the individual parts, compared to the good unit test examples above, because of the missing comments
 
bad test example code 2
 
What’s the issue here?
  • This test seems to be not up-to-date. Maybe, in an earlier version of the tested code unit, a newly created child had an age of 1. However, in the current version, age must be 0, which makes this test outdated.
  • Again, missing comments make it more difficult to observe the individual parts of the used Arrange-Act-Assert pattern.
 
bad test example code 3
 
What’s the issue here?
  • Although it seems like this case can increase productivity by covering several cases at once (check if code unit responds correctly to changing fertility of father and mother), you should by now already have a feeling that this is a bad thing when doing unit testing.
  • Imagine running this test case, and receiving a test failure as a result.
  • Can you tell why this test failed? What is the relevant part of the tested code unit that caused the failure?
  • You cannot anwer these questions, because you cannot even tell which parts of the tested code unit were executed (was an exception thrown? was a child created?)
  • If you think further, if this test passes in one run, and fails in the next run, you don’t know if this is because of a change in the tested code unit, or if a path of the code unit is executed that was incidentally ignored before
  • And most importantly, if this test passes, you cannot know whether all parts of the tested code unit work, because you only executed one arbitrary path (e.g. if an exception was thrown, you do not know if the code unit actually creates a child in case of both parents are fertile)
Common Issues
  • In these examples, it can be observed that test cases are named intuitively, without any pattern in mind that helps understand the contents of a test case. Most probably, a tester started implementing a first test case, not concerning about the name, and then iteratively continuing to find names that are not taen yet. Although comments are used to describe what happens in code blocks, there is no common pattern observable. This makes it difficult to (i) grasp the contents of a test case, and (ii) adapt its contents afterwards.
You can download the code examples as a .cs-file here: good-and-bad-unit-test-examples
 
What do you think about these examples?
Do you have better ones?
Let’s discuss in the comments section below.
 
written by Daniel Lehner

Leave a Comment

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Want to experience the magic of devmate?
Join A Live Product Demo
Get Free Access Now to
9 eBooks!
All about Automated Software Testing
Proven experts
Learn to save up to 75% of your test efforts
Get Free Access Now!
Get Access Now! & Save 50%
Personal Trainer FREE Nutrition Custom Workout App
Get Access Now!
eBook Download
Enter your details to get your free ebook!
All about Automated Software Testing
Download Free Ebook
SUBSCRIBE
MY WEB
NEWSLETTERS
Lorem ipsum dolor sit amet, consectetur adipiscing