Software testing is often done in a very intuitive way. In other words the testers follow their gut feeling when defining test cases. This can lead to redundant or missing test cases. To avoid this, formal testing methodologies have been developed that make sure that a system is fully tested with the least amount of test cases needed.

devmate builds upon those testing methodologies. In the next sections we will describe what theory you should know to use devmate as efficiently as possible.

Black Box vs. White Box Testing

White Box Testing

If a tester wants to perform white box testing techniques he first has to acquire knowledge of how the software is implemented. Based on the structure of the software the tester defines test cases that cover as much code as possible. E.g. a tester wants to test a method with a loop. So his goal is to derive test cases that cover the loop never, once and twice. All white box testing techniques are tightly coupled to the implementation. Most of the time the author of the code under test also implements the white box tests.

Common white box testing techniques are statement, decision/branch, condition testing or path testing.

In this whole process the specification does not necessarily play a role. White box testing tries to maximize the amount of code that is tested, but it doesn’t verify if the tested code is correct. Furthermore the tight coupling between code and white box tests causes tests to be fragile. Fragile tests tend to cause false positives when refactoring code.

Black Box Testing

Black Box Testing is called this way, because it views the software as a black box with inputs and output. A tester has no knowledge of how the system is structured inside the box. He has to derive his test cases solely from the specification. That’s why black box testing is also called specification testing. It forces the tester to concentrate on what the software does, not how it does it. Because black box testing techniques require no knowledge of the systems implementation, testing and implementation can be done by separate people. In some cases the tester doesn’t even have to have programming knowledge.

The most widespread black box testing techniques are equivalence partitioning, boundary value analysis, state transition tables and decision table testing. Two of them we will discuss in more detail in the next sections.

With black box testing we don’t know if all code is tested, but we can make sure, that it doesn’t contradict the specification.

Comparison

Equivalence Class Partitioning

Equivalence class partitioning is a black box testing technique, that can be applied at any level of testing. The idea behind it is to find subdomains for each input parameter of the system under test where the system has the same behavior.

This subdomains are called equivalence classes and input parameters are called input factors. Equivalence class partitioning assumes that each representant (e.g. specific value) of an equivalence class of one input factor is handled equivalently by the system under test, hence the name equivalence class partitioning.

The elements of equivalence classes are concrete inputs which can be passed to the system under test. Those elements are called representatives.

Equivalence classes can be valid or invalid, which means they contain representatives that the system under test can process or representatives which will lead to an expected error in the testcase execution.

For systems with multiple input factors one representative of each input factor has to be selected to define the inputs of one test case.

Test cases with representatives taken from valid equivalence classes should not result in an error.

If a representative of an invalid equivalence class is used for a test case the result of the test execution should be an (expected) error.

Example

Assume you want to test a method which calculates the average speed. This method has two input factors. The first represents duration and the second represents distance.

Both input factors are numbers. Durations smaller or equal than 0, or distances smaller than 0 lead to an error (the method returns -1). Otherwise the result for distance/time span is returned.

One possible equivalence class partitioning looks like this:

Based on this we can derive the following test cases:

Boundary Value Analysis

The idea behind boundary value analysis is that errors are often located at the boundaries of an equivalence class.

Boundary value analysis therefore focuses on testing these boundaries. This analysis represents an addition to equivalence class partitioning as it helps to find errors at the boundaries of the defined equivalence classes.

Which boundaries should be considered?

You should consider the following values for each equivalence class:

• The specified values of the lower and upper boundaries.
• Exceeding values near the lower and upper boundaries (e.g. upper boundary value +1, lower boundary value -1).
• Normal values near the lower and upper boundaries (e.g. upper boundary value -1, lower boundary value +1).

Let’s try to find some good boundary values for our example defined in the previous chapter ‘Equivalence Class Partitioning’.