D E V M A T E

Loading

Testing Methodology

You are here:
Estimated reading time: 3 min

Software testing is often done in a very intuitive way. In other words the testers follow their gut feeling when defining test cases. This can lead to redundant or missing test cases. To avoid this, formal testing methodologies have been developed that make sure that a system is fully tested with the least amount of test cases needed.

devmate builds upon those testing methodologies. In the next sections we will describe what theory you should know to use devmate as efficiently as possible.

Black Box vs. White Box Testing

White Box Testing

If a tester wants to perform white box testing techniques he first has to acquire knowledge of how the software is implemented. Based on the structure of the software the tester defines test cases that cover as much code as possible. E.g. a tester wants to test a method with a loop. So his goal is to derive test cases that cover the loop never, once and twice. All white box testing techniques are tightly coupled to the implementation. Most of the time the author of the code under test also implements the white box tests.

Common white box testing techniques are statement, decision/branch, condition testing or path testing.

In this whole process the specification does not necessarily play a role. White box testing tries to maximize the amount of code that is tested, but it doesn’t verify if the tested code is correct. Furthermore the tight coupling between code and white box tests causes tests to be fragile. Fragile tests tend to cause false positives when refactoring code.

Black Box Testing

Black Box Testing is called this way, because it views the software as a black box with inputs and output. A tester has no knowledge of how the system is structured inside the box. He has to derive his test cases solely from the specification. That’s why black box testing is also called specification testing. It forces the tester to concentrate on what the software does, not how it does it. Because black box testing techniques require no knowledge of the systems implementation, testing and implementation can be done by separate people. In some cases the tester doesn’t even have to have programming knowledge.

The most widespread black box testing techniques are equivalence partitioning, boundary value analysis, state transition tables and decision table testing. Two of them we will discuss in more detail in the next sections.

With black box testing we don’t know if all code is tested, but we can make sure, that it doesn’t contradict the specification.

Comparison

Black Box TestingWhite Box Testing
Tests if the code works as specifiedEnsures that all code is covered by tests
Testers can be independent testersTesters should be the developers of the software
No knowledge of implementation is neededKnowledge of implementation is required
Stable against refactoringsTend to produce false positives during refactoring
Expert has to map a specification to a test modelAutomatic generation of tests possible

Equivalence Class Partitioning

Equivalence class partitioning is a black box testing technique, that can be applied at any level of testing. The idea behind it is to find subdomains for each input parameter of the system under test where the system has the same behavior.

This subdomains are called equivalence classes and input parameters are called input factors. Equivalence class partitioning assumes that each representant (e.g. specific value) of an equivalence class of one input factor is handled equivalently by the system under test, hence the name equivalence class partitioning.

The elements of equivalence classes are concrete inputs which can be passed to the system under test. Those elements are called representatives.

Equivalence classes can be valid or invalid, which means they contain representatives that the system under test can process or representatives which will lead to an expected error in the testcase execution.

For systems with multiple input factors one representative of each input factor has to be selected to define the inputs of one test case.

Test cases with representatives taken from valid equivalence classes should not result in an error.

If a representative of an invalid equivalence class is used for a test case the result of the test execution should be an (expected) error.

Example

Assume you want to test a method which calculates the average speed. This method has two input factors. The first represents duration and the second represents distance.

Both input factors are numbers. Durations smaller or equal than 0, or distances smaller than 0 lead to an error (the method returns -1). Otherwise the result for distance/time span is returned.

One possible equivalence class partitioning looks like this:

Input FactorEquivalence ClassRepresentative
duration> 0 (valid)100
<= 0 (invalid)-10
distance>= 0 (valid)20
< 0 (invalid)-5

Based on this we can derive the following test cases:

Test Case 1Test Case 2Test Case 3
duration100-10100
distance2020-5
expected result0.2-1 (errorcode)-1 (errorcode)

Boundary Value Analysis

The idea behind boundary value analysis is that errors are often located at the boundaries of an equivalence class.

Boundary value analysis therefore focuses on testing these boundaries. This analysis represents an addition to equivalence class partitioning as it helps to find errors at the boundaries of the defined equivalence classes.

Which boundaries should be considered?

You should consider the following values for each equivalence class:

  • The specified values of the lower and upper boundaries.
  • Exceeding values near the lower and upper boundaries (e.g. upper boundary value +1, lower boundary value -1).
  • Normal values near the lower and upper boundaries (e.g. upper boundary value -1, lower boundary value +1).

Let’s try to find some good boundary values for our example defined in the previous chapter ‘Equivalence Class Partitioning’.

Representatives
Input FactorEquivalence ClassExc. Lower
Boundary
Lower
Boundary
NormalUpper
Boundary
Exc. Upper
Boundary
duration> 0 (valid)covered by ec “<= 0 (invalid)”Double.Epsilon(*)100.0Double.MaxValueDouble.PositiveInfinity
<= 0 (invalid)Double.NegativeInfinityDouble.MinValue-100.00covered by ec “> 0 (valid)”
distance>= 0 (valid)covered by ec “< 0 (invalid)”020.0Double.MaxValueDouble.PositiveInfinity
< 0 (invalid)NegativeInfinitymin_int-100.0-1covered by ec “> 0 (valid)”
Was this article helpful?
Dislike 0
Views: 78

Continue reading

Previous: Process Overview

We use cookies to give you the best online experience. By agreeing you accept the use of cookies in accordance with our cookie policy.

Privacy Settings saved!
Privacy Settings

When you visit any web site, it may store or retrieve information on your browser, mostly in the form of cookies. Control your personal Cookie Services here.

GetResponse, Google Analytics

We use Google Analytics for marketing purposes. You can disable these cookies.
  • __utmz
  • __utma
  • _ga
  • _gat

We use GetResponse for marketing purposes. This service cannot be disabled, otherwise the website functions will be limited.

Decline all Services
Accept all Services
Get Free Access Now to
9 eBooks!
All about Automated Software Testing
Proven experts
Learn to save up to 75% of your test efforts
Get Free Access Now!
Get Access Now! & Save 50%
Personal Trainer FREE Nutrition Custom Workout App
Get Access Now!
eBook Download
Enter your details to get your free ebook!
All about Automated Software Testing
Download Free Ebook
SUBSCRIBE
MY WEB
NEWSLETTERS
Lorem ipsum dolor sit amet, consectetur adipiscing