By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.

How DevMate changes your approach to unit and integration tests without changing how you work.

We facilitate requirements led testing in the pursuit of long term quality.

Problems and challenges

Before we explain how DevMate works, it's worth highlighting some of the key challenges that Developers and Test Engineers face.
Test Driven Development, Code First Testing or Lip Service Testing?

Whichever category you fall into, DevMate provides a new, requirements led approach to testing. This helps not only developers but also Product Owners, QA Engineers and Domain Experts participate in the unit and integration testing process.

A problem shared is a problem halved, so developers can spend more time writing functional code and less time writing test code.

What’s wrong with 100% Code Coverage?

High code coverage is no guarantee of quality.

  • How do you know if your code really covers all requirements?
  • Have you written tests that don’t cover important test cases?
  • Have you wasted time by writing tests that cover unnecessary or duplicated test cases?
I can write tests by hand very quickly. Is DevMate quicker?

For simple scenarios, you might be able to write tests as quickly but not for more complex ones.

However, there’s a lot more to quality than throwing together a test.

  • Have your tests really met the requirements?
  • How easy is it to visualize the background and requirements for new developers and even Product Owners, QA Staff and Domain Experts?
  • How understandable will your code and tests be a year or more from now when things aren’t fresh in anyone’s mind?
How many test cases should a test have?

Most people do this by gut feeling. For anything other than the most trivial cases, you will either end up missing an important test case or two or covering unnecessary ones.  

DevMate uses Equivalence Classes and Representatives to automatically generate the optimal number of test cases using Pairwise Combination, a state of the art approach to software testing.  

Is requirements led testing hard?

Less than 4% of companies wholeheartedly follow Test Driven Development practices. And that’s because it is indeed hard! It demands change and serious discipline during the development process.

However, you don't need to adopt TDD to see big gains in quality. DevMate will help any company, whether following TDD principles or not, to achieve significant quality gains in a way that is genuinely approachable.

How DevMate works

Requirements are key

Code should be based on a set of requirements, hopefully well documented. DevMate helps you to

  • take those requirements
  • map them to so-called Equivalence Classes and Representatives
  • generate test cases automatically or manually
  • generate instantly runnable tests.

We'll skip to ahead and see what a configured test definition looks like. An experienced DevMate needs five to ten minutes to fully configure.

fully configured test definition

Let's look at how we got there.

Testing a method

From your IDE, you right click the method(s) you want to test and select Test with DevMate. Let's start configuring.

Below is the basic structure when we start to configure a test.

Newly created test
  1. DevMate detects the name of the parent class AllTogether.
  2. The method to you want to test (checkCredit).
  3. The method's arguments

Input Equivalence Classes

You then define Equivalence Classes. Within each Equivalence Class, you define one or more Representative values.

For our example, we are offering credit to people between 18 and 25, and 26 and older. We offer different levels of credit and maximum repayment periods depending on their age.

We instantiate AllTogether and then call checkCredit() with the amount and duration of the loan.

Instances and inputs definitions
  1. We create an Equivalence Class for the 18-25 range. Below that we've also done the same for 26+.
  2. Using a popup dialog, we set the Representative value using AllTogether's constructor.
  3. We do the same for the creditSought parameter. We have first created a valid Equivalence Class that is valid for any age.
  4. We then created invalid ones with suitable Representatives.

This process is repeated as for all parameters in order to meet the requirements.

Output Equivalence Classes

We then define Equivalence Classes for the output or result of the method call.

Output definitions
  1. The Output section deals with everything that can happen when the method terminates.
  2. DevMate knows that checkCredit() returns an object of type CreditReturnStatus. We've added a valid and an invalid Equivalence Class and defined a Representative for each one.
  3. DevMate also handles Exceptions very elegantly.
  4. You can also code your own Custom Assertions if you want to handle more complex examples in code.

Auto Generating Test Cases

Once the Equivalence Classes have been defined, we can automatically or manually generate the test cases. You will usually let DevMate do this automatically. It uses a best practice approach called Pairwise Combination. Put simply, it only generates the test cases you need. No more and no less.

Auto generate test cases
  1. We can auto generate the test cases by pressing this button
  2. It has generated six test cases for us as columns. We have given friendly labels to each one. This makes the test definition easy to understand, even years later.
  3. For each test case, we click on the radio button in each of the Instances, Input and Output sections, thereby specifying what values we want to use and what we expect to be output.
  4. That's it. We can now simply generate the test code with another press of a button.

Generated Test Code

DevMate generates parameterized test code. This is a highly effective and best practice way of testing. Each test case is passed as a set of parameters into standard testing functions.

Auto generated test code
  1. Shows an example of how a Custom Assertion can be specified and quickly edited in your IDE.
  2. This example has two Scenarios (see below). This one is called ValidatePersonTest.
  3. And the second Scenario is the CheckCreditTest we showed earlier.
  4. Everything below this line does not really need to be looked at, but it's easy to understand if you want or need to.
  5. We're showing IntelliJ and Java but the same principle applies to Visual Studio and C#. We can run an individual Scenario's test, if you have more than one.
  6. Or you can run all Scenarios.

Running the Test

So let's run the test from the IDE. This uses whatever test runner you like to use. This also means that it fits into CI/CD pipelines as we are not performing any black magic that can get in the way.

run test

This example shows the test having run and two Scenarios and each one's test cases having executed and passed.

Notice how the tests are nicely labelled. Any labelling we do in DevMate not only makes it easy to understand in DevMate, but the code and test output is automatically documented as well.

Multiple Scenarios

You will often need flexibility when defining a test. This means you will want to perform various sub-tests within the context of a single, larger test. This is where Scenarios come in.

A common situation is where you want to validate an instance with all possible valid and invalid cases. After this, you want to perform a method test but only with valid cases. If you combined everything into a single scenario, you would end up with a large number of test cases and a more complex setup.

Multiple scenario configuration
  1. The currently selected Scenario is ValidatePerson.
  2. We've defined a set of valid and invalid Equivalence Classes for instantiating AllTogether.
  3. We then call the validatePerson() method and specify what outputs match each instantiation for each test case.
  4. We would then switch to the CheckCredit scenario, which tests the checkCredit() method we looked at earlier.

Multiple Methods

Within a scenario, you can call different methods sequentially. These methods can run independently or the output of one method can be passed to a later method.

Multiple method configuration
  1. We first call multiplyPositiveByTen() and pass in a value of 3.
  2. We then call multiplyPositiveByTen() again (you can call any method you like).
  3. Notice how we are taking the return value from the first method call (labelled as M1) and using this as the input value for the int input parameter. Methods do not have to be linked like this.

Tutorials

We have a large collection of tutorials that explain how to use DevMate step by step.

Of magic and asymmetric relationships

We're bigs fans of cutting edge technology. However, we also know that too much magic, used the wrong way, can create many more problems than solutions.
Witch on broomstick
Pulling back the curtain
Expand text
1 million tests ≠ Quality
Expand section arrow
An example AI generated test
Expand section arrow
The problem with magic tools
Expand section arrow
Asymmetric relationships
Expand section arrow