Frequently Asked Questions
What are the prerequisites for devmate?
To use devmate your PC should meet the following requirements:
- Windows 8 or newer
- VisualStudio 2019 Version >= 16.3.0
VSIX Installer states: “This extension is not installable on any currently installed products.”
devmate currently only supports Visual Studio 2019. Make sure you have Visual Studio 2019 installed.
What is a test model?
From a method you can open an editor in which you can define equivalence classes, representatives and test cases. This information is called test model.
What is a .tmdl file?
The tmdl file (short for test model file) contains the data for the method you want to test. It includes input factors, equivalence classes, representatives and test cases with all it’s values and properties. Edit it with devmate in Visual Studio.
What is equivalence class partitioning?
Equivalence class partitioning is a software testing technique that divides the input data of a software unit into partitions of equivalent data from which test cases can be derived. https://en.wikipedia.org/wiki/Equivalence_partitioning
How do I generate tests with devmate?
To generate tests with devmate you have to do three steps.
- First you generate a test model for a method by right clicking into the method and choosing ‘Test with devmate’. devmate should open.
- Here you perform the second step. Add input factors, equivalence classes, representatives and use them to generate test cases. For each test case you have to enter the expected value and possible exceptions that are thrown. Optional you can also check for side effects.
- The final step is to generate test code. First select for which framework you want to generate test code (XUnit, NUnit, MSTest). Then click the ‘Generate Test Code’ button in the equivalence class editor.
Should I add the generated test code to my VCS?
Yes. We encourage you to check-in the test code into your VCS. devmate allows you to modify generated code and merge any changes in the test model upon newly generating code. You just have to make sure not to touch the test-data in the code itself or dramatically changing the structure of the code.
How can I set a string parameter to null?
You can set the value of a string to null by entering the text null. If you want to pass the string “null”, enter the text “null” (with the quotes).
How can I set a char variable to a non-alphanumeric character?
Characters can be passed in two ways. You can either enter a character under single-quotes (e.g. ‘2’) or you can enter the numeric value of a character (e.g. 50). A non-alphanumeric character can only be passed by its numeric value.
How do I perform asserts on input parameter?
We currently don’t support checking of input parameters.
How do I check for exceptions that are thrown?
Every method can potentially throw exceptions. For each test case you can therefore define an exception as a potential return value in the footer of each test case next to the ‘Throws:’ label.
How do I check for side effects?
Every method parameter that is annotated by ‘ref’ or ‘out’ in C# can potentially have side effects within the method execution. Because of that devmate allows you to check for potential side effects. Each test case therfore has a field ‘Expected Side Effects’.
In the side effects dialog you can check which reference input parameter should be checked for side effects and set the expected value that this parameter should have after the execution of the method.
How do I create a Boilerplate template?
Choose a method for which you would like to create a boilerplate template. Right click on it and choose ‘Create Boilerplate with devmate’. Select the favoured unit testing framework (XUnit, NUnit, MSTest).
“Generate Test Case” does not work?
Generating test cases requires at least one positive equivalence class for each input parameter and at least one representative for each equivalence class.
Which unit testing frameworks does devmate support?
devmate currently supports XUnit, NUnit and MSTest.
“Generate Test Code” does not work?
Generating code requires that all your test cases have a selected representative for each input factor.
Why can’t I create a test model?
You can create a test model by doing a right click inside a method. Select ‘Test with devmate’. We do not yet support:
- getters and setters for properties
- generic methods or methods in generic classes
I closed the introduction video tab. How can I reopen it?
Click in the top task bar in Visual Studio on Extensions>devmate>Introduction Video.
Why can’t I define expected side effects?
Expected side effects for a test case can only be defined if your test case has a selected representative for each input factor.
Why should I use black-box testing techniques?
Testing software can be roughly divided into two approaches. White-box testing and black-box testing. White-box focuses on the implemented code. It makes sure that as much code as possible is reached by test at least once. What this approach does not cover is, if the implementation meets the requirements. Checking this condition is the main goal of black-box testing. Here you ignore the implementation and only use the requirements to derive test cases. Therefore if you want to check if a piece of code does what its supposed to do you should use black-box testing.
How can devmate improve my software development process?
By focusing solely on black-box testing, devmate encourages you to implement tests that are independent from your code. This results in tests that have not been changed when you refactor your code. Refactoring is major part of the software development process and adapt the test suite to this changes can be very hard. devmate will:
- Decrease your testing effort during refactoring.
- Make sure that your software behaves the same before and after refactoring.
I found a bug or have a feature request, how can I report it?
Please send us an email to: firstname.lastname@example.org
Where can I find a more detailed documentation of devmate? Do you have tutorials?
Yes on our webpage we have created a whole section that offers a detailed description of devmate and tutorials. You can find it under https://www.devmate.software/docs/