dotCover 2024.1 Help

Continuous Testing

Continuous testing streamlines your test-driven development practices in a number of ways. It detects changes that affect unit tests, rebuilds all affected projects, and re-runs all new and outdated tests. Everything is done automatically in the background. Only affected projects are rebuilt; only new and outdated tests are re-run.

Any unit test session can be switched to one of the continuous testing modes in the Unit Test Sessions window. To disable continuous testing, toggle the mode button off.

dotCover: Selecting continuous testing mode

Outdated tests

Continuous testing uses the concept of 'outdated tests' to define tests with currently unknown execution result. All tests that have never been executed, were aborted, or did not finish fall into this category.

Tests that have finished with a definite result (passed or failed) are relevant (not outdated), but only until you make any change in your code. To make sure that there are no more outdated tests after the change, you either need to re-run all tests or re-run tests that were affected by the change.

When continuous testing is enabled, dotCover calculates coverage of all unit tests and monitors changes in the covered code to detect whether any tests become outdated after a change. When a test becomes outdated, dotCover adds the question mark to its icon, for example icon_dirty_test.png, and changes the status indicator as well.

dotCover: Outdated tests notification

You can visualize code coverage to see which statements in your code are covered by unit tests and navigate from code to covering tests.

At any time, you can clear the information about test execution result by clicking Drop coverage results Drop coverage results on the toolbar.

Modes of continuous testing

To select the continuous testing mode means answering two questions:

1. How test run will be triggered?

You can select the preferred trigger in the ReSharper options using the main menu ReSharper | Options… | dotCover | Continuous Testing.

Continuous testing settings

There are two options available:

  • Build: tests will be started each time you build the solution.

  • Save: tests will be started each time you save the solution. In this case, dotCover has to build the related projects on its own. To help you find the balance between the build time and the relevance of the coverage data, dotCover offers you three build options:

    • Only projects related to continuous testing sessions: dotCover will build only unit test projects and projects referenced by these unit test projects.

    • Projects related to continuous testing sessions and projects included in coverage analysis: dotCover will build projects that fall into the Only projects related to continuous testing sessions category and also projects with code that is not excluded from the coverage analysis.

    • All projects in solution: dotCover will always build the entire solution.

  • Shortcut: tests will be started after you apply a shortcut (by default, unassigned).

2. What tests will be run and how?

You can select the preferred way on the toolbar in the Unit Test Sessions window using the Autostart Tests toggle button. The possible options are self-explanatory:

  • Autostart Tests: Run All Tests: all tests are run with no coverage analysis.

  • Autostart Tests: Run New and Outdated Tests: only new and outdated tests are run with no coverage analysis. This is the most resource-saving mode.

  • Autostart Tests: Cover New and Outdated Tests: new and outdated tests are run with enabled coverage analysis. This is the most resource-consuming but the most reliable mode. dotCover always keeps coverage information up-to-date.

Relevancy of coverage data

Autostart Tests: Run All Tests and Autostart Tests: Run New and Outdated Tests were created specifically to reduce the performance impact of continuous testing. When one of these modes is selected, dotCover collects no new coverage information. Therefore, it makes sense to turn them on only in case you already have relevant coverage info. If you change the code covered by tests, the Unit Test Coverage window will explicitly tell you that coverage info for some tests is outdated:

dotCover: Outdated coverage info

Coverage highlighting will also use pale colors to emphasize that it is based on possibly outdated coverage information.

dotCover: Outdated coverage highlighting

Nevertheless, if you are sure that the coverage info is still relevant, you may use these modes and postpone coverage until the info becomes really outdated.

Scope of continuous testing

Any session in the Unit Test Sessions window can be switched to the continuous testing mode. Therefore, you can have as many continuous testing scopes as many unit test sessions you have.

Continuous testing in action

As soon as you enable one of the continuous testing modes, dotCover will prompt you to run unit tests. After making sure that all your tests are green, you can close the Unit Test Sessions window and go on working on your project. You will be able to see current test status on the status bar of the main Visual Studio window.

Continuous testing status on Visual Studio toolbar

The status icon will notify you if any tests in any session with the enabled continuous testing scope fail or become outdated after your changes.

Last modified: 09 April 2024