dotCover 2020.1 Help

Continuous Testing

Continuous testing streamlines your test-driven development practices in a number of ways. It detects changes that affect unit tests, rebuilds all affected projects, and re-runs all new and outdated tests. Everything is done automatically in the background. Only affected projects are rebuilt; only new and outdated tests are re-run.

Any unit test session can be switched to one of the continuous testing modes in the Unit Test Sessions window. To disable continuous testing, toggle the mode button off.

dotCover: Selecting continuous testing mode

Outdated tests

Continuous testing uses the concept of 'outdated tests' to define tests with currently unknown execution result. Obviously, all tests that have never been executed, were aborted, or did not finish for some reason fall into this category.

Tests that have finished with a definite result (passed or failed) are relevant (not outdated), but only until you make any change in your code. To make sure that there are no more outdated tests after the change, you either need to re-run all tests or re-run tests that were affected by the change.

When continuous testing is enabled, dotCover calculates coverage of all unit tests and monitors changes in the covered code to detect whether any tests become outdated after a change. When a test becomes outdated, dotCover adds the question mark to its icon, for example icon dirty test png, and changes the status indicator as well.

dotCover: Outdated tests notification

You can visualize code coverage to see which statements in your code are covered by unit tests and navigate from code to covering tests.

At any time, you can clear the information about test execution result by clicking Drop coverage resultsDrop coverage results on the toolbar.

Modes of continuous testing

To select the continuous testing mode means answering two questions:

  1. How continuous testing will be triggered?
    Or, in other words, when will be tests autostarted? After you explicitly build your solution or after you save it? You can select the preferred trigger in the ReShaper options using the main menu ReSharper | Options… | dotCover | Continuous Testing.

  2. What tests will be autostarted and how?
    You can select the preferred way on the toolbar in the Unit Test Sessions window using the Autostart Tests toggle button. Possible options are self-explanatory:

    • Autostart Tests on [Build | Save]: Run All Tests: all tests are run with no coverage analysis.

    • Autostart Tests on [Build | Save]: Run New and Outdated Tests: only new and outdated tests are run with no coverage analysis. This is the most resource-saving mode.

    • Autostart Tests on [Build | Save]: Cover New and Outdated Tests: new and outdated tests are run with enabled coverage analysis. This is the most resource-consuming but the most reliable mode. dotCover always keeps coverage information up-to-date.

Important! Autostart Tests on [Build | Save]: Run All Tests and Autostart Tests on [Build | Save]: Run New and Outdated Tests were created specifically to reduce the performance impact of continuous testing. When one of this modes is selected, dotCover collects no new coverage information. Therefore, it makes sense to turn them on only in case you already have relevant coverage info. If you change the code covered by tests, the Unit Test Coverage window will explicitly tell you that coverage info for some tests is outdated:

dotCover: Outdated coverage info

Coverage highlighting will also use pale colors to emphasize that it is based on possibly outdated coverage information.

dotCover: Outdated coverage highlighting

Nevertheless, if you are sure that the coverage info is still relevant, you may use these modes and postpone coverage until the info really becomes outdated.

Scope of continuous testing

Any session in the Unit Test Sessions window can be switched to the continuous testing mode. Therefore, you can have as many continuous testing scopes as many unit test sessions you have.

Continuous testing in action

As soon as you enable one of the continuous testing modes, dotCover will prompt you to run unit tests. After making sure that all your tests are green, you can close the Unit Test Sessions window and go on working on your project. You will be able to see current test status on the status bar of the main Visual Studio window.

Continuous testing status on Visual Studio toolbar

The status icon will notify you if any tests in any session with the enabled continuous testing scope fail or become outdated after your changes.

Last modified: 08 May 2020