dotCover 2016.3 Help

Continuous Testing

Continuous testing streamlines your test-driven development practices in a number of ways. It detects changes that affect unit tests, rebuilds all affected projects, and re-runs all affected (dirty) tests. Everything is done automatically in the background. Only affected projects are rebuilt; only dirty tests are re-run.

You can control continuous testing in the Continuous Testing Session window ( ReSharper | Windows | Continuous Testing Session ).


In this topic:

Dirty Tests

Continuous testing uses the concept of 'dirty tests' to define tests with currently unknown execution result. Obviously, all tests that have never been executed, were aborted, or did not finish for some reason fall into this category.

Tests that have finished with a definite result (passed or failed) are not dirty, but only until you make any change in your code. To make sure that there are no more dirty tests after the change, you either need to re-run all tests or re-run tests that were affected by the change.

When continuous testing is enabled, dotCover calculates coverage of all unit tests and monitors changes in the covered code to detect whether any tests become dirty after a change. According to the selected mode of continuous testing, dotCover can then automatically re-run dirty tests or just detect them. In 'detection' modes, dotCover adds the question mark to the icons of unit tests that become dirty after your changes, e.g. /help/img/dotnet/2016.3/icon_dirty_test.png, and changes the status icon to /help/img/dotnet/2016.3/status_icon_dirty_tests.png.

You can visualize code coverage to see which statements in your code are covered by unit tests and navigate from code to covering tests.

At any time, you can clear the information about test execution result by clicking Drop Results /help/img/dotnet/2016.3/ThemedIcon.JetBrainsDotCoverPresentationVsResourcesContinuousTestingTrash.Screen.[Gray].png on the toolbar.

Modes of Continuous Testing

By default, continuous testing is disabled. To enable it, you need to choose one of the modes of continuous testing, which define when to recalculate dirty tests and what to do with them. You can switch modes of continuous testing on the toolbar. Names of the modes are self-explanatory:

  • Build and run on save - this is the recommended mode, however it is the most resource intensive.

    Note that dotCover can build affected projects either with native Visual Studio build management or with ReSharper Build (which is disabled by default). You can choose your build preference on the Tools | ReSharper Build | General page of dotCover options. It is recommended to use ReSharper Build, especially on medium-sized and large solutions.

  • Build and detect on save - this mode allows you to save resources by not running unit tests. However, you will be able to see the number of dirty tests in the window and on the status icon. You can always run the dirty tests by right-clicking the status icon and choosing Run Dirty Tests /help/img/dotnet/2016.3/ThemedIcon.JetBrainsDotCoverPresentationVsResourcesContinuousTestingRunDirtyTests.Screen.[Gray].png, or clicking the corresponding button on the toolbar.
  • Run on build - this mode allows you to save resources by not building related projects every time you save changes. dotCover will calculate which tests become dirty and run them after you build one of the related projects.
  • Detect on build - this mode is the most resource-saving. In this mode, dotCover calculates which tests become dirty after you build one of the related projects. You can recognize and run dirty tests the same way as you would do in the Build and detect on save mode.
  • Detect and run on explicit command - in this mode, continuous testing is effectively disabled. dotCover does nothing until you click Detect Dirty Tests /help/img/dotnet/2016.3/ThemedIcon.JetBrainsDotCoverPresentationVsResourcesContinuousTestingDetect.Screen.[Gray].png or Run Dirty Tests /help/img/dotnet/2016.3/ThemedIcon.JetBrainsDotCoverPresentationVsResourcesContinuousTestingRunDirtyTests.Screen.[Gray].png buttons on the toolbar.

Scope of Continuous Testing

By default, continuous testing session includes all unit tests in your solution. If some tests do not appear in the session, just build your solution once and they will be discovered.

If you do not want to monitor some tests, you can exclude them from scope. The status of unit testing session is not affected by excluded tests either.

To exclude tests from continuous testing session, select tests or any grouping items in the test tree and click Exclude from Scope /help/img/dotnet/2016.3/ThemedIcon.JetBrainsDotCoverPresentationVsResourcesContinuousTestingExcludeFromScope.Screen.[Gray].png on the toolbar.

On the status bar, right under the toolbar, you will see the number of in-scope /help/img/dotnet/2016.3/ThemedIcon.JetBrainsDotCoverPresentationVsResourcesContinuousTestingInScope.Screen.[Gray].png and out-of-scope /help/img/dotnet/2016.3/ThemedIcon.JetBrainsDotCoverPresentationVsResourcesContinuousTestingOutOfScope.Screen.[Gray].png tests. If you want to add some out-of-scope tests back into scope, click the Show out of Scope Tests /help/img/dotnet/2016.3/ThemedIcon.JetBrainsDotCoverPresentationVsResourcesContinuousTestingOutOfScope.Screen.[Gray].png on the status bar, choose tests that you want to add to scope, and then click include in Scope /help/img/dotnet/2016.3/ThemedIcon.JetBrainsDotCoverPresentationVsResourcesContinuousTestingIncludeInScope.Screen.[Gray].png on the toolbar.

Continuous Testing in Action

As soon as you enable one of the continuous testing modes, dotCover will prompt you to run unit tests. After making sure that all your tests are green, you can close the Continuous Testing Session window and go on working on your project. You will be able to see current test status on the status bar of the main Visual Studio window:

Continuous testing status on Visual Studio toolbar

Depending on the selected continuous testing mode, the status icon will notify you if any tests in scope of the continuous testing session fail or become dirty after your changes.

  • /help/img/dotnet/2016.3/status_icon_ct_passed.png - all tests in scope of the continuous testing session have passed, and there were no changes that affect them.
  • /help/img/dotnet/2016.3/status_icon_ct_failed.png - some tests have failed in the last test run, and there were no more changes that affect tests in scope of the continuous testing session.
  • /help/img/dotnet/2016.3/status_icon_dirty_tests.png - some tests became dirty after the last test run. this status icon may appear together with the 'failed' icon if there were failed tests in the previous run. You can run dirty tests by right-clicking the status icon and choosing Run Dirty Tests /help/img/dotnet/2016.3/ThemedIcon.JetBrainsDotCoverPresentationVsResourcesContinuousTestingRunDirtyTests.Screen.[Gray].png.
  • /help/img/dotnet/2016.3/status_icon_ct_in_progress.png - discovering or running dirty tests is in progress.

For status bar icons other than 'passed', dotCover displays the number of tests in this status. You can subsequently click this number to navigate to these tests in the editor.

You can also double-click on the status icon to see details. dotCover will open the Continuous Testing Session window; If the status icon is 'dirty' or 'failed', dotCover will automatically filter tests to display only the relevant ones.

If necessary, you can observe and control execution of unit tests and study coverage analysis data right in the Continuous Testing Session window.

Studying execution log

Continuous Testing Session window includes test execution log that lets you separate problems related to test execution process from unit tests code.
You will normally need to check execution log if some of your tests have the 'Inconclusive' state after execution, which could be a sign of an error occurring in the test runner.

If you have errors reported either by the test runner or dotCover, the number of errors is displayed on the right side of the window's status bar, else OK is shown.
To display or hide the execution log, click the Log /help/img/dotnet/2016.3/ThemedIcon.SessionLog.Screen.[Gray].png button.

By default, only events with the 'Error' severity level are logged. If necessary, you can change the minimum severity level. To do so, either right-click the log area and choose the desired severity level or use the Show log entries... selector on the Tools | Unit Testing page of dotCover options.

You can also copy the whole log to the clipboard by choosing the corresponding command in the context menu.

See Also

Last modified: 11 January 2017