dotCover 2020.2 Help

Coverage Analysis from the Command Line

dotCover console runner is a command-line tool distributed free of charge as an archive or as a NuGet Package ( Windows, macOS, Linux). The tool allows you to:

  • Control execution of any test runner (MSTest, NUnit, xUnit, MSpec, and so on) and record coverage of executed tests in coverage snapshots.

  • Merge coverage snapshots.
    You may need to merge snapshots if, for example, you use several different formats of unit tests in your project or solution. In this case, you will have to start console runner for all corresponding test runners and get several coverage snapshots. Then, you can use the merge command to merge the snapshots.

  • Generate coverage reports in different formats.
    Use report command to generate reports in desired formats from coverage snapshots.

Next in this topic, you will find several walkthroughs that would help you understand how to perform coverage analysis of your unit test projects using the Console Runner.

Set up Console Runner

Set up dotCover Console Runner on a server

  1. Download dotCover command line tools package from JetBrains website.

  2. Copy the archive to the machine where you are going to set up the Console Runner.

  3. Extract files from the archive to a directory, from which you are going to start the Console Runner executable file dotCover.exe.

Optionally, you can get the Console Runner as a NuGet package.

Basic scenario for .NET Framework

The following procedure illustrates the simplest case of running coverage with the Console Runner on .NET Framework projects.

Cover a unit test project (.NET Framework)

  1. Build your unit test project.

  2. Navigate to the directory where dotcover.exe is located.

  3. Type the following command in the console:

    dotcover cover /TargetExecutable="D:\Program Files\NUnit 2.6\bin\nunit-console.exe" /TargetArguments="D:\Projects\TheApplication\bin\Debug\AppTests.dll" /Output="AppCoverageReport.html" /ReportType="HTML"

    Where:

    • TargetExecutable is the path to the unit testing framework runner

    • TargetArguments are arguments passed to the runner - the compiled unit test assembly. Note that if these arguments contain paths with spaces, you should escape them with additional double quotes and a backslash, for example, ... /TargetArguments="\"D:\My Projects\My Application\bin\Debug\AppTests.dll""

    • Output is the report filename.

    • ReportType is the type of the report (in this case, we generate an HTML report)

  4. When the Console Runner exits, the AppCoverageReport.html file should appear in the same directory with the dotCover console runner. Open this file to explore the coverage results in your web browser.

  5. Alternatively, you can pass parameters in an XML file. To do so, type dotcover help cover coverage.xml in the command line.

  6. Open the generated coverage.xml file and specify the configuration parameters. For example, the configuration file can look as follows:

    <?xml version="1.0" encoding="utf-8"?> <AnalyseParams> <TargetExecutable>D:\Program Files\NUnit 2.6\bin\nunit-console.exe</TargetExecutable> <TargetArguments>D:\Projects\TheApplication\bin\Debug\AppTests.dll</TargetArguments> <Output>AppCoverageReport.html</Output> <ReportType>html</ReportType> </AnalyseParams>
  7. Use dotcover cover coverage.xml to run the coverage with the specified parameters.

Basic scenario for .NET Core

For .NET Core projects, the Console Runner provides a separate cover-dotnet command (or a shorter version dotnet). It runs dotnet.exe under coverage analysis (as if you specified the path to dotnet in --TargetExecutable).

Cover a unit test project (.NET Core)

  1. Build your unit test project.

  2. Navigate to the directory where dotcover.exe is located.

  3. Type the following command in the console:

    dotCover.exe dotnet --output=AppCoverageReport.html --reportType=HTML -- test "C:\MyProject\MainTests.csproj"

    where:

    • --output is the report filename.

    • --reportType is the type of the report (in this case, we generate an HTML report)

    • -- test "C:\MyProject\MainTests.csproj" at the end is an argument passed to dotnet. Actually, you can replace it with --TargetArguments="test \"C:\MyProject\MainTests.csproj\"" but that is much longer.

    • You've probably noticed that we're using Unix-style syntax for command-line arguments. You can use this syntax on Windows as well. For example, the following is interchangeable on Windows (but not on macOS and Linux): /TempDir:"../myDir" and --tempDir="../myDir"

  4. When the Console Runner exits, the AppCoverageReport.html file should appear in the same directory with the dotCover console runner. Open this file to explore the coverage results in your web browser.

Use dotnet.exe to run coverage analysis of .NET Core unit tests

There is an alternative to the Basic scenario of performing coverage analysis in .NET Core projects. You can do it directly with the general Microsoft driver: dotnet.exe. The main benefit here is that you don't need to download the dotCover console runner (this is a plus in, say, Docker containers).

All you have to do is to open the working directory of your project and run: dotnet dotcover test

Cover a unit test project (.NET Core) using dotnet.exe (steps 1-4 are made once per project)

  1. On the official nuget.org site, find the JetBrains.dotCover.DotNetCliTool package.

  2. Do not reference this package in your unit tests project. Instead, open its .csproj file and add a line that contains the package name and its version into an ItemGroup element. For example:

    <ItemGroup> <DotNetCliToolReference Include="JetBrains.dotCover.DotNetCliTool" Version="2019.2.0.1" /> </ItemGroup>

  3. In the command line, go to the directory containing your unit tests project.

  4. Run

    dotnet restore
    This will download dotCover command line tools to your computer.

  5. Run tests with coverage analysis:

    dotnet dotcover test

Important notes

  1. The suggested way of using the console runner is not a replacement but an addition to dotCover.exe.

  2. If you want to specify any argument you've used before with dotCover.exe, you can do this by adding the dc prefix to the argument name. For example, to specify a report type, use --dcReportType instead of --ReportType:

    dotnet dotcover test --dcReportType=XML

  3. If you configured dotCover.exe via an XML file and want to continue using it, specify a path to the file:

    dotnet dotcover test --dcXML="C:\config\config.xml"
    All parameters in the XML file that are not applicable to this runner type will be ignored.

Apply filters

If the coverage report contains some information that you are not interested in, you can apply filters, which tell the Console Runner, what should be included or excluded from the coverage report.

  • You can specify include and exclude filters in any order.

  • Independently of the order, Console Runner first applies include filters and then exclude filters.

  • If no include filters is specified explicitly, Console Runner first includes everything and then excludes what specified in exclude filters.

  • If there is some include filter, then Console Runner first excludes everything that does not match to include filter, and then applies explicit exclude filters, if any.

By default, whether you specify any exclude filters or not, Console Runner adds the following filters for system assemblies:

  • mscorlib

  • System

  • System.*

  • Microsoft.*

If necessary, you can disable these default filters with the DisableDefaultFilters command-line parameter.

There are two ways to specify coverage filters:

Set up coverage filters using XML configuration file

  • To exclude some items (modules, classes, methods) from the coverage report while keeping all others, add the corresponding entries to the ExcludeFilters node. For example:

    ... <Filters> <ExcludeFilters> <FilterEntry> <ModuleMask>AdditionalTests</ModuleMask> </FilterEntry> <FilterEntry> <ClassMask>MainTests.Unit*</ClassMask> </FilterEntry> <FilterEntry> <ClassMask>MainTests.IntegrationTests</ClassMask> <FunctionMask>TestFeature1</FunctionMask> </FilterEntry> </ExcludeFilters> </Filters>

    Here:

    • <ModuleMask>AdditionalTests</ModuleMask> - all tests from the corresponding module will be excluded. AdditionalTests here is an assembly name without extension.

    • <ClassMask>MainTests.Unit*</ClassMask> - all classes whose name starts with MainTests.Unit will be excluded.

    • <ClassMask>MainTests.IntegrationTests</ClassMask> <FunctionMask>TestFeature1</FunctionMask> - the TestFeature1 method of the MainTests.IntegrationTests class will be excluded. Note that FunctionMask must always contain method's short name. If you omit any element in FilterEntry, dotCover will consider this as a wildcard. For example, if you remove ClassMask in this particular case, dotCover will exclude all TestFeature1 methods (belonging to all modules and classes).

  • Alternatively, to include only the desired items while excluding all others of the same kind, add the corresponding entries to the IncludeFilters node.

  • Another option, is to filter out classes and methods based on their attributes. For example, to filter out methods marked with the System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute attribute, we can add the following to the coverage.xml configuration file:

    ... <AttributeFilters> <AttributeFilterEntry> <ClassMask>System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute</ClassMask> </AttributeFilterEntry> </AttributeFilters>

Alternatively, you can set up coverage filters using dotCover.exe command-line arguments.

Set up coverage filters using command-line arguments

  • To exclude/include items from the coverage analysis, you should run the console runner with the /Filters parameter. For example (for simplicity, we omit parameters related to the coverage target):

    dotCover.exe cover ... /Filters=-:module=AdditionalTests;-:type=MainTests.Unit*; -:type=MainTests.IntegrationTests;function=TestFeature1;

    This example is equivalent to the XML configuration example from above. Note that the semicolon (;) separates not only filter entries but also items inside filter entries.

  • An entry that starts with -: is responsible for excluding, and vice versa, an entry that starts with +: - for including.

  • If you need to exclude/include only a module, you can omit the module keyword:

    dotCover.exe cover ... /Filters=-:AdditionalTests;

  • To filter out classes and methods based on their attributes, you should use the /AttributeFilters parameter. For example, to filter out methods marked with the System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute attribute:

    dotCover.exe cover ... /AttributeFilters=System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute;

    The semicolon (;) works as a separator.

Change scope of coverage results

By default, when the coverage snapshot is ready, it only includes information on assemblies that were loaded for analysis, that is the assemblies that were not filtered out and that have tests. This can make the overall coverage percentage incorrect.

If necessary, you can change the scope of assemblies whose information should be added to the snapshot. To do so, use the Scope parameter. For example, to add all assemblies from your project to the snapshot, you can add the following to the configuration file:

<Scope> <ScopeEntry>ProjectFolder/**/*.dll</ScopeEntry> <ScopeEntry>ProjectFolder/**/*.exe</ScopeEntry> </Scope>

Note that everything excluded with filters is excluded anyway regardless of the specified scopes.

Cover multiple test projects

If there are several unit test projects in your solution, you can run coverage for all of them at once as described in the basic scenario. In this case, to avoid specifying full path to each assembly, you could use the TargetWorkingDir parameter when specifying test assemblies. For example:

<?xml version="1.0" encoding="utf-8"?> <AnalyseParams> <TargetExecutable>D:\Program Files\NUnit 2.6\bin\nunit-console.exe</TargetExecutable> <TargetArguments>AppTests.dll AppTests2.dll AppTests3.dll</TargetArguments> <TargetWorkingDir>D:\Projects\TheApplication\bin\Debug</TargetWorkingDir> <Output>AppCoverageReport.html</Output> <ReportType>html</ReportType> </AnalyseParams>

However, sometimes this approach may not work. For example, when unit test projects use different unit testing frameworks. In such cases we can breakdown the coverage, merging and reporting into individual steps.

Let's suppose that we have two unit test projects, TestProject1 that uses MSTest and TestProject2 that uses NUnit. To run coverage on both projects and get a single report, we perform the following steps:

dotCover: Merging coverages snapshot into a single report

Run coverage for multiple projects in separate steps

  1. Create two configuration files for running the cover (c) command on each of the tests projects. testProject1.xml for the TestProject1 that uses MSTest:

    <?xml version="1.0" encoding="utf-8"?> <CoverageParams> <TargetExecutable>C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\MSTest.exe</TargetExecutable> <TargetArguments>TestProject1.dll</TargetArguments> <TargetWorkingDir>D:\Projects\TheApplication\bin\Debug</TargetWorkingDir> <Output>Snapshot1.dcvr</Output> </CoverageParams>

    and testProject2.xml for the TestProject2 that uses NUnit:

    <?xml version="1.0" encoding="utf-8"?> <CoverageParams> <TargetExecutable>D:\Program Files\NUnit 2.6\bin\nunit-console.exe</TargetExecutable> <TargetArguments>TestProject2.dll</TargetArguments> <TargetWorkingDir>D:\Projects\TheApplication\bin\Debug</TargetWorkingDir> <Output>Snapshot2.dcvr</Output> </CoverageParams>
  2. Run the cover (c) command on each of the tests projects using the prepared configuration files: dotcover c testProject1.xml and dotcover c testProject2.xml. As a result you'll get two coverage snapshots, Snapshot1.dcvr and Snapshot2.dcvr.

  3. Run the merge (m) command to merge both snapshots:
    dotcover m merge.xml
    where the merge.xml is the configuration file:

    <?xml version="1.0" encoding="utf-8"?> <MergeParams> <Source>Snapshot1.dcvr</Source> <Source>Snapshot2.dcvr</Source> <Output>MergedSnapshots.dcvr</Output> </MergeParams>

  4. To build an HTML test report from the merged snapshots, run the report (r) command

    dotcover r report.xml
    where the report.xml is the configuration file:
    <?xml version="1.0" encoding="utf-8"?> <ReportParams> <Source>MergedSnapshots.dcvr</Source> <Output>CoverageReport.html</Output> <ReportType>html</ReportType> </ReportParams>

Find symbol files (PDB)

Locating symbol files (PDB) for the target binaries is vital for calculating coverage. If you cover unit tests or cover the startup project, dotCover easily locates symbol files using the structure of the current solution.

By default, dotCover search symbol files in the following places:

  • in the same directory where the binary file resides;

  • in the debug directory that is specified inside the binary file;

  • in all directories specified in the _NT_SYMBOL_PATH environment variable and in the registry;

If necessary, you can specify other places to look for symbol files. To do so, use the following parameters when using the cover command:

  • Use the SymbolSearchPaths parameter to provide a semicolon separated list of paths to search for symbol files. Each path can be either a directory path or a symbol server path (for example, srv*C:\LocalSymbols*http://symbolserver:33417/).

  • Use the AllowSymbolServerAccess parameter to allow dotCover access symbol servers specified either in the SymbolSearchPaths parameter, or in _NT_SYMBOL_PATH environment variable.

Also in this section

Last modified: 28 August 2020