Running Coverage Analysis from the Command LIne
This topic provides several walkthroughs that would help you understand how to apply the Console Runner to your unit test projects.
In this topic:
- Setting up Console Runner
- Basic scenario
- Applying filters
- Changing scope of coverage results
- Covering multiple test projects
Setting up Console Runner
To set up dotCover Console Runner on a server
-
Do one of the following:
- In Visual Studio, go to and click Get Server Package. dotCover will generate an archive with all necessary files.
- Download dotCover command line tools package from JetBrains website.
- Copy the archive to the machine where you are going to set up the Console Runner.
- Extract files from the archive to a directory, from which you are going to start the Console Runner executable file (dotCover.exe).
Optionally, you can add the path to the Console Runner executable as a system path
to make the
dotCover
command available everywhere.
Basic scenario
The following procedure illustrates the simplest case of running coverage with the Console Runner.
To cover a Single Unit Test Project
- Build your unit test project.
- Navigate to the directory where dotcover.exe is located.
-
Type the following command in the console:
dotcover analyse /TargetExecutable="D:\Program Files\NUnit 2.6\bin\nunit-console.exe" /TargetArguments="D:\Projects\TheApplication\bin\Debug\AppTests.dll" /Output="AppCoverageReport.html" /ReportType="HTML"
Where:
-
When the Console Runner exits, the
AppCoverageReport.html
file should appear next todotcover.xml
. Open this file to explore the coverage results in your web browser. -
Alternatively, you can pass parameters in an XML file.
To do so, type
dotcover help analyse coverage.xml
on the command line. -
Open the generated coverage.xml file and edit ti to specify the configuration parameters. For example, the configuration file can look as follows:
<?xml version="1.0" encoding="utf-8"?> <AnalyseParams> <TargetExecutable>D:\Program Files\NUnit 2.6\bin\nunit-console.exe</TargetExecutable> <TargetArguments>D:\Projects\TheApplication\bin\Debug\AppTests.dll</TargetArguments> <Output>AppCoverageReport.html</Output> <ReportType>html</ReportType> </AnalyseParams>
-
Type
dotcover analyse coverage.xml
to run the coverage with the specified parameters.
Applying filters
If the coverage report contains some information that you are not interested in, you can apply filters, which tell the Console Runner, what should be included or excluded from the coverage report.
You can specify include and exclude filters in any order.
Independently of the order, Console Runner first applies include filters and then exclude filters.
If no include filters is specified explicitly, Console Runner first includes everything and then
excludes what specified in exclude filters.
If there is some include filter, then Console Runner first excludes everything that does not match to
include filter, and then applies explicit exclude filters, if any.
By default, whether you specify any exclude filters on not, Console Runner adds the following filters for system assemblies:
If necessary, you can disable these default filters with the
DisableDefaultFilters
parameter.
In the first steps of the
previous procedure
we have generated the configuration file with the
dotcover help analyse coverage.xml
command.
In this file, there are two nodes commented out:
Filters
and
AttributeFilters. You can use these nodes to apply filters.
To set up coverage filters
-
To exclude some items from the coverage report while keeping all others,
add the corresponding entries to the
ExcludeFilters node.
For example, to exclude the IntegrationTests class, we can add the following to the
coverage.xml configuration file:
... <Filters> <ExcludeFilters> <FilterEntry> <ClassMask>Company.Product.Integration*</ClassMask> </FilterEntry> </ExcludeFilters> </Filters>
- Alternatively, to include only the desired items while excluding all others of the same kind, add the corresponding entries to the IncludeFilters node.
-
Another option, is to filter out classes and methods based on their attributes.
For example, to filter out methods marked with the
System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute
attribute, we can add the following to the coverage.xml configuration file:... <AttributeFilters> <AttributeFilterEntry> <ClassMask>System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute</ClassMask> </AttributeFilterEntry> </AttributeFilters>
Changing scope of coverage results
By default, when the coverage snapshot is ready, it only includes information on assemblies that were loaded for analysis, i.e. the assemblies that were not filtered out and that have tests. This can make the overall coverage percentage incorrect.
If necessary, you can change the scope of assemblies whose information should be added to the snapshot.
To do so, use the
Scope
parameter.
For example, to add all assemblies from your project to the snapshot,
you can add the following to the configuration file:
<Scope>
<ScopeEntry>ProjectFolder/**/*.dll</ScopeEntry>
<ScopeEntry>ProjectFolder/**/*.exe</ScopeEntry>
</Scope>
Note that everything excluded with filters is excluded anyway regardless of the specified scopes.
Covering multiple test projects
If there are several unit test projects in your solution, you can run coverage for all of them at once as described in the basic scenario. In this case, to avoid specifying full path to each assembly, you could use the TargetWorkingDir parameter when specifying test assemblies. E.g.:
<?xml version="1.0" encoding="utf-8"?>
<AnalyseParams>
<TargetExecutable>D:\Program Files\NUnit 2.6\bin\nunit-console.exe</TargetExecutable>
<TargetArguments>AppTests.dll AppTests2.dll AppTests3.dll</TargetArguments>
<TargetWorkingDir>D:\Projects\TheApplication\bin\Debug</TargetWorkingDir>
<Output>AppCoverageReport.html</Output>
<ReportType>html</ReportType>
</AnalyseParams>
However, sometimes this approach may not work. For example, when unit test projects use different unit testing frameworks. In such cases we can breakdown the coverage, merging and reporting into individual steps.
Let's suppose that we have two unit test projects, TestProject1 that uses MSTest and TestProject2 that uses NUnit. To run coverage on both projects and get a single report, we perform the following steps:
To run coverage for multiple projects in separate steps
-
Create two configuration files for running the
cover (c)
command on each of the tests projects.
testProject1.xml for the
TestProject1 that uses MSTest:
and testProject2.xml for the TestProject2 that uses NUnit:
<?xml version="1.0" encoding="utf-8"?> <CoverageParams> <TargetExecutable>C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\MSTest.exe</TargetExecutable> <TargetArguments>TestProject1.dll</TargetArguments> <TargetWorkingDir>D:\Projects\TheApplication\bin\Debug</TargetWorkingDir> <Output>Snapshot1.dcvr</Output> </CoverageParams>
<?xml version="1.0" encoding="utf-8"?> <CoverageParams> <TargetExecutable>D:\Program Files\NUnit 2.6\bin\nunit-console.exe</TargetExecutable> <TargetArguments>TestProject2.dll</TargetArguments> <TargetWorkingDir>D:\Projects\TheApplication\bin\Debug</TargetWorkingDir> <Output>Snapshot2.dcvr</Output> </CoverageParams>
-
Run the
cover (c)
command on each of the tests projects using the prepared configuration files:dotcover c testProject1.xml
anddotcover c testProject2.xml
. As a result you'll get two coverage snapshots,Snapshot1.dcvr
andSnapshot2.dcvr
. -
Run the
merge (m)
command to merge both snapshots:
dotcover m merge.xml
where the merge.xml is the configuration file:<?xml version="1.0" encoding="utf-8"?> <MergeParams> <Source>Snapshot1.dcvr</Source> <Source>Snapshot2.dcvr</Source> <Output>MergedSnapshots.dcvr</Output> </MergeParams>
-
To build an HTML test report from the merged snapshots, run the
report (r) command
dotcover r report.xml
where the report.xml is the configuration file:<?xml version="1.0" encoding="utf-8"?> <ReportParams> <Source>MergedSnapshots.dcvr</Source> <Output>CoverageReport.html</Output> <ReportType>html</ReportType> </ReportParams>