Code coverage

Code coverage uses generated reports to calculate the overall code coverage inside a method, a class, and a file. It also reports on the issues connected with the missing coverage in these entities.

This feature is available under the Ultimate and Ultimate Plus licenses in the following linters:

Linter	Code coverage tool	Supported report formats
Qodana for JVM	IntelliJ IDEA Code Coverage Agent is the recommended tool	`ic` is the preferable format. `XML` is also supported.
Qodana for JVM	JaCoCo	`xml`
Qodana for JS	Jest	`lcov`
Qodana for PHP	PhpUnit	`xml`
Qodana for .NET	coverlet.msbuild	`lcov`
Qodana for Python	Coverage.py	`xml`
Qodana for Go	`go test`	`out`

How it works

For the missing code coverage issues, the predefined threshold in Qodana is currently set to 50%.

Code coverage employs several inspections that are already included in the qodana.recommended and qodana.starter default inspection profiles, so you do not need to enable them:

Linter	Employed inspection
Qodana for JVM	`JvmCoverageInspection`
Qodana for JS	`JsCoverageInspection`
Qodana for PHP	`PhpCoverageInspection`
Qodana for Python	`PyCoverageInspection`
Qodana for Go	`GoCoverageInspection`
Qodana for .NET	`NetCoverageInspection`

Once the inspection is complete, reports are available in Qodana reports and JetBrains IDEs.

Code coverage calculation

Qodana calculates a code coverage based on the number of code lines containing logic with function, method, and class statements being ignored. Here is the snippet containing comments on how it works:

function divide(a, b) { // Not analyzed by the code coverage
  return a / b; // Analyzed by the code coverage
}
module.exports = divide; // Analyzed by the code coverage

Prepare the project

Configure a code coverage tool. While configuring Jest, note that all files in coverage reports should have the relative paths inside the project. For example, if your codebase files are contained in the <project-root>/src/ directory, then file paths in code coverage reports should be src/<file-name>.
If you run Qodana locally, use your code coverage tool to generate a code coverage report. Save the report to the directory where Qodana can read it. If you run Qodana in your GitHub pipeline, configure the workflow as shown in the Run the code coverage section.

For the Qodana for .NET linter, configure coverlet.msbuild for the test project.

Run the code coverage

To learn about running code coverage using the Qodana for .NET linter, skip to the Qodana for .NET section of this page.

Map the directory containing code coverage reports to the /data/coverage directory and the project token using the QODANA_TOKEN variable. Here are the Docker and Qodana CLI command samples:

                    docker run \
                        -v $(pwd):/data/project/ \
                        -v /directory/with/coverage/report/:/data/coverage \
                        -e QODANA_TOKEN="<qodana-cloud-token>" \
                        jetbrains/qodana-<linter>
                

                    qodana scan \
                       -v /directory/with/coverage/report/:/data/coverage \
                       -e QODANA_TOKEN="<qodana-cloud-token>"
                

Create the pipeline that will store all code coverage output files in the <project-root-dir>/.qodana/code-coverage directory. You can find various examples of the GitHub Actions configurations on the GitHub website.

Below is the pipeline configuration example for the Qodana for JS linter:

name: JavaScript - Jest Test

on:
  workflow_dispatch:
  pull_request:
  push:
    branches:
      - main
      - 'releases/*'

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout Code
      uses: actions/checkout@v3
      with:
        ref: ${{ github.event.pull_request.head.sha }}
        fetch-depth: 0

    - name: Use Node.js 18.x
      uses: actions/setup-node@v2
      with:
        node-version: 18.x

    - name: Install dependencies
      run: npm ci
      working-directory: JS/jest

    - name: Run tests
      run: npm test
      working-directory: JS/jest
      
    - name: Archive coverage data # Archive data for using by Qodana
      uses: actions/upload-artifact@v2
      with:
        name: jest-coverage-data
        path: JS/jest/.qodana/code-coverage

    - name: Qodana Scan # Run Qodana
      uses: JetBrains/qodana-action@main
      env:
        QODANA_TOKEN: ${{ secrets.QODANA_TOKEN_JS }}
      with:
        args: "-i,JS/jest,--linter,jetbrains/qodana-js:2024.1"
        pr-mode: false

Create a GitLab CI/CD pipeline that will store all code coverage output files in the <project-root-dir>/coverage directory:

        qodana:
           image:
              name: jetbrains/qodana-<linter>
              entrypoint: [""]
           cache:
              - key: qodana-2023.3-$CI_DEFAULT_BRANCH-$CI_COMMIT_REF_SLUG
                fallback_keys:
                   - qodana-2023.3-$CI_DEFAULT_BRANCH-
                   - qodana-2023.3-
                paths:
                   - .qodana/cache
           variables:
              QODANA_TOKEN: $qodana_token           -
           script:
              - qodana --save-report 
              --results-dir=$CI_PROJECT_DIR/.qodana/results 
              --coverage-dir=$CI_PROJECT_DIR/coverage
           artifacts:
              paths:
                 - .qodana/results/report
              expose_as: 'Qodana report'
        

This uses the QODANA_TOKEN variable to contain the project token. The --coverage-dir=$CI_PROJECT_DIR/coverage in the script block runs Qodana with the code coverage directory.

Qodana for .NET

Here is an example of the qodana.yaml file configuration for the Qodana for .NET linter:

dotnet:
  solution: <your-solution-file>

bootstrap: dotnet build, cd <path-to-dir-with-test-project-file> && \\
  dotnet add package coverlet.msbuild && \\
  ((dotnet test /p:CollectCoverage=true /p:CoverletOutput=<your-project-folder>/.qodana/code-coverage /p:CoverletOutputFormat=lcov))

Here, the dotnet option configures the solution file.

The bootstrap command performs several steps before running Qodana and explained in the table:

Command step	Description
`dotnet build`	Build a project or a solution
`cd <path-to-dir-with-test-project-file>`	Navigate to the directory containing the project test file
`dotnet add package coverlet.msbuild`	Add the `coverlet.msbuild` package to the project. This command needs to be repeated for each
`dotnet test ...`	Execute tests in the project, and:
`/p:CollectCoverage=true`	Enable code coverage
`/p:CoverletOutput=<your-project-folder>/.qodana/code-coverage`	Collect code coverage results to a specific directory
`/p:CoverletOutputFormat=lcov`	Specify the code coverage output format

Code coverage inspection results for the Qodana for .NET linter are available in Qodana Cloud.

Fresh code

Fresh code is the code contained in a GitHub pull request. Qodana can calculate fresh code coverage and display the results.

To enable the fresh code feature, in your GitHub workflow configure the PR-mode. Here is the sample for inspecting the JavaScript fresh code:

name: Code coverage fresh code

on:
  workflow_dispatch:
  pull_request:
  push:
    branches:
      - main
      - 'releases/*'

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Code
        uses: actions/checkout@v3
        with:
          ref: ${{ github.event.pull_request.head.sha }}
          fetch-depth: 0

      - name: Use Node.js 18.x
        uses: actions/setup-node@v2
        with:
          node-version: 18.x

      - name: Install dependencies
        run: npm ci

      - name: Run tests
        run: npm test
      
      - name: Archive coverage data
        uses: actions/upload-artifact@v2
        with:
          name: jest-coverage-data
          path: .qodana/code-coverage
      
      - name: Qodana Scan
        uses: JetBrains/qodana-action@main
        env:
          QODANA_TOKEN: ${{ secrets.QODANA_TOKEN_JS }}
        with:
          pr-mode: true # Enable the pull-request mode

Report overview

After you have prepared the project and ran the code coverage, you can view code coverage reports in Qodana Cloud or using your IDE.

Qodana Cloud

You can find code coverage statistics in the upper-right corner of the Qodana report UI. It also enumerates the inspections that were employed by the feature.

IDE

You can view code coverage reports using IntelliJ IDEA, WebStorm, PhpStorm, PyCharm, and GoLand IDEs starting from version 2023.2. This feature is available for reports retrieved from Qodana Cloud after linking, or reports from local storage.

Open reports from Qodana Cloud

In your IDE, navigate to Tools | Qodana | Log in to Qodana.
In the Settings dialog, click Log in.
This will redirect you to the authentication page.
In the Settings dialog, search for the project you would like to link with.

View coverage reports in IDE

You can view locally-based code coverage reports directly in JetBrains IDEs.

In your IDE, navigate to Run | Show coverage data and open the file containing a code coverage report.

The Choose Coverage Suite to Display dialog

In the Coverage tool window, you can view the test coverage report. This report shows the percentage of the code that has been executed or covered by tests.

Report overview

The IDE highlights the codebase test coverage using color marking. By default, the green color means that a particular line was covered, and the red color means the uncovered line of code.

The report shows coverage for the lines that implement the logic of a method, function, or a class, but not for the function, method, or class declaration. The image below shows that code coverage is not applicable to line 7, while line 8 is not covered.

Last modified: 26 April 2024