Docker

Prerequisites

A Dockerfile that defines the Docker image is stored in the project sources.

Run environment

If you want to run a job on a self-hosted worker, ensure it has Docker installed.
If you want to run a job on a Space Cloud worker, ensure that the Space Automation Cloud pool is used. Learn more about worker pools

CI/CD scenarios for Docker

Automated Docker builds: All image dependencies already exist and all you need is to build and publish a Docker image to Space Packages.
Creating dependencies before the build: An Automation job must create image dependencies before building and publishing the image.
Publishing an image to an external repository, for example, Docker Hub.

Ways to run Docker builds

Automation provides two different ways for building and publishing Docker images depending on a run environment:

job.host.dockerBuildPush

job.host.dockerBuildPush lets you run Docker builds on a self-hosted or Space Cloud worker. Technically, this is a DSL wrapper for the job.host.shellScript block that runs the docker build and docker push commands.

Pros over job.kaniko:

It's possible to build Windows Docker images (if the host machine runs on Windows).
It's easier and faster to build image dependencies – As you can pre-install all necessary tools to the host machine, the same host step can be used not only to build and publish an image but also to build artifacts required for this image.
In case of job.kaniko, you must build image artifacts in a separate step and provide them to job.kaniko via the file share.
Some things might work differently with Kaniko compared to classic Docker.

job.kaniko

job.kaniko lets you run Docker builds in a container with the pre-installed Kaniko tool. Technically, this is a job.container step that runs in a container based on a special image. The main reason of using job.kaniko over host.dockerBuildPush is when using of the latter is not possible. For example, if your company uses Space On-Premises and doesn't allow using self-hosted workers.

Build and publish a Docker image to Space Packages

The job below first builds and then publishes an image defined in ./docker/config/Dockerfile. Note that Automation jobs don't require authentication in a Space Packages registry, given that the registry belongs to the same project.

job("Build and push Docker") {
    host("Build and push a Docker image") {
        dockerBuildPush {
            // by default, the step runs not only 'docker build' but also 'docker push'
            // to disable pushing, add the following line:
            // push = false

            // path to Docker [[[context|https://docs.docker.com/engine/reference/commandline/build/#extended-description]]] (by default, context is working dir)
            context = "docker"
            // path to Dockerfile relative to the project root
            // if 'file' is not specified, Docker will look for it in 'context'/Dockerfile
            file = "docker/config/Dockerfile"
            // [[[build-time variables|https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg]]]
            args["HTTP_PROXY"] = "http://10.20.30.2:1234"
            // [[[image labels|https://docs.docker.com/config/labels-custom-metadata/]]]
            labels["vendor"] = "mycompany"
            // to add a raw list of additional build arguments, use
            // extraArgsForBuildCommand = listOf("...")
            // to add a raw list of additional push arguments, use
            // extraArgsForPushCommand = listOf("...")
            // [[[image tags|https://docs.docker.com/engine/reference/commandline/tag/]]]
            tags {
                // use current job run number as a tag - '0.0.run_number'
                +"mycompany.registry.jetbrains.space/p/prjkey/mydocker/myimage:1.0.${"$"}JB_SPACE_EXECUTION_NUMBER"
            }
        }
    }
}

job("Build and push Docker") {
    // special step that runs a container with the Kaniko tool
    kaniko {
        // build an image
        build {
            // path to Docker [[[context|https://docs.docker.com/engine/reference/commandline/build/#extended-description]]] (by default, context is working dir)
            context = "docker"
            // path to Dockerfile relative to 'context'
            // this option is equivalent to the Kaniko's --dockerfile argument
            dockerfile = "config/Dockerfile"
            // [[[build-time variables|https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg]]]
            args["HTTP_PROXY"] = "http://10.20.30.2:1234"
            // [[[image labels|https://docs.docker.com/config/labels-custom-metadata/]]]
            labels["vendor"] = "mycompany"
        }
        // push the image to a Space Packages repository (doesn't require authentication)
        push("mycompany.registry.jetbrains.space/p/prjkey/mydocker/myimage") {
            // [[[image tags|https://docs.docker.com/engine/reference/commandline/tag/]]]
            tags {
                // use current job run number as a tag - '0.0.run_number'
                +"0.0.\$JB_SPACE_EXECUTION_NUMBER"
            }
            // see [[[example|https://www.jetbrains.com/help/space/automation-environment-variables.html#example]]] on how to use branch name in a tag
        }
    }
}

Create an image dependency, build, and publish a Docker image to Space Packages

The job below implies that a Gradle build generates artifacts in the ./build directory. The job then builds a Docker image that includes the artifacts, for example, the artifacts can be added using the ADD directive in Dockerfile (not shown). The Dockerfile is located in the project root.

job("Build and push Docker") {
    // both 'host.shellScript' and 'host.dockerBuildPush' run on the same host
    host("Build artifacts and a Docker image") {
        // Gradle build creates artifacts in ./build
        shellScript {
            content = """
                ./gradlew build
            """
        }

        dockerBuildPush {
            // Note that if Dockerfile is in the project root, we don't specify its path.
            // We also imply that Dockerfile takes artifacts from ./build and puts them to image
            // e.g. with 'ADD /build/app.jar /root/home/app.jar'

            val spaceRepo = "mycompany.registry.jetbrains.space/p/prjkey/mydocker/myimage"
            tags {
                +"$spaceRepo:0.${"$"}JB_SPACE_EXECUTION_NUMBER"
                +"$spaceRepo:lts"
            }
        }
    }
}

In case of job.kaniko the job must contain two steps: The first step (job.container) builds artifacts and puts them to the file share, the second step (job.kaniko) takes the files from the file share and builds the image.

job("Build and push Docker") {
    container(displayName = "Run gradle build", image = "amazoncorretto:17-alpine") {
        // run gradle build and copy artifacts to the file share
        shellScript {
            content = """
                ./gradlew build
                cp -r build ${'$'}JB_SPACE_FILE_SHARE_PATH
            """
        }
    }

    kaniko {
        // This is a shellScript that runs before 'docker build' and 'docker push'.
        // Here we use it to copy the Gradle output from the file share to the
        // context directory ('build'). Initially, the 'build' directory in this
        // container is empty (we ran 'gradlew build' in another step, i.e. container).
        beforeBuildScript {
            content = "cp -r ${'$'}JB_SPACE_FILE_SHARE_PATH build"
        }

        // We imply that Dockerfile takes artifacts from ./build and puts them to image
        // e.g. with 'ADD /build/app.jar /root/home/app.jar'
        build {
            context = "build"
        }

        push("mycompany.registry.jetbrains.space/p/prjkey/mydocker/myimage") {
            val spaceRepo = "mycompany.registry.jetbrains.space/p/prjkey/mydocker/myimage"
            tags {
                +"$spaceRepo:0.${"$"}JB_SPACE_EXECUTION_NUMBER"
                +"$spaceRepo:lts"
            }
        }
    }
}

Publish a Docker image to a remote registry (Docker Hub)

The example below shows publishing an image to Docker Hub. However, the instructions would be identical for any external registry requiring authentication.

To publish to a registry that requires authentication, your script must first log in to this repository. You can do this either by running docker login in the script itself or create a connection to the registry and use it in the dockerRegistryConnections block (recommended).

In Docker Hub, create an access token with the Write permission. Save the created token to a safe location.
Create a connection to Docker Hub:
1. Open the desired project.
2. On the project sidebar menu, choose Settings, then Docker Registry Connections.
3. Click New connection and specify connection settings:
  - Key: a unique connection name that you will use to reference this connection in jobs, e.g., docker_hub or some_registry
  - Docker registry server: a URL of the remote Docker registry. For Docker Hub, it's index.docker.io
  - Username and Password: the username and the password (token) created previously.
Edit the project's .space.kts:
job("Publish to Docker Hub") { host("Build artifacts and a Docker image") { // Before running the scripts, the host machine will log in to // the registries specified in connections. dockerRegistryConnections { // specify connection key +"docker_hub" // multiple connections are supported // +"one_more_connection" } dockerBuildPush { labels["vendor"] = "mycompany" tags { +"myrepo/hello-from-space:1.0.${"$"}JB_SPACE_EXECUTION_NUMBER" } } } }

Note that as a job.kaniko runs in a container that is isolated from the host machine, it's not possible to use dockerRegistryConnections for logging in to Docker Hub. Instead, you must provide Docker Hub credentials in the config.json file.

In Docker Hub, create an access token with the Write permission. Save the created token to a safe location.
In Space, create two secrets:
- dockerhub_user: your Docker Hub username.
- dockerhub_token: the Docker Hub token you've created in step 1.
Edit the project's .space.kts:
job("Publish to Docker Hub") { kaniko("Docker build and push") { // assign project secrets to environment variables env["HUB_USER"] = Secrets("dockerhub_user") env["HUB_TOKEN"] = Secrets("dockerhub_token") // put auth data to Docker config beforeBuildScript { content = """ B64_AUTH=${'$'}(echo -n ${'$'}HUB_USER:${'$'}HUB_TOKEN | base64 -w 0) echo "{\"auths\":{\"https://index.docker.io/v1/\":{\"auth\":\"${'$'}B64_AUTH\"}}}" > ${'$'}DOCKER_CONFIG/config.json """ } build { labels["vendor"] = "mycompany" } //in push, specify repo_name/image_name push("myrepo/hello-from-space") { tags{ +"1.0.\$JB_SPACE_EXECUTION_NUMBER" } } } }

Last modified: 19 June 2023