Run a Step in a Container

Short overview:

You can use container images either from Docker Hub or from your own Space Packages registries.
Depending on your task, in a container, you can run a shell script, arbitrary Kotlin code, or a container image command.

All these three options are mutually exclusive.
During the run, the container has access to the project repository.
You can adjust container resources for resource-consuming steps.
A job can contain up to 50 steps (containers).
The default and maximum allowed job timeout is 2 hours.

Selecting a container image

To run something in a container, you should use the container keyword and specify a container image:

An image from Docker Hub (an image name is enough): container("hello-world")
An image from your Space Packages registry (an image URL is required): container("mycompany.registry.jetbrains.space/p/projectkey/mydocker/hello-world")

Note that Space caches the images downloaded from Docker Hub. The cache lifetime is no longer than three hours. The disk space required for storing cached images is provided free of charge.

Running a shell script

Using the shellScript keyword, you can run an arbitrary shell script. This could be a one-liner or a multiline script. The default script interpreter /bin/sh can be changed with interpreter.

For example:

            job("run shell script") {
                container("ubuntu") {
                    shellScript {
                        interpreter = "/bin/bash"
                        content = """
                            echo Hello
                            echo World!
                        """
                    }
                }
            }
        

If you want to run an existing script file (e.g., stored in the project repository), you should specify its location. It could be an absolute path, or a path that is relative to a working directory . If your script file requires arguments, you can provide them with the args function.

            job("run shell script") {
                container("ubuntu") {
                    shellScript {
                        location = "./myscript.sh"
                        args("arg1, arg2")
                    }
                }
            }
        

Running Kotlin code

As the Automation DSL is based on Kotlin, you can run any Kotlin code inside job steps. In case of a container, you should put your code inside the kotlinScript block. For example:

            job("build and publish") {
                container("gradle:6.1.1-jre11") {
                    kotlinScript { api ->
                        api.gradle("build")
                        try {
                            api.gradle("publish")
                        } catch (ex: Exception) {
                            println("Publishing failed")
                        }
                    }
                }
            }
        

How it works under the hood (with some simplification): Space compiles the script into a .jar file and runs it with a command like java -jar script.jar. This means that the image you use to run the script must include JRE/JDK (version 9 or later).

APIs for working with Space modules and external tools

In the kotlinScript block, you have access to various APIs: Space module APIs and APIs for various external tools (Gradle, dotnet, and others).

                job("build and publish") {
                    container("gradle") {
                        kotlinScript { api ->
                            try {
                                api.gradle("build")
                            } catch (ex: Exception) {
                                val recipient = MessageRecipient.Channel(ChatChannel.FromName("CI-channel"))
                                val content = ChatMessage.Text("Build failed")
                                api.space().chats.messages.sendMessage(recipient, content)
                            }
                        }
                    }
                }
            

In this example, we use api.gradle to run Gradle commands and api.space().chats to access the Chats module. For the full list of APIs, refer to job.container.kotlinScript.

Using external packages in kotlinScript

Kotlin script lets you use functions from external packages. To do this, you should reference the required package with the @file:DependsOn("$package_name") annotation. The $package_name is the name of the package available on Maven Central.

For example, the following script gets a random joke from icanhazdadjoke.com using the OkHttp client and prints it to the job log.

                @file:DependsOn("com.squareup.okhttp:okhttp:2.7.4", "org.json:json:20200518")

                import com.squareup.okhttp.*
                import org.json.JSONObject

                job("Get random joke") {
                    container("openjdk:11") {
                        kotlinScript {
                            val client = OkHttpClient()
                            val request = Request.Builder()
                                    .url("http://icanhazdadjoke.com")
                                    .addHeader("Accept", "application/json")
                                    .build()
                            val response = client.newCall(request).execute()
                            val jData = response.body().string()
                            val jObject = JSONObject(jData)
                            val joke = jObject.get("joke").toString()
                            println(joke)
                        }
                    }
                }
            

Running a container image command

To run a default image command, provide its arguments in the args array. Note that special characters must be escaped.

For example:

                job("Example") {
                    container("alpine") {
                        args("echo", "Hello World!")
                    }
                }
            

If you want to override the default image command, you can do this by using entrypoint. In this case, args provide arguments for the entrypoint command. Note that special characters must be escaped.

For example:

                job("Example") {
                    container("gradle:latest") {
                        entrypoint("/bin/sh")
                        args("echo", "Hello World!")
                    }
                }
            

Is it possible to run kotlinScript, shellScript, and entrypoint in the same container?

No, it's not possible. The kotlinScript, shellScript, and args | entrypoint items are mutually exclusive. If you specify more than one inside a container, only one of them will take effect according to the following priority: kotlinScript, shellScript, args | entrypoint.

Correct	Wrong
job("Example") { container("openjdk") { kotlinScript { api -> // Do smth. } } container("alpine") { shellScript { content = """ echo Do smth. """ } } }	job("Example") { container("openjdk") { kotlinScript { api -> // Do smth. } shellScript { content = """ echo Do smth. """ } } } Only `kotlinScript` will run.

Container resources

A job can contain not more than 50 containers (steps). Each container has the following resources constraints:

	Default	Max	Min
CPU	2 virtual CPUs	4 virtual CPUs	0.2 virtual CPU
Memory	7800 MiB (approx. 8 GB)	16 GiB	–

Default

Max

Min

CPU

2 virtual CPUs

4 virtual CPUs

0.2 virtual CPU

Memory

7800 MiB (approx. 8 GB)

16 GiB

–

Note that all containers within a job use the same disk volume (it contains the project repository). The default volume size is 5 GB and the maximum allowed size is 30 GB. To specify resources and volume size, you should use resources and volumeSize correspondingly. Note that all items support units:

cpu: You can set a value in .cpu or .mcpu (millicpu), for example, cpu = 250.mcpu is the same as cpu = 0.25.cpu.
memory and volumeSize: You can set their values in .mb (MB) and .gb (GB), for example, volumeSize = 10.gb.

            job("Example") {
                // 10 GB
                volumeSize = 10.gb

                container("alpine") {
                    resources {
                        // 500.mcpu = 0.5.cpu
                        cpu = 0.5.cpu
                        // 2000 MB
                        memory = 2000.mb
                    }
                }
            }
        

Last modified: 12 January 2021