Share Data Between Steps
Sometimes it is necessary to pass an execution context (for example, an access token, script run results, and so on) from one Automation step to another. Depending on the data type, you can share it using a parameter or a file.
Sharing parameters between steps
For this purpose, all steps in a job have access to a common parameter storage. To access the storage, you should use the
parameters API. The API is available only inside the
kotlinScript block: the container you use to run the step must include JRE/JDK 9 or later.
Note that you can remove parameters from the storage by using
parameters().remove(key: String), where
key is a parameter name.
Sharing files between steps
In some cases, a file created in one step may be required in another step. For example, files generated during the build may be required by tests executed in a separate container. For this purpose, Automation provides a file share: a separate external storage that is mounted to a container.
Accessing file share directly
The file share is mounted to a step container at
$mountDir/share (in general case, it is
/mnt/space/share ). You can work with this directory directly using container commands. In the following example, the first container creates
file.txt in the file share, and the second container shows the file share contents.
Accessing file share in Kotlin code
Alternatively, there's a more high-level approach of accessing the file share: If your step runs Kotlin code in the
kotlinScript block, you can use the special file-sharing API. Note that the container must include JVM/JDK 8 or later.
In this script:
The first container creates
file.txtin the user's directory.
fileShare().put()copies the file to the file share. Note that you can change the file name during copying.
The second container uses
fileShare().locate()to get the path to the file and then it prints the contents of the file.
Because the steps (containers) run sequentially,
file.txtis guaranteed to be available in the file share when the second container starts.
import java.io.*imports the java.io package as the script uses Java functions to work with files.
File share size limit
The maximum capacity of the file share is limited to 2 GB per job. Let's use an example to see how this limit works:
A job consists of two steps (i.e. containers): A and B.
The steps run sequentially one after another.
None of the steps is allowed to allocate more than 2 GB in the file share.
Step A allocates 1.5 GB.
Let's suppose step B allocates 1 GB (the step is allowed to do this as it is less than the 2 GB limit). The size of the file share is 2.5 GB now.
If at the end of step B, the file share size is still 2.5 GB, the job will fail.
If step B cleans the file share so that its size doesn't exceed the 2 GB limit, the job will finish successfully.
You can remove files from the file share either directly (using container commands) or using the API:
Monthly limit on file share usage
Automation calculates file share usage twice for each step: First, when a file share volume is mounted to a step, and, for the second time, when the step finishes working (Automation calculates the size of new data allocated by the step). The monthly limit on file share usage is 200 GB for all subscription plans. Automation will warn you once the file share usage limit is reached.
Let's use an example to see how this limit works:
A job consists of three steps (i.e. containers): A, B, and C.
The steps run sequentially one after another.
Step A allocates 1 GB in the file share. Now, the file share size is 1 GB, file share usage is 1 GB.
Automation starts step B and mounts the file share to the step container. Even if step B does nothing with the file share, this is treated as read access to the file share. Now, the file share size is 1 GB, file share usage is 2 GB.
Step B allocates 0.5 GB in addition to the existing data. Now, the file share size is 1.5 GB, file share usage is 2.5 GB.
Step C gets the file share and does not put any new data to it. Now, the file share size is 1.5 GB, file share usage is 5 GB.
Job finishes its work with the resulting 5 GB file share usage. This means that you can run such a job only 40 times a month.
Parallel steps and file sharing
By default, steps in a job run sequentially: the next container starts only after the previous container finishes its work. This prevents conflicts in the file share. But what if two steps (A and B) run in parallel and put the same file in the file share? This behavior is undetermined: the file share will contain the result of either A or B. Consider an example:
Both containers run in parallel: the first container writes
file.txt while the second one writes
World to the same file. When the job finishes,
file.txt will contain either
World but neither
Hello World nor
File attributes and file sharing
File attributes are preserved when a file is copied to/from the file share.
fileShare() function provides access to the file sharing API:
|Method or Property||Description|
|(Read-only) Path to the file share (by default, |
Returns full path to the file.
Locates a file in the file share by its
Returns file path.
Deletes a file from the file share by its