Amazon S3 and S3-compatible Storages

TeamCity comes bundled with the Amazon S3 Artifact Storage plugin which allows storing build artifacts in Amazon S3 buckets, as well as S3-compatible buckets such as MinIO, Backblaze B2, and others. S3-compatible storages can be hosted in both AWS and non-AWS environments.

Create and Set Up a New AWS S3 Storage

Navigate to the Administration | <Your_Project> page and switch to the Artifacts Storage tab.
- Open settings of a <Root project> if you want your new storage to be available for all TeamCity projects.
- Edit one specific project if your new storage should be available only for this project and its sub-projects.
The built-in TeamCity artifacts storage is displayed by default and marked as active. Click Add new storage button to create a new storage.
Specify the custom storage name and, if needed, its internally used ID.
Set the Type field to "AWS S3".
Choose an existing AWS Connection that TeamCity should use to access your Amazon resources. If no suitable AWS connection exists, click the "+" icon to add one.
You cannot use AWS Connections that belong to a parent project and have their Available for sub-projects settings disabled.
However, if a parent project has an S3 storage configured using a non-shared connection, this storage is still available to child projects.
A user whose credentials the selected AWS Connection uses (or an IAM Role it assumes) to access the S3 buckets should have the following permissions:
- ListAllMyBuckets
- GetBucketLocation
- GetObject
- ListBucket
- PutObject
- DeleteObject
- GetAccelerateConfiguration (if Transfer Acceleration is enabled)
In previous TeamCity versions, the Artifacts Storage dialog allowed you to explicitly specify connection settings: access key credentials, user IAM role, and whether TeamCity should look for credentials in the default AWS locations (the Default provider chain setting).
Starting with version 2023.11, these settings are exclusive to AWS Connections. If you're migrating from an older version of TeamCity and your existing AWS S3 storages used any of these settings, click the Convert to AWS Connection link. This action transfers AWS-related settings to a new AWS Connection, and selects this new connection as the source connection of your storage.
TeamCity uses the selected AWS Connection to retrieve the list of available S3 buckets. Open the Bucket drop-down menu to choose a specific item from the list.

(Optional) Specify the path prefix if you want to use the same S3 bucket for all TeamCity projects and configure prefix-based permissions.
Amazon S3 buckets support two options to speed up file uploads and downloads:
- AWS CloudFront — a content delivery network (CDN) that allows TeamCity to transfer arifacts using low-latency CloudFront servers nearby.
- Transfer Acceleration — a bucket-level feature designed to optimize transfer speeds from across the world into centralized S3 buckets. It enables fast, easy, and secure transfers of files over long distances between your client and an S3 bucket.
If your bucket is configured to use Transfer Acceleration or CloudFront, choose the corresponding option under the Transfer speed-up section. Otherwise, if you wish TeamCity to transfer files in the regular mode, choose the None type.
Choosing the AWS CloudFront option requires setting up additional fields. See this section for more information: Transferring Artifacts via CloudFront.
Using Transfer Acceleration applies the following restrictions:
- The virtual host addressing is always on. Path-style requests are not supported.
- Transfer Acceleration must be enabled on your S3 bucket.
- Buckets cannot have dots (.) in their names.
- User credentials or IAM role the AWS Connection uses to access your bucket must include the GetAccelerateConfiguration permission.

To optimize the upload of large files to the storage, you can enable the multipart upload. To do this, tick the Customize threshold and part size setting and set the multipart upload threshold. The minimum allowed value is 5MB. Supported suffixes: KB, MB, GB, TB. If you leave this field empty, multipart upload will be initiated automatically for all files larger than 8 MB (8MB is the default value).
Additionally, you can configure the maximum allowed size of each uploaded file part. The minimum value is 5MB. If left empty, TeamCity will use 8MB as the default value.
We recommend that you configure a bucket lifecycle policy to prevent incomplete multipart uploads.

Uncheck the Force virtual host addressing option to turn off the corresponding feature (enabled by default). Currently, both hosted-style and path-style requests are supported by TeamCity. Note that Amazon stopped supporting path-style access for new buckets since September 2020.
This setting cannot be disabled if your storage uses Transfer Acceleration.
Tick Verify file integrity after upload to allow TeamCity to perform an additional check-up on uploaded files. If the integrity verification fails, TeamCity writes a corresponding message to the build log.
Click Save to save your new storage and return to the list of available storages.

When viewing a list of storages available for a project, click Make Active to start using the corresponding storage for all new builds of this project. The has N usages link allows you to view which builds used this storage to upload their artifacts.

Create and Set Up a New S3-Compatible Storage

Navigate to the Administration | <Your_Project> page and switch to the Artifacts Storage tab.
- Open settings of a <Root project> if you want your new storage to be available for all TeamCity projects.
- Edit one specific project if your new storage should be available only for this project and its sub-projects.
The built-in TeamCity artifacts storage is displayed by default and marked as active. Click Add new storage button to create a new storage.
Specify the custom storage name and, if needed, its internally used ID.
Set the Type field to "Custom S3".
Specify the Access key ID and Secret access key values. See documentation for your S3-compatible storage vendor for the information on how to issue access keys.
Specify the storage endpoint TeamCity should use to access your bucket.
(Optional) Specify the path prefix if you want to use the same S3 bucket for all TeamCity projects and configure prefix-based permissions.
To optimize the upload of large files to the storage, you can enable the multipart upload. To do this, tick the Customize threshold and part size setting and set the multipart upload threshold. The minimum allowed value is 5MB. Supported suffixes: KB, MB, GB, TB. If you leave this field empty, multipart upload will be initiated automatically for all files larger than 8 MB (8MB is the default value).
Additionally, you can configure the maximum allowed size of each uploaded file part. The minimum value is 5MB. If left empty, TeamCity will use 8MB as the default value.
Uncheck the Force virtual host addressing option to turn off the corresponding feature (enabled by default). Currently, both hosted-style and path-style requests are supported by TeamCity. Note that Amazon stopped supporting path-style access for new buckets since September 2020.
Tick Verify file integrity after upload to allow TeamCity to perform an additional check-up on uploaded files. If the integrity verification fails, TeamCity writes a corresponding message to the build log.
Click Save to save your new storage and return to the list of available storages.

Transferring Artifacts via CloudFront

Amazon CloudFront is a content delivery network that offers low latency and high transfer speeds. Enabling its support for an S3 storage will allow TeamCity to transfer artifacts through the closest CloudFront server. If your S3 bucket is located in a different region than your TeamCity infrastructure, this could significantly speed up the artifacts' upload/download and reduce expenses.

Prerequisites

TeamCity can set up CloudFront integration for you, or you can set up all the settings manually.

The CloudFront integration requires configuring:

an OAI user
two CloudFront distributions, one for uploading and downloading artifacts.
a trusted key group
an SSH-2 RSA key pair (public key + private key) in PEM format

CloudFront Settings

When you switch the storage type to AWS CloudFront, four new settings appear.

Use the Download distribution and Upload distribution drop-down menus to choose manually created distributions.
The Public key field and Upload private key... button allow you to specify corresponding keys.

Alternatively, you can click the icon to let TeamCity configure all four settings automatically.

For Cloudfront settings to work properly, TeamCity needs the following permissions:

cloudfront:ListDistributions
cloudfront:ListKeyGroups
cloudfront:ListPublicKeys

Automatic CloudFront Setup

TeamCity can configure the settings automatically. This involves:

Generating a key pair and uploading a public key to CloudFront.
Creating a new key group in CloudFront.
Creating two new distributions with:
- the Use all edge locations price class.
- a new Origin Access Identity that can access the current bucket.
- the default behaviour defining
  - allowed HTTP methods for uploading artifacts: GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE, for downloading: GET, HEAD, OPTIONS;
  - a new key group with the viewer access;
  - a custom cache policy that allows passing all query strings.
Adding a new policy to an S3 bucket to allow the new distributions access to it. See the policy example.

Automatic setup requires giving Teamcity additional permissions:

cloudfront:CreateDistribution
cloudfront:CreateKeyGroup
cloudfront:CreatePublicKey
cloudfront:CreateOriginRequestPolicy
cloudfront:CreateCloudFrontOriginAccessIdentity
cloudfront:CreateCachePolicy
cloudfront:DeleteKeyGroup
cloudfront:DeletePublicKey
cloudfront:ListCloudFrontOriginAccessIdentities
cloudfront:ListCachePolicies
cloudfront:ListOriginRequestPolicies
cloudfront:GetDistribution
cloudfront:GetPublicKey
s3:GetBucketPolicy
s3:PutBucketPolicy

Example policy providing all necessary permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Action": [
                "cloudfront:CreatePublicKey",
                "cloudfront:CreateOriginRequestPolicy",
                "cloudfront:ListCloudFrontOriginAccessIdentities",
                "cloudfront:DeleteKeyGroup",
                "cloudfront:GetPublicKey",
                "cloudfront:ListCachePolicies",
                "cloudfront:CreateDistribution",
                "cloudfront:ListOriginRequestPolicies",
                "cloudfront:DeletePublicKey",
                "cloudfront:CreateCloudFrontOriginAccessIdentity",
                "cloudfront:CreateKeyGroup",
                "cloudfront:CreateCachePolicy",
                "cloudfront:GetDistribution",
                "cloudfront:ListPublicKeys",
                "s3:ListAllMyBuckets",
                "cloudfront:ListKeyGroups",
                "cloudfront:ListDistributions"
            ],
            "Resource": "*"
        },
        {
            "Sid": "2",
            "Effect": "Allow",
            "Action": [
                "s3:PutBucketPolicy",
                "s3:GetBucketPolicy"
            ],
            "Resource": "arn:aws:s3:::<YOUR_BUCKET_NAME>"
        }
    ]
}

Manual CloudFront Setup

For security reasons, we recommend configuring two separate distributions for uploading and downloading artifacts. For each distribution:

Generate a key pair in SSH-2 RSA key format.
Upload the public key from the pair to CloudFront.
Add a new key group in CloudFront and add the created public key to this group.
Create a new cache policy with Cache key settings | Query strings set to All.
If you use a private bucket, create a new OAI user.
Create a distribution and attach your key group to it:
- Make sure to choose the same S3 bucket as specified in TeamCity.
- Allowed HTTP methods for uploading artifacts GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE; for downloading GET, HEAD, OPTIONS
- Restrict viewer access: yes
- Trusted authorization type: trusted key groups
- Cache key and origin requests: Cache policy and origin request policy
- For private buckets, enable the use OAI option and configure OAI with the following setting:
  - Bucket policy: No, I will update the bucket policy
- For public buckets, disable the Block public access option.
Add a new policy to your S3 bucket. See the policy example.

When configured, the distributions should automatically appear in the Download distribution and Upload distribution drop-down menus.

Select the target CloudFront distribution.
In Public key, select the public key associated with this distribution.
Click the Upload private key... button upload the private key from the pair.
Save the storage settings.

S3 Policy Example

For accessing a private bucket with OAI:

{
"Sid": "1",
"Effect": "Allow",
"Principal": {
      "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity <OAI ID"
  },
  "Action": [
      "s3:GetObject",
      "s3:PutObject",
      "s3:DeleteObject"
  ],
  "Resource": "arn:aws:s3:::<S3 bucket name/*"
}

For accessing a public bucket:

{
"Sid": "PublicRead",
"Effect": "Allow",
"Principal": "*",
  "Action": [
        "s3:GetObject",
        "s3:GetObjectVersion"
    ],
  "Resource": "arn:aws:s3::<BUCKET_NAME>/*"
}

Kotlin DSL

The following sample illustrates how to add an S3 bucket as custom storage for project artifacts, and set this storage as primary (default).

import jetbrains.buildServer.configs.kotlin.*
import jetbrains.buildServer.configs.kotlin.projectFeatures.activeStorage
import jetbrains.buildServer.configs.kotlin.projectFeatures.s3Storage

project {
    // ...
    features {
        activeStorage {
            id = "PROJECT_EXT_37"
            activeStorageID = "PROJECT_EXT_4"
        }
        s3Storage {
            id = "PROJECT_EXT_4"
            awsEnvironment = default {
                awsRegionName = "eu-west-1"
            }
            connectionId = "AwsPrimary"
            storageName = "S3 Transfer Acceleration"
            bucketName = "dk-s3ta"
            enableTransferAcceleration = true
            forceVirtualHostAddressing = true
            verifyIntegrityAfterUpload = true
        }
    }
    // ...
}

Migrating Artifacts To a Different Storage

TeamCity server ships with a command-line tool that transfers build artifacts from one storage to another. You can download this tool on the Project Settings | Artifacts Storage page.

Currently, the tool supports the following migration routes:

From a local directory to an Amazon S3 bucket and vice versa
From one Amazon S3 bucket to another

We're working on supporting other cloud storage options as well.

Learn more: Artifacts Migration Tool.

Last modified: 04 April 2024