Datalore Help

Install Datalore Enterprise on AWS

The instruction below explains how to install Datalore on-premises in your AWS infrastructure.

Prerequisites

To install Datalore on-premises in AWS, you need the following tools installed on your machine:

You also need to download this repository to your computer and configure AWS CLI.

Spot instances

By default, Datalore uses spot AWS instances. To ensure you can use your account to start spot instances, execute the following command before installation:

aws iam list-roles | grep spot.amazonaws.com >/dev/null || aws iam create-service-linked-role --aws-service-name spot.amazonaws.com

It will create the necessary role (AWSServiceRoleForEC2Spot) in your account if it's missing.

Installation procedures

Deploy your AWS infrastructure

  1. Go to the aws/terraform directory in the downloaded repository.

  2. Change the parameters in the terraform.tfvars file if needed. It is important to use the name of your AWS key pair in ssh_keypair.

  3. Run terraform init.

  4. Run terraform apply to get the following important information:
    • datalore_ip is the ip address of the machine where Datalore will be installed. This IP will be used for accessing Datalore.

    • init_script is the script that you will need to execute on the machine which the terraform script made for you.

Configure Datalore

  1. SSH to the machine with the IP (datalore_ip) from the previous step. Use ubuntu as a user and the private key from your AWS key pair that you specified in ssh_keypair.

    ssh -i <path_to_my_key> ubuntu@<datalore_ip>

  2. Log into the Hub repository:

    sudo docker login -u datalorecustomer

  3. Download the datalore.sh script:

    wget https://raw.githubusercontent.com/jetbrains/datalore-configs/on-premises-0.3.0/aws/datalore.sh chmod +x datalore.sh

  4. Execute the init_script command from this step. (eg ./datalore.sh <a_lot_of_params>).
  5. Find the ADMIN_API_AUTH_TOKEN value in the output printed by the script. Save this value to use later when creating an admin user.

  6. Execute the following command to start Datalore:

    ./datalore.sh start-datalore

Now you have access to Datalore at http://<datalore_ip>:8080 and Hub at http://datalore_ip>:8082. Your admin account in Hub is admin/changeme.

Change AMI settings

By default, Datalore is configured to work in the eu-west-1 AWS region. If your deployment is in another region, do the following:

  1. In the /home/ubuntu/datalore/agents_config.yaml file, edit the AMI values according to your region.
    • eu-west-1:

      cpuAmi: ami-0669fe7edb74297be gpuAmi: ami-051f0cc2350133b73

    • us-east-1:

      cpuAmi: ami-0af1fe2e95a20289c gpuAmi: ami-0873c82141e9861cb

  2. If your region is not listed yet, contact us at datalore-enterprise@jetbrains.com, and we will add an AMI for your region.

  3. To apply the changes, restart Datalore.

(Optional) Set up email notifications

  1. In the /home/ubuntu/datalore/datalore.env file (later referred to as datalore.env), set the MAIL_ENABLED parameter to true. This enables Datalore to send emails (welcome emails, sharing invitations, etc).

  2. Edit the following parameters:
    • MAIL_SENDER_EMAIL: sender's email

    • MAIL_SENDER_NAME: sender's name

    • MAIL_SENDER_USERNAME: username of SMTP user

    • MAIL_SENDER_PASSWORD password of SMTP user

    • MAIL_SMTP_SERVER: SMTP server host

    • MAIL_SMTP_PORT: SMTP server port

(Optional) Set up a hostname

If you don't want to access Datalore via ip and want to assign a human readable DNS name for it and add a load balancer with https, update the configuration using the steps belows.

  1. Go to http://<datalore_ip>:8082/hub/services (as admin user) and replace the ip addresses in Home URL, Base URLs, and Redirect URIs with a DNS name for both Datalore and Hub services.

  2. Go to the datalore.env file and change the values of HUB_PUBLIC_BASE_URL and FRONTEND_URL.

Start/stop commands

Use the following commands:

  • To stop Datalore and Hub: ./datalore.sh stop

  • To start Datalore and Hub: ./datalore.sh start

  • To check Datalore logs: sudo docker logs datalore

  • To check Hub logs: sudo docker logs hub

    Admin user and licenses

    Set up an admin user

    To use the admin panel feature, you need a user with admin rights. Use the admin API token from this procedure to create the first admin user.

    1. Log into Hub (http://<datalore_ip>:8082/) as the user you want to grant an admin role to. Make sure this user's email is set.
    2. Log into Datalore with the same user (http://<datalore_ip>:8080/) and accept Terms of Service.
    3. Send a POST request to http://<datalore_ip>:8080/api/admin/user/role?email=<EMAIL_OF_ADMIN_USER>&role=<NEW_USER_ROLE> with the header Authorization: <ADMIN_API_AUTH_TOKEN> to change the user role.

    4. Choose of the following user roles:
      • REGULAR: regular user. Can be used to demote the user from the admin role.

      • ADMIN: admin user with access to the admin panel.

      • SUPER_ADMIN: admin user who can also change other users' roles via the admin panel.

    Add a license

    To use Datalore, you need to activate your license (provided in license.key). Only with an activated license, you can start computations and create more than one user. You have the following options:

    • (Recommended) Use the admin panel:
      1. Set up your admin user.

      2. Open http://<datalore_ip>:8080/admin/license.

      3. Add your license.key in the Add new license field.

      Once submitted and verified, the license will be immediately activated (no restart needed). Licenses are persisted in the database, so they will work even after restart.

    • Use the admin REST API as an alternative to the admin panel:
      1. Send a POST request to http://<datalore_ip>:8080/api/admin/license with the header Authorization: ADMIN_API_AUTH_TOKEN (token from this procedure).

      2. Place your license.key in the request body.

    Last modified: 16 September 2021