Datalore 2024.1 Help

Security

This section covers the following security-related procedures and advisories:

Runtime permissions

Datalore notebook agent relies on two things which require elevated access to the runtime: CRI-U and FUSE mounts within the containers. Both of these things require at least SYS_ADMIN capability granted to the runtime, otherwise the reactive mode and attached files won't work properly.

For the same reason, Datalore operational capacity is limited on environments with limited permission scope, like AWS Fargate.

We are looking into ways of reducing the scope of the permissions required. If Datalore is planned to be operated within the communal infrastructure, it's advised to provision a dedicated set of host machines specifically for Datalore compute agents.

    Configure TLS certificates for Datalore

    Datalore does not provide any TLS-related options to its end-users. Instead, it relies on a third-party load balancers (or reverse proxies) to perform such a termination. As a consequence, the Datalore app itself is not normally expected to be user-faced directly without some intermediary proxy deployed next to Datalore.

    Configure TLS certificates for Docker-based deployment

    This procedure describes creating another container with Nginx that will work as a reverse proxy with SSL termination.

    1. Edit the docker-compose.yaml file as shown in the example below.

      version: "3.9" services: datalore: image: jetbrains/datalore-server:2023.6 expose: [ "8080", "8081", "5050", "4060" ] networks: - datalore-agents-network - datalore-backend-network volumes: - "datalore-storage:/opt/data" - "/var/run/docker.sock:/var/run/docker.sock" environment: DATALORE_PUBLIC_URL: "https://datalore.example.com" #change to your domain name DB_PASSWORD: "changeme" #change to your password postgresql: image: jetbrains/datalore-postgres:2023.6 expose: [ "5432" ] networks: - datalore-backend-network volumes: - "postgresql-data:/var/lib/postgresql/data" environment: POSTGRES_PASSWORD: "changeme" #change to your password DATABASES_COMMAND_IMAGE: "jetbrains/datalore-database-command:2023.6" nginx: image: nginx:1.25 networks: - datalore-backend-network volumes: - nginx-selfsigned.crt:/etc/ssl/certs/nginx-selfsigned.crt #change to your cert - dhparam.pem:/etc/nginx/dhparam.pem #change to your DH parameters - nginx-selfsigned.key:/etc/ssl/private/nginx-selfsigned.key #change to your cert key - ssl.conf:/etc/nginx/conf.d/ssl.conf #change to your nginx config ports: - 80:80 - 443:443 volumes: postgresql-data: { } datalore-storage: { } networks: datalore-agents-network: name: datalore-agents-network datalore-backend-network: name: datalore-backend-network
    2. Edit the nginx ssl.conf file as shown in the example below.

      server { listen 443 ssl; server_name datalore.example.com; ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt; #change to your cert ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key; #change to your cert key ssl_protocols TLSv1.3; ssl_prefer_server_ciphers on; ssl_dhparam /etc/nginx/dhparam.pem; #change to your DH parameters ssl_ciphers EECDH+AESGCM:EDH+AESGCM; ssl_ecdh_curve secp384r1; ssl_session_timeout 10m; ssl_session_cache shared:SSL:10m; ssl_session_tickets off; # commented for self-signed certificate ssl_stapling on; #comment this parameter if you have self-signed certificate ssl_stapling_verify on; location / { proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header Host $host; proxy_pass http://datalore:8080; } } server { listen 80 default_server; server_name _; return 301 https://$host$request_uri; }

    Configure TLS certificates for Helm-based deployment

    We suggest that you use one of the following methods:

    • Self-acquired certificate and private key

    • Let's Encrypt

    1. Perform the following steps based on a selected method:

      1. Create a Kubernetes TLS secret, following the official Kubernetes guidance.

      2. Adjust the datalore.values.yaml file, as follows, replacing datalore.example.com with your actual FQDN you're going to use with Datalore.

        ingress: enabled: true tls: - secretName: datalore-tls hosts: - datalore.example.com hosts: - host: datalore.example.com paths: - path: / pathType: Prefix annotations: nginx.ingress.kubernetes.io/proxy-body-size: "8m" kubernetes.io/ingress.class: "nginx"
      1. Install CertManager into your Kubernetes cluster.

      2. Create a letsencrypt.yaml with the following content:

        apiVersion: cert-manager.io/v1 kind: Issuer metadata: name: letsencrypt-prod spec: acme: # The ACME server URL server: https://acme-v02.api.letsencrypt.org/directory # Email address used for ACME registration email: [PLACE YOUR EMAIL HERE] # Name of a secret used to store the ACME account private key privateKeySecretRef: name: letsencrypt-prod # Enable the HTTP-01 challenge provider solvers: - http01: ingress: ingressClassName: nginx
      3. Apply the manifest: kubectl apply -f letsencrypt.yaml

      4. Check the kubectl get issuer. Eventually, it should become as follows:

        kubectl get issuer 1 ↵ NAME READY AGE letsencrypt-prod True 14d
      5. Adjust the datalore.values.yaml file, as follows, replacing datalore.example.com with your actual FQDN you're going to use with Datalore.

        ingress: enabled: true tls: - secretName: datalore-tls hosts: - datalore.example.com hosts: - host: datalore.example.com paths: - path: / pathType: Prefix annotations: nginx.ingress.kubernetes.io/proxy-body-size: "8m" kubernetes.io/ingress.class: "nginx" cert-manager.io/issuer: "letsencrypt-prod"
    2. Set the DATALORE_PUBLIC_URL parameter in the same datalore.values.yaml file. Using the same value you provided to replace "https://datalore.example.com" in the step above.

      dataloreEnv: DATALORE_PUBLIC_URL: "https://datalore.example.com"
    3. Apply the configuration and restart Datalore.

    4. Check whether ingress controller registered the changes: kubectl get ingress. The expected result is a datalore ingress with the 443 port exposed.

    5. Check whether the certificate is issued: kubectl get certificates. The expected output is similar to the one below.

      kubectl get certificates NAME READY SECRET AGE datalore-tls True datalore-tls 8m5s

    Use Kubernetes native secrets for storing the database password

    1. Modify the databaseSecret block in your datalore.values.yaml, as follows:

      databaseSecret: create: false name: datalore-db-password key: DATALORE_DB_PASSWORD

    2. Create a Kubernetes secret, using the value from the name key above as the secret name, and the desired password as the secret value.

    3. (if applicable) Remove the password key with its value from the databaseSecret block.

    4. Proceed based on whether this is your fresh deployment or Datalore is already installed.

      Proceed with the installation. No further action is required.

      Apply the configuration

      helm upgrade --install -f datalore.values.yaml datalore datalore/datalore --version 0.2.15

    Database password rotation

    Datalore requires a permanent connection to a PostgreSQL database to operate properly. Once Datalore is deployed, the database password is saved within the environment so Datalore can re-use it later once restarted.

    However, you might want to change this password later due to various compliance or operational reasons.

    1. Locate the values.yaml file being used for the deployment.

    2. Depending on the method used: either replace the password within the databaseSecret block, OR update the secret value, if the Kubernetes secret is used instead of the plain-text value.

    3. Update the Datalore deployment: helm upgrade --install -f datalore.values.yaml datalore datalore/datalore --version 0.2.15

    4. Restart Datalore. For further operational guidance, refer to Server lifecycle events.

    1. Locate the docker-compose.yaml file being used for the deployment.

    2. Update the DB_PASSWORD block in environment block.

    3. Restart Datalore. For further operational guidance, refer to Server lifecycle events.

      Configure TLS between server and agent

      1. Click the avatar in the upper right corner and select Admin panel from the menu.

      2. From the Admin panel, select Configuration.

      3. Select the Force agent SSL checkbox.

      1. Click the avatar in the upper right corner and select Admin panel from the menu.

      2. From the Admin panel, select Configuration.

      3. Click the Reset secrets button.

      Reset secrets
        Last modified: 13 February 2024