Datalore 2024.3 Help

Google BigQuery

This procedure explains how to configure a Google BigQuery database connection. It is assumed that you are working on the General tab of the New Google BigQuery connection dialog and need to provide connection details relevant for this particular database type.

Configure a Google BigQuery connection

  1. Open the New Google BigQuery connection dialog. Select the tab for one of the two options:

    • From the editor, to attach the new database to a specific notebook

    • From the Home page, to add a database connection to a workspace

    • Go to Main menu | Tools | Attached data or click the Attached data icon on the left-hand sidebar.

    • Click New connection and select New database connection.

    • In the New database connection dialog, select Google BigQuery.

    • On the Home page, select the workspace where you want to create the connection.

      Selecting a workspace
    • Select Databases from the menu under the workspace name.

    • Click the Add connection button in the upper right corner. The New database connection dialog will open.

    • In the New database connection dialog, select Google BigQuery.

  2. On the General tab, select the connection type.

    • default: to connect by specifying the Host, Port, and Database.

    • URL only: to connect by providing the URL of a pre-built connection.

  3. (For default) In the Host field, type your server address.

  4. (For default) In the Port field, type the port of BigQuery. The default port is 443.

  5. Select an authentication method:

    • Google Service account: to connect using a Google service account. This method requires a JSON service account key. Refer to this page for more details.

    • OAuth: to connect using the BigQuery Google OAuth method.

    • Access and Refresh Tokens: to connect using access and refresh tokens.

    • Application Default Credentials: to connect using the credentials from a service account key.

    • No auth: to connect without authentication.

  6. Proceed based on the selected authentication method:

    Obtain a credentials file

    1. Create a service account by following the official instructions.

      On the Grant this service account access to the project step in the wizard, select roles for this service account.

      For example, for read-only access, select BigQuery Data Viewer, BigQuery Job User, and BigQuery User from the BigQuery menu. Alternatively, select BigQuery | BigQuery Admin to access all resources within the project.

    2. Generate and download the service account key file by following the official instructions.

    Continue in the New connection dialog in Datalore

    1. In the Project ID, specify your project ID.

      Usually, it is a part of the service account email that goes after the at sign (@). For example, bigqueryproject-322409. For the project ID's format, refer to the official instructions on creating a service account.

    2. In the Service account email, type the service account's name. You can find the service account's name as Email on the Service accounts page (IAM & Admin | Service accounts) of the Google Cloud Platform. For more information about creating a service account for the name's format, refer to the official instructions.

    3. Under Key file, click Select file, and add the required file using the browser window.

    Obtain credentials

    1. Create an OAuth consent screen by following the official instructions.

      For more information about user consent, refer to the this article.

    2. Get a client ID and a client secret by following the official instructions.

      For the URI, use:

      • https://datalore.jetbrains.com/api/databases/v1/auth (for the Cloud version)

      • [your address]/api/databases/v1/auth (for the Enterprise version)

    Continue in the New connection dialog in Datalore

    1. In the Project ID, specify your project ID.

      Usually, it is a part of the service account email that goes after the at sign (@). For example, bigqueryproject-322409. For the project ID's format, refer to the official instructions on creating a service account.

    2. In the Client ID field, paste your client ID.

    3. In the Client secret field, paste your client secret.

    Obtain credentials

    1. Create an OAuth consent screen by following the official instructions.

      For more information about user consent, refer to the this article.

    2. Get a client ID and a client secret by following the official instructions.

      For the URI, use:

      • https://datalore.jetbrains.com/api/databases/v1/auth (for the Cloud version)

      • [your address]/api/databases/v1/auth (for the Enterprise version)

    3. Click Authenticate and fill tokens.

    Continue in the New connection dialog in Datalore

    1. In the Project ID, specify your project ID.

      Usually, it is a part of the service account email that goes after the at sign (@). For example, bigqueryproject-322409. For the project ID's format, refer to the official instructions on creating a service account.

    2. In the Client ID field, paste your client ID.

    3. In the Client secret field, paste your client secret.

    4. Click Authenticate and fill tokens.

    Obtain a credentials file

    1. Create a service account by following the official instructions.

      On the Grant this service account access to the project step in the wizard, select roles for this service account.

      For example, for read-only access, select BigQuery Data Viewer, BigQuery Job User, and BigQuery User from the BigQuery menu. Alternatively, select BigQuery | BigQuery Admin to access all resources within the project.

    2. Generate and download the service account key file by following the official instructions.

    Continue in the New connection dialog in Datalore

    1. In the Project ID, specify your project ID.

      Usually, it is a part of the service account email that goes after the at sign (@). For example, bigqueryproject-322409. For the project ID's format, refer to the official instructions on creating a service account.

    2. Under Key file, click Select file, and add the required file using the browser window.

    No special steps are required for this option.

  7. Based on the selected connection type:

    • In the Default dataset field, type the dataset name to which you want to connect.

    • In the URL field, provide the URL of the pre-built connection you want to establish.

  8. Click the Test connection button at the bottom of the dialog.

  9. Once the connection is successfully tested, click the Save and close button.

Results

  • If created for a specific notebook, the new connection will be automatically added to the workspace. You can later attach this database to any other notebook from this workspace.

  • If created for a workspace, this connection is added to the workspace databases and can be attached to any notebook from this workspace.

  • If OAuth authentication was used for this connection, report users and notebook collaborators may be required to provide their credentials to access the database.

  • To retrieve and process data from the connected database, use Query data with SQL cells.

Last modified: 11 July 2024