Query data with SQL cells

Use SQL cells to query the following data sources:

Attached databases
DataFrames
Attached .csv files

Query a database in an SQL cell

Add an SQL cell:
- Go to Tools | Attached data and click Connect SQL cell for the attached database that you want to query.
- Hover over the bottom border of a cell and select More | SQL. If there are several attached databases, in the cell, click Select database and select the required data source.
Enter an SQL statement.
The cell provides code completion based on the introspected database schema.
(Optional) You can also use the following options on the cell toolbar:
- Browse schema: to view the queried database schema, which will be shown in the Attached data tool.
- Limit 500: to set the limit of returned rows to 500.
Run the query (Run icon or Ctrl+Enter shortcut). The result set will be shown in the output and saved to a DataFrame. The resulting DataFrame name is shown in the cell toolbar.
By clicking the Stop button or interrupting/restarting the kernel, you explicitly stop querying the database.
(Optional) To rename the data object, click the object name next to RESULT SAVED TO on the SQL toolbar.

Example 1

The image below shows an SQL cell querying a PostgreSQL database, 500 row limit enabled.

Query a DataFrame or .csv file in an SQL cell

The procedure queries a DataFrame or a .csv file from notebook or workspace files and saves the result set to a DataFrame.

Hover over the bottom border of a cell and select More | SQL.
In the added SQL cell, click Select database and select Dataframes.
(Optional) To rename the resulting DataFrame:
1. Click the name next to RESULT SAVED TO on the SQL cell toolbar.
2. Enter a new name and press Enter.
Enter an SQL statement. Run the query. The result set will be shown in the output and saved to a new DataFrame under the name shown on the SQL cell toolbar.
By clicking the Stop button or interrupting/restarting the kernel, you explicitly stop querying the database.
(Optional) To get a visual representation of the retrieved data, switch to the Visualize tab. Find more details in Use automatic plotting.

Example 2

The image below shows a code cell that generates a DataFrame, and an SQL cell that queries that DataFrame.

Example 3

The image below show an SQL cell that queries a .csv file stored in Notebook files.

Parameterized SQL queries

You can use Python variables in your SQL queries in Datalore. Such queries can be reused repeatedly with different values, which helps make your reports more interactive.

Supported variable types are: strings, numbers, booleans, and lists. Make sure you place your variable inside {} brackets.

Parameterized SQL query example

In the image below, you can see two cells:

Dropdown interactive control cell using the method variable.
SQL cell where the value in payment column equals the method variable value selected from the dropdown list.

Table parametrisation for SQL cells

Starting from this version (2023.5), Datalore allows for table paramterization by supporting unsafe parameters for SQL cell statements. The syntax is as follows:

select * from {table_name | unsafe} limit {n}

In the code example above, the {table_name | unsafe} part will assume the value of the table_name variable.

String variables for unsafe parameters

To pass a string variable to an unsafe parameter, use quote escaping quotes. See the examples below:

Unsafe parameter defined in a code cell: product = "'Product'"
Unsafe parameter referenced in an SQL cell: SELECT * from releases where jb_product in ({product | unsafe})
Resulting SQL query: SELECT * from releases where jb_product in ('Product')

Run SQL cells as functions from other cells

Datalore allows you to wrap queries from SQL cells into Python code and use them as functions with parameters. The function with an SQL cell statement has the following syntax: {DataFrame_name}_execute_sql_cell, where {dataframe_name} is the name of the DataFrame that saves the result of the SQL cell you want to use.

Example

Cell 1, Python

import pandas as pd
idd = [1,2]
lim = 10
        

Cell 2, SQL (result saved to a df_1 DataFrame)

            select * from small_iris_df
            where small_iris_df."Id" in {idd}
            limit {lim}
        

Cell 3, Python (function using the df_1 DataFrame from Cell 2)

            frames = [await df_1_execute_sql_cell(i,5,2)  for i in range(5) ]
            # Number of the function parameters matches the count of values of variables in the SQL statement: idd - 2 values , lim - 1 value.
            pd.concat(frames)

Keywords

sql statement, sql query, retrieve data, query table, query database

Last modified: 11 September 2024