IntelliJ IDEA 2019.3 Help

Big Data tools

The Big Data Tools plugin is available for IntelliJ IDEA 2019.2 and later. It provides specific capabilities to process data with Zeppelin and Spark. You can create new or edit existing local or remote Zeppelin notebooks, execute code paragraphs, preview the resulting tables and graphs, and export the results to various formats.

Big data tools UI overview

The plugin supports many IDE features to work with notebooks:

Coding assistance for Scala

Notebook features

Getting started with Big Data Tools in IntelliJ IDEA

The basic workflow for big data processing in IntelliJ IDEA includes the following steps:

Configure your environment

  1. Install the required plugins:

  2. Create a new project in IntelliJ IDEA.

  3. Configure a connection to the target server.

Work with notebooks

  1. Create and edit a notebook.

  2. Execute the notebook.

  3. Analyze your data:

Get familiar with the user interface

When you install the Big Data Tools plugin for IntelliJ IDEA, the following user interface elements appear:

Big Data Tools window

The Big Data Tools window appears in the rightmost group of the tool windows.

Big Data Tools window

The Big Data Tools window displays the list of the configured servers and user notebooks structured by folders. For the basic operations with the servers, use the window toolbar:

Item

Description

Add connection

Adds a new connection to a server.

Refresh Connection

Refreshes connections to all configured servers.

Connection settings

Opens the connection settings for the selected server.

Notebook editor

Zeppelin notebook editor

In the notebook editor, you can add and execute Scala and SQL code paragraphs. When editing your code paragraph, you can use all the coding assistance features available for a particular language. Code warnings and errors will be highlighted in the corresponding code constructs in the scrollbar. The results of paragraph execution are shown in the preview area below each paragraph.

Use the notebook editor toolbar for the basic operations with notebooks:

Item

Description

Run all

Executes all paragraphs in the notebook.

Stop execution

Stops execution of the notebook paragraphs.

Clear all outputs

Clears output previews for all paragraphs.

Interpreter bindings

Opens the Interpreter Bindings dialog to configure interpreters for the selected notebook.

Open in a browser

Opens the notebook in the browser.

Module settings

Opens the module settings.

Minimap

Shows the minimap for quick navigation through the notebook.

The notebook editor toolbar also shows the status of the last paragraph execution.

Status of the paragraph execution
Execution with errors occurred
.
Execution with errors occurred
.

Zeppelin tool window

Zeppelin tool window

The Zeppelin tool window shows the connection status for configured Zeppelin servers and provides links to the server for quick navigation.

Last modified: 2 December 2019