Configure Big Data Tools environment

Mandatory parameters:

• URL: the path to the target server.

• Login and Password: your credentials to access the target server.

Optionally, you can set up:

• Name: the name of the connection to distinguish it between the other connections.

• Login as anonymous: select to login without using your credentials.

• Per project: select to enable these connection settings only for the current project. Deselect it if you want this connection to be visible in other projects.

• Enable connection: deselect if, for some reasons, you want to restrict using this connection. By default, the newly created connections are enabled.

• Library Versions: Scala Version, Spark Version, and Hadoop Version: these values are derived from the plugin bundles. If needed, specify any alternative version values.

• Enable HTTP basic authentication: connection with the HTTP authentication using the specified username and password.

• HTTP Proxy: connection with the HTTP or SOCKS Proxy authentication. Select if you want to use IDEA HTTP Proxy settings or use custom settings with the specified host name, port, login, and password.

Mandatory parameters:

• Bucket: a globally unique Amazon S3 bucket name.

Optionally, you can set up:

• Name: the name of the connection to distinguish it between the other connections.

• Region: an AWS region of the specified bucket. You can select one from the list or let IntelliJ IDEA to auto detect it.

• Root path: a path to the root directory in the specified bucket.

• Authentication type: the authentication method. You can use your AWS account credentials (by default), or opt to entering the access and secret keys.

• Per project: select to enable these connection settings only for the current project. Deselect it if you want this connection to be visible in other projects.

• Enable connection: deselect if, for some reasons, you want to restrict using this connection. By default, the newly created connections are enabled.

• Use custom endpoint:

Mandatory parameters:

• URL: the path to the target server.

Optionally, you can set up:

• Name: the name of the connection to distinguish it between the other connections.

• Per project: select to enable these connection settings only for the current project. Deselect it if you want this connection to be visible in other projects.

• Enable connection: deselect if, for some reasons, you want to restrict using this connection. By default, the newly created connections are enabled.

• Enable HTTP basic authentication: connection with the HTTP authentication using the specified username and password.

• Enable HTTP proxy: connection with the HTTP proxy using the specified host, port, username, and password.

• HTTP Proxy: connection with the HTTP or SOCKS Proxy authentication. Select if you want to use IDEA HTTP Proxy settings or use custom settings with the specified host name, port, login, and password.

• Kerberos authentication settings: opens the Kerberos authentication settings.

  

  Specify the following options:

  • Enable Kerberos auth: select to use the Kerberos authentication protocol.

  • Krb5 config file: a file that contains Kerberos configuration information.

  • JAAS login config file: a file that consists of one or more entries, each specifying which underlying authentication technology should be used for a particular application or applications.

  • Use subject credentials only: allows the mechanism to obtain credentials from some vendor-specific location. Select this checkbox and provide the username and password.

  • To include additional login information into IntelliJ IDEA log, select the Kerberos debug logging and JGSS debug logging.

    Note that the Kerberos settings are effective for all you Spark connections.

Mandatory parameters:

• Root path: a path to the root directory.

Optionally, you can set up:

• Name: the name of the connection to distinguish it between the other connections.

• Per project: select to enable these connection settings only for the current project. Deselect it if you want this connection to be visible in other projects.

• Enable connection: deselect if, for some reasons, you want to restrict using this connection. By default, the newly created connections are enabled.

Mandatory parameters:

• Root path: a path to the root directory on the target server.

• Config path a path to the HDFS configuration files directory. See the samples of configuration files.

• File system URI an explicit uri of an HDFS server. Once you select this option, you need to specify the file system URI, for example localhost:9000 and a username to connect.

Optionally, you can set up:

• Name: the name of the connection to distinguish it between the other connections.

• Per project: select to enable these connection settings only for the current project. Deselect it if you want this connection to be visible in other projects.

• Enable connection: deselect if, for some reasons, you want to restrict using this connection. By default, the newly created connections are enabled.

Note that the Big Data Tools plugin uses the HADOOP_USER_NAME env variable to login to the server. It this variable is not defined then the user.name property is used.

Mandatory parameters:

• Bucket: a name of the basic container to store your data in Google Storage.

• Cloud store JSON location: a path to the Cloud Storage JSON file.

Optionally, you can set up:

• Name: the name of the connection to distinguish it between the other connections.

• Base directory (root by default): storage base directory.