Databricks
Connect Erathos to Databricks.
Last updated
Connect Erathos to Databricks.
Last updated
You can connect the Erathos platform to any Databricks instance, regardless of the cloud environment it is hosted in (AWS, Azure, or Google Cloud).
To connect to your Databricks, the following information is necessary : Server Hostname, HTTP path and Access token. You can read the tutorial bellow on how to obtain these credentials.
The first step is to create a new SQL Warehouse. If you already have an existing one, you can skip this step.
To create a new SQL Warehouse, go to New and select SQL Warehouse from the options.
Name your SQL warehouse and define the configurations accordingly to your use, for example the cluster size, auto stop, scaling and type. When the configuration is done, click on Create.
Navigate to the Connection details tab to obtain the Server Hostname and HTTP path credentials.
To identify the Catalog Name select Catalog in the left bar. The list of available catalogs will be exhibited. Choose one of the options and copy the value, you will need it later to complete the Erathos platform.
In case you don't now your AWS Account ID, enter the databricks accounts page and select the Cloud resources option in the side bar. In the Role ARN column you can find the numeric ID between iam:: and :role.
There are two possible ways for supported authentication: Personal Access Token and OAuth. You can follow the steps down below to obtain both informations. Remember, only one of these ways is necessary to establish the connection.
In the Connection Details tab, in the selected SQL Warehouse, select the "Create a personal access token" option to create a new Access Token:
Select the Generate new token button and name the new token for identification purposes.
Warning: copy the token value, it will not be displayed again.
In order to iniciate you must create a new Service Principal. To do so, navigate to the databricks accounts inicial page and click on User Management in the side bar.
After that, select the Service Principals tab and click on the Add service principal button.
In the service principal creation page, make sure to name it so you can identify the user that will be connectes to Erathos. Click on Add to complete the creation. Now, returning to the list, click on the created service principal.
Click on the Generate secret button to create new values for Client ID and Client Secret (the Secret value will not be exhibited again, so make sure to copy and paste it somewhere safe)
Next, you will need to confirm the workspace's access permissions for the new service principal. To do so, select Workspaces in the side bar and click on the chosen workspace in the list of loades workspaces.
In the configurations, select the Permissions tab and click on Add permissions.
A text box will be exhibited, in which in must search the service principal's given name. Then, select the Admin permission level and click on save to finish.
To connect Erathos to your Databricks:
Create a New Destination
Go to Settings > Data Warehouse
Select Databricks.
Fill Out the Form
Set the Server Hostname, HTTP path and Catalog Name.
Add either an Access Token or Client ID and Client Secret for authentication.
Complete the Setup
If you need help connecting to your Databricks, reach out to our team at support@erathos.com.
