> For the complete documentation index, see [llms.txt](https://docs.erathos.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.erathos.com/~/changes/58/destinations/databricks.md).

# Databricks

Connect Erathos to Databricks.

{% hint style="info" %}
You can connect the Erathos platform to **any Databricks instance**, regardless of the cloud environment it is hosted in (AWS, Azure, or Google Cloud).
{% endhint %}

To connect to your Databricks, the following information is necessary : **Server Hostname**, **HTTP path** and **Access token**. You can read the tutorial bellow on how to obtain these credentials.

## Databricks Setup 

### Create a SQL Warehouse (Optional) 

The first step is to create a new SQL Warehouse. If you already have an existing one, you can skip this step.

To create a new SQL Warehouse, go to **New** and select **SQL Warehouse** from the options.

![](https://help.erathos.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBYWs9IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--2780fca85fd0e73ab92efeb39a65c993b528bbbd/Untitled%20\(75\).png)

Name your SQL warehouse and define the configurations accordingly to your use, for example the cluster size, auto stop, scaling and type. When the configuration is done, click on **Create**.

![](https://help.erathos.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBYW89IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--9529c7242f19c95d221767dfd6a2c0a22d4f583c/Untitled%20\(76\).png)

### Credentials

Navigate to the Connection details tab to obtain the **Server Hostname** and **HTTP path** credentials.

![](https://help.erathos.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBYXM9IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--64cb4527c1e1ec2c7fcea80d9c25c702ab20ef98/Untitled%20\(77\).png)

To identify the **Catalog Name** select **Catalog** in the left bar. The list of available catalogs will be exhibited. Choose one of the options and copy the value, you will need it later to complete the Erathos platform.

![](https://help.erathos.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBZzBCIiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--93cc87f763601a45eaccedd3d284e71cc3f90628/Screenshot%20from%202024-09-11%2013-07-07.png)

In case you don't now your **AWS Account ID,** enter the databricks accounts page and select the **Cloud resources** option in the side bar. In the **Role ARN** column you can find the numeric ID between **iam**:: and :**role.**

![](https://help.erathos.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBZ3dCIiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--dd31e052b1e9894dcf7f9f40101522289728f7e6/Screenshot%20from%202024-09-11%2013-01-05.png)

There are two possible ways for supported authentication: Personal Access Token and OAuth. You can follow the steps down below to obtain both informations. Remember, only one of these ways is necessary to establish the connection.

#### Personal Access Token:

In the **Connection Details** tab, in the selected SQL Warehouse, select the "Create a personal access token" option to create a new **Access Token:**

![](https://help.erathos.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBYWM9IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--fefbcfac654464909136d8201e913b8072131e08/Untitled%20\(73\).png)

Select the **Generate new token** button and name the new token for identification purposes.

**Warning**: copy the token value, it will not be displayed again.

![](https://help.erathos.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBYWc9IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--afd5fe77dc776870993f6dd5d8c7cad0b136d2e9/Untitled%20\(74\).png)

#### OAuth

In order to iniciate you must create a new Service Principal. To do so, navigate to the databricks accounts [inicial page](https://accounts.cloud.databricks.com/login?tuuid=b8f52cec-89bf-4f80-ba8a-946b775d21b7) and click on **User Management** in the side bar.

After that, select the **Service Principals** tab and click on the **Add service principa**l button.

![](https://help.erathos.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBZ2NCIiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--acc7ae6ff4958ce5b363a091caa9e53f8948dfb0/Screenshot%20from%202024-09-11%2012-23-42.png)

In the service principal creation page, make sure to name it so you can identify the user that will be connectes to Erathos. Click on **Add** to complete the creation. Now, returning to the list, click on the created service principal.

![](https://help.erathos.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBZ2dCIiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--d0cab8e60e46a3799c8ff69fff53799ea56aae79/Screenshot%20from%202024-09-11%2012-25-36.png)

Click on the **Generate secret button to create new values for** Client ID **and** Client Secret (the Secret value will not be exhibited again, so make sure to copy and paste it somewhere safe)

![](https://help.erathos.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBZ3NCIiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--6237708ae70a8915b3467e529c72cb0c94c4fcb8/Screenshot%20from%202024-09-11%2012-48-24.png)

Next, you will need to confirm the workspace's access permissions for the new service principal. To do so, select **Workspaces** in the side bar and click on the chosen workspace in the list of loades workspaces.

![](https://help.erathos.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBZ2tCIiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--e6f4c4e033f6595055d47713d817529fcd84109c/Screenshot%20from%202024-09-11%2012-29-51.png)

In the configurations, select the **Permissions** tab and click on **Add permissions.**

![](https://help.erathos.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBZ29CIiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--4612aeedd5fb3de96bc9e4d5ea2475bf31a879c0/Screenshot%20from%202024-09-11%2012-34-14.png)

A text box will be exhibited, in which in must search the service principal's given name. Then, select the **Admin** permission level and click on save to finish.

## Connect to Erathos

To connect Erathos to your Databricks:

1. **Create a New Destination**
   * Go to **Settings** > **Data Warehouse**
   * Select **Databricks.**
2. **Fill Out the Form**
   * Set the **Server Hostname, HTTP path** and **Catalog Name**.
   * Add either an **Access Token** or **Client ID** and **Client Secret** for authentication.
3. **Complete the Setup**

{% hint style="info" %}
If you need help connecting to your Databricks, reach out to our team at <support@erathos.com>.
{% endhint %}