Databricks

How to set up the Databricks connection and additional notes on using it

With Codatum, you can connect, query, and manage data of Databricks.

This feature is currently in preview. Functionality and availability may change without notice.

We'd love your feedback.

How to Set Up

Databricks integration targets Serverless SQL Warehouses.

1. Necessary preparations in Databricks Console

Creat a service principal for integration and assign the appropriate permissions.

Account Admin Page

• From the menu, go to User Management > Service Principals and create a service principal for integration

• In the service principal's Credentials and Secrets tab, generate an OAuth secret. Save the client ID and secret at this time

• In the service principal's Roles tab, turn OFF all settings

• From the menu, select Workspaces, choose the workspace to integrate, go to the Permissions tab, and add the created service account

Workspace Admin Page

• Grant the service principal "Can Use" permission for the SQL Warehouse to connect

  • From the menu, go to SQL Warehouses and select the warehouse to integrate

  • Click the Permissions button in the upper right and grant the service principal Can Use permission

• Grant the service principal permissions to use the catalog

  • From the menu, go to Catalogs and select the catalog to integrate

  • In the Permissions tab, click the Grant button, select the service principal, and grant SELECT, USE CATALOG, and USE SCHEMA permissions

2. Register Databricks connection in Codatum

• From Workspace Settings, go to Connections and select New Connection

• Enter the Connection Name

• Enter the Access Level

• Enter the Server Host Name

  • In Databricks Workspace Console, go to SQL Warehouses from the menu, select the warehouse to integrate, and find the Server Hostname in the Connection Details tab (e.g., <your_instance_name>.cloud.databricks.com)

• Enter the HTTP Path

  • In Databricks Workspace Console, go to SQL Warehouses from the menu, select the warehouse to integrate, and find the HTTP Path in the Connection Details tab (e.g., /sql/1.0/warehouses/<your_warehouse_id>)

• Enter the Warehouse ID

  • In Databricks Workspace Console, go to SQL Warehouses from the menu, select the warehouse to integrate, and find the ID next to the Name in the Overview tab

• Enter the Client ID

  • The client ID of the service principal created in step 1

• Enter the Client Secret

  • The client secret of the service principal created in step 1

Once you've entered all the information, run a connection test and save the connection.