Skip to main content

Setting up Databricks as a data source

To connect to Databricks, you need to provide GrowthBook with the connection details for your SQL Warehouse along with credentials that let GrowthBook authenticate and run queries against it.

1. Find the connection details for your SQL Warehouse​

In your Databricks instance, navigate to the SQL Warehouses, select your SQL Warehouse that stores the data you want GrowthBook to access, and the click Connection Details. You should see the following page, which contains most of the fields needed to connect GrowthBook to your Databricks SQL Warehouse.

Finding SQL Warehouse Connection Details in Databricks

From this page you will use the Server Hostname, Port, and HTTP Path values when adding the data source to GrowthBook.

2. Choose an authentication method​

GrowthBook supports two ways to authenticate against Databricks:

  • OAuth (machine-to-machine / service principal) — the recommended method. GrowthBook uses a Databricks service principal's Client ID and OAuth secret to obtain short-lived access tokens automatically at query time.
  • Personal Access Token (PAT) — the legacy method. Databricks now recommends OAuth over personal access tokens because OAuth provides stronger security.

Whichever method you choose, make sure the service principal or user has permission to execute SQL queries against your SQL Warehouse.

Option A: OAuth (machine-to-machine)​

OAuth M2M authenticates using a Databricks service principal. GrowthBook exchanges the service principal's Client ID and OAuth secret for short-lived tokens (via the client_credentials grant) and refreshes them automatically, so you never store a long-lived token.

  1. Create a Databricks service principal (or reuse an existing one). See the Databricks docs on managing service principals.
  2. Generate an OAuth secret for that service principal. Databricks shows the Client ID (the service principal's application ID) and lets you generate a Secret. Copy both now — the secret is only shown once. See OAuth machine-to-machine (M2M) authentication.
  3. Grant the service principal permission to use your SQL Warehouse and access the data you want to query.
  4. In GrowthBook, you will enter the Client ID and OAuth Secret in step 3 below.

Option B: Personal Access Token​

There are two kinds of access tokens that will work to connect GrowthBook to Databricks:

  • A personal access token for a user account
  • A personal access token for a service principal

Follow these instructions to create a service principal and create an access token for the service principal: https://docs.databricks.com/en/dev-tools/service-principals.html

Alternatively, you can create a personal access token for a Databricks workspace user following these instructions: https://docs.databricks.com/en/dev-tools/auth.html#pat

3. Add credentials to GrowthBook​

Now that you have the connection details and credentials, you can add the data source in GrowthBook.

Click "Add a Datasource" from the Data Sources page under Metrics and Data and either select your event tracker or click "custom data source". Then you can select Databricks as your data warehouse and enter the above credentials.

Note: We only use the event tracker (e.g. Snowplow, Segment) to help us build queries for you. This information does not impact how or whether we are able to connect to Databricks.