Understanding Databricks SQL Warehouse Types

Databricks offers three SQL warehouse types: Serverless, Pro, and Classic. All three differ in their operational characteristics and cost implications.

Serverless warehouses are fully managed by Databricks. Rather than using your compute (in AWS or GCP), Databricks uses their own shared infrastructure pool. This delivers a critical operational advantage: spin-up times of 4-6 seconds compared to approximately 4 minutes for Pro and Classic warehouses. This dramatic difference comes from Databricks' ability to distribute demand across their larger compute pool, eliminating most of the cold-start penalty.

What This Benchmark Measures

This benchmark uses the industry-standard TPC-H dataset (scale factor 10) that ships with Databricks by default, making these results easily reproducible. The dbt workflow processes multiple tables through base models, an operational data store (ODS) layer, and a dimensional warehouse with fact and dimension tables—a realistic representation of modern data transformation pipelines.

The workload completes in approximately 2 minutes, making it ideal for observing the performance characteristics of each warehouse type under real-world conditions. We test these workloads in both cold (i.e. warehouse just spun up) and warm (i.e. warehouse just ran the same workload) scenarios. 

Benchmark Results

| Warehouse Type | Warehouse Size | Run #      | Warm/Cold | Total Time (s) |
| -------------- | -------------- | -----------| --------- | -------------- |
|  Serverless    |  X-Small       | 1          | N/A       | 129            |
|                |                | 2          | N/A       | 127            |
|                |                | 3          | N/A       | 127            |
|                |                | Average    | N/A       | 127.66         |
|                |                |            |           |                |
|  Pro           |  X-Small       | 1          | Cold      | 147            |
|                |                | 2          | Warm      | 102            |
|                |                | 3          | Cold      | 156            |
|                |                | 4          | Warm      | 116            |
|                |                | 5          | Warm      | 119            |
|                |                | 6          | Cold      | 158            |
|                |                | Cold Avg   | Cold      | 153.67         |
|                |                | Warm Avg   | Warm      | 112.33         |
|                |                |            |           |                |
|  Classic       |  X-Small       | 1          | Cold      | 159            |
|                |                | 2          | Warm      | 129            |
|                |                | 3          | Cold      | 143            |
|                |                | 4          | Warm      | 102            |
|                |                | 5          | Warm      | 107            |
|                |                | 6          | Cold      | 160            |
|                |                | Cold Avg   | Cold      | 154            |
|                |                | Warm Avg   | Warm      | 112.67         |
|                |                |            |           |                |

Key Findings

The results reveal important performance patterns:

Serverless Beats Classic/Pro Cold: Serverless warehouses completed the workflow in an average of 127.66 seconds. In contrast, Pro and Classic warehouses starting from a cold state averaged 153.67s and 154s respectively. This represents approximately 20% better performance for Serverless on cold starts

Warm Cache Helps: Pro and Classic warehouses with warm caches performed slightly better at 112.33s and 112.67s respectively—roughly 12% faster than Serverless. This demonstrates the value of data caching when compute remains continuously available.

Classic Largely Matches Pro: Pro and Classic warehouses performed exactly the same on this benchmark, even though Classic warehouses are much cheaper. While Databricks docs claim that Classic "provides less performance than a Pro warehouse," this may not be true for all workload patterns.

The Cost-Performance Trade-off

While Pro and Classic warehouses can achieve marginally faster execution times when warm, maintaining that warm state requires keeping the warehouse running continuously — a costly proposition.

To achieve that 12% performance improvement, you must either:

- Keep your warehouse running 24/7, incurring compute costs during idle periods

- Accept the 4-minute cold start penalty every time the warehouse auto-suspends, potentially negating any performance gains

Serverless warehouses eliminate this trade-off. They start in seconds and perform better than cold-start Pro/Classic warehouses. Thus, for most data engineering workflows where workloads are bursty or scheduled, Serverless offers superior performance, all else equal.

Recommendations

1) For teams who want better performance without drastically increasing cost, Serverless warehouses deliver the best overall value. 

2) Reserve Pro or Classic warehouses only for sustained loads where cache benefits can be maximized. In these cases, Classic warehouses are likely to be cheaper for the same performance.

3) To save up to 50% on your Databricks automatically, book a call with our team.