The Case For Apache Iceberg: Moving Storage Off Snowflake Can Cut Your Bill In Half

We’ve seen Snowflake customers cut their total bill in half by moving storage into Iceberg. This might be hard to believe: your Snowflake bill is probably 90%+ compute, and the fraction you pay for storage isn’t even expensive - it’s basically the price you’d pay your cloud provider directly.

So then, how do those massive savings happen? Well, the real opportunity here isn’t cutting down storage costs: it’s saving money on compute by moving your ELT jobs out of Snowflake.

‍ELT vs. ETL: Why ELT Inside Snowflake is Expensive

ETL jobs - extract, transform, load - can be a good fit for data warehousing: the transform step is an analytical batch job that can spike compute requirements and therefore benefit from scaling cloud compute.

Ironically, as data warehouses shift best practice away from ETL and towards ELT - extract, load, transform - the load step just turns into a regular batch write. That means you’re paying a big premium for elastic compute just to shuffle bytes into storage.

The Case for Moving ELT Off Snowflake

Pushing bytes over the network is about as simple as it gets when it comes to data pipelines. Batch writes are easy to set up, scale well, and are simple to manage. With a little bit of investment in cluster administration, you can replace your Snowflake load jobs with something like Spark or dbt - at a fraction of the cost.

There are a lot of optimizations to be had here: depending on your cost structure, operational needs, and the level of bandwidth your team has for engineering complexity, you can save 90% or more on load costs when compared to Snowflake. As just one example, a fault-tolerant job with checkpoints can run on spot instances, which can be 70-90% cheaper than on-demand instances - and on-demand instances are already much cheaper than Snowflake clusters.

Benefits of Apache Iceberg

Iceberg, of course, isn’t just about cost savings. Moving your data to Iceberg will let you access it directly, so you can start experimenting with tools and solutions outside of Snowflake. If you’re already using multiple data warehouses, this may even decrease overall operational complexity: you can keep your data in one place instead of having multiple copies.‍

Next Steps‍

Here’s an exercise: break down your Snowflake compute bill by job, and figure how much of your bill is just going towards bulk writes. If you’re spending $100k here or more, think through what it would take to move that workload off Snowflake - it might be easier than you think.

‍