Cloud Costs Optimization

Snowflake: New kid on block in the world of Cloud Cost Optimization

Introduction

Unlike the world of AWS/GCP optimization, data clouds like Snowflake are a different beast to tame.

Chaos Genius is a pioneer in the field of Data Cloud Cost optimization starting with Snowflake. Chaos Genius provides solutions to organizations to reduce their spend on Snowflake and maximize query performance.

In this talk, we want to talk about the growing footprint of data warehouses like Snowflake, Databricks, Bigquery in enterprises and share our learnings from working with some of the largest Snowflake customers in the world and how Chaos Genius helped them reduce their Snowflake costs while improving their query performance.

Snowflake: How do costs work?

Here we will talk about how Snowflake has a unique cost structure where it divides your consumption across Compute, Storage and Data Transfer costs. For most parts, Compute costs make up for majority of Snowflake spend. These costs comprise of warehouse consumption, and usage of various serverless services like Snowpipe, Auto-clustering, Search Optimization, Query Acceleration etc. This is mostly followed by Storage costs which can slowly stack up over time specially a large number of tables which go unused.

What does Snowflake optimization comprise?

Here we will talk about our pioneering approach towards data cloud optimization consisting of:
Visibility
Why & How tracking down costs down to user and query level is the most important aspect of reducing your data cloud costs
Optimization
There are 3 legs to optimizing your data cloud costs
Warehouse Right-sizing: Finding your over-provisioned and under-provisioned warehouses using unique metrics like Warehouse Utilization, Disk Spillage, warehouse configs, clustering, etc.
Workload Optimization: Identifying most expensive workloads using unique technology like Query Fingerprinting and tuning them.
Storage Optimization: Identify storage cost structure across active storage, time-travel, fail-safe, cloning etc. and also identifying dormant tables.
Observability
Cost and performance optimization is not a one time exercise and we will talk about the importance of on-going observability using techniques like anomaly detection and alerting

Learnings from Snowflake Summit 2023: Generative AI & data cost implications

Here we will talk about the impact of generative AI on data costs and why it will make data costs one of the most important pillars of keeping your infra costs under control while maximising data ROI.

Speaker: Preeti Shrimal, Co-Founder & CEO, Chaos Genius

Outline Draft: https://docs.google.com/document/d/1ZGolk6c2mVkjUTzL_QZ4o7hVG2TjUxHM2OKR9p4TQec/edit?usp=sharing