BEGIN:VCALENDAR VERSION:2.0 PRODID:-//HasGeek//NONSGML Funnel//EN DESCRIPTION:Stories of SQL server\, CosmosDB and all that makes infrastruc ture at Udaan X-WR-CALDESC:Stories of SQL server\, CosmosDB and all that makes infrastru cture at Udaan NAME:The battle of scale versus costs X-WR-CALNAME:The battle of scale versus costs REFRESH-INTERVAL;VALUE=DURATION:PT12H SUMMARY:The battle of scale versus costs TIMEZONE-ID:Asia/Kolkata X-PUBLISHED-TTL:PT12H X-WR-TIMEZONE:Asia/Kolkata BEGIN:VEVENT SUMMARY:The battle of scale versus costs DTSTART:20230721T120000Z DTEND:20230721T133000Z DTSTAMP:20260421T130832Z UID:session/SQpibnVKjVxFXNiqgrVHW4@hasgeek.com SEQUENCE:19 CREATED:20230705T092838Z DESCRIPTION:# Context\nUdaan reduced >75% in database costs over two years while growing their business.\nIs your organization burning money because of wrong infrastructure choices? Or\, because of a gap in understanding w hat technologies to use?\n**This talk focuses on understanding key concept s relevant to DB cost optimization and practical strategies to cut down co sts by leveraging the right tools.**\n\n## Key takeaways for the audience\ nThere are better ways to improve the performance and efficiency of a data base than throwing money at it.\nUnderstanding certain key database concep ts can help all engineers reduce costs - how to choose the right cloud off erings\, right indexes\, DB maintenance ops and tools. It doesn't have to be hard if we leverage the right tools like Snorql which the team built at Udaan.\n\n## Who should attend/participate?\n* Anyone who wants to unders tand their SQL/cosmos databases well and how to make the best out of it wi thout splurging money.\n* Engineers interested in learning cost optimizati on strategies for their databases.\n* DBAs who want to simplify their life by using tools built by us.\n\n## Talks\n### SQL - by Lokesh\nTalking abo ut the kind of problems and surprises we encountered when we started optim izing our SQL server instances. \n\n**Cloud offering specific optimization s - choosing the right offering**\nTo help choose between serverless vs pr ovisioned\, elastic pool vs isolated\, business-critical vs general purpos e vs hyper-scale\, and Local disk vs network disk. Understanding why certa in decisions have a high reversal cost & should be thought through during design.\n\n**Role of database maintenance**\nRebuilding\, reorganizing\, u pdating index statistics\, redundant index analysis\, and removal\, archiv al/purge.\n\n**Understanding indexes and query gotchas**\nIO vs storage tr adeoff\, clustered vs unclustered\, order and specificity of columns\, key lookups and included columns\, conditional indexes\, overlapping and dupl icate indexes\, identifying implicit type\, exploiting page compression.\n \n**Leveraging Snorql and tools**\nHow we took a tool-first approach to so lve the above problems.\nStory of how we built a diagnostics and optimizat ion tool to save our cost and time and decided to open-source this. And ho w to leverage this tool to do optimizations without being a hardcore DBA.\ n\n### Cosmos by Rohit\n**Scale vs Provisioning** 
\nUnderstanding the s cale and traffic pattern to choose right model of pricing. Autoscale prici ng model vs Manual fixed Pricing model trafeoffs\n2. **Partition key**
 \nChoice of partition key is one of the most important choices we make in NoSQL. It should be serve majority of queries and at the same time preven t hot partitions\, and bottlenecks. 
Cosmos specific - > How spike in si ngle partition can cause entire container cost to spike\, some quick remed ies. Saving costs with materialised view (multiple partition keys support for same DB) .\n3. **Indexes and composite indexes** 
\nChoice of index and optimised composite indexes for expensive queries. Cost saving in % wi th real examples using composite index. Like 25000RU(request units) to 300 RU
Cosmos specific -> order of columns in query matters in composite ind exes for cosmos. Like if we create composite index for column (A\,B\,C) an d use query like where A=a1\, C=c1 and B=b1 will not work\, as order of co lumn B and C are not proper. Extending this\, to reuse composite indexes a cross queries which have common columns\, we need to maintain the relative order in query
\n4. **Misc strategies**
 \nLike 
- sharing throughp ut across multiple containers in an account which have low throughput requ irement.
- Dynamic real time scale change in autoscale models for heavy cron jobs (upto 70% cost saving depending on useacase)
- use of synapse analytics for cheap and price effective bulk ingestion for corncobs\n5. ** In-house customised analysis and recommendation system**
\nThis understa nds our usage and traffic pattern. It can alert us if there is sudden incr ease in resources like due to some unoptimised queries or potential hot pa rtitions
It further recommends strategy like
- identifying low usage collections and sharing throughput across them
- How much resources to p rovision and what pricing model to choose\n\n## About the speakers\n[Lokes h Devnani](https://www.linkedin.com/in/lokeshdevnani/) is software enginee r at Udaan. \n[Rohit Sinha](https://www.linkedin.com/in/rohit-sinha-48b803 1b3/) is software engineer at Udaan.\n\n## About the reviewers\nAnand Chit ipothu is editor at Rootconf. \nSwanand Pagnis is the organizer of [Papers WeLove Bangalore](https://hasgeek.com/pwl_bangalore/) chapter. \n\n## Supp ort Rootconf's community activities with a membership\nRootconf is a commu nity funded organization. If you like the work that Rootconf does and want to support it\, consider contributing by picking up a membership. \n\n## Contact\nJoin the Rootconf Telegram group at or fo llow [@rootconf on Twitter](https://twitter.com/rootconf).\nFor inquiries\ , contact Rootconf at +91-7676332020. LAST-MODIFIED:20230902T111357Z LOCATION:Online/Bangalore - https://hasgeek.com/rootconf/infra-at-udaan/ ORGANIZER;CN=Rootconf:MAILTO:no-reply@hasgeek.com URL:https://hasgeek.com/rootconf/infra-at-udaan/ BEGIN:VALARM ACTION:display DESCRIPTION:The battle of scale versus costs in 5 minutes TRIGGER:-PT5M END:VALARM END:VEVENT END:VCALENDAR