The ingredients of good software engineering, and more - February 2024 newsletter
The bread and butter of good software engineering
IN THE SPOTLIGHT
THE FIFTH ELEPHANT
The secret ingredient: How good datasets shape state-of-the-art AI models
A good dataset is essential whether you’re pre-training LLMS or fine-tuning them. To know what a good dataset is, it might be easier to start with what bad datasets are and work backwards from that. Dealing with ill-formatted or truncated data can either be done programmatically or could be automated using smaller language models like tiny Llama. Wrangling data in a post-LLM world is majorly easier and cost effective. Anthropomorphizing the dataset and curriculum learning are some new ideas in the space - and these have been getting promising results.
Preserving the dataset diverse after wrangling, having low perplexity for the LLM and having cohesivity within the training data are some of the key points of focus Abhishek looks at while developing state of the art models.
THE FIFTH ELEPHANT
How much math do you really need for LLMs?
By Harshad Saykhedkar
Apparently, not much - if you ever did your math homework diligently during school or college days, the techniques used to run Large Language Models faster and cheaper are not hard to understand.Turns out, quantising a range of larger numbers to a range of smaller ones is most of what one needs for getting our Llamas and Falcons running inference faster without much drop in quality - terms and conditions apply, as it is with any engineering decision.
Evaluate Quantized LLMS and understand whether or not it’s worth it for your workflows as Harshad breaks down the Lego blocks of generative transformers. He shares how current state of the art might look like for running LLM powered inference at the edge - all while maintaining a simplicity in the math; you might just question whether that PhD you did for Math was really worth it - it was, it just isn’t needed to understand LLM quantization. :P
ROOTCONF
Optimising Cloud Infrastructure with a Hybrid Strategy at Flipkart
By Sudhir Reddy
Sudhir chalks out Flipkart’s infrastructure journey from starting out from on prem to moving to private cloud and trying out public cloud on a per service basis, with an eventual move to hybrid. Serving major sales events such as “Big Billion Days,” “Independence Day,” and “Republic Day” year on year, with Flipkart’s data platform growing at 50%, required taking a careful look at things such as data compression, event batching and rolling out custom solutions.
Automating cloud cost savings recommendations gave cost visibility to application developers, and Flipkart’s decision to stick with a hybrid setup gave the flexibility to either stick with on-prem racks or go with the cloud provider’s service.
With a ruthless approach to improving unit costs, from increasing resource utilization to improving electricity footprint, to scrapping and rebuilding data centers to gain better compute density. Sudhir narrates and elucidates how Flipkart gains to get the most bang for their buck.
HAPPENINGS AT HASGEEK
THE FIFTH ELEPHANT
[Meetup]: AI and Risk Mitigation in Edtech session
Sunayana Sitaram, Principal Researcher, and Tanuja Ganu, Principal RSDE Manager at Microsoft Research India will discuss use cases of AI in Edtech, and risks and caveats associated with the same.
To join the session, RSVP here
Link to previous talk
ROOTCONF
[AMA] SQL and Schema Design - Will ChatGPT make SQL obsolete?
Sunayana Sitaram, Principal Researcher, and Tanuja Ganu, Principal RSDE Manager at Microsoft Research India will discuss use cases of AI in Edtech, and risks and caveats associated with the same.
To join the session, RSVP here
FRESH: VIDEO HIGHLIGHTS
Ethical Concerns in Use of AI in Healthcare
Unlocking SRE success: a two-decade journey
MEMBERS IN ACTION
Sreeram Venkitesh - member, has been talking very fervently about Kubernetes at DevOps conferences such as Kubernetes Community Days (KCD) Kerala and more recently Rootconf SRE. He has also received the Dan Kohn scholarship for attending KubeCon Paris. He is part of the Kubernetes Release Team and also writes for Last Week in Kubernetes Development.
COMMUNITIES IN FOCUS
Rust Bangalore
Organises regular meetups exploring better application of the Rust programming language. Discover more>>
Papers We Love
Initiated by a Member of The Fifth Elephant group, this group gets serious about technology papers and their potential impact. Discover more>>
UPCOMING EVENTS / AROUND THE CORNER
[MIXER] Rootconf mixer for engineering managers | 23 February 2024, 5:30 PM onwards | Read more ...
[MEETUP] Data privacy by design - strategies and patterns | 29 February 2024, 5:30 PM onwards | Read more ...
[HACKATHON] The Fifth Elephant Open Source AI hackathon | Submission close 28 February 2024 at 9 PM | Read more ...