In the last 5 months, Generative AI Community has organised:
- Demo Day Meetup in February, a Deep Coffee Hack,
- A Hackathon with 5L INR in cash prizes, 43 submissions from almost 100 participants!
- April meetup which saw a stellar demo on a chatbot for farmers, evaluating question answering from a Llama Index contributor and an Intro to Model Quantisation from Amod Malviya
- May meetup that saw talks on unet architecture by Vignesh, CTO hexo.ai and composability with stable diffusion by Amogh Vaishampayam, Founding Team @ dashtoon.ai
- June Meetup which saw a discussion with Amod Malviya, Kailash Nadh and Charu Tak.
Pratyush is a researcher at Microsoft Research and AI4Bharat (IIT M) with a focus on systems and deep learning for language technologies. He is deeply interested in realising AI as a force of social good.
Talk Theme: Brief story of AI4Bharat - the mission and what we have achieved. Will then move on to LLMs, provide some technical insight into intrinsic dimensionality in deep learning, and his take on how innovation will develop in building custom LLMs
Sachin Dharashivkar will speak about LLM Finetuning and RLHF
Sachin is a founder who is exploring use cases of AI agents. He enjoys training Reinforcement Learning agents and exploring novel applications of Large Language Models.
Talk Theme: Introduction to Supervised and Reinforcement Finetuning.
Three steps of training chatGPT style models. How to perform supervised finetuning. Why is Reinforcement Learning from Human Feedback important and How to train Reward and Policy models.
Location: Bengaluru. Exact location is shared in the invite on approval.
Developing User-Friendly LLMs: Introduction to Supervised and Reinforcement Finetuning
Three years ago, OpenAI’s GPT-3 model created a buzz in the Machine Learning community and garnered some media attention. However, it had limited impact on regular users. In contrast, the launch of chatGPT two and a half years later became a viral sensation. This was due to the user-friendly and enjoyable experience it provided. In this talk, we’ll explore model finetuning approaches that contributed to the excitement around chatGPT. We’ll cover when and who should do this finetuning and give a quick overview of notebooks used to customize Large Language Models- LLMs for specific purposes.
Developing a system like chatGPT involves three stages: pretraining the base language model, supervised finetuning, and finetuning with Reinforcement Learning. The first stage is crucial for good performance, but it’s expensive ($100,000 - $10,000,000) and requires deep expertise. On the other hand, finetuning existing models is cheaper ($100 - $1,000) and easier. With the availability of many open source large language models, we’ll show hackers how to customize them for their specific needs. We’ll also discuss different situations and the corresponding finetuning strategies.
This talk is mainly geared towards Machine Learning engineers, as we will go through some code snippets. However, we will also discuss high-level concepts to make it useful for Generative AI enthusiasts.
- I have been training Deep Learning models using Supervised Learning and Reinforcement Learning since 2016.
- Over the years, I’ve trained Deep Reinforcement Learning agents to play games like Doom and Overcooked, as well as execute equity trades efficiently.
- In 2021, I finetuned the T5 language model for my startup, enabling it to generate questions and answers from paragraphs.
- I also write about training Large Language Models on my blog at https://sachindharashivkar.substack.com/
Slides of the talk are at- https://docs.google.com/presentation/d/1Unw6GsoCHA2KmFMHOOhenamU3Do1VflMVksbmAancLs/