What is a system?

A system is a set of things working together as parts of a mechanism or an interconnecting network; it is a complex whole. The definition comes from the Oxford Dictionary.

Background

When we talk about GenAI applications, there is a lot going on under the hood. First, we have a large language model (yes, we are going to be talking in the context of NLP). Then, we have the hardware required to run these models. What follows is a complex system to marry the two efficiently to derive the highest throughput and efficiency.

What to expect in this talk?

For this talk, I assume that the audience can come from different backgrounds, so I will start by discussing the basics, like what a GPU is and why do we need it.
After setting the context, I will explain how LLM runs on GPUs.
Finally, I will discuss about the systems that marry the two enabling efficient execution of LLMs on GPUs deriving higher throughput and efficiency.

About the speaker

Simrat Hanspal is an applied Data Scientist with over 14 years of experience. She specializes in Natural Language Processing (including Generative AI). She has worked with multiple product companies like Amazon, VMware, Epifi, Nirvana, Hasura, etc., and has led multiple initiatives working with founders and leadership.
She also heads the AI division at CodeBlossom to help women upskill/enter tech.