The Fifth Elephant 2025 Annual Conference CfP

Introduction

Did you know that almost all modern Large Language Model (LLM) applications have Retrieval-Augmented Generation (RAG) as a core component? This workshop is designed to help developers and ML engineers build powerful LLM applications leveraging RAG and explore techniques to enhance them beyond basic proof-of-concepts. Participants will gain a solid understanding of the core RAG architecture and learn strategies to improve the retrieval performance that feeds into LLMs, enabling them to handle more complex queries using advanced methods and AI agents. The hands-on approach ensures that learners grasp both the foundational theory and its practical application.

The course focuses on bridging the gap between simple prototypes and more effective RAG implementations for LLMs. It provides essential techniques for optimizing the retrieval mechanisms that inform LLM generation and enhancing overall system performance. By focusing on advanced optimization techniques such as hybrid search, reranking, metadata filtering, and the use of agentic workflows, participants gain skills needed to improve system reliability and the quality of LLM-generated responses for many common use cases.

Throughout the workshop, participants engage in hands-on exercises that build upon each other. Starting with basic RAG implementations for LLMs, participants will explore the limitations of “naive” RAG, followed by practical improvements using advanced techniques. The exercises demonstrate how the performance and reliability of these LLM systems can be improved step-by-step, ensuring participants learn how to build more robust RAG-powered applications.

By the end of the workshop, participants will be equipped with the knowledge and skills to develop and refine LLM applications using RAG, moving them towards more capable and reliable systems ready to address real-world challenges.

Link to Source:

• Slides
• Exercises

Agenda:

RAG Workshop Schedule (~4 hours)

Segment 1: Foundations of Retrieval-Augmented Generation (RAG)

Duration: 2 Hours (Including Breaks)
Objective: Establish a foundational understanding of RAG, its necessity, and core components.

• Poll: Pulse check on understanding of Retrieval Augmented Generation

1.1 The Foundations of RAG (30 min)

• Dive into the RAG architecture: Semantic Search, Embeddings, and Vector Stores.
• Simple demo for embeddings

1.2 The “Naive” RAG Architecture (30 mins)

• Introduction to a basic RAG setup.
• Discuss the interplay between retrieval mechanisms and generative models.

1.3 Q&A

1.4 Building a Basic RAG Example (30 min)

• Hands-On/Exercise: Create a simple RAG application using Python.

1.5 Challenges with “Naive” RAG Implementations (20 min)

• Discuss common issues with naive RAG.
• Explore the impact of these challenges on overall LLM Application performance.

1.6 Q&A

Break: 10 Minutes

Segment 2: From PoC to Production - Enhancing RAG Responses

Duration: 1 Hour 45 mins (Including Breaks)
Objective: Transition from a Proof-of-Concept to a production-ready MVP by implementing advanced RAG techniques.

2.1 Techniques to Improve “Naive” RAG Performance (30 mins)

• Advanced Approaches: Optimization strategies for better retrieval
  • Reranking
  • Hybrid search
  • Metadata filters
• Q&A

2.2 Hands-On: Refined RAG implementation with advanced techniques (30 mins)

• Hands-On/Exercise: Refine the earlier RAG implementation with advanced techniques.
• Evaluate and practically showcase the improved performance.

2.3 AI Agents and Orchestrating complex workflows (25 mins)

• How can we further improve the performance and reliability of our application?
• Introduction to AI Agents
• Discuss examples - Self RAG, Corrective RAG

2.4 Q&A

Prerequisites

• Basic knowledge of Python will be needed to follow along with the coding exercise.
• OpenAI API key and basic knowledge of using OpenAI API
• A bit of Experience with playing around tools like ChatGPT

Target Audience

• You’re a developer with a basic understanding of Generative AI
• You’re a developer working on proof of concept RAG application and want to deploy it to make it production-ready
• You’re an ML/AI Engineer
• You want to become an AI Engineer

About the Trainer:

Sarang Sanjay Kulkarni leads a healthcare client account at Thoughtworks, spearheading projects using generative AI to develop advanced research assistants designed to expedite drug discovery timelines. He has over 13 years of experience that spans the entire development stack, equipping him with expertise across multiple roles, including developer, DevOps engineer, data engineer, and AI engineer.

Sarang also serves as a trainer and has facilitated numerous developer bootcamps, data engineering programs, and generative AI training, sharing his knowledge and guiding teams through complex projects.

He is also a trainer at O’reilly where he conducts a workshop on building RAG based LLM application