BEGIN:VCALENDAR VERSION:2.0 PRODID:-//HasGeek//NONSGML Funnel//EN DESCRIPTION:Hands-on workshop on Agentic Evals X-WR-CALDESC:Hands-on workshop on Agentic Evals NAME:Stop guessing. Start measuring. X-WR-CALNAME:Stop guessing. Start measuring. REFRESH-INTERVAL;VALUE=DURATION:PT12H SUMMARY:Stop guessing. Start measuring. TIMEZONE-ID:Asia/Kolkata X-PUBLISHED-TTL:PT12H X-WR-TIMEZONE:Asia/Kolkata BEGIN:VEVENT SUMMARY:Stop guessing. Start measuring. DTSTART:20260717T083000Z DTEND:20260717T114500Z DTSTAMP:20260705T193121Z UID:session/KjPocWqjvCsP3TfYJvSo8g@hasgeek.com SEQUENCE:11 CREATED:20260701T071756Z DESCRIPTION:## πŸ“˜ Workshop overview\nIn this workshop we will cover the following points: \n- Why AI agents failβ€”and how to measure them\n- Desi gn metrics that matter for AI agents\n- Generate synthetic test data at sc ale\n- Perform error analysis and improve prompts\n- Build production-read y evaluation pipelines\n\n---\n\n## 🎯 Target audience\nTechinical produ ct managers\n- AI Engineers\n- Data Scientists\n- Data Analysts\n- Data En gineers\n- Software developers\n \n---\n\n## Background knowledge prerequ isites\nFamiliarity with python coding\n\n---\n\n## βœ… Learning outcomes\ nSystematic agentic evaluations\n\n---\n\n## πŸ›  Software installation re quirements\nAccess to google colab\n\n---\n\n## Workshop duration and cont ent plan\nThis is a two-hour workshop\, covering:\n\n* Why do Agents make mistakes - 3 Gulfs [Comprehension\, Specification and Generalization]. (10 min)\n* Challenges of evaluating agent responses . Why is it different fr om standard software testing on ML system testing (10 min)\n* Component wi se evaluation of agents (What is equivalent of module level testing in Age nts) (30 min)\n* How to generate synthetic data to evaluate your agents - Hands on activity (20 min)\n* How to come up with metrics to evaluate an a gent that generates linkedin posts automatically - Error analysis - Group Activity hands on (50 min)\n* How to deal with subjectivity among reviewer s? (15 min)\n* LLM as a judge to evaluate Agents at scale (30 min)\n* Wrap up - 15 min\n\n---\n\n## About the instructor\nAbhijith Neerkaje is a dat a science and AI leader with over 20 years of experience building AI produ cts across retail\, semiconductor manufacturing\, and energy. He is the Co -founder of Beyond Vectors AI Pvt Ltd\, where he helps mid-career professi onals and engineering leaders build expertise in Generative AI and Agentic AI.\n\nPreviously\, Abhijith led the Data Science & Analytics function at Falabella India\, driving AI-powered search\, recommendations\, pricing\, and seller intelligence for one of Latin America's largest e-commerce pla tforms. Earlier\, at Target\, he built machine learning solutions for pric ing\, merchandising\, and supply chain optimization.\n\nAbhijith holds eng ineering degrees from PES University (formerly PESIT) and the Indian Insti tute of Science (IISc)\, Bangalore\, and an MS in Engineering and Manageme nt from the Massachusetts Institute of Technology (MIT). He was an MIT Tat a Fellow and a winner of the MIT Clean Energy Prize (Renewable Energy Trac k\, 2013).\n\n---\n\n## How to attend this workshop\nThis workshop is open to:\n🎟️ Fifth Elephant community members β€” https://hasgeek.com/fif thelephant#memberships\n🎟️ Ticket holders for The Fifth Elephant annu al conference β€” https://hasgeek.com/fifthelephant/enterprise-ai-in-produ ction-meetup#tickets\n\n**This workshop is open to 30 participants (in-per son) & hybrid access for remote attendees. Seats for in-person participant s will be available on first-come-first-served basis. 🎟️**\n\n---\n\n ## Need more info?\n☎️ Call: (91) 7676332020\nπŸ“§ Email: info@hasgeek .com LAST-MODIFIED:20260701T073711Z LOCATION:Bangalore - https://hasgeek.com/fifthelephant/ai-evals-workshop/ ORGANIZER;CN="The Fifth Elephant":MAILTO:no-reply@hasgeek.com URL:https://hasgeek.com/fifthelephant/ai-evals-workshop/ BEGIN:VALARM ACTION:display DESCRIPTION:Stop guessing. Start measuring. in 5 minutes TRIGGER:-PT5M END:VALARM END:VEVENT END:VCALENDAR