BEGIN:VCALENDAR VERSION:2.0 PRODID:-//HasGeek//NONSGML Funnel//EN DESCRIPTION:Battle stories from engineers who have shipped with Open Weigh ts/Sovereign AI models X-WR-CALDESC:Battle stories from engineers who have shipped with Open Weig hts/Sovereign AI models NAME:Open weights\, real stakes X-WR-CALNAME:Open weights\, real stakes REFRESH-INTERVAL;VALUE=DURATION:PT12H SUMMARY:Open weights\, real stakes TIMEZONE-ID:Asia/Kolkata X-PUBLISHED-TTL:PT12H X-WR-TIMEZONE:Asia/Kolkata BEGIN:VEVENT SUMMARY:Check-in & registration DTSTART:20260711T043000Z DTEND:20260711T050000Z DTSTAMP:20260726T004833Z UID:session/8KHAT4QjKQVyazMteceUxi@hasgeek.com SEQUENCE:5 CREATED:20260708T133513Z LAST-MODIFIED:20260708T134133Z LOCATION:ThoughtWorks\, Pune ORGANIZER;CN="The Fifth Elephant":MAILTO:no-reply@hasgeek.com BEGIN:VALARM ACTION:display DESCRIPTION:Check-in & registration in 5 minutes TRIGGER:-PT5M END:VALARM END:VEVENT BEGIN:VEVENT SUMMARY:Addressing the elephant in the room - feature engineering DTSTART:20260711T050000Z DTEND:20260711T051500Z DTSTAMP:20260726T004833Z UID:session/QqTTY5SKnbu9t2fkAJnMMH@hasgeek.com SEQUENCE:3 CREATED:20260708T134425Z DESCRIPTION:# The Elephant in the Room: Feature Engineering\n\n## Descript ion\n\nWe pretend it's easy. Fifty tables\, hundreds of columns\, and some how you're supposed to know exactly which variable determines if a transac tion is fraud. You spend days browsing schemas\, running describe queries\ , guessing at joins — and this is before you've written a single line of ML code. Feature engineering and data understanding eat up 70% of the ML lifecycle\, yet we keep treating it like a solved problem. Meanwhile\, fro ntier models could help — but we don't trust them with our data\, and ri ghtly so.\n\nWhat if the model never saw your data at all? We built a tool that points a local LLM (Ollama\, fully offline\, open weight models) at your databases\, discovers schemas\, computes statistics using native SQL\ , and runs EDA — all without a single row leaving your machine. Then it suggests production-grade ML features — entity keys\, transformations\, types — and you review them in a table. Keep what makes sense\, throw o ut what doesn't. One click generates a complete Feast feature store repo. Another click runs `feast apply`. Schema to deployed feature store in unde r a minute. No API keys\, no cloud\, no trust compromise. Open source mode ls solving the problem that actually takes up your time.\n\n## Key Takeawa ys\n\n1. **Feature engineering is the real bottleneck\, not modeling** — and open weight models are good enough to help\, if you design the right guardrails. The LLM never touches raw data. It only sees schema metadata a nd pre-computed statistics. You review every suggestion before anything ge ts applied.\n2. **The pattern generalizes beyond Feast** — "LLM suggests \, system validates\, human decides" works for any ML platform. We'll show real outputs from 3B and 8B models\, where structured prompting works\, a nd where small models break down.\n\n## Target Audience\n\nML engineers\, data scientists\, and platform engineers who spend more time wrangling dat a than building models — and are curious about using open weight models to fix that\, without sending data to the cloud.\n\n## Bio\n\nAniket Palus kar — Curious Engineer at Red Hat AI working on Feast & Data.\nChaitany Patel — Associate Software Engineer at Red Hat working on Feast (#1 ope n source feature store). \n\n## Github\nhttps://github.com/patelchaitany/s chema-discovery-eda\n LAST-MODIFIED:20260708T154014Z LOCATION:ThoughtWorks\, Pune ORGANIZER;CN="The Fifth Elephant":MAILTO:no-reply@hasgeek.com URL:https://hasgeek.com/fifthelephant/open-source-indian-ai-models-pune-me etup/schedule/addressing-elephant-in-the-room-feature-engineering-QqTTY5SK nbu9t2fkAJnMMH BEGIN:VALARM ACTION:display DESCRIPTION:Addressing the elephant in the room - feature engineering in 5 minutes TRIGGER:-PT5M END:VALARM END:VEVENT BEGIN:VEVENT SUMMARY:Hard-earned lessons from training domain-specific LLMs: from vocab ulary extension to efficient inference DTSTART:20260711T051500Z DTEND:20260711T055000Z DTSTAMP:20260726T004833Z UID:session/VBx1vxCPUdyhYnBEKnBerp@hasgeek.com SEQUENCE:7 CREATED:20260708T135951Z DESCRIPTION:Training and serving domain-specific LLMs is fraught with hidd en challenges. In this talk\, we will walk through the entire LLM lifecycl e—from continued pretraining and supervised fine-tuning (SFT) to optimiz ed inference—sharing practical\, hard-earned lessons along the way. We w ill explore the "tokenization bottleneck" in continued pretraining\, the n uances of domain-specific SFT\, and pitfalls of serving models using specu lative decoding. LAST-MODIFIED:20260709T064414Z LOCATION:ThoughtWorks\, Pune ORGANIZER;CN="The Fifth Elephant":MAILTO:no-reply@hasgeek.com URL:https://hasgeek.com/fifthelephant/open-source-indian-ai-models-pune-me etup/schedule/from-vocabulary-extension-to-speculative-decoding-hard-earne d-lessons-in-training-and-serving-llms-VBx1vxCPUdyhYnBEKnBerp BEGIN:VALARM ACTION:display DESCRIPTION:Hard-earned lessons from training domain-specific LLMs: from v ocabulary extension to efficient inference in 5 minutes TRIGGER:-PT5M END:VALARM END:VEVENT BEGIN:VEVENT SUMMARY:From LLM to SLM: enterprise fine-tuning that keeps open models imp roving in production DTSTART:20260711T055000Z DTEND:20260711T060500Z DTSTAMP:20260726T004833Z UID:session/AAg1cYzQdXiaCEvj6WeddR@hasgeek.com SEQUENCE:6 CREATED:20260708T134705Z DESCRIPTION:# Open Models That Keep Getting Better: A Self-Improving Disti llation Pipeline on Kubeflow ecosystem\n\n---\n\n## Description\n\nMost op en-model demos stop at a single fine-tuning run and a benchmark score. Thi s project goes further: an autonomous pipeline on kubeflow pipeline and th e whole kubeflow ecosystem that distills a 32B open-weight (or any LLM) te acher (Qwen2.5-Coder) into a 1.5B SLM student\, evaluates it with a struct ured judge framework\, generates targeted training data from its weaknesse s\, retrains through **SFT → DPO → GRPO**\, and promotes the model to production only when it passes a quality gate \, then repeats the cycle. T he pipeline is not a notebook or a one-shot script\; it is a Kubeflow DAG with orchestrated components\, checkpoint chaining across cycles\, and eva l-gated canary deployments via KServe.\n\nThe session focuses on **product ion mechanics and real engineering learnings** from running this on a live cluster: preference pairs extracted from real eval failures\, GRPO reward functions aligned with evaluation judges\, automatic checkpoint selection between training stages\, and an outer loop that analyzes weak categories and feeds the next iteration. Everything runs on-cluster with open models \, Kubeflow Pipelines\, KServe\, MLflow\, and MinIO — no proprietary API s\, no data leaving your infrastructure. The talk is built around one ques tion: **how do you turn open models into production assets that improve ov er time?**\n\n\n## Production challenges & learnings\n1. **GPU scheduling* * — Training\, Ollama (teacher)\, and KServe (student) competed for the same GPU pool. We automated KServe scale-to-zero before TrainJobs and rese rved node capacity.\n2. **Multi-stage fine-tuning (SFT → DPO → GRPO)** — DPO didn't always beat SFT (AB_2: 73.3% vs 60%). We added checkpoint routing between stages\, aligned train/serve prompts\, and built preferenc es from real eval failures only.\n3. **Model promotion across cycles** — Continual learning needed durable S3 gate markers and version resolution logic\, not just MLflow tags\, so each cycle starts from the previous best checkpoint.\n\n## What the live walkthrough will show\n\nA pipeline cycle starts from the previous cycle's best checkpoint. The audience will see t he full progression in real infrastructure:\n\n- The Kubeflow DAG executin g end-to-end on OpenShift AI\n- MLflow training curves across SFT\, DPO\, and GRPO — loss decreasing\, rewards climbing\n- The eval harness scorin g correctness\, conciseness\, format compliance\, and review quality\n- Au tomatic checkpoint routing between DPO and GRPO so the RL stage always sta rts from the strongest model\n- The outer loop identifying weak categories and generating targeted training data for the next cycle\n- Three complet e cycles (AB_1 → AB_2 → AB_3) with visible knowledge retention — eac h cycle's SFT loss starts lower than the last\n- How the production fixes above show up in the actual DAG\, metrics\, and promotion flow\n\n\n## Tak eaways from session\n\n1. **Open models plus open infrastructure can power a closed-loop improvement system today** — Kubeflow\, KServe\, MLflow\, and Hugging Face TRL on OpenShift AI give you a full distillation flywhee l with data sovereignty and no vendor lock-in.\n\n2. **Eval-driven deploym ent is the bridge from experimentation to production** : the same judges t hat gate promotion can align with GRPO rewards\, turning evaluation from a post-hoc check into the engine that drives continuous improvement.\n\n3. **Staged fine-tuning needs production guardrails** :GPU orchestration\, c heckpoint routing between stages\, and explicit promotion state across cyc les are what make SFT → DPO → GRPO reliable in practice\, not just on paper.\n\n\n## Which audiences is this session going to be beneficial for? \n\nThis session is especially useful for ML engineers\, platform engineer s\, and technical leads running or planning to run open-weight models in p roduction on Kubernetes or OpenShift.\n\nIt will also be relevant for:\n\n - Teams moving from one-shot fine-tuning to iterative model improvement pi pelines\n- Engineers exploring DPO and GRPO in production settings\n- AI i nfrastructure teams building on Kubeflow\, KServe\, MLflow \n- Organizatio ns that need data sovereignty: everything runs on-cluster\n- Architects de signing agentic systems where specialized open models replace large genera l-purpose ones\n\n## About me\n\nI am Sridhar Pillai a Software Engineer a t Red Hat\, working on AI/ML infrastructure for OpenShift AI. I built the agentic continual learning pipeline spanning Kubeflow Pipelines\, the Trai ning Operator v2\, KServe\, and MLflow \, focused on making open-model tr aining and serving production-ready on Kubernetes.\n\n- **Demo Slides** [C anva](https://canva.link/1tx11fq2p5ef08w)\n- **LinkedIn:** [linkedin.com/i n/sridharpillai](https://www.linkedin.com/in/sridharpillai)\n- **GitHub:** [github.com/Sridhar1030](https://github.com/Sridhar1030)\n- **Blog on thi s:** [Building a Self-Improving Distillation Pipeline on Red Hat OpenShift AI](https://medium.com/@sridharpillai/building-a-self-improving-distillat ion-pipeline-on-red-hat-openshift-ai-byoa-meets-a2a-5268e3330a62) LAST-MODIFIED:20260710T135139Z LOCATION:ThoughtWorks\, Pune ORGANIZER;CN="The Fifth Elephant":MAILTO:no-reply@hasgeek.com URL:https://hasgeek.com/fifthelephant/open-source-indian-ai-models-pune-me etup/schedule/from-llm-to-slm-enterprise-fine-tuning-that-keeps-open-model s-improving-in-prod-AAg1cYzQdXiaCEvj6WeddR BEGIN:VALARM ACTION:display DESCRIPTION:From LLM to SLM: enterprise fine-tuning that keeps open models improving in production in 5 minutes TRIGGER:-PT5M END:VALARM END:VEVENT BEGIN:VEVENT SUMMARY:How far can you get without fine-tuning? Model quality as architec ture DTSTART:20260711T060500Z DTEND:20260711T064000Z DTSTAMP:20260726T004833Z UID:session/7zCMTazFUKqrkbkmrBtJ8t@hasgeek.com SEQUENCE:3 CREATED:20260708T134607Z DESCRIPTION:## Describe your session (2 paragraphs)\nThis session is a dee p dive on building architecture and guardrails around self-hosted open-wei ght models to deliver quality. We run a production LLM helpline for Gujara ti dairy farmers (phone-line and chat) answering questions on animal healt h\, breeding\, milk\, and schemes\, on open-weight models we host ourselve s.\n\nWe share how far you can get without fine-tuning: an English-reasoni ng core wrapped in a swappable translation layer\, a domain glossary that beats frontier models on terminology\, a streaming pipeline that hides tra nslation latency\, and the guardrails that catch the failure modes you inh erit with open weights. We close on the few things that genuinely need fin e-tuning\, and how running the product first is what made fine-tuning chea p.\n\n## Takeaways (1–2)\n\n 1. A reusable\, eval-gated escalation ladd er for LLM product quality (prompts → glossary → deterministic guards → architecture → fine-tuning) with a measured stopping rule at each g ate\, so you know how far the cheap\, portable interventions get you befor e you spend on weights. (Most of our quality was bought by orchestration\, not training.)\n 2. Concrete\, transferable patterns: the swappable "tra nslation-sandwich" for model optionality\; a tiny domain glossary that out -performs frontier models on terminology\; a generate∥translate∥speak streaming pipeline to hide translation latency on self-hosted inference\; and a clear test for when fine-tuning is actually the right tool (and how production traffic bootstraps the data for it).\n\n## Who is this benefici al for?\n\n - Platform / infra engineers running self-hosted LLM inferenc e (vLLM\, GPU serving\, multi-model orchestration) who need quality and la tency without per-token API lock-in.\n - Applied-AI / ML engineers buildi ng LLM products (especially multilingual / Indic-language and voice) facin g domain-terminology\, translation\, and streaming-latency problems.\n - Founders / tech leads deciding between fine-tuning and orchestration\, or planning to scale one product across many languages.\n - Anyone evaluatin g open-weight vs managed models for production and wanting an honest accou nt of the failure modes you inherit.\n\n## Bio\nI'm Kanav Dwevedi\, a soft ware engineer at [The Flywheel](https://www.linkedin.com/company/theflywhe el-in/) leading infrastructure and application development for [AmulAI](ht tps://amulai.in/chat) in partnership with [COSS](https://www.linkedin.com/ company/coss.org.in).\n\n[Github](https://github.com/KDwevedi) \n\n## Draf t Slides\n[Link to the draft slides](https://docs.google.com/presentation/ d/1UPLw46Kk_6HWlxnIshJCb6lrki9GxB4WeDXDEIH4t18/edit?usp=sharing) LAST-MODIFIED:20260708T154738Z LOCATION:ThoughtWorks\, Pune ORGANIZER;CN="The Fifth Elephant":MAILTO:no-reply@hasgeek.com URL:https://hasgeek.com/fifthelephant/open-source-indian-ai-models-pune-me etup/schedule/how-far-can-you-get-without-fine-tuning-model-quality-as-arc hitecture-7zCMTazFUKqrkbkmrBtJ8t BEGIN:VALARM ACTION:display DESCRIPTION:How far can you get without fine-tuning? Model quality as arch itecture in 5 minutes TRIGGER:-PT5M END:VALARM END:VEVENT BEGIN:VEVENT SUMMARY:Break DTSTART:20260711T064000Z DTEND:20260711T070000Z DTSTAMP:20260726T004833Z UID:session/Hx6JkWLVkZd9pApuhYtt6S@hasgeek.com SEQUENCE:1 CREATED:20260708T140004Z LAST-MODIFIED:20260708T140006Z LOCATION:ThoughtWorks\, Pune ORGANIZER;CN="The Fifth Elephant":MAILTO:no-reply@hasgeek.com BEGIN:VALARM ACTION:display DESCRIPTION:Break in 5 minutes TRIGGER:-PT5M END:VALARM END:VEVENT BEGIN:VEVENT SUMMARY:Sovereign AI at scale: engineering a secure\, private inference pl atform DTSTART:20260711T070000Z DTEND:20260711T071500Z DTSTAMP:20260726T004833Z UID:session/K6pTwcGjcjckpWDDMXWiLs@hasgeek.com SEQUENCE:5 CREATED:20260708T134846Z DESCRIPTION:Enterprises are currently trapped between the urgent need for AI acceleration and the systemic risks of relying on foreign\, public SaaS models. This over-reliance is not just a regulatory hurdle—it is an eco nomic and operational vulnerability that mirrors global supply chain depen dencies. As "Shadow AI" usage spreads\, teams are often forced to build fr agmented\, insecure tools that lack governance\, safety\, and central over sight\, leaving technical leadership without the visibility or control req uired to scale.\n\nThis session cuts through the hype to show the engineer ing reality of running any model in production. We will take you through a hands-on journey\, rotating through the perspectives of a Platform Engine er\, AI Application Developer\, SRE\, and Technical Decision Maker. By bui lding a private "Model-as-a-Service" (MaaS) platform on Kubernetes\, you w ill learn how to unify your internal AI APIs\, implement strict governance via LlamaGuard\, and leverage Agentic AI for safe cluster diagnostics as an example use case—all while keeping your data and compute strictly loc alized and cost-effective.\n\nKey Takeaways:\nArchitecture as an Anchor: L earn how to bridge the gap between "experimental AI" and "production-ready service" by bringing compute into direct proximity with your data to ensu re true sovereignty.\nOperationalizing Governance: Master the patterns for standardized AI governance\, token-based security\, and observability tha t transform isolated\, risky AI experiments into a compliant\, enterprise- wide service.\n\nTarget Audience:\nThis session is designed for Platform E ngineers and SREs responsible for infrastructure stability\, AI Applicatio n Developers looking to integrate local models securely without external d ependencies\, and Technical Architects or Decision Makers who need to bala nce the speed of AI innovation with stringent data sovereignty and complia nce requirements.}\n\nRitesh Shah is Senior Principal Architect with Red H at's Product Portfolio Team and focused on building AI based solutions.\n\ nCo-Speaker: Abhijit Roy (abroy@redhat.com)\n\nDraft slides: https://docs. google.com/presentation/d/1MDRpIeOxuOyBuWq7bIxHTwSot1FY5Wgq/edit?usp=shari ng&ouid=101735931142205186115&rtpof=true&sd=true LAST-MODIFIED:20260710T135153Z LOCATION:ThoughtWorks\, Pune ORGANIZER;CN="The Fifth Elephant":MAILTO:no-reply@hasgeek.com URL:https://hasgeek.com/fifthelephant/open-source-indian-ai-models-pune-me etup/schedule/sovereign-ai-at-scale-engineering-a-secure-private-inference -platform-K6pTwcGjcjckpWDDMXWiLs BEGIN:VALARM ACTION:display DESCRIPTION:Sovereign AI at scale: engineering a secure\, private inferenc e platform in 5 minutes TRIGGER:-PT5M END:VALARM END:VEVENT BEGIN:VEVENT SUMMARY:Serving open models in production: an inference playbook DTSTART:20260711T071500Z DTEND:20260711T075000Z DTSTAMP:20260726T004833Z UID:session/QNkPst5LsqXueS8CyjcgN1@hasgeek.com SEQUENCE:6 CREATED:20260708T134806Z DESCRIPTION:Running an open-weight model on a laptop is easy\; getting it to serve reliably in production - on-prem/on-cloud GPUs - is where most te ams stall. This session is a hands-on playbook for that journey: model sel ection\, quantization\, choosing a serving stack (vLLM\, TensorRT-LLM)\, b atching and KV-cache behaviour\, the latency-vs-throughput tradeoff\, eval s\, and routing. I'll use an open-weight model (Nemotron) as the worked ex ample and walk through the decisions with real numbers\, not spec sheets.\ nThe cost and data-control case for open weights only pays off if you can actually operate them - and with local and on-prem inference now realistic even on small footprints\, the bottleneck isn't the model\, it's the oper ational know-how to serve it well and cheaply. I'll be honest about what w orked\, what broke\, and what I'd do differently\, including the failure m odes that never make it into launch posts. The goal is that you leave with a repeatable approach you can apply to your own deployment the next day.\ n\n1. A repeatable checklist for taking an open-weight model from prototyp e to production\, with real latency / throughput / cost numbers from actua l runs.\n2. How to choose between open-source serving stacks for your work load - and the common failure modes (and fixes) to avoid along the way.\n\ nEngineers\, data scientists\, and technical founders who have shipped - o r want to ship - something using open-weight models in production. If you' ve deployed or are planning to deploy open models on-prem.\n\nhttps://driv e.google.com/file/d/1etLjfWohPDNY_80P3-yPjZutIgvtCaBm/view?usp=sharing\n\n Nilesh Kumar\, Solution Architect at NVIDIA\, working with teams on deploy ing open-weight model inference in production. https://www.linkedin.com/in /nileshkr1203/ LAST-MODIFIED:20260709T064408Z LOCATION:ThoughtWorks\, Pune ORGANIZER;CN="The Fifth Elephant":MAILTO:no-reply@hasgeek.com URL:https://hasgeek.com/fifthelephant/open-source-indian-ai-models-pune-me etup/schedule/open-weights-in-production-a-practitioners-inference-playboo k-QNkPst5LsqXueS8CyjcgN1 BEGIN:VALARM ACTION:display DESCRIPTION:Serving open models in production: an inference playbook in 5 minutes TRIGGER:-PT5M END:VALARM END:VEVENT END:VCALENDAR