Open weights, real stakes

Enterprises are currently trapped between the urgent need for AI acceleration and the systemic risks of relying on foreign, public SaaS models. This over-reliance is not just a regulatory hurdle—it is an economic and operational vulnerability that mirrors global supply chain dependencies. As “Shadow AI” usage spreads, teams are often forced to build fragmented, insecure tools that lack governance, safety, and central oversight, leaving technical leadership without the visibility or control required to scale.

This session cuts through the hype to show the engineering reality of running any model in production. We will take you through a hands-on journey, rotating through the perspectives of a Platform Engineer, AI Application Developer, SRE, and Technical Decision Maker. By building a private “Model-as-a-Service” (MaaS) platform on Kubernetes, you will learn how to unify your internal AI APIs, implement strict governance via LlamaGuard, and leverage Agentic AI for safe cluster diagnostics as an example use case—all while keeping your data and compute strictly localized and cost-effective.

Key Takeaways:
Architecture as an Anchor: Learn how to bridge the gap between “experimental AI” and “production-ready service” by bringing compute into direct proximity with your data to ensure true sovereignty.
Operationalizing Governance: Master the patterns for standardized AI governance, token-based security, and observability that transform isolated, risky AI experiments into a compliant, enterprise-wide service.

Target Audience:
This session is designed for Platform Engineers and SREs responsible for infrastructure stability, AI Application Developers looking to integrate local models securely without external dependencies, and Technical Architects or Decision Makers who need to balance the speed of AI innovation with stringent data sovereignty and compliance requirements.}

Ritesh Shah is Senior Principal Architect with Red Hat’s Product Portfolio Team and focused on building AI based solutions.

Co-Speaker: Abhijit Roy (abroy@redhat.com)

Draft slides: https://docs.google.com/presentation/d/1MDRpIeOxuOyBuWq7bIxHTwSot1FY5Wgq/edit?usp=sharing&ouid=101735931142205186115&rtpof=true&sd=true