Aishwarya Raimule

@aishwarya1

Leverage Envoy AI Gateway To Operate MCP Servers At Scale

Submitted Jun 25, 2026

As organizations adopt AI agents, the number of Model Context Protocol (MCP) servers inside the enterprise is growing rapidly. Teams are deploying MCP servers for GitHub, Jira, Kubernetes, observability platforms, internal APIs and business systems. While MCP standardizes tool integration, operating dozens of MCP servers introduces new challenges around authentication, authorization, discoverability, resiliency, observability and governance.

This session demonstrates how Envoy AI Gateway can act as a centralized MCP Gateway, enabling organizations to securely expose and manage multiple MCP servers through a single unified endpoint. Attendees will learn how Envoy AI Gateway aggregates tools from multiple MCP backends, performs intelligent tool routing, enforces OAuth-based security policies, filters tool access and leverages Envoy’s production-grade capabilities such as rate limiting, circuit breaking, retries, and observability. Through architecture diagrams and a live demo, we will show how platform teams can build a scalable, secure, and resilient enterprise AI platform using Envoy AI Gateway.

Key Takeaways

  • Learn how to aggregate and expose multiple MCP servers through a single Envoy AI Gateway endpoint while enforcing centralized authentication, authorization, and tool governance.
  • Understand how production-ready gateway capabilities such as intelligent routing, circuit breaking, rate limiting, retries and observability can be applied to MCP traffic to operate AI infrastructure at enterprise scale.

Target Audience

  • AI/ML Platform Engineers
  • Cloud Native Architects
  • Open source contributors interested in AI infrastructure and Gateway API

Bio

Aishwarya Raimule is a ML Systems Engineer at Nutanix, where she owns the Observability and MCP Infrastructure charter for the Enterprise AI Platform. Over the past 8+ years, she has built and operated large-scale Data, ML, platform systems across organizations such as Nutanix, Swiggy.
Her current interests lie at the intersection of AI infrastructure and observability. She is an active contributor to Envoy AI Gateway and is passionate about building secure, scalable infrastructure for AI applications.

Presentation

https://docs.google.com/presentation/d/13ZtsbgQRva_fiCACH6lPryYIpprA5zT5h2WW43ClTWQ/edit?usp=sharing

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

Jumpstart better data engineering and AI futures