Rootconf 2026 Annual Conference

Session title

Grounding the Agent: How to Let LLMs Reliably Operate Infrastructure

One-line summary

We built an MCP server that sits between AI agents (Cursor, Claude Desktop, custom agents) and Nutanix Prism Central’s v4 REST API. It serves pruned schemas, code samples and live state on demand, so the agent can only call operations that exist in the v4 spec, and writes stay off until an operator turns them on.

What problem are you addressing?

LLMs are probabilistic. Infrastructure APIs are not. Connecting the two breaks in predictable ways:

• Stale training data. Ask a model for a Nutanix automation script and it writes against the older v3 API it memorised. System prompts saying “use v4” are a suggestion, not a constraint.
• Specs that don’t fit. The v4 API is 1,000+ operations across 19 namespaces. Request bodies are deeply nested $ref graphs; a single operation can expand to tens of thousands of tokens. Pasting the OpenAPI spec into context costs latency and money, and often exceeds the window outright.
• No live state. RAG over docs can tell you what listClusters means. It cannot tell you the live CPU headroom on cluster X during an incident.
• Blast radius. The obvious fix, giving the agent an HTTP tool, gives it un-sandboxed access to a production datacenter, with a model that confidently invents parameters.

The engineering question: how do you expose a large enterprise API to an IDE agent or an SRE agent in real time, without handing it the keys to everything?

Who is the intended audience?

Platform Engineering / SRE / DevOps / Infrastructure / Developers building agent tooling / Technical architects

Level

Intermediate/Advanced

List one or two practical takeaways

1. A pattern for putting an MCP server between an LLM and a REST gateway as a schema-pruning, policy-enforcing adapter: a small fixed tool set (23 tools, not 1,000) with discovery, lazy schema expansion and server-side safety.
2. Concrete techniques for API determinism and context budgeting: ranking operations from keyword tokens, resolving $ref only on demand, and making the spec itself the allowlist, so a hallucinated endpoint fails before it reaches the datacenter.

What will you share?

• Architecture and design decisions: why 23 tools (4 discovery + 19 per-namespace executors) instead of one tool per operation; why dynamic tool calls beat a static vector index for live infrastructure; where the server’s job ends and the agent’s job begins.
• Implementation details (from the repo): listOperations ranks operations with field-weighted token scoring (_score_operation in src/generators/tool_generator.py); getOperationSchema expands $ref lazily with a cycle-safe resolver (src/generators/schema_resolver.py); getCodeSample and getOperationPermissions give the agent a verified example and the required roles before it executes anything; src/tools/runtime_dispatcher.py validates every call against the loaded v4 operation set.
• Safety mechanics: READ_ONLY_MODE enforced at the dispatcher rather than in the prompt; 401/403 bodies sanitised so credential and realm details never reach the model; Prism Central’s own RBAC bounding what the configured credential can do.
• Production challenges: token bloat from schemas (stripping readOnly fields and $UNKNOWN/$REDACTED enum noise); PC errors that were reaching the model as successful results; and how we think about concurrency for an I/O-bound server.
• Live demo: Cursor or Claude Desktop connected to a live Prism Central through the server: discover an operation from a plain-English request, pull its pruned schema and sample, query live state, and refactor a legacy v3-style script into a v4-compliant one.

What is your experience with this problem?

Internal engineering project / Open-source project / Hard-earned engineering lesson

What approaches failed, disappointed, or created unexpected problems?

1. Registering every operation as a tool. 1,000+ tool definitions swamp the context and degrade tool selection. Abandoned for the fixed 23-tool model with ranked discovery.
2. Expanding schemas eagerly. One operation’s $ref graph could dominate the whole prompt. Lazy resolution plus pruning fixed it.
3. Trusting the prompt to enforce v4. Instructions help, but a model can still reach for remembered v3 shapes. What worked was removing the option: the executor only accepts operation IDs from the v4 artifacts.
4. Returning Prism Central errors as normal results. 4xx/5xx responses came back as ok=true, so the agent kept going after a failed call. This is a known gap we are closing, and a good example of how an agent can only recover from failures it can see.
5. A new HTTP client per call. Fine for a demo; connection churn under concurrent agents. Moving to a pooled async client.

What will you do differently today?

1. Make error fidelity part of the tool contract from day one: a failed infrastructure call must look like a failure to the model.
2. Put safety in the server (read-only mode, and a per-operation allow/deny policy which is on our GA backlog), never only in the prompt.
3. Keep the default scope to read/query and code generation, with write operations an explicit operator decision.
4. Resolve everything lazily (schemas today, the operation index next) so startup and per-call cost stay flat as the API grows.

What trade-offs did you consider?

• Open standard vs. custom wrappers: MCP gives native compatibility with Cursor, Claude Desktop and custom agents. We gave up proprietary, deeper integrations with any one client.
• Live API vs. pre-indexed RAG: live calls cost network latency but are the only way to answer operational questions about current state. RAG stays useful for docs, not for state.
• Lexical ranking vs. embeddings: token scoring has no model or vector-store dependency and is debuggable. We gave up synonym matching and plan to add an alias map first; embeddings only if golden-query tests show lexical ranking is not enough.
• Implementation language and runtime: the workload is almost entirely I/O wait on Prism Central, so the choice of language matters less than connection reuse and async I/O. We optimised for iteration speed and kept the option to revisit this as scale demands.
• Single service credential vs. per-user passthrough: simpler to operate and audit, but the agent acts with the credential’s role, not the end user’s. Operators have to scope that credential deliberately.

How can this help other practitioners?

• A reusable design for making an on-prem or cloud platform API usable by agents without flooding the context.
• A blueprint for bounding an LLM integration with existing IAM: scope the credential, enforce read-only in the server, sanitise auth errors.
• Operational practices for context budgeting on a large API surface, and a way to decide when ranking or embeddings are actually warranted.

Current state

In progress. Working open-source server in technical preview state moving toward GA.

• Repository: https://github.com/nutanix/ntnx-api-mcp-server/tree/main
• Deep-dive tech blog: https://www.nutanix.com/blog/mcp-server-for-nutanix-api-is-now-available-in-technology-preview#

Tags

#platformengineering #agents #infrastructure #sre #developerplatforms #automation #security #demo #mcp #workinprogress