Rootconf 2026 Annual Conference

One-line summary
How we took Slash, Razorpay’s agent platform, from one shared worker pool to isolated tenants, sandboxed runs and workload-shaped compute, told as eight things that broke at scale and the decision each one forced.

What problem are you addressing?
Platforms assume workloads are human-reviewed code, requests are short and stateless, and every tenant trusts the platform the same way. Agents break all three:

• Untrusted code. Each run executes code generated at runtime, with a shell, inside an agent harness we didn’t write. A list of allowed tools can’t fence a shell.
• Long, stateful work. Runs take ~10 minutes at the median and hours at the tail, while pods are scaled, deployed and evicted under them.
• Many tenants. Teams across Razorpay and external companies build their own agents on one platform. Keys, files and network access must never cross between them.

Slash today: 200+ agents, ~10K tasks a day, 98%+ success. About 3 in 10 tasks are started by a person; bots, schedules and CI start the rest.

The talk follows two questions:

1. How do you keep tenants, and runs inside a tenant, apart, when every agent has a shell?
2. How do you find compute for long jobs whose demand swings ~20x through the day, when CPU barely moves?

Intended audience
Platform engineering, SRE / DevOps, infrastructure and security engineers, and engineering leaders building internal AI platforms.

Level
Intermediate

Practical takeaways

• Enforce below the agent. Prompts and tool lists are not boundaries; the process, file system, network and credentials are. A run should hold references and short-lived grants, never keys.
• Make every default refuse. In a multi-tenant system a fallback is a tenant decision nobody made; routers and data access should fail closed.
• For long jobs on a queue, load is waiting plus in-flight work. Not visible queue depth, and not CPU.
• Every clock on a run must nest: workflow ceiling < run deadline ≤ claim lease ≤ queue visibility, with pod drain inside it. Check it at boot, or runs lose work or execute twice.

What will you share?

• Tenancy architecture: one shared control plane that holds no secrets, and one isolated data plane per tenant (namespace, queue, warm pool, workload role). Tenants are pre-built and declared like pods, so onboarding is four values, not ~15–20 PRs.

• Run containment: a sandbox in our agent gateway SDK that wraps any coding harness. It enforces:

  • a kernel file allow-list over a copied workspace
  • one network exit through a policy proxy with rules per calling program
  • blocked escape syscalls and a non-root user
  • model keys added outside the sandbox

  If it can’t enforce, the run doesn’t start.

• Prompt-injection defence: checks at four points in every turn (prompt, tool call, tool result, reply), with walls underneath that hold when a check misses. We’ll walk through how each finding from our pre-launch security review is now closed.

• Compute for long runs:

  • queues and warm pools split by workload shape
  • a per-turn setup budget
  • scale-down that counts in-flight work
  • a separate pool for fan-out workloads

• Before/after numbers for each fix, and the alternatives we rejected.

What is your experience with this problem?

• Production system
• Real incident or failure
• Hard-earned engineering lesson

What approaches failed, disappointed, or created unexpected problems?

• A tool list as the privilege boundary. In a pre-launch security review, one injected prompt had the agent copy cloud metadata into a repo, and others read system files. Security held the launch. We moved enforcement below the agent: credentials by absence, a kernel-enforced sandbox, one network exit.
• Fallbacks during a migration. While two generations of our dispatcher ran side by side, every convenience default (route by name, a default sandbox, a default account) silently picked a tenant. The router and the database guard now refuse instead of guessing.
• One queue for everyone. People asking quick questions waited ~27 minutes for ~9 minutes of work, stuck behind long CI jobs. Priority classes only reordered the line. Separate queues and pools by workload shape took that wait to ~0.
• A warm pod is not a warm run. A trivial “hi” took 30–45 s while the model needed ~6 s; the rest was sequential setup and tool health checks. Treating setup as a per-turn budget cut it by about a quarter.
• An autoscaler that couldn’t see work. Messages being worked on are hidden from the queue for hours, so the autoscaler saw an empty queue and cut ~100 runs mid-flight. Scale-down now counts in-flight work.
• CPU-based scaling. Agents mostly wait on models and tools, so CPU sat at ~15–25%, and most pods were parked above demand.
• Fan-out in a shared pod. One code review spawned ~6 agents within a second, ran its pod out of memory and took the neighbouring runs with it. Caps limit counts, not blast radius; reviews now get their own pool.
• Clocks out of order. Three-hour runs lost their work at the deadline. Runs now release and resume from a checkpoint, and ~95% of them finish.

What will you do differently today?

• Choose the isolation unit (a data plane per tenant) before the first external tenant.
• Keep keys out of runs from day one: model access only via a gateway, credentials only via brokers.
• Put containment below the harness: one policy for every harness, failing closed.
• Write the timing inequality for every clock on a run, and check it when a worker boots.
• Scale long jobs on waiting plus in-flight work.

What trade-offs did you consider?

• Warm shared pods vs. per-run microVMs: warm pods are fast and cheap, with isolation at the tenant; per-run containment comes from a kernel sandbox, not a VM.
• Copying the workspace into the sandbox vs. mounting it: we pay copy time on large repos so there is no path back to the host.
• Pod network policy vs. a per-sandbox proxy: pod rules see one address for several runs and can’t name hosts or programs.
• Shared pre-warmed tool connections vs. per-run setup: pooling is faster but carries stale per-user credentials, so we cut setup time instead.
• Bigger pods vs. a separate pool for fan-out work: bigger pods hit node limits, and the next wider fan-out hits the same wall.
• An off-the-shelf event-driven autoscaler vs. extending our own: the off-the-shelf one clashed with autoscalers other teams already run in shared clusters.

How can this help other practitioners?

• Treat agents as untrusted, long-running, multi-tenant jobs, and know which platform primitives stop holding: tool allow-lists, pod network policy, queue depth, CPU scaling, default routes.
• Copy the patterns:
  • credentials by absence
  • fail-closed chokepoints
  • kernel containment below the harness
  • in-flight-aware scaling
  • nested run clocks
• For leaders: decide the isolation unit and the key-handling model before the first external tenant; both are expensive to retrofit.

Current state
In production at Razorpay since March 2026, serving internal teams and external companies.

Tags

#agents #platformengineering #security #governance #developerplatforms #observability #inference #scalability #casestudy