Rootconf 2026 Proposal Draft

Session title

When the Client Disconnects but the Agent Keeps Running

One-line summary

We built a resumable streaming architecture for long-running AI agent interactions at Glance, where agent execution can continue independently of the client connection and a disconnected client can resume the stream from the last event it received.

What problem are you addressing?

SSE makes it straightforward to stream an LLM response to a client, but it does not solve resumability when that client disconnects mid-turn. In our architecture, that problem became more interesting because the client could reconnect to a different gateway pod while the original agent execution was still running.

We wanted to separate the gateway which owns authentication, conversation/session handling and client-facing concerns from the service running the agent. Once we did that, the lifetime of the client connection, gateway pod and agent execution were no longer the same.

A client could disconnect while an agent was still running and later reconnect to a different gateway pod. We needed to preserve the ordered history of that turn and resume from the last event the client had received, without requiring sticky sessions or coupling the agent to a particular gateway instance.

The engineering problem became less about “how do we stream tokens?” and more about where the state of an active stream should live and what exactly the unit of recovery should be.

Who is the intended audience?

Platform Engineering, Infrastructure, SRE, and backend developers building distributed or AI systems.

Level

Intermediate

One or two practical takeaways

1. Define the recovery unit and recovery contract before choosing a streaming technology. In our case, recovery was per agent turn, from a specific event ID.
2. Separating execution from delivery makes long-running agent systems much easier to operate across reconnects, pod changes and asynchronous work.

What will you share?

A production architecture story covering:

• Why SSE alone was insufficient for resumable streaming.
• Why separating the gateway and agent changed the problem.
• The trade-offs we saw with gRPC streaming, Kafka, and Redis Streams.
• How the final design enables ordered replay and recovery from a client-provided cursor.
• What we learned about operating this pattern for long-running agent workflows.

What is your experience with this problem?

• Production system
• Internal engineering project

What approaches failed, disappointed, or created unexpected problems?

None of the alternatives were fundamentally incapable of solving the problem, but they did not fit our recovery model cleanly.

Direct SSE coupled execution and delivery too closely when the system was split across gateway and agent services.

gRPC streaming gave us a good transport between services but did not by itself provide the replayable history required when a client reconnected through another gateway pod.

Kafka provided durability and replay, but its natural recovery position is partition plus offset, whereas our application contract was turn ID plus message ID. We could have built additional indexing and cursor translation around Kafka, but that introduced considerably more machinery for the problem we were trying to solve.

What will you do differently today?

We would start by defining the recovery contract rather than beginning with a transport choice.

The key questions would be:

• What exactly needs to survive a disconnect?
• What is the unit the client needs to resume — a session, turn, task or workflow?
• Who owns the cursor?
• How long does replay state need to exist?
• What state belongs to execution versus delivery?

Once those are explicit, the choice of streaming and state primitives becomes much easier.

We would also treat stream lifecycle and event granularity as first-class operational concerns from the beginning rather than as implementation details added later.

What trade-offs did you consider?

Our primary optimization was for simple per-turn recovery without pod affinity.

We considered:

Direct SSE: simplest architecture, but strongly couples the client connection to execution.

gRPC: excellent service-to-service streaming, but requires another system for replay and reconnection state.

Kafka: extremely capable for durable event streaming, but its partition-based recovery model was not naturally aligned with our per-turn addressability requirement.

Redis Streams: allowed us to give each turn its own ordered and replayable event history. A reconnecting gateway only needs the turn and the last received event ID to continue.

The trade-off is that this introduces Redis as part of the active delivery path and requires us to think about stream lifecycle, connection scale and event volume.

How can this help other practitioners?

The main pattern is broader than Redis or even LLMs:

Choose the state model around the unit you need to recover.

Teams building agents, coding assistants, media generation pipelines or other long-running asynchronous systems often start with streaming as a transport problem. Once execution can outlive the original connection, it becomes a distributed state-management problem.

The session gives practitioners a way to reason about that transition and evaluate technologies based on their actual recovery requirements rather than simply choosing a familiar streaming system.

Current state

Production experience

Tags

#agents #platformengineering #reliability #databases #scalability #distributed-systems #casestudy