The Fifth Elephant 2026 Annual Conference

Abstract

Building an agent that works in a demo is no longer the hard part. The challenge begins when the same system must operate reliably in production, handling evolving requirements, multiple agents, long-running workflows, changing prompts, evaluation, latency constraints, and operational failures. Every team encounters different problems, yet many of the underlying engineering questions are remarkably similar.

This Birds of a Feather session brings together practitioners who are designing, deploying, or operating agentic systems to exchange experiences from their journey to production. Rather than focusing on frameworks or model capabilities, the discussion will centre on architecture, design decisions, evaluation strategies, operational trade-offs, and the production failures that reshaped those systems. The objective is to surface recurring patterns, identify emerging best practices, and collectively understand what it takes to build agentic systems that remain reliable, maintainable, and adaptable as they evolve.

Key Questions That We’ll Explore Together

Designing Agentic Systems

• What differentiates a production-ready agentic system from a working prototype?
• How do you decide what should be deterministic code versus agentic reasoning?
• What architectural patterns have proven effective, and where have they broken down?
• When should a capability be implemented as a tool, an agent, or a workflow?

Consistency & Maintainability

• How do you prevent prompts, tools, schemas, and business rules from drifting over time?
• Who owns system state? The application, the agent, or a combination of both?
• How do you build reusable agent components that can evolve across multiple projects and use cases?
• What practices have improved maintainability as systems grow in complexity?

Evaluation & Reliability

• How do you define and measure success for inherently probabilistic systems?
• What evaluation strategies have proven effective in detecting regressions before deployment?
• Which failure modes have been the hardest to identify and debug?
• How much of your evaluation pipeline is automated versus human-driven?

Operating Agentic Systems

• What does observability look like for multi-agent systems?
• Which metrics, traces, and logs have been most valuable in production?
• How do you diagnose failures across multiple interacting agents?
• What governance and guardrails have become essential in production deployments?

Performance & Optimisation

• Where do you see the biggest trade-offs between quality, latency, and cost?
• Which optimization techniques have delivered the greatest practical impact?
• How do you decide when additional reasoning or context is worth the extra latency and cost?
• What architectural changes have significantly improved scalability or operational efficiency?

Lessons from the Field

• What surprised you most after moving from prototype to production?
• What production incident fundamentally changed your system design?
• If you were designing your agentic platform again today, what would you do differently?
• What emerging practices do you believe will become standard for production-grade agentic systems?

Prospective Participants

This session is intended for AI engineers, solution architects, software engineers, technical leads, platform engineers, and product teams who are designing, deploying, or operating agentic systems beyond the prototype stage. Whether you’re building internal copilots, enterprise automation, AI products, or multi-agent platforms, you’ll have the opportunity to exchange experiences with peers tackling similar production challenges.

I would also encourage participants to come with one production lesson to share, whether it’s a design decision that worked, a failure that reshaped their architecture, or an evaluation practice that caught a critical regression. That simple expectation often leads to far richer discussions than a typical Q&A.

Key Participants

TBD - WIP