The Fifth Elephant 2026 Annual Conference

Everyone is building agents. Most demos look impressive. Production systems tell a different story.

Let’s share the assumptions that turned out to be wrong, the architectural decisions we’d change, and the engineering lessons that only appear after running agentic systems in the real world.

Rather than discussing frameworks or model benchmarks, we’ll focus on lessons learned from building and operating agentic systems in production or at scale.

Topics may include evaluation, orchestration, multi-agent coordination, state management, tool use, observability, guardrails, cost, latency, and deciding when not to use an agent.

Discussion prompts

• What assumption about agentic systems turned out to be wrong?
• What production issue surprised you the most?
• Where did you replace LLM reasoning with deterministic code?
• When is a multi-agent architecture actually worth the complexity?
• How do you evaluate and debug agentic systems?
• What tooling, observability, or testing practices proved valuable?
• Looking back, what would you design differently today?
• How have your prompts evolved over time? Do prompts “rot” as the system grows, requiring constant maintenance?
• What techniques have helped make prompts more robust and maintainable?
• What retrieval strategy worked best in practice? Simple RAG, hybrid search, knowledge graphs, SQL, hierarchical retrieval, or something else?

Expected takeaways

Participants should leave with:

• Practical engineering lessons from real-world agentic systems
• Common architectural pitfalls and ways to avoid them
• Better intuition for when to use agents and when simpler approaches are sufficient
• A broader understanding of production challenges such as evaluation, orchestration, state management, observability, guardrails, and cost

Bio

Sujit Kamthe is a Solution Consultant at Sahaj Software based in Pune, India. He focuses on data engineering, large scale data processing, and architecting robust data platforms. He has previously presented at The Fifth Elephant on pragmatic guides to robust data quality checks, and his work centers on building enterprise systems that are scalable, governed, and ready for modern AI consumption.