In the Loop - harnesses, loops, graphs, and the evals that keep them honest

Abstract

We often treat multi-agent architecture as the solution to complex AI workflows. In production, the harder work is building the harness around the model: clear agent boundaries, useful retrieval, safe tool access, human-review points, and evaluation that measures real outcomes. In this talk, I will share lessons from a six-agent LangGraph workflow for prospect research and outbound operations. I will cover what worked, where orchestration introduced complexity, how improving semantic retrieval reduced irrelevant outreach, and how we used human review as a deliberate control point. Attendees will leave with a practical way to decide whether a task needs multiple agents, one agent, or conventional automation.

Takeaways

• When multi-agent graphs help
• Why retrieval and tool design matter
• How to evaluate relevance, task completion, review outcomes, latency, and cost

Audience

Software/ML engineers, AI platform teams, technical leads, and founders building production AI workflows.

Bio

Tushar Kumar Shah is an AI Engineer at Neusix, working on production agentic workflows, semantic retrieval, and backend systems. He has experience with LangGraph, FAISS, OpenAI embeddings, Node.js, TypeScript, and production APIs.

Presentation link:

https://canva.link/32mcmg6amfa2byy