In the Loop - harnesses, loops, graphs, and the evals that keep them honest

Evidence as first-class runtime state in long-running agent workflows.

An evidence-governed runtime and control layer that sits above observability infrastructure. It consumes same telemetry signals but turns verified, durable evidence into decisions about agent continuation, recovery, escalation and authorization. Observability tells us what happened. EGAH asks whether what happened is still sufficient evidence for what the agent is about to do.

EGAH RUNTIME
 
  DURABLE EXECUTION STATE
  Graph · Checkpoints · Memory · Recovery State
 
  DURABLE EVIDENCE STATE
  Provenance · Validity · Verification · Authorization
 
  POLICY / DECISION CONTROL
  Continue · Refresh · Escalate · Abstain
 
        |
        | telemetry
        v
 
  OBSERVABILITY PLANE
 
  OpenTelemetry / Langfuse / Phoenix / Datadog / etc.
  Traces · Logs · Metrics · Tokens · Cost · Latency

Long-running agents are designed to preserve execution state: graph state, checkpoints, memory, tool results and workflow progress. But in a production agent, state alone is not enough.

Modern AI observability is sophisticated. Platforms such as OpenTelemetry, Langfuse, Phoenix, and Datadog capture

• LLM generations, tool calls, retrieval operations
• Token usage, cost, latency per operation
• Hierarchical traces across agent steps
• Quality scores and evaluation metrics
• Prompts, completions, tool arguments and results

These tools answer: “What happened?”.

They are essential infrastructure. EGAH does not replace them.

What EGAH adds on top

EGAH introduces a decision-bearing evidence layer that sits above observability and answers a different question:

“Given what we know and have verified, what should the agent be allowed to do next?”

                    EGAH RUNTIME
┌──────────────────────────────────────────────────────┐
│  Durable Evidence State    →  What do we know?       │
│  Verification History      →  Was it verified?       │
│  Freshness Analysis        →  Is it still valid?     │
│  Policy Decision           →  ACT / REFRESH / ASK / ABSTAIN  │
└──────────────────────────────────────────────────────┘
                        │ telemetry
                        ▼
┌──────────────────────────────────────────────────────┐
│  OBSERVABILITY PLANE                                 │
│  OpenTelemetry / Langfuse / Phoenix / Datadog        │
│  → What happened?                                    │
└──────────────────────────────────────────────────────┘

An agent may resume at step 40 of 60 with its execution state fully intact while the evidence that justified its next action is stale, incomplete, unverified or no longer applicable. The harness has remembered where it was, but not necessarily what it knew, what it verified, what it accomplished, or what it is still allowed to do, which is the gap to be addressed.

EAGH introduces evidence as first-class runtime state alongside graph state, checkpoints, memory and tool execution. Instead of treating verification and governance as external layers bolted on after the fact, the harness weaves evidence checkpoints, evidence envelopes, verification history, outcome attribution and capability boundaries directly into the execution graph.

Core execution shift moves from

Plan --> Tool Call --> Result --> Next Step

to:

Plan --> Evidence Checkpoint --> Tool Call --> Result Verification --> Evidence Update --> Next Step

At each evidence checkpoint, the harness captures what is known, verifies it against a policy, records the decision, and persists the result durably before allowing the next consequential action.

REQUEST --> CONTEXT --+       +-- MODEL
                      |       |
                      v       v

  EGAH RUNTIME

    STATE        EVIDENCE        TELEMETRY

    PLAN --> CHECKPOINT --> POLICY GATE --> SUFFICIENT?
      ^              YES        STALE        NO
      |               |          |            |
    TOOLS --> ACT    REFRESH    ASK --> ESCALATE
      ^        |       |         |
    WORLD      v       v         v
    FAILURE --> RECOVERY --> SAFE RESUME    ABSTAIN

                      |
                      v
               VERIFIED RECORD

This becomes critical for long-running workflows. If an agent crashes at step 40 of 60, conventional checkpoint recovery restores execution state and resumes blindly.

An evidence-governed harness asks additional questions before continuing.

1. Which prerequisites were already verified, and when?
2. Has anything changed in the external world while the workflow was interrupted?
3. Which evidence remains fresh and valid within its time-to-live window?
4. Which evidence is stale, and does staleness affect downstream actions?
5. Does the agent still have authorization to perform the next action?
6. What must be revalidated before the next consequential step?
7. Should the harness resume, refresh, escalate to a human, or abstain entirely?

The same evidence-governed approach provides a practical mechanism for small-model deployment. Rather than asking whether a tuned 8B model is universally “good enough,” the harness establishes evidence-backed capability boundaries for specific task classes. Tasks within an established boundary execute with the smaller model. Tasks near the boundary trigger additional verification or human review. Tasks beyond the boundary escalate to a larger model or human intervention. Every boundary decision is recorded with the evidence that supported it.

Architecture

A production agent harness should persist not only where the agent was, but also what the agent knew, what it verified, what it accomplished, and what it is still authorized to do."

Evidence-enhanced execution graph, evidence-state model with durable envelopes, crash-and-resume scenario with evidence validity analysis, small-model capability boundaries with evidence-backed thresholds, resource and outcome accounting linked to verified evidence, and the trade-offs between additional verification overhead and runtime safety. It distinguishes clearly between the proposed architecture, implementation experiments demonstrated in the PoC, and production validation still required.

Takeaways

1. Execution state is not evidence state. A checkpoint can tell an agent where it was without telling it whether the evidence supporting its next action is still valid.
2. Make evidence a first-class harness primitive. Capture provenance, observation time, verification status, applicability, outcome and verification history alongside execution state.
3. Put verification at consequential boundaries. The answer is not to validate everything. Use stronger verification where stale or incorrect evidence can materially change the outcome.
4. Design recovery around semantic continuity. After a crash, resuming the graph should also consider what evidence needs to be revalidated before execution continues.
5. Evaluate small models by capability boundaries, not model size alone. Establish where an 8B model is demonstrably sufficient for a defined task class, where verification is needed, and when escalation is appropriate.
6. Measure whether the harness earns its complexity. Checkpoint overhead, recovery correctness, capability-boundary accuracy and outcome quality should determine whether an additional harness mechanism belongs in production.

Audience

Engineers and technical teams building or operating production AI agents, agent harnesses, execution graphs, tool-using agents and long-running workflows.

It is particularly relevant to teams moving beyond agent demos and asking what needs to change before an agent can reliably execute long-running, tool-using and consequential workflows in production.

Bio

Khaja Shaik is an engineering leader and AI practitioner with extensive experience in software engineering, distributed systems, enterprise platforms and AI-enabled applications. His recent work spans AI engineering, agentic AI, RAG, agent harnesses, evaluation, observability, AI resource economics and trustworthy AI execution.

His current research explores how AI systems can move beyond model-level capability toward evidence-driven, observable and accountable agent execution, with particular interest in context and harness engineering, long-running agent workflows, evaluation, resource/outcome attribution and AI governance.

He works across the intersection of software engineering and emerging AI systems, with a focus on understanding which abstractions genuinely improve production reliability, efficiency and trust — and which add complexity without providing measurable value.

LinkedIn: https://www.linkedin.com/in/khajashaik/
GitHub: https://github.com/khshaik