Instrumenting your coding agent’s harness against silent failure

Most teams that build a coding-agent harness (a CLAUDE.md, a few hooks, a CI gate) treat it as config they set once. It isn’t. The harness is supposed to guard your code changes. Those same changes erode the harness itself: the standing-rules file goes stale until it’s confidently wrong rather than usefully absent, rule density crosses an adherence cliff where even frontier models stop following it, and the agent doing the work can satisfy a failing gate by deleting the test that enforces it. Nobody’s watching the watcher, and a harness that fails silently is worse than no harness at all.

This talk gives you a failure taxonomy for the harness itself and the safety net for each failure. The core distinction: encoded guardrails (lint, tests, static analysis, anything the agent doing the work can edit or delete) versus structural guardrails (permission hooks the agent’s own session can’t switch off, default-deny network egress, a CI gate with no admin-bypass path). The first kind only holds if the agent cooperates; the second holds regardless of what the agent tries. “Coverage is theater” generalizes: every encoded guardrail is theater against a suggestible actor.

From there, three concrete practices for treating the harness as production infrastructure rather than settings: config-drift detection in CI, canary load-lines that confirm context actually loaded, and harness regression evals that gate every change to a prompt, tool description, or iteration cap before it ships.

Takeaways

1. Your harness has its own failure modes (context rot, adherence cliffs, guardrail erosion) and needs sensors pointed at itself, not just at the code it guards.
2. One question separates a real guardrail from a decorative one: can the agent doing the work delete this and still pass? If yes, you’re relying on an encoded check, not a structural one. Know which you have before you bet on it.

Audience

Engineers and teams running coding agents against real production codebases who’ve already written a CLAUDE.md/AGENTS.md or a hook and want to know whether it’s actually holding. Includes people building agent infrastructure, harness tooling, and evaluation frameworks.

Bio

Karun Japhet is an engineer and consultant at Sahaj Software, working on large-scale systems and distributed architecture. For the last two years his focus has shifted to how AI reshapes the full software development lifecycle, with a particular interest in harness design (the scaffolding and guardrails around coding agents, not just the prompts fed to them). Karun writes and speaks on codebase structuring for multi-agent reasoning, measuring AI-assisted development impact, and how much judgment to leave with humans as agents take on more of the work. He’s spoken previously on distributed systems, event-driven architecture, and AI’s role across the delivery lifecycle.

Others

Slides link (PDF, comments access): TODO

2-min elevator pitch video: TODO

Past recorded talks: YouTube Playlist