12,000 Lines of Agent Rules. Zero Enforcement.

Subtitle: How I audited my AI coding harness and moved repository conventions into mechanical gates

One-line summary

I built an AI coding harness that turned requirements into verifiable agent tasks, routed repository conventions into context, and still repeatedly violated those conventions. An audit revealed that, despite roughly 12,000 lines of conventions, the harness had no mechanical enforcement. This is the story of what failed, what I changed, and what I learned from testing the same tasks with enforcement enabled and disabled.

Abstract

In 2025, I built an AI coding harness for a production codebase. It interviewed stakeholders to clarify requirements, decomposed work into stories with acceptance criteria, and orchestrated coding agents across sequential and parallel tasks.

I also built a substantial conventions system to keep generated code consistent with the existing architecture. The root instructions were about 300 lines, with task-specific context loaded progressively. A routing agent fetched relevant conventions as needed.

Yet the coding agents kept making the same mistakes.

As problems accumulated, our conventions grew from roughly 5,000 to 12,000 lines. Every failure seemed to produce another written rule. Eventually, I audited the harness to answer a different question: which of these rules could actually prevent a violation, rather than merely ask the model not to make one?

The answer was uncomfortable. None of the checks in the harness were mechanical.

One experiment exposed why even our context router was insufficient: when I deliberately underspecified a task, the router never retrieved a relevant convention because nothing in the task signalled that it needed to look for it.

I began separating conventions into two categories: those that could be checked mechanically through linting, structural checks or dependency rules, and those that genuinely required engineering judgment.

I then tested enforcement hooks on the same codebase using the same prompts, with the hooks enabled and disabled. Without hooks, familiar violations recurred. With hooks, the agent encountered a failing check at commit time, corrected its work, and retried.

These were only five or six runs, not a large controlled benchmark. They demonstrated a useful behaviour in one environment, not a universal improvement in agent reliability.

This talk reconstructs the failure, explains where the enforcement boundary belongs, and demonstrates how familiar developer tooling can provide a feedback loop for probabilistic coding agents.

The larger question is not how to make agents read more rules. It is how to make the engineering constraints that matter enforceable without requiring someone to supervise every step.

What problem does this address?

Engineering teams adopting coding agents often encode accumulated architectural knowledge in Markdown rules, skills, system prompts and repository conventions.

But guidance is not enforcement.

As tasks become longer and more autonomous, the same repository constraints must hold even when the agent fails to retrieve, interpret or follow the relevant instructions.

The challenge is deciding which constraints belong in executable checks, which require judgment, and where to place the checks so a failed agent action becomes a recoverable feedback loop.

Intended audience

Platform engineers, developer productivity engineers, DevOps engineers, senior software engineers and engineering leaders building or maintaining AI-assisted development workflows.

Level: Intermediate. Familiarity with Git hooks, CI and basic static analysis is useful. No machine-learning background is required.

What attendees will learn

1. How to audit a coding-agent harness for enforcement gaps. Distinguish instructions that merely guide behaviour from checks that can actually reject a violation.

2. How to translate repository conventions into mechanical checks. Identify rules suited to linting, dependency constraints and structural validation, while leaving genuinely subjective decisions to human judgment.

3. How to build a repair loop around coding agents. Use failing checks at execution and commit boundaries to provide concrete feedback that an agent can act on, and understand what that approach still cannot guarantee.

Technical material and proposed session outline

1. The harness that looked complete (5 minutes)

Walk through the actual workflow: requirements interviews, story decomposition, acceptance criteria, progressive context loading, and the routing agent. Explain why we initially believed the system had sufficient guardrails.

2. Why more conventions did not solve the problem (7 minutes)

Examine how the conventions accumulated over time. Reconstruct the underspecified-task experiment where the router failed to retrieve the relevant rule. Explain why a retrieval system cannot guarantee that every applicable constraint will be present in an agent’s context.

3. From conventions to executable constraints (8 minutes)

Take two concrete examples from the original codebase:

• Constants were required to live in one shared module. This is mechanically decidable and belongs in an executable check.
• Simple queries could live in the facade layer, while complex queries belonged in service files. The distinction required contextual engineering judgment and could not be captured by a simple lint rule.

Show how the audit separated machine-checkable invariants from judgment-dependent guidance.

4. Testing hooks on versus hooks off (10 minutes)

Reconstruct the same-prompt experiments on the same codebase, using execution traces and a small reproducible demonstration.

With enforcement disabled, the agent could repeat a convention violation. With enforcement enabled, a commit-time check rejected the change, returned actionable feedback, and caused the agent to repair the violation before retrying.

Discuss where hooks, local checks and required CI checks fit, and why merely installing a check is different from making it unavoidable.

5. What I still do not know (5 minutes)

Discuss the limitations of the small experiment, the maintenance cost of custom constraints, possible false positives, the temptation to over-constrain the codebase, and cases where human judgment remains necessary.

Finish with a practical method attendees can use to audit their own coding-agent configurations.

Experience and evidence

This work comes from a coding harness I originally built for a real production application in 2025, followed by a retrospective audit and small-scale comparative experiments in 2026.

The original harness was used in production development. The enforcement investigation is an experimental case study based on one codebase and approximately five or six hooks-on/hooks-off runs. I am not presenting it as a statistically validated productivity study.

I published the original investigation in July 2026:

https://prasunsrivastava.com/writing/auditing-my-own-ai-coding-harness-for-enforcement/

What failed or surprised me?

I initially treated recurring violations as a context-engineering problem and responded by adding more written guidance.

The more surprising failure was the context router. It could retrieve rules relevant to a sufficiently descriptive task, but could not guarantee retrieval when the task omitted an important architectural constraint.

The key change was recognising that I had built an elaborate guidance system without building an enforcement system.

The tooling necessary to enforce many of these conventions—linting, static checks, Git hooks and CI—was already available.

What would I do differently?

I would design the enforcement layer alongside the initial agent workflow rather than add it after conventions had accumulated.

For each new rule, I would first ask whether compliance can be mechanically determined. If it can, I would implement and test that constraint as an executable check. If it cannot, I would keep the guidance concise, with examples and clear boundaries for human judgment.

How will this help practitioners?

Attendees can take the audit method back to their own repositories, identify convention rules that should be executable, and start replacing repeated corrective prompting with checks that produce explicit failure feedback.

The session is intended as a technical failure story and implementation discussion, not a product or consulting pitch.

Speaker bio

Prasun Srivastava is a Bengaluru-based hands-on software architect and engineering leader with 16 years of experience across data systems, machine learning and full-stack engineering.

Most recently, he served as Director of Engineering at a US digital-health startup, where his teams used AI-assisted development in a HIPAA- and SOC 2-regulated production environment.

He now runs Software Delivery Works, an independent consulting practice focused on making AI-assisted software development more reliable through measurable workflows, executable constraints and engineering verification.

He publishes first-hand engineering investigations at https://prasunsrivastava.com/.

Suggested tags

#platformengineering #agents #developerplatforms #devex #governance #failurestory #casestudy #devops

Current state: Production experience (original harness); experimental investigation (enforcement changes and comparative runs).