Aashish Bharadwaj

@fencio_ab

What Does an AI Firewall Actually Block? Building Portable Intent Signatures with Concepts + IQL

Submitted Oct 9, 2026

Session title

Concepts + IQL: Building Portable Intent Signatures for AI Agents

One-line summary

How do we define what an AI firewall should recognize and enforce across natural-language instructions, tool calls, and agent actions? This session explores Concepts + IQL, a research architecture for expressing behavioral constraints as reproducible, shareable, and configurable intent signatures.

What problem are you addressing?

AI agents can express the same behavior through different representations: a natural-language request, a structured API call, a browser interaction, or a generated command. Security controls need to recognize that behavior while preserving the details that determine whether it is permitted: who is acting, what resource is affected, where information is going, and under whose authority.

For example, “send this confidential report outside the organization” could become an email, a document-sharing operation, or a file upload. A reusable signature must describe the relevant behavior and specify the evidence required to evaluate it.

This creates several engineering challenges:

  • Semantic recognition can be uncertain even when policy evaluation is deterministic.
  • Identical observations can have different meanings under different context or permissions.
  • Different enforcement systems observe different parts of an agent’s activity.
  • A correct decision is ineffective if it arrives after the action executes.

The session examines how data normalization, semantic normalization, and a small policy language can establish a common contract between observations and enforcement.

Who is the intended audience?

Platform engineers, security engineers, developers building AI agents and tool integrations, infrastructure engineers, and engineering leaders evaluating agent security architectures.

Level

Advanced.

List one or two practical takeaways

  1. A design pattern for separating observation normalization, semantic recognition, policy evaluation, and action enforcement—with explicit handling of missing or uncertain evidence.
  2. An evaluation framework that distinguishes policy-authoring accuracy, recognition quality, deterministic conformance, and actual prevention of prohibited actions.

What will you share?

  • Architecture and design decisions: Concepts as versioned predicates with explicit roles; IQL as a language for composing behavioral constraints.
  • Normalization pipeline: How heterogeneous observations become structured evidence while preserving provenance, action identity, and uncertainty.
  • Formal semantics: The meaning of ALL, IN, and ALL EXCEPT, and how established violations, established nonviolations, and unresolved evidence produce operational decisions.
  • Implementation experience: Lessons from the current coding-agent implementation, used as a concrete example of the broader research architecture.
  • Preliminary measurements: An archived 293-case policy-authoring evaluation, including its limitations and why authoring accuracy does not establish runtime security.
  • Evaluation design: Tests for independent implementations, adversarial observations, context freshness, false blocks, latency, and enforcement failures.

The session will distinguish implemented capabilities from proposed extensions and experiments.

What is your experience with this problem?

  • Internal engineering project
  • Experiment or prototype
  • Research / investigation

I am developing Concepts + IQL and working through its implementation for coding agents. The research explores how the same semantic contract could support other agent domains through explicit domain profiles, context providers, and enforcement integrations.

Current implementation evidence comes from coding agents. Portability across other domains and independent enforcement systems remains an evaluation objective.

What approaches failed, disappointed, or created unexpected problems?

Implementation and design review exposed several limitations:

  • Natural-language policies alone leave interpretation underspecified. Models can disagree about the operation, its target, or the direction of information movement.
  • Unstructured collections of concept labels lose relationships. Recognizing “delete” and “configuration file” in the same event does not establish that the configuration file is being deleted.
  • Canonicalization has limited guarantees. Identical normalized expressions support artifact comparison; they do not establish that different recognizers interpreted an observation correctly.
  • Successful authoring can conceal enforcement gaps. A correctly constructed rule does not prove that the runtime obtains the required facts or stops the action.
  • Repeated model agreement is insufficient validation. Confirmation calls and cached judgments can preserve the original recognition error.

These are design limitations and observed implementation concerns, rather than claims from a completed comparative security benchmark.

What will you do differently today?

I would define the evidence contract and evaluation boundaries earlier.

Specifically, I would preserve distinct action occurrences, pin concept revisions, represent unknown outcomes explicitly, and attach provenance and freshness to contextual facts. I would also build conformance fixtures alongside the language specification and test the actual action-release boundary before drawing conclusions from aggregate accuracy scores.

What trade-offs did you consider?

  • Semantic flexibility versus predictability: Semantic models broaden coverage; exact adapters and parsers provide more constrained behavior within supported inputs.
  • Conservative enforcement versus task completion: Holding unresolved actions reduces exposure but can interrupt legitimate work.
  • Expressiveness versus complexity: A finite core makes evaluation easier to reason about, while leaving temporal policies and richer graph relationships to explicit extensions.
  • Shared definitions versus local requirements: Portable signatures need stable meanings, while organizations need configurable boundaries and versioned overrides.
  • Detailed evidence versus latency and cost: Structured evidence supports debugging and replay, but may require more processing than directly asking a model for a verdict.

How can this help other practitioners?

The session gives teams a concrete way to inspect an agent security system:

  • What observations does it actually capture?
  • How does it normalize those observations into behavioral evidence?
  • Which decisions are deterministic, and which depend on uncertain recognition?
  • What happens when context is missing or the evaluator is unavailable?
  • Does enforcement happen before the protected action becomes effective?

These questions support architecture reviews, integration testing, and comparisons between competing approaches. They also help teams avoid treating a detector score as proof of end-to-end protection.

Current state

A product has been built on top of Concepts + IQL, with a current implementation focused on coding agents. Research into broader applicability, interoperability, and end-to-end evaluation is ongoing.

Tags

#agents #security #semanticnormalization #platformengineering #runtimeenforcement #policylanguages #interoperability #observability #evaluation #workinprogress

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('Post a comment…') }}
{{ gettext('New comment') }}
{{ formTitle }}

{{ errorMsg }}

{{ gettext('No comments posted yet') }}

Hosted by

We care about site reliability, cloud costs, security and data privacy