Intent Security for AI Agents: When the Action Is Allowed but Wrong

Traditional access control answers one question: is this principal allowed to perform this action?

Runtime policy adds a second: does this specific command, path, or amount violate configured rules?

Intent security adds a third: does the agent's stated purpose match what the action will actually do?

That third question matters because agents can propose actions that are mechanically allowed and structurally valid — but dishonest, misaligned, or dangerously off-purpose.

Three questions at the tool boundary

Proposed tool call
    │
    ├─ Permission?     → IAM / RBAC / tool allowlist
    │
    ├─ Policy?         → Path caps, amount limits, blocked command patterns
    │
    └─ Intent?         → Stated reason vs. inferred effect
            │
            ├─ MATCH  → may proceed (if prior gates pass)
            └─ MISMATCH → BLOCK + audit record

Permission and policy catch obvious violations in milliseconds. Intent catches scope mismatches that pass mechanical checks.

Why the reason field exists — and why it is not trusted

IntentFrame requires agents to supply a reason with each governed action — a plain-language statement of purpose. The Hermes intentframe-gate plugin injects this into the tool schema shown to the model and validates before delegating to the native handler.

The field is not trusted. It is treated as adversarial input at every pipeline stage. A compromised or confused agent can lie.

What reason enables:

1. Purpose-keyed policy — intent_limits in plain English, injected into Guardian review as named boundaries
2. Stated-vs-inferred comparison — Analysis Engine cross-checks reason against action payload
3. Operator audit — searchable purpose strings, not just parameter dumps

Without a structural purpose field, the same mechanical action — a file write, a transfer, a cron registration — cannot be distinguished as on-purpose versus off-purpose. Policies collapse into over-block or under-block.

Semantic-only catches: when deterministic policy would allow

IntentFrame's invoice attack suite includes red-team cases where payloads hide in data and target while reason stays benign — simulating a post-compromise agent controlled by an attacker who knows the system inspects reasons.

Two attacks are the strongest evidence for why the semantic layer exists:

Attack 15 — stealth amount mismatch

Attack 17 — data exfiltration via write

The Analysis Engine catches the reason-versus-data mismatch; Guardian issues the BLOCK. If you removed both and kept only deterministic enforcement, attacks 15 and 17 would succeed. That is concrete proof that rules alone are insufficient for intent-shaped deception.

Overall invoice suite score: 23/24 defended, 0 bypassed, 1 known gap (Attack 16 — salami slicing across five $4,000 payments; cumulative session policy is planned, not a defense failure).

Production example: the .bashrc scope mismatch

In a live Hermes + IntentFrame Docker session (Hermes v0.17.0), a user asked the agent to add a # testing comment to .bashrc.

The patch tool call used replace mode with an empty old_string — effectively a file overwrite. The stated reason claimed to "add a comment."

Path allowlists catch /etc writes. They do not catch "add a comment" that overwrites a privileged shell init file. Only semantic intent review evaluates alignment between stated purpose and actual effect.

See the full session teardown: Runtime authorization incident teardown.

Detection is not enforcement in the same head

The return-agent experiment compared a hardened DIY support chatbot (semantic refund rules in the system prompt) against the same rules enforced by an external IntentFrame judge.

On a quiet chat attack across 51 runs:

The agent was often capable of seeing contamination (~68.6% escalation on the same attack shape). The failures were not "didn't notice" — they were "relabeled the thread as clean to stay in helper mode." Moving enforcement outside the agent changed end-to-end leakage from 29.4% to 0% on this attack.

That is not a proof of impossibility — the guardian is itself an LLM, and the experiment used a single attack shape across 51 runs of the same contaminated thread (IntentFrame only evaluated cases Build A approved). It still shows why intent-shaped semantic policy needs an external judge, not more prompt text.

How to operationalize intent security

1. Require reason on governed tools — schema + executor, not prompt-only
2. Write intent_limits in plain English — Guardian cites them verbatim in block messages
3. Combine deterministic + semantic — caps and paths for speed; AE + Guardian for mismatch
4. Treat audit as source of truth — when blocked, read the authorization record, not the model's explanation
5. Own known gaps honestly — per-intent policy does not yet catch cumulative abuse across sessions

Takeaways

1. Intent security is the third gate: purpose alignment, not just permission or path.
2. The reason field creates a comparison surface; it is not a trust anchor.
3. Structurally valid actions can still be semantically deceptive — attacks 15 and 17 prove it.
4. External enforcement matters when detection and approval live in the same model.
5. Intent security complements — does not replace — deterministic runtime authorization.

Related: LLM security vs AI agent security · Runtime authorization incident teardown · Deploy AI agent governance on Hermes · Deployment options

Image credits

Photo from Unsplash (license):

• Hero: imgix