Skip to content
Agent security· Guide 8 of 8Intermediate7 min readUpdated

Checking model text vs controlling tool calls ​

There are two places to stop an AI agent from doing harm. You can check the text: scan prompts for attacks, scan answers and tool arguments for secrets or unsafe content, and block what matches. Or you can control the actions: decide, outside the model, which tools this agent may call, with which arguments, for which caller, and which calls must wait for a person. Text checks judge what words mean, so they are probabilistic and an attacker can rephrase around them. A tool policyMCP tool policy: A set of rules, enforced outside the model, that decides whether a given agent or client may call a given MCP tool with given arguments: allow it, deny it, or hold it for a person to approve. judges a structured request against fixed rules, so it gives the same answer every time, whatever wording led to the call. You want both, but they are not interchangeable: for an agent with tools, the tool policy is the control that still holds when a text check misses.

Two questions, asked at different points ​

A text check asks: does this text look dangerous? It runs on strings: the user's prompt, a retrieved document, a tool result, a tool call's arguments, the final answer. It might be a pattern match for card numbers, a classifier for injection attempts, or a second model asked to judge the answer.

A tool policy asks: is this caller allowed to take this action, here? It runs on a structured request: tool name, server, argument values, the role or client behind the run, the workflow, and sometimes what data the run has touched. It does not try to understand language at all.

Where text checks and tool policy sit in an agent runText enters the agent from the user and from tool results, and text leaves as the answer; text checks sit on those flows. When the model asks for a tool call, the request passes a tool policy decision of allow, deny or require approval before the tool server runs it.User promptAgent + modelAnswertext checktext checkTool policyallow · deny · approvalTool servertool calltool result (text check)
Text checks sit on the flows of text in and out. The tool policy sits on the one path that changes other systems: the tool call.

Why text checks alone are not enough for agents ​

Text checks are useful, and for a chatbot they may be most of what you need. For an agent they have four limits that matter.

They can be talked around. Any check that judges meaning can be beaten by text that means the same thing in another form: a paraphrase, another language, an encoding, an instruction split across two documents. OWASP's entry on prompt injection is direct about the state of the art: "it is unclear if there are fool-proof methods of prevention for prompt injection." Its list of mitigations includes input and output filtering, and also least-privilege access with tool calls handled in code and human approval for high-risk actions, because filtering will sometimes miss.

A harmful call can be made of harmless words. refund_order({"order_id": "A-1042", "amount": 900}) contains no attack string, no secret and no unsafe content. Whether it is harmful depends on facts no text check sees: is this agent supposed to issue refunds, is 900 within its limit, did a customer actually ask? Those are authorization questions.

They decide nothing about who. The same call is fine for a support lead and wrong for an anonymous webhook. Text checks look at content, not at the caller.

A miss is silent. When a filter fails, nothing marks the miss. The action runs, and you learn about it afterwards.

OWASP's guidance on excessive agencyExcessive agency: OWASP's name for an LLM application that can take damaging actions because it has more functions, permissions or autonomy than its task needs. points the same way: "Implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not." A text check that asks a model whether a call looks safe is still relying on a model to decide.

Why tool policy alone is not enough either ​

A tool policy has the opposite blind spot. It sees the shape of a call, not the meaning of the values in it.

  • Allowed tool, harmful content. Policy can allow send_email only to addresses at your own domain. It cannot tell whether the body of that email holds a customer's card number. A text check on the arguments can.
  • Bad text with no action. An agent that answers a customer with something false, offensive or confidential calls no tool at all. Only a check on the answer sees it.
  • Rules only cover what you foresaw. A rule on env one_of ["dev", "staging"] is exact, but someone has to know that env is the argument that matters. That is why least-privilege rules are best written from the calls agents actually make (least privilege for agents covers how).

The comparison ​

QuestionText checksTool policy
What does it look at?Strings: prompts, documents, tool results, arguments, answersA structured call: tool, server, argument values, caller, workflow
How does it decide?Patterns, classifiers or a judge model; a score or a matchFixed rules: allow, deny or require approval
Same input, same answer?Not always, for model-based checksYes
Can wording get around it?Yes: paraphrase, encoding, splittingNot the prompt's wording. A deny-list on argument values can be dodged by spelling; an allow-list cannot
Knows who is calling?NoYes: role, client, project
Catches a secret in an allowed call?Yes, if it matches a detectorNo
Catches a harmful answer with no tool call?YesNo
Typical failureA miss that nobody noticesA rule that is too broad, or too narrow and blocks real work

How to combine them ​

Treat the tool policy as the boundary and text checks as the filters inside it.

  1. Decide the actions first. List what the agent must be able to do and allow only that: these tools, these argument values, for these callers. Where your policy engine allows anything no rule matches, deny by default with one broad deny rule, then carve out narrower allow rules; the calls you did not foresee are then refused, whatever the model is persuaded to try. The design patterns paper by Beurer-Kellner et al. (2025) explains why text checks cannot be the boundary: it calls detection-based defenses "fundamentally heuristic".
  2. Hold the calls that matter for a person. Writes above a threshold, sends to outside addresses and deletes go to an approval. A person sees the actual call, with its arguments, not the model's description of it.
  3. Add text checks where meaning matters. Detect secrets and personal data in arguments and answers, flag likely injection in retrieved text, and check answers against their sources. These catch what rules cannot express.
  4. Let what the run has read narrow what it may do. Once a run has read untrusted content or sensitive data, tighten the tool rules for the rest of the run. The same paper puts the principle plainly: once an agent has read untrusted input, "it must be constrained so that it is impossible for that input to trigger any consequential actions". This joins the two: content decides the labels, and the policy acts on them.

Then measure both. For tool rules, check how many real calls each rule would have changed before you enforce it. For text checks, track false positives and known misses against a test set.

In L2M ​

Tool policy is off until an admin adds a rule: a tool call that no rule matches is allowed. Text checks are on by default for agents: an agent with no rule set of its own, its workflow's or its project's uses the built-in default rule set, unless an admin turns auto-apply off. Neither needs any setup for a simple workflow.

  • MCP tool policy rules allow, deny or require approval by project, role, server, tool name and, optionally, argument values and the run's data labels. A tool that an unconditional deny blocks is removed from the agent's tool list; rules with conditions are decided at call time, with the call's arguments. Least privilege for agents covers precedence, conditions that fail closed, and testing rules before you enforce them.
  • Text checks. Content rules check prompts, tool arguments, tool results and answers, and can block, rewrite or pause at each point.
  • The order between them. Policy is checked first: a call a rule denies or holds for approval never reaches the argument checks. A workflow's Ask before write tools switch, on its own, runs after the argument checks, so a call they block is never put to a person, and the person approves the arguments that got through. The switch covers tools on an agent's tool port; a direct MCP Tool node needs an Approval Gate in front of it.
  • Labels join the two. A rule can match label_present on the run's data labels, so a send tool can be denied or held once untrusted data is in the run (or sensitive data, when label tracking is on in a rule set).

Try it in L2M ​

The govern MCP tools guide walks through a first set of rules, and human approval for agent tool calls sets up approvals. For the attack these controls defend against, read prompt injection and the lethal trifecta.

Further reading ​

Last updated:

This website (not the L2M product) uses Google Analytics, only after you accept cookies. Cookie settings