Tool Use Is Not Enough: Why AI Agents Need a Policy Layer
Function calling gives AI agents capability but not governance. A policy layer with pre-execution authorization, scope constraints, and action budgets is what separates a demo from a production system.
Function calling changed what AI agents can do. A model that can invoke APIs, query data, and trigger actions is categorically more useful than a text generator. But function calling solved the capability problem. It did not touch the governance problem.
In most agent architectures, tool use works like this: model decides which function to call, runtime executes it, result flows back. The model is both decision-maker and executor. No independent authority evaluates whether the call should happen -- only whether it can.
For healthcare, this is not enough. The gap between "the agent can call this function" and "the agent is authorized to call this function in this context for this patient" -- that gap is where governance lives. Closing it requires a policy layer.
The Authorization Gap
Consider a practical scenario. A healthcare AI agent has access to three tools:
get_patient_summary— retrieves a patient's clinical summarysend_message— sends a message to a patient via SMS or app notificationcreate_task— creates a follow-up task for a care coordinator
The agent has the capability to call all three functions. But authorization depends on context:
- A front-desk coordinator's agent should be able to retrieve summaries and create tasks, but not send clinical messages.
- A care navigator's agent should be able to do all three, but only for patients in their assigned panel.
- An automated after-hours agent should be able to retrieve summaries for context, but should not send messages or create tasks without human review.
In a standard tool-use architecture, all three tools are exposed to the model, and the model decides which to call based on its prompt and context. Role-based restrictions, if they exist at all, are encoded in the system prompt — a natural-language instruction to a probabilistic system.
System prompt-based guardrails are not access control. They are suggestions to a probabilistic system. A sufficiently complex prompt, an unexpected edge case, a subtle reasoning error -- any of these bypass them. In healthcare, this is not theoretical. It is the most common failure mode of deployed AI systems.
Where Tool-Use Architectures Fall Short
Function calling provides capability without governance
No Role Enforcement
Tool availability is binary — exposed or not. There is no mechanism for role-based, context-dependent authorization at the function level.
No Action Budgets
The agent can call tools unlimited times per session. There is no concept of rate limits, action quotas, or cost constraints per interaction.
No Temporal Constraints
Tool calls have no time-based rules — no quiet hours for patient messaging, no business-hours-only restrictions for certain action types.
No Pre-Execution Audit
The decision to call a function is made by the model and logged after execution. There is no independent evaluation before the call happens.
No Sequence Governance
The agent can chain tool calls in any order. There are no rules requiring data retrieval before action, or review before communication.
No Consent Integration
Tool execution has no awareness of patient consent preferences, communication opt-outs, or data sharing restrictions.
What a Policy Layer Does
A policy layer sits between the model's intent to call a function and the actual execution of that function. Every tool invocation passes through the policy layer, which evaluates a set of rules before permitting execution.
The policy evaluation considers:
Identity and role. Who initiated this interaction? What is their role? What authorization level does that role carry? A policy engine resolves these questions against a structured role definition — not a natural-language system prompt.
Patient context. If the action involves a specific patient, what are their consent preferences? Are they opted into automated messaging? Have they restricted data sharing? Is there an active do-not-contact flag?
Action classification. What type of action is this? Read-only data retrieval, communication dispatch, clinical record modification, and task creation are different action classes with different authorization requirements.
Scope constraints. Is this action within the permitted scope for this role-context combination? A care coordinator's agent can create tasks for patients in their panel but not for patients in another department.
Budget limits. Has this session exceeded its action budget? Action budgets prevent runaway agents — a model stuck in a reasoning loop that calls the same function repeatedly, or an interaction that triggers an unexpectedly long chain of actions.
Temporal rules. Is this action permitted at this time? Patient-facing messages may be restricted outside business hours. Certain administrative actions may require weekday execution.
If the policy evaluation returns a permit decision, the function executes. If it returns a deny decision, the agent receives a structured explanation of why the action was not permitted. If it returns an escalate decision, the action is routed to a human reviewer with full context.
Atomic Interaction Execution
A policy layer needs a unit of work to govern. Individual function calls are too granular — a single user interaction may involve multiple data retrievals, reasoning steps, and actions. Governing each function call independently misses the interaction-level context.
The alternative is interaction-level execution. Each interaction is an atomic unit that encompasses a complete agent exchange: from the initial user request through reasoning, data retrieval, action execution, and response generation. The entire interaction operates under a single policy evaluation.
Each interaction has four phases:
-
Authorization. Before the model runs, the policy engine evaluates the request context — user role, patient scope, action type classification — and produces a permit set: the specific tools, data sources, and action types the agent is authorized to use for this Turn.
-
Execution. The model reasons and calls tools, but only tools in the permit set. Any tool call outside the permit set is blocked before execution, and the agent is informed of the boundary.
-
Completion. The interaction produces its output — a response to the user, a set of actions taken, or an escalation to a human reviewer.
-
Recording. The complete interaction — request, policy evaluation, tool calls, model output, actions, and outcome — is written as an immutable audit record.
Policy-Governed Agent Execution
Every interaction passes through authorization before any tool executes
Why Post-Hoc Filtering Does Not Work
The most common alternative to a policy layer is post-hoc filtering: let the model do whatever it wants, then review the output before it reaches the user or triggers a downstream action.
This approach fails in healthcare for three reasons:
Data exposure happens at inference time. If the model retrieves patient records it should not access in order to generate a response, the data exposure has already occurred — even if the response is filtered before delivery. The model has processed the data. It may influence subsequent responses in the same session. Post-hoc filtering does not prevent unauthorized data access; it only prevents unauthorized data display.
Filtering is less capable than the model. A post-hoc filter must anticipate every possible policy violation the model might produce. The model is a general-purpose reasoning system capable of novel output. The filter is a rule-based system designed for known violation patterns. The model will always be more creative than the filter. This is not a solvable arms race.
Latency and cost compound. When a model generates a response that is then filtered and rejected, the inference cost and latency are wasted. In a healthcare system handling thousands of AI interactions per hour, wasted inference is both expensive and slow. Pre-execution authorization prevents wasted work by ensuring the model only attempts actions it is permitted to complete.
What Policy-Governed Agents Enable
A policy layer transforms AI agents from capable-but-ungoverned systems into deployable clinical infrastructure:
Predictable behavior. The same role, in the same context, with the same patient, always has the same authorized action set. The model's reasoning may vary, but the boundaries of what it can do are deterministic and inspectable.
Graduated autonomy. Organizations can start with conservative policies — most actions require human approval — and expand autonomous scope as they build confidence. The policy layer makes this expansion controlled and reversible.
Multi-role deployment. A single agent platform serves different roles with different authorization levels. The agent's capabilities are constant; its permissions vary by role. This eliminates the need to build and maintain separate agent implementations for each user type.
Compliance by architecture. Regulators do not need to audit model behavior — they audit the policy engine. The policy definitions are declarative, versioned, and inspectable. They define what the system is authorized to do in plain, structured rules — not in probabilistic model behavior.
The Bottom Line
Tool use gave AI agents hands. A policy layer gives them judgment -- or more accurately, ensures that authorization judgment does not rest with a probabilistic model. The model reasons. The policy engine authorizes. The audit system records. Three separate concerns, three separate systems.
For healthcare AI, this separation is not an architectural preference. It is the difference between a demo and something you can actually deploy.
THB's AI Platform implements policy-governed execution with pre-execution authorization, action budgets, and role-scoped tool access. AI Agents built on this platform operate within structured policy boundaries from day one. Explore the architecture.