On this page
Retrieval-augmented generation and tool-using agents expand what an AI application can do. They also expand the number of places where untrusted information can influence a consequential action. A retrieval system selects context; a model interprets it; a tool executor may then read, modify, or transmit real data. Each transition needs its own authority boundary.
This guide develops a defensive design for an assistant that researches project documents and proposes task updates. It is an original review workflow, not a guarantee that one configuration secures every agent. The core principle is to let the model propose useful work while the application retains control of permissions, destinations, budgets, and execution.
Draw the data path and the action path separately
A data-flow diagram shows documents moving into retrieval and model context. An action-flow diagram shows proposed operations moving toward a tool executor. Keep both. A system may correctly restrict document retrieval while still exposing a tool that can edit records outside the user’s project.
The OWASP AI Agent Security Cheat Sheet discusses tool permissions, context security, human oversight, and independent validation of high-impact actions. Those ideas support separating model reasoning from execution authority. In the example design, the assistant can draft an update, but a policy-aware execution service decides whether that exact update is allowed. The model’s explanation is not treated as evidence of permission.
Filter retrieval with current authorization context
Apply tenant and resource permissions before retrieved text reaches the model. The user’s question is not a trustworthy authorization filter. A request that mentions a restricted project should not cause the retrieval service to search that project and then ask the model to keep the result confidential.
For the example assistant, derive the permitted document scope from verified user context and current project membership. Carry document identifiers and provenance alongside each passage. Recheck access when serving stored answers or source links if permissions may have changed. Include caches and search indexes in the review: they can retain content or decisions after the original access rule changes. A correct database permission check is insufficient if an unpartitioned answer cache returns another tenant’s previous result.
Keep retrieved instructions inside the content boundary
Documents and tool responses may contain text that resembles instructions to the assistant. Treat that text as source material, not application policy. Preserve the distinction when summarizing, translating, chunking, or combining documents. A transformed representation should not inherit a higher trust level merely because another model produced it.
In the project assistant, source passages can inform a draft task description but cannot grant permission to change the project owner. Put privileged instructions in a separate, controlled configuration path. Use clear provenance in prompt assembly and downstream records. This helps reviewers locate where an external sentence entered the workflow, while enforced tool permissions limit its consequences if the model nonetheless interprets it incorrectly.
Design small tools with narrow arguments
Prefer tools that express a bounded business operation over generic tools that accept arbitrary commands or destinations. A function that proposes a task status change is easier to constrain than a function that executes any database query. The useful design question is what the tool can do with valid arguments, not only whether invalid JSON is rejected.
For each tool, define the authorized actor, target resource, allowed fields, value constraints, and expected side effects. Reject extra arguments unless the contract explicitly permits them. Resolve tenant identifiers from trusted context where appropriate rather than allowing the model to supply a different tenant. Keep read and write capabilities separate. A research session can often operate with read-only tools until a user explicitly requests a specific change.
Bind approval to the exact action
When an operation requires human approval, show a trustworthy preview of the actual normalized action: what changes, which resource is affected, and where information will go. Do not rely only on a natural-language summary generated by the same model that proposed the action. The user should approve the operation the executor will perform.
Bind the approval record to the actor, tool, target, normalized arguments, and an appropriate expiration. If any important field changes, require a new approval. Check for replay where repeated execution would be harmful. In the example assistant, approving one task update must not authorize a batch update across a project. If membership or task state changes between approval and execution, re-evaluate the relevant conditions rather than assuming the earlier approval remains sufficient.
Treat memory as stored input
Agent memory can improve continuity, but it also preserves content across time and sessions. Define who can write memory, which sources are eligible, what scope it belongs to, and when it expires. Do not store an external document’s instructions as a permanent user preference or administrative rule simply because the assistant found them useful in one conversation.
For the project assistant, distinguish a user-approved preference from a temporary summary of a document. Partition memory by the appropriate user and tenant context, and provide deletion or expiry behavior consistent with the feature. Keep sensitive source text out of memory unless it is necessary and authorized. When memory is reused, treat its provenance and current access rights as relevant inputs, not facts that were settled forever at the time of storage.
Bound execution and outbound destinations
A tool-capable assistant needs limits on the whole task: steps, retries, elapsed time, concurrent work, and external calls. Each component should respect a trusted remaining budget rather than starting its own independent unlimited retry loop. A task that is cancelled or denied should not leave hidden tool work running indefinitely.
Review outbound destinations separately. A generated URL can become a sensitive boundary when a backend fetches it or sends data to it. Prefer approved destinations and narrowly scoped integrations over arbitrary network access. For the example assistant, task updates stay inside the project service and document citations point only to authorized records. Adding an email or webhook tool requires a new review of recipients, payloads, approval, and data exposure.
Evaluate end-to-end consequences
Test the full workflow, including retrieval, prompt assembly, memory, proposed tools, approvals, execution, and response rendering. A model refusing an instruction is useful behavior, but the decisive question is whether the application performed an unauthorized action or disclosed protected data. Keep those outcomes separate from the wording of the generated answer.
Build fixtures for cross-project access, a stale permission, an altered approved action, a tool timeout, a repeated execution request, and a task that reaches its step budget. Include legitimate work that should succeed. Record the model version, tool definitions, policy version, retrieval settings, and memory state used in each run. This gives the team a reproducible basis for reviewing changes rather than relying on one reassuring demonstration.
Plan capability-level containment
Prepare controls that can disable a particular tool, connector, or memory feature without necessarily disabling every read-only function. Keep ownership and rollback instructions beside the capability definition. During an incident, preserve minimized decision records that identify the involved resources and actions. Avoid copying private documents into broad incident channels. Restore the capability only after its affected boundary has been tested, not simply after changing the prompt that happened to precede the failure.
Conclusion: the executor must own authority
A defensible RAG and agent design keeps document relevance separate from access permission, and keeps model proposals separate from executable authority. Narrow tools, current authorization, exact approvals, scoped memory, and task budgets make that separation concrete. They also make failures easier to understand and contain.
The LLM API shield guide covers the surrounding prompt and response boundaries. Continue with the AI shield API architecture to connect retrieval and tools to the wider policy pipeline. A useful agent can be powerful within a well-defined task without receiving unrestricted authority over the systems around it.
