Prompt injection is what happens when text that a model reads is treated as instructions rather than as data. It is first on the OWASP Top 10 for Large Language Model Applications, and it is the vulnerability class that best explains why securing AI systems is genuinely different.
The uncomfortable part is that there is no equivalent of parameterized queries here. In SQL injection you can cleanly separate code from data. With a language model, the instructions and the data are the same thing: text.
Direct Injection: Talking the Model Out of Its Rules
Direct injection is a user telling the model to ignore its instructions. This is what most people mean when they say jailbreak.
It matters less than it sounds in most business contexts, because the person doing it is the person who would receive the output anyway. If a user convinces a chatbot to answer a question it was told not to answer, the harm is usually reputational or policy related rather than a breach.
It matters a great deal when the model has permissions the user does not have, which is increasingly common.
Indirect Injection: The Real Problem
Indirect injection is when the malicious instructions arrive inside content the model processes on someone else's behalf. A web page it browses. A document it summarizes. A support ticket, an email, a code comment, a log entry, or a calendar invitation.
The user did not write those instructions and cannot see them. The model reads them, and if it has been given tools, it may act on them.
This is why the risk scales with capability. A model that only answers questions can be made to say something wrong. A model that can send email, query a database, open tickets, or run code can be made to do something wrong, using permissions that belong to someone else.
Why It Is Hard to Fix
Every proposed fix runs into the same wall. Filtering malicious instructions requires reliably recognizing them, and the space of ways to phrase an instruction is effectively unbounded, including in other languages, in encodings, and in text hidden from human view.
Instruction hierarchy training, where models are taught to weight system instructions above content, reduces the success rate meaningfully. It does not eliminate it, and vendors are generally careful to describe it as mitigation rather than prevention.
So the practical approach is architectural. Assume injection will sometimes succeed, and design so that a successful injection cannot cause serious harm.
Defenses That Actually Reduce Risk
Least privilege for the model. Give an assistant the narrowest set of tools and permissions that lets it do its job. If it does not need to send email, it should not be able to.
A human approval step for consequential actions. Reading is one risk tier. Writing, sending, paying, and deleting are another. Approval should show the actual action, not a summary the model wrote of the action.
Separation of trust levels. Treat anything the model reads from the outside world as hostile input, the same way you treat form data in a web application.
Output handling. If model output reaches a browser, escape it, because a model can be induced to produce cross site scripting payloads. If it reaches a shell or a database, the same rules apply as for any other untrusted string.
Logging. Record prompts, retrieved context, tool calls, and outputs. When something goes wrong you need to be able to reconstruct why the system did what it did.
The Rest of the OWASP LLM Top 10
Prompt injection is first, but the list covers the whole application. Sensitive information disclosure, where the model reveals data from its context or training. Supply chain risk in models, adapters, and datasets. Data and model poisoning. Improper output handling, which is the classic injection problem in reverse. Excessive agency, which is the permissions problem named directly. System prompt leakage. Weaknesses in vector and embedding stores used for retrieval. Misinformation. And unbounded consumption, which covers both cost and denial of service.
The full list with descriptions is at OWASP. For attacker techniques against AI systems organized in ATT&CK style, see MITRE ATLAS.
Retrieval Systems Deserve Special Attention
Retrieval augmented generation, where a system searches a document store and feeds the results to a model, is the most common enterprise AI pattern. It is also a direct path for indirect injection, because the retrieved documents are exactly the untrusted content the model will read.
There is a second and quieter problem: permissions. If the retrieval index was built by crawling everything, and the model answers everyone's questions from that index, you have built a system that cheerfully leaks documents across permission boundaries. Access control has to be enforced at retrieval time, per user, not applied afterwards.
Testing Your Own AI Systems
Red teaming an AI application is a growing specialty. The practical version is trying injection through every input path, checking what the model can reach, and testing whether the approval steps can be bypassed by making an action look routine.
The NIST AI Risk Management Framework and its generative AI profile provide structure for this work, and are covered in our AI governance guide. For building assistants with tools safely, see securing AI agents.
Learn This at HackRange
Week 11 of the mentorship covers prompt injection as part of a hands on AI assisted security review, including finding places where the assistant itself was the weak point.