Author: Alexander Ivanyuk, Senior Director, Technology
Generative AI changes how people work with information. A
user can ask a question, upload a document, summarize a ticket, draft an email
or ask an AI assistant to help with a workflow. That is useful because the
interaction feels natural. But the same natural-language interface also creates
a new security problem: instructions and data can become mixed together.
That is the basic idea behind prompt injection. An attacker
tries to place malicious instructions where an AI system may read them, so the
model follows the attacker’s goal instead of the user’s or the company’s
intended rules. Sometimes the malicious instruction is typed directly into the
prompt. Sometimes it is hidden inside a document, email, webpage, support
ticket, calendar invite, code comment, metadata or tool response. For a normal
application, that text may just be content. For an LLM-powered application, it
may become something the model interprets as an instruction.
According to
OpenAI, “Robustness to
adversarial attacks is a longstanding challenge for machine learning and AI,
making this a hard, open problem.” It also asserts that defending against
prompt injection is “a challenge across the AI industry” and that adversaries
are expected to keep developing attacks.
Anthropic
is even more explicit: “Prompt injection is far from a solved problem, particularly
as models take more real-world actions.” Anthropic also states: “No browser
agent is immune to prompt injection.”
This is why prompt injection protection is becoming a core
part of GenAI security. It is not only about stopping obvious prompts like
“ignore all previous instructions.” That is the easy example. The harder
problem is detecting harmful intent when the attack is subtle, hidden in
business content, or designed to manipulate an agent that can use tools and
access data.
Figure 1. Prompt injection attack chain
Why prompt injection is different from ordinary input abuse
Traditional security already knows how to deal with many
types of malicious input. SQL injection, command injection and cross-site
scripting all involve attacker-controlled input being interpreted in the wrong
way. Prompt injection follows a similar high-level pattern, but it works
through language and context instead of only code syntax.
That makes it harder to control. A malicious prompt may not
look like malware. It may look like a normal instruction, a polite request, a
note inside a document or a hidden message inside content that the AI is asked
to summarize. The model is not simply parsing a command. It is trying to
interpret meaning. That is why a prompt injection attack can be phrased in many
different ways while still aiming for the same outcome.
OWASP ranks prompt
injection as the first risk
in its LLM Application Top 10 for 2025. The important point is
not just that prompt injection exists, but that OWASP separates direct prompt
injection, where the user directly enters malicious instructions, from indirect
prompt injection, where hostile instructions are placed in external content
that the model later processes. That distinction matters a lot for business use
because employees often ask GenAI tools to summarize, analyze or act on content
they did not write themselves.
For managed service providers (MSPs) and small and medium IT
deployments (SMDs), the practical explanation is simple: Prompt injection is
not only something a malicious user types into a chatbot. It can arrive through
the content the business already handles every day.
Acronis GenAI Protection is now live: Secure the AI era
Try now
Direct and indirect prompt injection
A direct prompt injection happens when someone intentionally
gives the AI a harmful or manipulative instruction. The classic example is a
user telling the model to ignore safety rules, reveal confidential information
or bypass the intended task. These attacks can be crude, but they still matter
because users may test limits, employees may misuse tools and attackers may
interact with customer-facing AI systems.
An indirect prompt injection is often more dangerous in
business settings. In this case, the malicious instruction is not entered by
the user as a direct request. It is hidden inside something the AI reads: an
email, a webpage, a document, a support ticket, a database field or content retrieved through RAG. The user may simply ask, “Summarize this message,”
while the message contains hidden instructions telling the AI to ignore
previous rules, extract sensitive information, or send data somewhere else.
The difference is important because indirect prompt
injection attacks the trust relationship between user intent and external
content. The user wants the AI to process the content. The attacker wants the
AI to obey the content. Without protection, the model may not clearly separate
those two roles.
This is why prompt injection becomes more serious as AI
becomes more connected. A chatbot with no access to company data can still
produce a bad answer. But an AI assistant with access to email, documents,
tickets, CRM, file storage or model context protocol (MCP) tools has a much
bigger attack surface. If it can read sensitive context and call tools, a
malicious instruction hidden in the wrong place can become a data leak or an
unsafe action.
Figure 2. Indirect prompt injection attack chain
Real cases show why this matters
This is not only a theoretical risk. Several public cases
already show how prompt injection can affect connected AI systems.
One of the clearest examples is EchoLeak,
a zero-click vulnerability reported by Aim Labs in Microsoft 365 Copilot. The
researchers said the attack chain could automatically exfiltrate sensitive and
proprietary information from Microsoft 365 Copilot context without user
awareness and without relying on specific victim behavior. Microsoft mitigated
the issue, and public reporting said there was no evidence of real-world
exploitation, but the case still matters because it showed how AI connected to
enterprise context can become a data-exfiltration path when prompt handling,
retrieved data, and output channels interact in the wrong way.
Another useful example is Reprompt,
reported by Varonis Threat Labs. The attack used a personal (not 365) Microsoft
Copilot URL parameter to automatically populate a prompt from a link. Varonis
said the attack required only a single click on a legitimate Microsoft link and
could maintain control even when the Copilot chat was closed, allowing session
data to be exfiltrated with no further interaction. The case shows a practical
lesson: prompt injection does not always look like someone typing into a chat
window. It can be delivered through ordinary user behavior such as clicking a
link.
Agentic AI makes the same problem more serious. Trail of Bits
demonstrated how prompt injection can lead toward remote code execution in AI agents that are allowed
to run system commands. Their research focused on agent designs where some
commands are treated as safe and allowed automatically, while others require
approval. The lesson is simple: once an AI agent can call tools or execute
commands, prompt injection is no longer only about manipulating text output. It
can become a way to misuse the agent’s permitted actions.
These cases point to the same pattern. Prompt injection
becomes dangerous when three things come together: the AI can see sensitive or
trusted context, it can process untrusted content, and it can communicate
outward or take action. That combination is exactly why prompt injection
protection needs to sit inside the business workflow, not only inside the
model.
Why MSPs and SMDs should care
For MSPs, prompt injection is a customer-environment issue.
Customers may not describe it in technical language, but they will feel the
risk when AI becomes part of daily work. A customer may use AI to summarize
tickets, search internal documents, draft replies, review contracts or help
with support workflows. Each of those use cases can involve untrusted content
and sensitive business data at the same time.
For SMDs, the challenge is even more practical. Smaller
teams adopt AI because it saves time. They may not have a dedicated AI security
team, and they may not review every workflow in detail. That means employees
can start using LLM-powered apps before anyone has defined where prompts are
inspected, where sensitive data is blocked, and AI interactions should be
logged.
The business does not need to treat every prompt as hostile.
But it does need to understand that prompts are now part of the security
surface. A prompt can carry sensitive data. A prompt can carry malicious
instructions. A prompt can influence a model’s output. And in agentic systems,
a prompt can influence which tool is called next.
That is why prompt injection protection is not a “nice to have.”
It is part of making GenAI safe enough for normal business use.
What prompt injection protection needs to do, and how Acronis helps
Prompt injection protection cannot be just a simple keyword
filter. It has to deal with obvious attacks, subtle attacks, and attacks hidden
inside normal business content.
The first job is to detect direct instruction override
attempts, such as prompts that try to make the model ignore rules, reveal
secrets, bypass safeguards, or behave outside the approved purpose. Acronis
addresses this with Prompt Injection Protection that can detect potentially
harmful prompts, record details in the event log, block those prompts, and
notify the end user.
Prompt injections are addressed using a purpose-built engine
that analyzes prompts for patterns and techniques commonly associated with
attempts to manipulate LLM behavior. Acronis continuously tests and improves
this engine against a large corpus of legitimate and potentially harmful
prompts to help maintain effective detection while keeping false positives low
The second job is to catch more nuanced attacks where the
wording changes but the intent is still malicious. Attackers may frame the
prompt as a role play, a fictional scenario, a debugging request, a translation
task or a request to test the model’s hidden rules. This is why prompt
protection needs more than one detection method. Acronis uses an AI security
engine as a high-speed first line of defense, based on discovered attack
patterns, keywords, and regular expressions. The same security model also
includes additional layers such as NLP-based intent analysis, LLM checks for
higher-risk cases, and similarity search against known attack patterns.
The third job is to reduce the risk of indirect prompt
injection. This is where malicious instructions are hidden in content the AI is
asked to process, such as a document, webpage, email, ticket or tool response.
This is especially important for MSPs and SMDs because normal business
workflows often involve untrusted or semi-trusted content. Acronis treats
indirect prompt injection as part of the broader GenAI risk model, because LLMs
can be manipulated when they process attacker-controlled external data sources.
The fourth job is to connect prompt protection with data
protection. Many prompt injection attacks are dangerous because they try to
make the AI expose sensitive information or send it somewhere it should not go.
That is why prompt protection and DLP belong together. Acronis GenAI Protection can detect sensitive data sent to AI applications, record
details in the event log, block sensitive data from being sent, and notify the
user. This gives MSPs and customers both sides of the control: detect malicious
instructions and reduce the chance that sensitive data leaves the environment.
The fifth job is to support different enforcement levels.
Some customers may want to start with detection and logging so they can
understand what is happening. Others may want to block risky prompts
immediately. Acronis supports both modes: Prompt injection protection can be
configured to detect harmful prompts or block them, while recording the event
details in the log and notifying the end user when blocking is applied.
Finally, prompt injection protection needs to fit into the larger GenAI control model. A
harmful prompt is not always isolated. It may be part of shadow AI use,
sensitive-data exposure, agentic behavior, MCP tool access or abnormal
automated activity. Acronis brings prompt protection together with usage
monitoring, GenAI management for applications, agents, MCP servers, DLP, detection
of denial-of-wallet attacks, dashboard widgets and the Generative AI security report.
This makes prompt injection protection part of a managed security workflow
rather than a separate filter that is hard for MSPs to operate at scale.
Acronis GenAI Protection is now live: Secure the AI era
Try now
