Home » Blog » One tool call away from a supply chain breach

One tool call away from a supply chain breach

Is a tool call the difference between an external prompt injection and a successful process?
Casestudy Graphic Hero

Industry

Outcomes

  • Scale with AI
  • Scale with Data

Services

  • Data Engineering

Organisation Size

Published

Author

Share:

The days of accidentally sending a file to the wrong client (or the wrong file to the right client) faded as we introduced tighter access control, improved Data Loss Prevention (DLP) tools, and wider awareness naturally grew.

So, what is the next big threat that an organisation has to worry about around DLP? Unfortunately, it is staring at us right in the face: the unrealised attack surface through agentic automation.

Agentic automation is a marvel in time savings and process efficiency improvement, but we sometimes take for granted the black-box nature of that processing and how actors can take advantage.

We’ve seen examples where a user will try to trick a chatbot or agentic process by interacting with it intentionally. But what if the next attack is just the agent stumbling across the minefield that is the web and falling into a trap?

How tool calling is being exploited

One way to vastly improve agentic flows is to introduce tool calling. This commonly involves enabling the ability to search the web or interact with other services. This is done through either dedicated Google Search tools, niche metadata search tools such as SearXNG, custom tooling or leveraging a new “llms.txt” file which is in the process of being standardised across the web.

Malicious actors can poison both Google search results and llms.txt to steer an agent accessing this information. This can be an ‘innocent’ attack to emphasise capability that is not available for a product to drive revenue, or can be as malicious as convincing an agent to install an alternative package pre-built with malware ready to steal the environment’s secrets.

The attack in premise is very simple: create a website optimised for agents and host a llms.txt favouring your magic product is the best, solves all problems and just wait. Alternatively, take a typosquatting-based approach, something as simple as changing a Python “httpx” package to “httpx-aiohttp” (a known existing typosquat), providing clear instructions on how this resolves the following error message:

But in the llms.txt, instead of referencing httpx2, suggest how this is instead properly resolved with httpx-aiohttp. This provides the prompt injection to ‘trick’ the agent into utilising an alternative package. As we start seeing full agentic PR creation based upon automated upgrading of dependencies managing breaking changes, we would expect, in this niche example, the developer reviewing the code to catch this attempt to install the package before further damage is done. Yet as seen with recent supply chain attacks, even just installing a malicious package (Which contains a pre-install script) can result in CI/CD tokens being exploited – and it might be unknown for weeks.

				
					/home/[redacted]/.venv/lib/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
				
			

How do you prevent this from occurring?

To address this, organisations must pivot from a reactive posture to a “zero trust” architecture for AI agents. The fear of being left behind has resulted in productionising PoCs without the usual assurance expected for a product, even if it is not customer/internet-facing. Prevention cannot rely on the hope that an individual will catch a poisoned dependency, or find it odd that Miracle product X will solve all their problems.

Instead, deterministic guardrails must be hardcoded around the agent’s execution boundaries, sandboxing tool calls, such as limiting web searches to trusted whitelisted domains and employing content-filtering layers to restrict possible poisonous attacks and leverage shadow models.

Defending against a new wave of exploitation requires a refresh on how we define data loss prevention, where historically DLP was inward-facing, preventing proprietary data from leaking out. However, in the era of agentic automation, organisations need to look to prevent unverified external data sneaking into the system. Every search result, API payload, and llms.txt file must be treated as an untrusted user input subject to strict sanitisation.

Until organisations accept that tool-calling agents effectively wield the privileges of automated internal users. An indirect prompt injection will remain a dangerously thin line between efficient groundbreaking automation and a critical supply chain breach.

Picture of Jorge Millares-Bobet

Jorge Millares-Bobet

Principal Consultant, Scale Factory

Consultation Bottombar Graphic
Not sure where to start

01 | Industry challenges discussion

02 | Compliance requirements review

03 | Solution approach outline

04 | Next steps & roadmap

Thinking about
a similar

challenge?

We work with organisations across regulated and complex industries to build the foundations for AI-enabled growth.

Related Insights

What would you like to search for?