AI Prompts: Sensitive Secrets Circulate Among Providers
A study conducted by Dragos injected nearly 2 million trapped prompts into AI services. Result: leaks of secrets, query logs, and gaps between privacy promises and actual practices.
Researchers injected nearly 2 million trapped prompts into a chain of AI services to verify what happens to sensitive information. According to the study led by HD Moore, chief research officer at Dragos, secrets reappeared with third parties, sometimes after hijacking, sometimes through logging or data scraping practices contrary to client expectations. This issue matters for companies entrusting data to models via intermediaries, as passing from one router to another multiplies leakage points. It also reminds us that a "no data collection" flag does not always carry the weight of a concrete safeguard.
Quand le prompt traverse plusieurs intermédiaires, la confidentialité se dilue
"AI routers" or "AI gateways" act as routing points: they receive a request, read it in clear text to choose an inference engine, then forward it to one or more available providers. In this circuit, prompts may pass through actors like OpenRouter or Vercel before reaching a model hosted elsewhere, with at each step a risk of logging, retention, or reuse. Moore explains having sown valid AWS keys, SSH passwords, unique URLs, DNS names, and references to private Git repositories in a single prompt at a time, to precisely track where each secret reappeared. The method allowed linking leaks to specific links in the chain, rather than settling for a general observation on "privacy."
Des fournisseurs compromis, d’autres trop curieux
The results describe three types of problems. First, at least three AI providers were compromised by attackers who siphoned off received prompts. Next, several inference resellers recorded requests then passed them on to people using the exposed credentials. Finally, some platforms did not respect announced privacy settings, notably Zero Data Retention (ZDR) or "no data collection" options. Moore writes: "We sent nearly two million AI prompts containing unique, trackable secrets to AI providers, then watched where those secrets surfaced." A stark, almost clinical finding. The digital canaries have spoken, and not in favor of the sector.
Pourquoi les flags de confidentialité ne suffisent pas
The most worrying point concerns compromised hosts. In one case, attackers reused "a couple hundred" AWS keys, some stolen even before a model ran. This suggests copying requests right at the provider's entry point, not just further down the chain. In two other cases, secrets marked ZDR were nonetheless recovered and exploited. Moore also indicates that stolen keys were then used "tens of thousands" of times in AWS API calls, with attempts to explore S3 buckets, secrets managers, and parameter stores, then maintain access via IAM users. In other words, the leak did not just expose data: it opened doors.
Un laboratoire chinois mis en cause, des clients américains embarqués avec lui
The report also targets a Chinese lab, accused of retrieving secret URLs from more than half of the prompts sent between July and October, including about two-thirds of requests marked "no data collection" or ZDR. According to Moore, the scraper usually appeared two or three days after the request. Several American hosts reselling its models triggered the same behavior, meaning prompts supposed to remain in the United States were sent back to the original provider. In another case, a Git client on a different cloud tried to clone private repositories referenced in the prompts. When the promise of data localization relies on a reseller, the geographic boundary quickly becomes theoretical.
Ce que recommandent les chercheurs, et ce que cela coûte
Moore believes the best protection remains hosting on on-premises servers, thus controlled by the company itself. It is effective but costly. For those who cannot afford this option, he recommends a top-tier provider with a single contract and a clear privacy policy, as well as explicit choice of provider upstream rather than using default pools at intermediaries. The conclusion is harsh for the market: the AI value chain facilitates production deployment but complicates data control. Between convenience and control, companies can no longer claim to have seen nothing coming.
Key points
- Nearly 2 million prompts tested by HD Moore.
- Three compromised providers according to the Dragos study.
- ZDR secrets were nonetheless recovered.
- A Chinese lab triggered scrapers after requests.
- On-premises remains the safest option.
- OpenRouter and Vercel illustrate the role of AI routers.
By the numbers
- 2 million — prompts injected in the experiment, according to Dragos.
- 3 — compromised providers, according to Moore's investigation.
- Up to 2/3 — ZDR or "no data collection" requests affected by the scraper.
- 10,000+ — AWS API calls made with stolen keys.
- July to October — collection period mentioned for the Chinese lab.