Introduction
This post describes a data exfiltration vulnerability we found in collaboration with rez0, rhynorater and lupin in Google’s Bugswat Live Hacking Event in Tokyo, Japan in April 2025, which our team won overall. This specific issue was also awarded Most Creative Finding.
Data exfiltration vulnerabilities are quite common in LLM-based systems these days, and Google’s AI Security team is doing an excellent job of handling and fixing them.
Lately, we’ve observed a trend in the industry of nicknaming pretty typical markdown image-based data exfiltration vulnerabilities (that are already fixed, no less!). Considering this trend, we insist on giving our findings some nicknames as well. Given that we generally don’t name individual XSS vulnerabilities, we believe that we shouldn’t be naming prompt injection attacks either, except in extreme circumstances. These kinds of attacks are here to stay - we don’t need names for each one.
Anatomy of a Data Exfiltration Vulnerability
Generally, we can view a data exfiltration vulnerability as having two main components:
- The delivery - our means of delivering the prompt
- The exfiltration - using the delivery to exfiltrate data based on what’s accessible
There’s no point in having an exfiltration without being able to deliver it, and there’s no impact to a delivery without exfiltration or some other attack. Typically, indirect prompt injections are used to deliver payloads. Google’s threat model for LLMs prioritises Rogue Actions that could damage the integrity of the system, or Data Exfiltration.
Delivery and exfiltration in prompt injection exploits.
Initial Investigations
During our investigation of the event scope, we noted that the Gemini mobile app, and more specifically the Android app, had a significant number of tools that interacted with the phone itself. These tools are not available on the Web version. Google was positioning Gemini to replace Google Assistant as the primary assistant system on Android phones, and as part of this rollout, a number of tool calls were added to the Android app to allow users to control various settings on the phone directly from Gemini.
One of these mobile-only tool calls was the Notifications tool. Notifications granted Gemini the power to read the user’s current notifications, including the contents of text messages. This was a goldmine - you could use this tool to read 2FA codes! We had our means of getting PII or sensitive data. Next, we had to figure out how to get a prompt injection payload into the app itself.
Delivery
In a stroke of genius, Rhynorater very quickly found a gadget in the APK that solved this problem. He discovered an Intent in the APK that allowed us to populate the contents of the text input field in the chat (but not press the button!), and proposed a clickjacking-type attack using a malicious app to fool the victim into sending the message.
As such, we crafted a malicious app with a (very unconvincing) “captcha”.
Our POC app.
This “tapjacking” attack fooled the victim into clicking the button five times, but on the third tap, the intent would trigger, and the victim’s fourth or fifth tap would send the message within Gemini. The “captcha button” is conveniently placed exactly where the “Send” button is in the Gemini mobile app.
Stages of tapjacking.
Exfiltration
Sequencing Tools
The Gemini app on Android is optimised towards allowing you to manage your phone using Gemini with as little friction as possible. As such, there was a feature that allowed you to run tool calls in sequence. As long as the tool you were sequencing didn’t prompt you for confirmation, you could sequence as many tools as you wanted to without any user interaction. This feature existed (and still exists) so users could, for example, take photos with a timer through Gemini. This is not possible on the web interface of Gemini - only the mobile app. So we had a way forward to read the notification contents, and encode the contents somehow in another tool, or to commit rogue actions.
With the pieces for a full chain in place, we set about looking for an exfiltration mechanism. One of the more interesting tool calls available to us was the Phone tool call, which allowed Gemini to make phone calls on behalf of the user. There was also an SMS tool call, but this required the user to manually approve the message before it was sent; incredibly the Phone tool call did not suffer this limitation, and prompted no further confirmation. In theory, we could make the victim phone anyone. Naturally, this alone has relatively limited impact - you could leak the victim’s phone number to the attacker, but that was about it.
So we dug deeper, and eventually, we found DTMF.
DTMF
DTMF stands for Dual Tone Multi Frequency. In short, when you’re on a call with customer support and you hear “press 1 for XYZ” and you hear a tone after pressing it - that’s DTMF at work. It’s a system for assigning numbers to various tones. It’s called “dual-tone” because it plays a high tone and a low tone in various combinations, and there are 16 possible combinations. Phone tones have a long history in the hacking space - the origins of hacker culture date to the phone “phreakers” in the 70s, who used to use tones to achieve a kind of delimiter attack to get free calls. Those were single tones (2600Hz), not dual tones, but it gives this finding a poetic aspect.
Tone combinations in the DTMF system.
To play tones on the receiving phone, we can specify dial strings. This is a syntax that allowed us to specify what to play - a comma denotes a two-second pause. Crucially, DTMF tones play on the receiving phone; since Gemini allowed us to specify a phone number to call, it also allowed us to specify dial strings after the phone number.
This meant that we could now encode the user’s PII (or 2FA codes, for example) into the dial string, and call the attacker’s phone (with no further confirmation). The attacker could then simply record the tones that play upon picking up the call, and decode them to retrieve the data!
The flow of our attack.
Attack Scenario
In our proposed attack scenario, the victim would have our POC app installed; upon falling for the tapjacking, the prompt would send and no further interaction would be required - Gemini would subsequently phone the attacker and leak the victim’s Saved Info through the dial string.
Several months after this Live Hacking Event, Google released shareable Gems: this completely removed the necessity for a malicious app in the delivery component, but we will discuss that further in the next blog post, in which will talk about one of our findings in the Mexico City Live Hacking Event in October 2025.
Timeline
- April 10th 2025: Reported to Google
- April 10th 2025: Accepted
- April 22nd 2025: Rewarded $9,137 ($7,800 + $1,337 bonus for Most Creative)
- July 22nd 2025: Notification to Google of intent to talk about the finding at HackAIcon
- September 25th 2025: Disclosed at HackAIcon in Lisbon, Portugal
- November 19th 2025: Marked as Fixed
- February 9th 2026: Publication
Conclusion
We’ll be publishing some more research next week. The links will be posted on Linkedin and X, so you’re welcome to follow along if you liked this. This finding had a significant number of obstacles to exploitation, but in our next works, we pushed the limits of our creativity even further, so look forward to it.