Siri's 1.2T-Parameter Model Opens Your Data to Prompt Injection

Apple’s new Siri can read your messages, photos, and calendar to plan a watch party — but security researchers say the same conduit that makes that possible also makes prompt injection inevitable. Aim Security already demonstrated this attack class on Microsoft 365 Copilot with EchoLeak, a zero-click exploit where a single email plants instructions that the AI executes later, exfiltrating data through a silently loaded image.

The Lethal Trifecta

Simon Willison describes the core risk as the “lethal trifecta”: any assistant that can read private data, ingest untrusted content, and transmit information can be tricked into handing that data to a stranger. Apple’s new Siri, powered by the System Orchestrator layer and a 1.2 trillion-parameter AFM Cloud Pro model derived from Google Gemini, brings all three together. That model runs on servers inside Google data centers — reportedly on Nvidia chips — though Apple insists the deployment is separate from Google’s consumer stack.

Natalie Shapira at Northeastern University puts it bluntly: “Autonomous agents significantly expand the attack surface for prompt injection. The challenge is the chain of permissions and actions that connects the model to multiple applications and services.”

A Billion-Dollar Model With a Blind Spot

Apple licensed this Gemini-derived model for about $1 billion a year, according to Bloomberg. The company markets Private Cloud Compute as inspectable — researchers can verify servers don’t retain data — but that inspection promise doesn’t cover the data routing between the on-device Spotlight Semantic Index, the App Toolbox, and the cloud model. Encryption protects data in transit and at rest, but it can’t stop the model from misusing access it already has. A single email, webpage, or shared document can inject commands into the same stream as the user’s intent.

Apple’s WWDC demo showed Siri pulling tournament schedules from the internet, scanning Messages for a friend’s cookie recipe, drafting an invitation, and sending it — all without the user touching an app. That same pipeline can carry hidden instructions. EchoLeak proved it’s not theoretical on a production assistant. Apple patched nothing yet because the new Siri hasn’t shipped.

Regulatory Walls, Not Technical Ones

Apple addressed the risk through policy rather than architecture: Siri AI won’t launch on iPhones and iPads in the European Union, citing the Digital Markets Act. The company claims the law would force it to give rival assistants the same deep access. In China, the features await regulatory approval. The public release is scheduled for this fall.

Until independent researchers test the live system, Apple’s security claims rest on trust — and trust breaks when a single crafted document turns your personal assistant into an exfiltration pipeline.

Source: Inside the new Siri AI and the privacy paradox of Apple Intelligence
Domain: scientificamerican.com