AI Agent, AI Spy - The Privacy Crisis of Agentic AI Systems

Explore the critical privacy and security threats posed by agentic AI systems being integrated into operating systems and applications in this 41-minute conference talk from 39C3. Examine how AI-enabled systems designed to complete complex tasks autonomously are fundamentally transforming operating systems and browsers from neutral resource managers into active, goal-oriented infrastructure controlled by corporations rather than users. Analyze specific examples like Microsoft's "Recall" feature, which creates comprehensive surveillance of all user activity under the guise of productivity enhancement, and understand how these systems create centralized, high-value targets for attackers while posing existential threats to privacy-focused applications like Signal. Learn how OS-level surveillance renders application-level privacy features, including end-to-end encryption, effectively useless by capturing data before encryption or after decryption. Discover the "blood-brain barrier" analogy that explains how secure applications rely on neutral operating systems as stable foundations, and how agentic AI breaks this critical barrier. Review an actionable four-point framework for addressing these immediate dangers, including empowering developers with APIs to designate sensitive applications, implementing granular user control over AI access permissions, mandating radical transparency from OS vendors, and supporting adversarial research efforts. Gain insights into the political economy of this emerging privacy crisis and understand what the hacker community can do to preserve developer agency, user control, and fundamental privacy rights in the age of pervasive AI surveillance.