SoK: Data Reconstruction Attacks Against Machine Learning Models - Definition, Metrics, and Benchmark

Learn about data reconstruction attacks against machine learning models through this 14-minute conference presentation from USENIX Security '25. Explore a comprehensive systematization of knowledge (SoK) that addresses the critical gap in formal definitions and evaluation metrics for attacks that aim to recover training datasets from target models with limited access. Discover how researchers from the Institute of Science Tokyo and CISPA Helmholtz Center for Information Security propose a unified attack taxonomy and formal definitions specifically for the vision domain. Examine the development of quantitative evaluation metrics that prioritize quantifiability, consistency, precision, and diversity in measuring attack quality. Understand how large language models (LLMs) are leveraged as substitutes for human judgment to enable visual evaluation with emphasis on high-quality reconstructions. Gain insights into the unified framework for systematically evaluating existing attack strengths and limitations while establishing benchmarks for future research. Review empirical results from a memorization perspective that validate the effectiveness of proposed metrics and provide valuable guidance for designing new data reconstruction attacks in machine learning security.