Security Risks Related to Downloading and Running LLMs Locally

Learn about the cybersecurity risks associated with downloading and running large language models (LLMs) locally in this 18-minute conference talk. Explore why the common assumption that "models are only data" is incorrect and discover the various formats used to distribute models, including Pickle objects, GGUF, and Python implementations, some of which contain executable code that runs when downloaded and executed by inference engines. Examine what actually gets downloaded when pulling an LLM from internet repositories and understand the potential security vulnerabilities this creates. Gain insights into cybersecurity risks specific to local LLM deployment and discover potential security controls and mitigation strategies, with particular focus on how open source models provide additional approaches for risk assessment and reduction.