Researchers have devised a novel method for manipulating machine learning (ML) models by embedding malicious code during the serialization process. This technique targets the “pickling” process, commonly used to store Python objects in bytecode. Despite the well-known risks associated with Pickle files, ML models are often distributed in this format.
According to a recent blog post by Trail of Bits, Pickle files provide attackers with cover to inject malicious bytecode into ML programs. This injected code could potentially lead to various consequences, such as altered output or data theft, without being easily detected like other supply chain attack methods.
David Brauchler, a principal security consultant with NCC Group, warns that this approach allows malicious behavior to be subtly integrated into applications at runtime, potentially evading detection for extended periods.
One example of such an attack is the “Sleepy Pickle” technique, which can be executed using tools like Flicking. By convincing a target to download a tainted .pkl file, attackers can trigger a malicious operation code during deserialization, executing it as a Python payload.
This method offers several advantages for stealth, as it does not require direct access to the target’s system and leaves no trace on the disk. Additionally, the dynamic nature of the attack during deserialization makes it resistant to static analysis, unlike static malicious models in repositories like Hugging Face.
The serialized model files contain substantial data, allowing the malicious code to represent only a fraction of the total file size. These attacks can be tailored in various ways to evade detection and analysis, similar to traditional malware attacks.
While these attacks can potentially cause significant harm, implementing controls such as sandboxing, isolation, and traffic control can mitigate the impact on user systems and data.
To mitigate these risks, organizations can opt for safer file formats like Safetensors, which exclusively handle tensor data without the risk of arbitrary code execution during deserialization. Alternatively, models distributed in Pickle format can be uploaded to secure sandboxes like AWS Lambda for conversion into Safetensors files.
However, Brauchler emphasizes the importance of trust management within systems, urging organizations to separate data retrieval from the code used by ML models. This architectural approach ensures that even if models malfunction, the application users and assets remain unaffected.
