Understanding Model Threats
This resource is designed to provide detailed information on various threat categories, helping you understand and mitigate potential risks in AI and machine learning systems.
Deserialization Threats
Deserialization threats occur when untrusted data or code is used to reconstruct objects, leading to potential exploitation. In AI and machine learning systems, this can result in malicious actors injecting harmful code during the deserialization process, exploiting vulnerabilities to gain unauthorized access or manipulate your systems behavior. Understanding deserialization threats is crucial for securing data integrity and preventing unauthorized code execution in your AI models.
Overview
Deserialization threats in AI and machine learning systems pose significant security risks, particularly when using Keras models with Lambda Layers. This article outlines the specific threat, its potential impact, and provides actionable steps to mitigate these risks.
Models flagged for this threat meet the following criteria:
- Built using Keras
- Incorporate Lambda Layers
- Contain potentially malicious code within Lambda Layers that executes upon model loading
Keras is an open-source deep learning library for developing machine learning models. It provides a high-level API that integrates with other machine learning libraries like TensorFlow, JAX, and PyTorch.
Lambda Layers in Keras allow custom operations or transformations on data within a neural network. While standard Keras layers (e.g., Dense, Conv2D) offer predefined functionality, Lambda Layers enable unique transformations when standard layers are insufficient.
The security threat arises from the flexibility Lambda Layers provide for customizing operations. This flexibility extends to allowing arbitrary code execution. As with all forms of code, users should only load and execute models from trusted sources.
Key Points
- Deserialization threats can lead to unauthorized code execution in AI models
- Keras models using Lambda Layers are particularly vulnerable
- Attacks can result in data theft, system compromise, and broader security breaches
- Mitigation strategies include using safe formats and thorough vetting processes
Impact
An attacker could exploit a compromised model to:
- Access sensitive information (e.g., SSH keys, cloud credentials)
- Execute malicious code on your system
- Use the compromised system as a vector for broader attacks
The code within Lambda Layers executes at model load time, potentially compromising your system before the model is even used.
How The Attack Works:
Best Practices
You should:
- Only load and execute models from trusted sources
- Implement a vetting process for third-party models before use
- Use sandboxing techniques when loading untrusted models
- Regularly update Keras and related libraries to benefit from security patches
- Use model formats that don't allow arbitrary code execution, such as SafeTensors, which provides a safe alternative to traditional serialization formats
Remediation
If possible, use a different model format like SafeTensors in order to remove this type of code injection attack from impacting your work entirely.
If not possible, reach out to the model creator and alert them that the model has failed our scan. You can even link to the specific page on our Insights Database to provide our most up to date findings.
The model provider should also report what they did to correct this issue as part of their release notes.