Implementing Advanced Model Security for Custom Model Import in Amazon Bedrock

Integrating generative AI into enterprise workflows unlocks tremendous innovation potential, but it also introduces new and complex security challenges. Open source models, in particular, can contain hidden vulnerabilities including data leakage risks and susceptibility to adversarial attacks. To scale AI both rapidly and securely, it's critical to assess and mitigate these threats before deployment. We’ve seen that high quality, openly available machine learning (ML) models are becoming increasingly widespread, which is great for fostering innovation and knowledge sharing. However, just like there are malicious actors that take advantage of traditional OSS technologies, there’s been an increase in malicious actors taking advantage of AI/ML-specific technologies. For enterprises, this raises significant security considerations around machine learning model artifacts as they’re being shared internally and externally.

This blog builds on the prior work in Deploy DeepSeek-R1 distilled Llama models with Amazon Bedrock Custom Model Import. We’ll address the security implications that come from downloading external models in order to upload them to Amazon Bedrock Custom Model Import (CMI). The open source nature of these models paired with their competitive performance has encouraged enterprises to respond quickly, but possibly not securely.

We’ll explore how organizations can use Protect AI Guardian to ensure that new models, such as DeepSeek-R1, are secure before entering their environment and then continuously check for threats as models are fine-tuned or new vulnerabilities are found. 

Why Do We Need Model Security?

Models are software artifacts that require validation before deployment, just like other code components. This validation is especially critical as organizations increasingly depend on ML models for business critical decisions, making them high-value targets for attackers. Using ML models without implementing the right security checks puts your enterprise at risk. 

Models can be created internally, but oftentimes AI developers are pulling these models from external sources such as Hugging Face, Github, or other third-party sources. These models can contain malicious code that cannot be detected by traditional malware scanners, such as:

• Deserialization Threats: Malicious code embedded within models can execute during the loading process
• Backdoor Threats: Hidden triggers that, when activated, cause the model to produce unexpected or malicious outputs while appearing to function normally otherwise
• Runtime Threats: Threats that occur during model inference or execution 

These attacks can lead to harmful outputs, data leakage, and reputational damage for organizations deploying AI solutions. 

In the DeepSeek solution overview below, we’ll explore DeepSeek-R1-Distill-Llama-8B. This is a smaller, more efficient version of DeepSeek-R1 that has gone through the process of knowledge distillation.

Distilled models use a technique called knowledge distillation (KD), where a smaller “student” model learns to mimic the behavior of a larger “teacher” model. The distilled model is trained to mimic the predictions of the larger model while maintaining much of its accuracy with reduced computational requirements. Distilled models can help lower deployment costs and provide faster inference times, making them attractive model architectures. There are, however, security considerations to take into account, such as:

• Amplified Vulnerabilities: If the teacher model contains biases, adversarial weaknesses, or privacy leaks, the distilled model can inherit or even amplify them due to optimization trade-offs.
• Adversarial Transfer: Attacks that are crafted for the teacher model can transfer effectively to their distilled version, sometimes with higher success rates due to reduced complexity and robustness.
• Backdoor Inheritance Risk: A teacher model that includes a backdoor can be published to an open model hub (such as Hugging Face) and unsuspecting users looking to use it for distillation will unknowingly propagate the hidden vulnerabilities into their own model. 

Figure 1: DeepSeek-R1’s model repository on Hugging Face Hub showing Protect AI’s integrated security scan results available to all users.

Through Protect AI’s strategic partnership with Hugging Face, Hugging Face users can access security insights for over one million models on Hugging Face Hub. This provides users with immediate confidence that models have been thoroughly vetted against the major vulnerabilities discussed in this blog. However, once a model enters your internal environment, responsibility shifts to your organization to ensure that subsequent modifications don’t introduce new security risks. 

Next, we’ll demonstrate how Protect AI Guardian creates a comprehensive security framework that extends protection seamlessly into your internal AI/ML development processes, establishing end-to-end ML security across your entire model lifecycle. 

Securing Models with Guardian

Protect AI’s Guardian is a comprehensive security tool designed specifically for machine learning models. Insights from our in-house threat research team and the huntr community of over 16,000 ethical hackers enable us to maintain the broadest and deepest feed of model vulnerabilities available today. 

Our security coverage extends well beyond traditional software vulnerabilities that appear in the National Vulnerability Database (NVD). After scanning over one million ML models on Hugging Face Hub—both traditional and generative—we’ve identified critical deserialization, backdoor, and runtime threats that informs Guardian’s ongoing enhancements, as documented on Insights DB. 

By continuously scanning both first-party and third-party models before deployment, Guardian adds a critical security layer to your ML development pipeline. This ensures only models that meet your security standards reach production environments, including platforms like Amazon Bedrock CMI. With Guardian, you can confidently pursue AI innovations knowing your models are protected against malicious code and novel security threats. 

Guardian secures your machine learning ecosystem through two components that integrate directly into your existing MLOps pipelines via SDK, CLI, or RESTful endpoints. We maintain a centralized repository of third-party model scans (Guardian Gateway) from sources like Hugging Face, while also providing a scanning system that operates within your environment (Guardian Scanner). 

Guardian Gateway: This proxy service creates a secure perimeter around third-party model access by intercepting requests to sources like the Hugging Face Hub through simple environment configuration. Acting as a gating mechanism, Gateway prevents potentially malicious external models from entering your environment. 

Guardian Scanner: Deployed within your infrastructure, Guardian Scanner provides a dedicated API endpoint for validating your internal first-party models as they move through development or undergo customization. 

Both Gateway and Scanner work with administratively configured policies that allow you to tailor your security posture to organizational requirements. This enables you to gate model deployments and implement a zero trust approach that ensures only secure, compliant models operate within your enterprise network. 

Solution Overview

Now, we’ll demonstrate how Guardian Gateway and Guardian Scanner seamlessly integrate into existing ML pipelines to establish a secure workflow that protects both first- and third-party models throughout their lifecycle. 

Figure 2: Guardian integrates directly into your existing machine learning pipelines.

We will implement a secure model deployment process through three key workflows:

1. Scan an incoming third-party model using Guardian Gateway: Gateway acts as a secure proxy to validate external models before they enter your environment.
   1. Enable relevant Managed Policies within Guardian to match your desired security posture:
      • By default, Guardian enables all policies to protect against deserialization, backdoor, and runtime threats.
      • Security administrators can customize which policies are enabled and configured.
   2. Retrieve DeepSeek-R1-Distill-Llama-8B from Hugging Face Hub only if it passes the Gateway scan.
      • Models failing security checks are blocked from entering your environment.
2. Process model within an internal environment:
   1. Upload the validated model to our internal Amazon S3 bucket.
   2. Simulate a malicious attack by an internal actor.
      • Once a model is stored internally, they’re often fine-tuned or otherwise customized before they’re promoted to production environments.
3. Scan first-party model using Guardian Scanner: Scanner validates internally developed or modified models before deployment.
   1. Deploy Scanner in your environment to maintain data privacy.
      • No proprietary model data leaves your infrastructure during scanning.
   2. Scan the modified DeepSeek model stored in S3.
   3. Take action based on scan results:
      • If PASSED: Proceed with deployment to Amazon Bedrock Custom Model Import.
      • If FAILED: Halt deployment and initiate remediation based on organizational practices.
   4. Repeat scanning process whenever changes are made to the model.

Prerequisites

1. Clone the following Github repo and navigate to the Guardian folder.
2. Have an existing Amazon S3 bucket with the proper permissions to upload and download.
3. Create a Bedrock Service Role for model import: See here.
4. Have AWS CLI v2 installed and configured with the proper roles to access the S3 bucket above.
5. Have a Protect AI Guardian instance.

Guardian Managed Policies

Once you’ve gotten access to the Guardian dashboard, you can begin to configure policies to match your company's security posture. 

1. Login to the Guardian dashboard. 
2. Navigate to the Managed Policies tab.
3. Under the third party models section in the policies list:
   1. Look for the “License is Valid For Use” policy and click edit policy.
   2. Confirm that the policy is Enabled.
   3. Add “mit” as a valid license and click Save.
   4. Continue with the notebooks.

Figure 3: Configurable Guardian policies allows security admins to ensure that models adhere to the company’s security policies.

Scan and Download a Model: DeepSeek-R1-Distill-Llama-8B

Make sure you’ve downloaded Secure-DeepSeek-R1-Distill-Llama-Noteb.ipynb.

In this section, we will attempt to download the DeepSeek-R1-Distill-Llama-8B from Hugging Face to our working machine:

1. Ensure you’ve gotten access to the Guardian Gateway endpoint. 
2. Point the HF_ENDPOINT environment variable to your Guardian Gateway endpoint.
3. Run the cells and continue within the notebook.

Note: The Guardian Gateway scan evaluation happens before the model ever enters our internal network. This means that the model is never downloaded locally to initiate a scan. Only models that have passed your policy checks will get downloaded.

Inspecting a Guardian Gateway Scan Result

PASS:

{

    "uuid": "59071aea-5c0d-4d87-9081-a611e5cdeae3",

    "model_uri": "https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B",

    "license": "mit",

    ...

    "global_threshold_snapshot": "MEDIUM",

    "enabled_policy_count_snapshot": 10,

    "created_at": "2025-03-01T00:09:01.375318Z",

    "updated_at": "2025-03-01T00:09:04.791719Z",

    "namespace": "deepseek-ai",

    "model_name": "DeepSeek-R1-Distill-Llama-8B",

    "revision": "6a6f4aa4197940add57724a7707d069478df56b1",

    "policy_violations": null,

    "model_uuid": "18e10fae-f978-4d7e-9386-c1a923cb5701",

    "model_version_uuid": "ffb1ab5e-9c0d-4907-9f5a-42a99c0a858d"

}

• license: The license found for the model
• global_threshold_snapshot: The global threshold that we set across all third-party model scans. A MEDIUM threshold is more restrictive because a model with lower severity violation will be blocked. 
• revision: Scans are done against specific revisions. This means that the “same” model can potentially have different scan results due to different revisions.
• policy_violations: Since no policy violations were found, there’s nothing to list, and therefore, null

FAIL: To demonstrate blocking behavior, we can simply set the “License is Valid For Use” policy to only allow a different license type (in this case we set it to apache-2.0).

...    

"policy_violations":[

  {

      "policy_name": "License Is Valid For Use",

      "remediation": {

            "description": "Ensure that the model includes a license",

            "steps": [

"Check with model owner to ensure that the model has a license",

"Ensure that the license is one of the approved licenses"

            ],

      },

      "severity": "MEDIUM",

      "compliance_issues": [

     "input data contains invalid license: apache-2.0"

      ],

      ...

  }

],

"model_uuid": "18e10fae-f978-4d7e-9386-c1a923cb5701",

"model_version_uuid":"ffb1ab5e-9c0d-4907-9f5a-42a99c0a858d"

Notice here that the policy_violations key now lists out the violations.

• In this example, the issue is that the input data, deepseek-ai/DeepSeek-R1-Distill-Llama-8B contains a “mit” license and NOT an “apache-2.0” license.
• This is classified as a MEDIUM severity violation, and since our global threshold is set to block any violations that are MEDIUM or higher, we block this model from being downloaded.

Inspecting a Guardian Scanner Scan Result

FAIL: We can demonstrate a failed first-party (internal) model scan by modifying the deepseek-ai/DeepSeek-R1-Distill-Llama-8B model that we previously downloaded. To do this, we can simply create a new “malicious” file called extra_data.pkl and then upload it to the same S3 bucket that the current model lives in. 

This simulates a real-world scenario where once a model is pulled from external sources, it’s modified internally before being sent to another location. In this case, we’re modifying the DeepSeek model that lives in S3 before we send it to Amazon Bedrock Custom Model Import. 

"policy_violations": [

     {

         "policy_name": "Detected No Arbitrary Code Execution At Model Load Time",

         "remediation": {

             "description": "Ensure that the model does not contain any potentially malicious operations",

             "steps": [

                 "Ensure that the model does not contain any potentially malicious operations",

                 "Use model formats that disallow arbitrary code execution"

             ],

             "url": "https://docs.protectai.com/guardian/policies#detected-no-arbitrary-code-execution-at-model-load-time"

         },

         "severity": "CRITICAL",

         "compliance_issues": [

             "The model will execute remote code since it contains operator `system` from module `posix`."

         ],

         "issues_by_file": [

             {

                 "file": "DeepSeek-R1-Distill-Llama-8B/extra_data.pkl",

                 "threat": "PAIT-PKL-100"

             }

         ]

     }

],

"global_threshold_snapshot": "MEDIUM",

...

• global_threshold_snapshot: The global threshold that we set across all first-party model scans.
• policy_violations: In this example, the violation comes from the fact that the Scanner found a potential threat in the extra_data.pkl which used the system operator and loaded it as a pickle file.

PASS: In order to remediate the policy violation found, we can follow the suggested remediation steps of ensuring that the model does not contain any potentially malicious operations and is using model formats that disallow arbitrary code execution. In other words, we must remove the system call and not use pickling.

{

"uuid": "702a45ea-d975-43e9-86fb-2f0fe470b61a",

"model_uri": "s3://pai-guardian-demo04-models/DeepSeek-R1-Distill-Llama-8B/",

...

"policy_violations": null,

"global_threshold_snapshot": "MEDIUM",

...

}

• policy_violations: Since no policy violations were found, there’s nothing to list, and therefore, null

Upload to Amazon Bedrock Custom Model Import

Let's recap our journey so far. We began by downloading the deepseek-ai/DeepSeek-R1-Distill-Llama-8B model from Hugging Face using Guardian Gateway, which automatically scanned it for malicious code during the transfer process. We then made modifications to the model and stored it in our internal Amazon S3 bucket—these changes simulate what might happen if a malicious actor compromised our internal infrastructure.

Following a zero trust security approach, we established a protocol to scan all models before they move to any new environment. In this case, we used Guardian Scanner to examine our internally modified model before uploading it to Amazon Bedrock Custom Model Import. This additional scan revealed a new threat: a vulnerability introduced through unsafe pickling and system calls.

Fortunately, we caught the security issue before deployment, allowing us to remediate the model. After fixing the vulnerability, we ran another scan with Guardian Scanner to verify the fix. With a clean security report confirmed, we can now confidently proceed with uploading our model to Custom Model Import.

Conclusion

As we look ahead to a future where AI capabilities continue to expand, the need for robust security measures becomes not just important, but essential. With Protect AI Guardian, you can enable your teams to harness the power of machine learning without compromising on safety.

The seamless integration of Guardian into existing workflows means security doesn't have to come at the cost of innovation or speed. By implementing automated scanning processes that work with Amazon Bedrock and other model sources, organizations can maintain their development velocity while significantly reducing risk exposure.

Perhaps most importantly, Guardian helps bridge the gap between security teams and AI practitioners. As these two disciplines increasingly overlap, having tools that speak both languages becomes invaluable. Guardian provides security professionals with the visibility they need while giving AI teams the freedom to explore and create.

Ready to learn more? Start securing your AI artifacts today by reaching out to our Sales team.