Protect AI's February 2024 Vulnerability Report

At Protect AI we are taking a proactive approach to identifying and addressing security risks in AI systems, to provide the world with critical intelligence on vulnerabilities and how to fix them. 

Protect AI’s huntr is the world's first AI/ML bug bounty program. Our community of 15,000+ members hunt for impactful vulnerabilities across the entire OSS AI/ML supply chain. Through our own research and the huntr community, we’ve found the tools used in the supply chain to build the machine learning models that power AI applications to be vulnerable to unique security threats. These tools are Open Source and downloaded thousands of times a month to build enterprise AI Systems. They also likely come out of the box with vulnerabilities that can lead directly to complete system takeovers such as unauthenticated remote code execution or local file inclusion. 

Below you will find a full list of vulnerabilities discovered by our community this month, including a summary of the recently published critical vulnerability in the Triton Inference Server, and a Remote Code Execution (RCE) in Hugging Face transformers. 

It is important to note that all vulnerabilities were reported to the maintainers a minimum of 45 days prior to publishing this report, and we continue to work with maintainers to ensure a timely fix prior to publication. The table also includes our recommendations for actions to take immediately, if you have these projects in production. If you need help mitigating these vulnerabilities in the meantime, please reach out, we’re here to help. community@protectai.com.

Triton Inference Server --model-mode=explicit Remote Code Execution

https://huntr.com/bounties/b27148e3-4da4-4e12-95ae-756d33d94687/

Impact: Server takeover, sensitive information loss

Triton is an API server which allows users to query machine learning models for inference. For example, a user may send a request to the server with a customer’s purchases and receive product recommendations back. An engineer may run Triton using the --model-mode=explicit argument which allows Triton to serve a different model without restarting the server. This argument opened the door to a file overwrite vulnerability which could be turned into direct remote code execution on the Triton server. More details can be found in  this security notice or the Triton Inference Server user guide

Local File Include in Gradio

https://huntr.com/bounties/25e25501-5918-429c-8541-88832dfd3741/

Impact: Loss of sensitive data such as credentials

An API call in Gradio included a file path that when tampered with, could be used to read arbitrary files on the Gradio server. Sensitive files such as SSH or cloud keys could be stolen by an attacker as well as other data or code.

RCE in Hugging Face transformers https://huntr.com/bounties/423611ee-7a2a-442a-babb-3ed2f8385c16/

Impact: takeover of user’s system, loss of sensitive data

Through a creative chain of actions, huntr zpbrent was able to display potential for a wormable exploit hosted on Hugging Face via a deserialization attack in the RagRetriever object of the transformers library.