Protect AI's January 2024 Vulnerability Report

Introduction

The landscape for AI and the security of AI is evolving at breakneck speed. At Protect AI we are taking a proactive approach to identifying and addressing security risks in AI systems, to provide the world with critical intelligence on vulnerabilities and how to fix them. 

Protect AI’s huntr is the world's first AI/ML bug bounty program. Our community of 13,000+ members hunt for impactful vulnerabilities across the entire OSS AI/ML supply chain. Through our own research and the huntr community, we’ve found the tools used in the supply chain to build the machine learning models that power AI applications to be vulnerable to unique security threats. These tools are Open Source, meaning they likely come out of the box with vulnerabilities that can lead directly to complete system takeovers such as unauthenticated remote code execution or local file inclusion. What does this mean for you? You are likely at risk of theft of models, data, and credentials. 

Below you will find a list of vulnerabilities discovered by our community this month, with the key ones being focused on MLFlow. These issues only affect the open-source MLflow project and do not impact Databricks MLFlow users. 

It is important to note that all vulnerabilities were reported to the maintainers a minimum of 45 days prior to this publication. The maintainers of MLflow are also constantly working to identify and address security issues promptly, and all identified vulnerabilities below have been fixed in MLflow 2.9.2 and above. The table also includes our recommendations for actions to take immediately, if you have these projects in production. If you need help mitigating these vulnerabilities in the meantime, please reach out, we’re here to help. community@protectai.com.

This Month's Top Vulnerabilities

MLflow Arbitrary File Overwrite via Malicious Source URLhttps://huntr.com/bounties/93e470d7-b6f0-409b-af63-49d3e2a26dbc/

Impact: Server takeover, sensitive information loss

MLflow, a tool for storing and tracking models, had an arbitrary file overwrite vulnerability in the code used to pull down remote data storages. Users could be fooled into using a malicious remote data source which could execute commands on the user’s behalf. This could be used to get remote code execution on the server running MLFlow.

MLflow Arbitrary File Overwrite via Path Validation Bypass

https://huntr.com/bounties/2408a52b-f05b-4cac-9765-4f74bac3f20f/

Impact: Potential for system takeover, denial of service, destruction of data

A bypass in an MLflow function which validates that a file path is safe was found, allowing a malicious user to remotely overwrite files on the MLflow server. This can lead to remote code execution with additional steps such as overwriting the SSH keys on the system or editing the .bashrc file to run arbitrary commands upon next user login.

MLflow Local File Include

https://huntr.com/bounties/fe53bf71-3687-4711-90df-c26172880aaf/

Impact: Loss of sensitive information, potential for system takeover

MLflow hosted on certain types of operating system could be tricked into displaying the file contents of sensitive files through a file path safety bypass. There is potential for system takeover if SSH keys or cloud keys were stored on the server and MLflow was started with permissions to read them.

Table