Protect AI's December 2023 Vulnerability Report

At Protect AI, we are taking a proactive approach to identifying and addressing security risks in AI systems, to provide the world with critical intelligence on vulnerabilities and how to fix them. Knowing the specific threats that exist, is just one step to enable your entire organization, from AppSec to ML teams, to build systems that are “secure-by design”.

This is the second monthly vulnerability report from us, supporting our mission to create a safer AI-powered world. Through our research and the Protect AI huntr community, we continue to find critical vulnerabilities in the tools used in the supply chain to build your AI applications. Many of these OSS tools, frameworks, and artifacts, come out of the box with vulnerabilities that can lead directly to complete system takeovers such as unauthenticated remote code execution or local file inclusion. This month we will cover a number of critical vulnerabilities found in popular AI Open Source projects used by millions of users per month.

It is important to note that all vulnerabilities were reported to the maintainers a minimum of 45 days prior to this publication, and the fixed status reflects the situation at the time of publication. We have also been working very closely with some of the maintainers, to ensure they have had sufficient time to address the vulnerabilities found. Our repo of vulnerability findings and exploits can be found here. We will publish the exploits for December’s vulnerabilities in the coming weeks. Our recommended remediation actions pertaining to December’s report, are listed in the table below. If you need help mitigating these vulnerabilities in the meantime, please reach out, we’re here to help. community@protectai.com.

This Month's Top Vulnerabilities

H2O-3 Arbitrary File Overwrite

Impact: Denial of service.

H2O-3 stands up a webserver and allows the importation of data for automatic creation of machine learning models based on the data. It contains a network-accessible API call which saves files to the server’s disk and includes the ability to overwrite arbitrary files. The data written to disk is in CSV format preventing this attack from being used to replace sensitive configuration files such as SSH keys, but can lead to denial of service by overwriting system files necessary for the server to function. 

https://huntr.com/bounties/a5d003dc-c23e-4c98-8dcf-35ba9252fa3c/ 

Kubeflow External HTTP Interaction

Impact: Server can be used as a proxy to internal resources.

Kubeflow contained a flaw that allowed users of the webserver to force the webserver to make arbitrary outbound requests. Attackers could abuse this to query internal network resources that may have not been accessible to them or to use the Kubeflow server as a proxy to perform malicious actions on other resources.

https://huntr.com/bounties/82d6e853-013b-4029-a23f-8b50ec56602a/ 

Gradio GitHub Action Command Injection

Impact: takeover of official GitHub repository

Gradio maintainers responded and fixed an issue that would’ve allowed the private GitHub tokens to be stolen via a command injection vulnerability in GitHub Workflows. An attacker could fork the repo, inject a payload, then submit a pull request and automatically gain access to the repo’s private token allowing them to take over the official repository.

Table: