Protect AI's October 2024 Vulnerability Report

Executive Summary

At Protect AI we are taking a proactive approach to identifying and addressing security risks in AI systems, to provide the world with critical intelligence on vulnerabilities and how to fix them. 

Protect AI’s huntr is the world's first AI/ML bug bounty program. Our community of 15,000+ members hunt for impactful vulnerabilities across the entire OSS AI/ML supply chain. Through our own research and the huntr community, we’ve found the tools used in the supply chain to build the machine learning models that power AI applications to be vulnerable to unique security threats. These tools are Open Source and downloaded thousands of times a month to build enterprise AI Systems. They also likely come out of the box with vulnerabilities that can lead directly to complete system takeovers such as unauthenticated remote code execution or local file inclusion. This report contains 34 vulnerabilities You can find all the details of this month's vulnerabilities in the table below, or you can head over to protectai.com/sightline, to search the comprehensive database of all huntr findings, and download tools to detect, assess and remediate them within your organization's AI Supply chain. 

It is important to note that all vulnerabilities were reported to the maintainers a minimum of 45 days prior to publishing this report, and we continue to work with maintainers to ensure a timely fix prior to publication. The table also includes our recommendations for actions to take immediately, if you have these projects in production. If you need help mitigating these vulnerabilities in the meantime, please reach out, we’re here to help. community@protectai.com.

Top Vulnerabilities

Timing Attack in LocalAI

https://sightline.protectai.com/vulnerabilities/3fbcf95a-4a19-45ec-9463-c7858274047b

Impact: An attacker can potentially determine valid API keys by analyzing the response time of the server.

The vulnerability allows an attacker to perform a timing attack, which is a type of side-channel attack. By measuring the time taken to process requests with different API keys, the attacker can infer the correct API key one character at a time.

Insecure Direct Object Reference (IDOR) in Lunary

https://sightline.protectai.com/vulnerabilities/a8580293-cbec-4e97-8b6f-aec2c557f8ea

Impact: Unauthorized users can view or delete internal user data by manipulating user-controlled ID values.

The vulnerability arises because the application does not properly validate user-controlled ID values. This allows an attacker to access or delete data of other users by simply changing the ID in the request.

Insecure Direct Object Reference (IDOR) in Lunary

https://sightline.protectai.com/vulnerabilities/88d7a4f7-fb4f-40ff-8c32-276befb2dd78

Impact: Unauthorized users can update other users' prompts by manipulating user-controlled ID values.

The vulnerability is due to the application using a user-controlled ID parameter without proper validation. This allows an attacker to update prompts belonging to other users by changing the ID in the request.

Table