Protect AI's July 2024 Vulnerability Report

Executive Summary:

At Protect AI we are taking a proactive approach to identifying and addressing security risks in AI systems, to provide the world with critical intelligence on vulnerabilities and how to fix them. 

Protect AI’s huntr is the world's first AI/ML bug bounty program. Our community of 15,000+ members hunt for impactful vulnerabilities across the entire OSS AI/ML supply chain. Through our own research and the huntr community, we’ve found the tools used in the supply chain to build the machine learning models that power AI applications to be vulnerable to unique security threats. These tools are Open Source and downloaded thousands of times a month to build enterprise AI Systems. They also likely come out of the box with vulnerabilities that can lead directly to complete system takeovers such as unauthenticated remote code execution or local file inclusion. This report contains 20 vulnerabilities which you can find all the details of in the table below. Or, you can head over to protectai.com/sightline, to search the comprehensive database of huntr findings, and download tools to detect, assess and remediate them within your organizations AI Supply chain. 

It is important to note that all vulnerabilities were reported to the maintainers a minimum of 45 days prior to publishing this report, and we continue to work with maintainers to ensure a timely fix prior to publication. The table also includes our recommendations for actions to take immediately, if you have these projects in production. If you need help mitigating these vulnerabilities in the meantime, please reach out, we’re here to help. community@protectai.com.

Privilege Escalation (PE) in ZenML

https://sightline.protectai.com/vulnerabilities/f278804c-97ae-446b-9a70-b79606a19372/assess

Impact: Unauthorized users can escalate their privileges to the server account, potentially compromising the entire system. A vulnerability in ZenML allows users with normal privileges to escalate their privileges to the server account by sending a crafted HTTP request. This can be exploited by modifying the is_service_account parameter in the request payload.

Local File Inclusion (LFI) in lollms

https://sightline.protectai.com/vulnerabilities/ec267473-45fc-4f4c-bb02-409a1465a498/assess

Impact: Attackers can read or delete sensitive files on the server, potentially leading to data breaches or denial of service.

The sanitize_path_from_endpoint function in lollms does not properly sanitize Windows-style paths, making it vulnerable to directory traversal attacks. This allows attackers to access or delete sensitive files by sending specially crafted requests.

Path Traversal in AnythingLLM

https://sightline.protectai.com/vulnerabilities/dbd917bb-6676-4410-ad72-1787f6f74a62/assess

Impact: Attackers can read, delete, or overwrite critical files, leading to data breaches, application compromise, or denial of service.

A bypass in the normalizePath() function allows attackers to perform path traversal attacks. This can be exploited to read, delete, or overwrite files in the storage directory, including the application's database and configuration files.