Written by Dan McInerney for Dark Reading

With artificial intelligence (AI) and machine learning (ML) adoption evolving at a breakneck pace, security is often a secondary consideration, especially in the context of zero-day vulnerabilities. These vulnerabilities, which are previously unknown security flaws exploited before developers have had a chance to remediate them, pose significant risks in traditional software environments.  

However, as AI/ML technologies become increasingly integrated into business operations, a new question arises: What does a zero-day vulnerability look like in an AI/ML system, and how does it differ from traditional contexts?

Understanding Zero-Day Vulnerabilities in AI

The concept of an "AI zero-day" is still nascent, with the cybersecurity industry lacking a consensus on a precise definition. Traditionally, a zero-day vulnerability refers to a flaw that is exploited before it is known to the software maker. In the realm of AI, these vulnerabilities often resemble those in standard Web applications or APIs, since these are the interfaces through which most AI systems interact with users and data. 

However, AI systems add an additional layer of complexity and potential risk. AI-specific vulnerabilities could potentially include problems like prompt injection. For instance, if an AI system summarizes one's email, then an attacker can inject a prompt in an email before sending it, leading to the AI returning potentially harmful responses. Training data leakage is another example of a unique zero-day threat in AI systems. Using crafted inputs to the model, attackers may be able to extract samples from the training data, which could include sensitive information or intellectual property. These types of attacks exploit the unique nature of AI systems that learn from and respond to user-generated inputs in ways traditional software systems do not.

The Current State of AI Security

AI development often prioritizes speed and innovation over security, leading to an ecosystem where AI applications and their underlying infrastructures are built without robust security from the ground up. This is compounded by the fact that many AI engineers are not security experts. As a result, AI/ML tooling often lacks the rigorous security measures that are standard in other areas of software development. 

From research conducted by the Huntr AI/ML bug bounty community, it is apparent that vulnerabilities in AI/ML tooling are surprisingly common and can differ from those found in more traditional Web environments built with current security best practices.

Challenges and Recommendations for Security Teams

While the unique challenges of AI zero-days are emerging, the fundamental approach to managing these risks should follow traditional security best practices but be adapted to the AI context. Here are several key recommendations for security teams: 

  •  Adopt MLSecOps: Integrating security practices throughout the ML life cycle (MLSecOps) can significantly reduce vulnerabilities. This includes practices like having an inventory of all machine learning libraries and models in a machine learning bill of materials (MLBOM), and continuous scanning of models and environments for vulnerabilities. 
  •  Perform proactive security audits: Regular security audits and employing automated security tools to scan AI tools and infrastructure can help identify and mitigate potential vulnerabilities before they are exploited. 

Looking Ahead

As AI continues to advance, so too will the complexity associated with security threats and the ingenuity of attackers. Security teams must adapt to these changes by incorporating AI-specific considerations into their cybersecurity strategies. The conversation about AI zero-days is just beginning, and the security community must continue to develop and refine best practices in response to these evolving threats.