Introduction
Since 2024, we’ve witnessed enterprises rapidly adopt LLMs and rush to secure these systems in production. Alongside this, we’ve seen a surge in tools and companies offering solutions to safeguard LLMs in production. Many of these solutions mirror our own open-source security toolkit, LLM Guard, which focuses on LLM firewalling. These firewalls enable companies to detect, redact, or sanitize risks in real-time by acting as a proxy between the model and the application, meaning all prompts and outputs pass through the firewall. While this approach is effective for POC deployments, it becomes unsustainable at scale due to the exponential increase in computational costs associated with low-latency scanning.
As LLMs evolve with retrieval-augmented generation and agentic capabilities, their operational scope and potential exploitation paths continue to expand. Traditional LLM firewalls, which focus solely on prompts and outputs, lack visibility into the interactions and calls LLMs make to connected systems, creating significant blind spots. This limitation highlights the need for a more future-proof approach to LLM security.
LLM Security with eBPF
This is where eBPF (Extended Berkeley Packet Filter) comes in, offering a shift in how we secure LLMs at scale and more importantly at runtime. Yet, eBPF is not new. In fact, it has been a driving force in the security industry where it transformed different verticals such as cloud, app, and network security and gave rise to some well known players such as Wiz, DataDog, and Red Canary and emerging players like Armis and Upwind. Many well-known enterprises have also extensively been using eBPF tools such as AWS, with Falco, and Cilium.
eBPF provides full visibility into every event within applications running in your infrastructure, functioning like an advanced security system embedded within your kernel. It offers real-time monitoring without the need for manual instrumentation of each LLM application, significantly reducing overhead. Unlike traditional in-line scanning methods, eBPF enables out-of-line runtime monitoring, delivering unparalleled visibility across the entire LLM value chain. This allows AI security teams to secure all internally deployed LLM applications without needing the AI engineering team to instrument their apps—one of the key drawbacks of LLM firewalls.
By offering a shorter time to value and security, eBPF minimizes overhead and friction between development and LLM adoption without compromising security. It captures every session and interaction, including API calls, system calls, and database queries, allowing for deeper insights and broader security coverage. Moreover, the rich telemetry data collected by eBPF enables advanced anomaly detection, policy creation, and enforcement. Security teams can quickly identify and respond to threats that traditional LLM firewalls often miss, thereby enhancing the overall security posture. eBPF is not just a tool; it’s an entirely new approach to securing LLM applications, providing deep, real-time visibility and control at runtime.
LLM Detection and Response with Layer
At Protect AI, we are actively future-proofing LLM security with eBPF with Layer. Layer provides unparalleled visibility and security for your LLM applications, tailored specifically for security teams. We are working closely with several partners to secure their LLMs in production environments.
Reach out to us to discover how Layer can secure your LLM applications at runtime with LLM Detection and Response capabilities.
