A Chief Information Security Officer (CISO) who overlooks prompt injection due to a low Common Vulnerabilities and Exposures (CVE) count is misinterpreting the situation. Prompt injection has consistently ranked as the top threat on the OWASP Top 10 for AI and Machine Learning (LLM) Applications for three years. However, an analysis conducted by Kyriakos “Rock” Lambros and Steve Wilson, two leaders of the OWASP Top 10 for LLM Applications project, revealed that when compared against 6,639 real-world incidents, prompt injection dropped to No. 12. This discrepancy highlights the visibility challenges associated with detecting this type of attack.
The study conducted by Lambros and Wilson involved analyzing 7,714 LLM security incidents from various databases, including CVE, GitHub Security Advisories, and others. The comparison between expert judgment and the public incident record showed weak agreement, indicating that more comprehensive measures are required to address prompt injection.
Prompt injection involves hiding malicious instructions within legitimate content, making it difficult for traditional vulnerability scanners to detect. As such, defenses against prompt injection should focus on adversarial testing and limiting the reach of AI agents within the system.
Wilson emphasized the importance of implementing an authorization gate to prevent AI agents from executing unauthorized actions based on injected prompts. This control ensures that the agent can propose changes but cannot grant itself the authority to implement them without proper authorization.
Despite prompt injection being a well-understood attack, the discrepancy between expert opinion and incident data underscores the ongoing challenges in mitigating this threat effectively. Wilson noted that incident data, while valuable, may not always reflect the most critical risks present in current systems.
The study also highlighted the disagreement between expert opinion and incident data regarding other LLM threats, such as misinformation. This discrepancy underscores the complexity of assessing and prioritizing AI-related risks.
Addressing these challenges requires a nuanced approach that goes beyond relying solely on incident data or expert judgment. By reevaluating the weighting of different risk factors and implementing tailored controls, organizations can better manage the evolving landscape of AI security threats.
Overall, the study’s findings emphasize the need for continuous monitoring, robust defenses, and a proactive approach to addressing AI-related risks in modern cybersecurity environments.
