AI models block 87% of single attacks, but just 8% when attackers persist

The Vulnerability of Open-Weight AI Models in Real-World Attacks

When it comes to defending against malicious attacks, open-weight AI models show a significant vulnerability when faced with sustained adversarial pressure. While these models may perform well in blocking single-turn attacks, the success rates plummet when attackers employ multi-turn strategies.

The research conducted by the Cisco AI Threat Research and Security team reveals that the gap between single-turn and multi-turn attack success rates can be as high as 80%. This shift in success rates highlights the critical importance of evaluating the resilience of open-weight models for enterprise deployment.

According to DJ Sampath, SVP of Cisco’s AI software platform group, the increase in vulnerability from single-turn to multi-turn attacks is a significant concern. While models may be able to withstand isolated adversarial inputs, they struggle to maintain contextual defenses over extended dialogues, allowing attackers to refine prompts and bypass safeguards.

The Impact of Conversational Persistence on AI Security

The study conducted by the Cisco research team demonstrates that open-weight AI models are particularly susceptible to attacks that leverage conversational persistence. By extending the conversation and employing techniques such as information decomposition and reassembly, contextual ambiguity, crescendo attacks, role-play, and refusal reframe, attackers can significantly increase their success rates.

These multi-turn attack strategies exploit the natural flow of conversation, making them familiar and effective in bypassing the defenses of open-weight models. The research findings emphasize the need for enterprises to understand the vulnerabilities of these models and implement appropriate guardrails to mitigate the risks.

Addressing the Security Gap in Open-Weight AI Models

The research conducted by Cisco highlights the need for enterprises to prioritize security measures that can enhance the resilience of open-weight AI models. Context-aware guardrails, model-agnostic runtime protections, continuous red-teaming, hardened system prompts, comprehensive logging, and threat-specific mitigations are essential capabilities that can help defend against multi-turn attacks.

It is crucial for enterprises to act swiftly and proactively in implementing these security measures, as the window for action is narrowing. By understanding the vulnerabilities of open-weight AI models and taking steps to enhance their security, enterprises can unlock the full potential of AI adoption while safeguarding against malicious attacks.