According to Philippa Cogswell, Vice President & Managing Partner, Unit 42 – Asia Pacific & Japan at Palo Alto Networks, the latest research by Unit 42 highlights significant vulnerabilities in large language models (LLMs) that could pose risks to organizations. DeepSeek jailbreaking research conducted by the unit reveals that LLMs can be manipulated, indicating that companies cannot always trust these models to function as intended. Cogswell emphasized the importance of considering these potential vulnerabilities when incorporating open source LLMs into business processes.
Cogswell also noted that as organizations look to leverage these models for various applications, they must assume that threat actors are also utilizing them for malicious purposes. She warned that the manipulation of LLMs could lead to more advanced cyber attacks, especially as nation-state threat actors are already using OpenAI and Gemini to enhance phishing attacks, develop malware, and improve the effectiveness of cyber threats. As these threat actors refine their techniques, the sophistication, speed, and scale of cyber attacks are expected to increase.
To mitigate these risks, Cogswell stressed the need for organizations to build safeguards at the organizational level. Given the possibility of LLM guardrails being broken, it’s crucial that businesses incorporate additional protections into their security strategies. The evolving landscape of AI-driven cyber threats means that organizations must be vigilant and proactive in safeguarding their operations from increasingly sophisticated AI attack agents.
As the use of AI and LLMs continues to grow, understanding and addressing their potential vulnerabilities will be key to preventing their exploitation by malicious actors.
