Last week, PinnacleOne examined how contractors like I-Soon (上海安洵) fit into the larger Chinese hacking ecosystem and highlighted key implications for business leaders.

This week, we focus on China’s application of emerging AI tools to augment their rapidly improving cyber capabilities and emphasize the urgency for defenders to keep pace.

Please subscribe to read future issues – and forward this newsletter to your colleagues to get them to sign up as well.

Feel free to contact us directly with any comments or questions: [email protected]

Insight Focus: China’s AI-Enabled Cyber Capabilities

Highly capable nation state threat actors like China are looking to leverage AI to augment and accelerate their cyber operations. While we make this assessment with high confidence, the specific real-world effects will remain hard to discern and attribute. The UK’s National Cyber Security Centre found in a recent assessment that:

“AI is likely to assist with malware and exploit development, vulnerability research and lateral movement by making existing techniques more efficient. However, in the near term, these areas will continue to rely on human expertise, meaning that any limited uplift will highly likely be restricted to existing threat actors that are already capable. AI has the potential to generate malware that could evade detection by current security filters, but only if it is trained on quality exploit data. There is a realistic possibility that highly capable states have repositories of malware that are large enough to effectively train an AI model for this purpose.”

While use cases like deep fakes and synthetic media for influence operations are overt and more easily detectable, we believe that technical indicators that an attacker like China is using AI to augment other cyber activities may be sparse for some time. Instead, AI tools may improve offensive operations in a way not easily observed by the defender. This is owing to how adversaries are considering using AI for offense.

How China is Using AI for Cyber

Public research indicates that some universities connected to People’s Republic of China (PRC) security services host research institutes and PhDs working on applying AI to “APT attack and defense”. Among the topics covered by some of these schools include using AI to improve the pace at which software vulnerabilities are discovered – a capability that would improve PRC operational tempo, but which would not be easily discernible as an impact of AI by the defenders.

OpenAI’s recent blog post identifying activities by specific threat actors on ChatGPT supports this analysis. Hacking teams used ChatGPT to help debug or write code, perform open source research on foreign intelligence agencies, and translate technical documents. None of the actions outlined by the blog would appear in technical indicators seen by defenders.

Similarly, China has begun hosting competitions to automate vulnerability discovery, exploitation, and patching – another process that would improve operational efficiency but go unseen by the defenders. The timeline below shows the competitions held to automate this process, including through the use of machine learning techniques. Many of the universities conducting research on AI and cyber attack and defense participated in these competitions.

Finally, it is clear that the PRC has built cyber ranges to build and test these capabilities. Peng Cheng Labs hosts a cyber range with significant computational resources, ties to the security services, and an interest in automating attack path decision making with AI. Another cyber range in China, Zhejiang Labs, had a researcher publish about using AI to improve attacks on ICS systems.

Security Impact on Western Firms

None of the technologies being researched by actors in the PRC and covered here would provide technical indicators that AI was used to enable the attack. Instead, vulnerabilities discovered and exploited – and the attack paths taken by attackers – will continue to look “normal.”

Near term, evidence of AI in offensive operations may only be discernable in the operational pace and efficiency of operations – analysis that would require more complete knowledge of PRC hacking operations than any one cybersecurity firm may possess.

The impact of China’s efforts will be to accelerate the pace and effectiveness of their overall cyber operations. This will exacerbate the existing significant challenge the U.S. and its allies already face in confronting broad-scale and aggressive PRC cyber activity. It should motivate a sense of urgency in driving development and adoption of AI-enabled defensive tools and capabilities by public and private organizations across the Western world.