How suspected Chinese agents exploited AI to refine surveillance strategies
OpenAI has revealed that suspected Chinese government operatives misused ChatGPT to help design large-scale surveillance systems and propaganda materials, raising new alarms over how artificial intelligence is being repurposed to enhance authoritarian control.
In its latest security report, the company detailed cases in which users—believed to be tied to Chinese government entities—asked ChatGPT to write proposals for monitoring Uyghur Muslims and other so-called “high-risk” groups, and to create promotional content for tools scanning social media for “extremist” speech.
The revelations underscore a growing concern: AI models are no longer just powering innovation—they’re also being weaponized for repression and cyber operations.
What OpenAI uncovered in its latest investigation
According to the report, one ChatGPT user, “likely connected to a [Chinese] government entity,” requested help drafting a proposal for a surveillance tool that could analyze police records and travel histories of ethnic minorities, including Uyghur Muslims.
Another user reportedly sought assistance in crafting marketing materials for a platform designed to scan major social media sites—such as Facebook and X—for politically sensitive or religious content.
OpenAI permanently banned both accounts after identifying their misuse.
“There’s a push within the People’s Republic of China to get better at using artificial intelligence for large-scale things like surveillance and monitoring,” said Ben Nimmo, OpenAI’s principal investigator. “They’ve realized AI could make those operations more efficient.”
These cases, while alarming, also highlight how AI can magnify existing state capabilities rather than create entirely new ones. The Chinese Communist Party has long deployed sophisticated surveillance systems, particularly in Xinjiang. The use of generative AI simply makes those systems faster, cheaper, and more polished.
Why AI misuse is becoming a global security threat
The OpenAI report paints a broader picture of how both state-backed hackers and private scammers are adapting AI for exploitation.
- State-linked actors from Russia, North Korea, and China have reportedly used ChatGPT to improve the grammar and authenticity of phishing messages or refine malicious code.
- Cybercriminals—including groups based in Myanmar—have employed AI to manage fraudulent operations, from researching legal loopholes to tracking payments.
- Scammers are using AI for deception, while ordinary users are increasingly turning to AI tools like ChatGPT to spot scams before falling victim.
OpenAI estimates that ChatGPT is now being used three times more often to detect scams than to create them, suggesting AI is becoming a double-edged sword—empowering both criminals and their potential targets.
“Adversaries are using AI to refine existing tradecraft, not to invent new kinds of cyberattacks,” said OpenAI security expert Michael Flossman.
The U.S.-China AI rivalry enters a new phase
The incident comes amid an intensifying AI arms race between the United States and China. Both nations are investing billions to dominate the technology that could define global influence in the 21st century.
Earlier this year, Chinese firm DeepSeek unveiled its ChatGPT-like model “R1,” reportedly capable of similar performance at a fraction of OpenAI’s cost. The development sparked unease among U.S. officials and investors, who fear China’s AI breakthroughs could outpace Western innovation.
In response, President Donald Trump announced a plan in January to mobilize $500 million in private-sector investment toward AI infrastructure—signaling Washington’s determination to keep America ahead in the AI race.
How China defends its AI approach
Asked about OpenAI’s claims, Liu Pengyu, spokesperson for the Chinese Embassy in Washington, D.C., dismissed them as “groundless attacks.”
Liu emphasized that China is “rapidly building an AI governance system with distinct national characteristics,” balancing innovation and security.
According to his statement, Beijing has introduced new ethical and legal frameworks governing algorithmic services, generative AI, and data security—though critics argue that such rules often entrench state control rather than protect citizens.
The fine line between security and control
Experts say the cases highlighted by OpenAI illustrate the fine line between legitimate national security efforts and invasive surveillance.
While governments worldwide use AI to combat crime or terrorism, authoritarian regimes may apply the same tools to suppress dissent or target minorities.
The Chinese government has repeatedly denied accusations of human rights abuses against Uyghur Muslims in Xinjiang, but U.S. officials—including those in the first Trump administration—have accused Beijing of committing genocide and crimes against humanity.
These revelations about AI misuse could strengthen calls in Washington and Brussels for stricter export controls on advanced chips and AI models, as policymakers grow wary of how Western technology might empower repressive regimes.
The military dimension: AI and cyber warfare
The OpenAI report stops short of confirming whether U.S. intelligence or military agencies use ChatGPT for offensive cyber operations. However, the Pentagon’s U.S. Cyber Command has already published an “AI roadmap” pledging to “accelerate adoption and scale capabilities” in artificial intelligence.
That includes exploring how AI could be used to identify and exploit software vulnerabilities in adversarial networks—essentially, AI-assisted hacking for defense and offense.
This raises an unavoidable question: if democratic nations are adopting AI for cyber operations, can they realistically prevent authoritarian governments from doing the same?
What’s next: Toward responsible AI governance
OpenAI’s findings underscore the urgency of global AI governance frameworks that balance innovation with accountability. Without clear guardrails, powerful models like ChatGPT could become tools of manipulation, surveillance, or warfare.
Experts suggest three immediate steps:
- Transparency in AI usage — Governments and companies must disclose how AI systems are being deployed, especially in surveillance or defense contexts.
- Stricter API monitoring — AI developers should detect and ban state-linked misuse early, as OpenAI did.
- International cooperation — Democratic nations need to coordinate export controls and ethical AI standards to prevent abuse by authoritarian actors.
TL;DR
OpenAI has identified cases where suspected Chinese operatives misused ChatGPT to design surveillance tools targeting minorities and promote censorship systems. The findings highlight how AI, once seen as a symbol of innovation, is increasingly being exploited for control and cyber operations. The U.S. and China’s escalating AI rivalry now extends beyond economics—into the realms of ethics, national security, and human rights.
