中国人工智能安全全景报告(State of AI Safety in China)

Published May 14, 2024 Executive Summary Executive Summary (1) ➢Therelevance and qualityof Chinesetechnical research for frontier AI safetyhasincreasedsubstantially, with growing work on frontier issues such as LLM unlearning, misuse risksof AI in biology and chemistry, and evaluating "power-seeking" and "self-awareness" risks of LLMs. ➢There have been nearly15 Chinese technical papers on frontier AI safety per monthonaverage over the past 6 months.The report identifies 11 key research groups who have written asubstantial portion of these papers. ➢China’s decision to sign theBletchley Declaration, issue a joint statement on AI governance withFrance, and pursue an intergovernmental AI dialogue with theUSindicates agrowingconvergence of views on AI safety among major powerscompared to early 2023.➢Since 2022, 8Track 1.5 or 2 dialoguesfocused on AI have taken place between China andWestern countries, with 2 focused on frontier AI safety and governance. Executive Summary Executive Summary (II) ➢Chinesenational policy and leadershipshow growing interest indeveloping large modelswhile balancing risk prevention.➢Unofficialexpert draftsof China’sforthcoming national AI lawcontainprovisions on AIsafety, such as specialized oversight for foundation models and stipulating value alignment of AGI.➢Local governmentsin China’s 3 biggest AI hubs have issuedpolicies on AGI or large models,primarily aimed at accelerating development while also including provisions on topics such asinternational cooperation, ethics, and testing and evaluation.➢Several influentialindustry associationsestablishedprojects or committees to research AIsafety and security problems, but their focus is primarily on content and data security ratherthan frontier AI safety.➢In recent months,Chinese expertshave discussed several focusedAI safety topics, including“red lines”that AI must not cross to avoid “existential risks,”minimum funding levelsfor AIsafety research, and AI’s impact onbiosecurity. Table of Contents Section 1: Introduction and scopeSection 2:Technical safety researchSection 3: International governanceSection 4: Domestic governanceSection 5: Lab and industry practicesSection 6: Expert views on AI risksSection 7: Public opinion on AISection 8:Additional resourcesSection 9:About us Introduction and Scope Thanks to positive feedback on our first report and rapid AI developments sinceOctober 2023, we have decided to issue an update! ➢The 2023 version was published before the UK AI Safety Summit, and our CEO, Brian Tse, shared itwith other attendees at the summit.➢We provided briefings on the report to over a dozen organizations including the BrookingsInstitution, the Center for Strategic and International Studies, Google DeepMind, the FrontierModel Forum, and the Tony Blair Institute for Global Change.➢Media outlets including Politico and Sixth Tone have covered our report, and it has beenrecommended by leading AI experts, including Jeffrey Ding in his ChinAI newsletter. Introduction and Scope Our report focuses on “frontier AI risks.” ➢We share the focus of the 2023 UK AI Safety Summit, which emphasized risks from cutting-edge largemodels – “highly capable general-purpose AI models, including foundation models, that could perform awide variety of tasks” – as well as narrow AI systems in dangerous domains.1 ■We include both types of models when using the phrase “frontier AI.” Introduction and Scope Our report focuses on AI safety rather than AI security. ➢In English, risks from frontier AI are the subject of the discipline called AI “safety.” In Chinese, the term“人工智能安全” encompasses this definition, while also including AI “security.”2 ■AI “safety” is about protecting against broadly harmful consequences that could result from AIsystems such as accidents and misuse, whereas AI “security” is about preventing AI systems frombeing attacked and compromised. ■AI security includes topics such as cybersecurity of AI model weights, data security of AI models,and physical security of AI development facilities, which we exclude from the scope of the report. ■We exclude lethal autonomous weapons (LAWs) from the scope of this report to focus onnon-military AI risks. ➢In cases of ambiguity, we translate the term “人工智能安全” as “AI safety/security.” ➢Some AI safety topics can also be considered AI security issues and fallwithinour scope, such as: Table of Contents Section 1: Introduction and scopeSection 2:Technical safety researchSection 3: International governanceSection 4: Domestic governanceSection 5: Lab and industry practicesSection 6: Expert views on AI risksSection 7: Public opinion on AISection 8:Additional resourcesSection 9:About us Technical Safety Research Overview of key developments since October 2023 ➢Relevance and quantity of frontier AI safety researchhas risen substantially compared to 2023,with increasing interest in frontier aspects of AI safety. ➢We have identified with high confidence11key safety research g