教育生成式人工智能开发技术报告英

Irina Jurenka*,‡,1, Markus Kunesch*,†,1, Kevin McKee§,1, Daniel Gillick§,1, Shaojian Zhu†,1, Sara Wiltberger§,1,Shubham Milind Phal1, Katherine Hermann1, Daniel Kasenberg§,1, Avishkar Bhoopchand1, Ankit Anand1,Miruna Pîslar1, Stephanie Chan§,1, Lisa Wang§,1, Jennifer She1, Parsa Mahmoudieh1, Aliya Rysbek1, Wei-JenKo3, Andrea Huber1, Brett Wiltshire1, Gal Elidan‡,2, Roni Rabin2, Jasmin Rubinovitz†,4, Amit Pitaru4, MacMcAllister3, Julia Wilkowski3, David Choi8, Roee Engelberg2, Lidan Hackmon2, Adva Levin2, Rachel Griffin5,Michael Sears5, Filip Bar6, Mia Mesar3, Mana Jabbour3, Arslan Chaudhry1, James Cohan3, SridharThiagarajan1, Nir Levine1, Ben Brown1, Dilan Gorur§,1, Svetlana Grant1, Rachel Hashimoshoni3, LauraWeidinger1, Jieru Hu1, Dawn Chen3, Kuba Dolecki3, Canfer Akbulut1, Maxwell Bileschi1, Laura Culp1,Wen-Xin Dong3, Nahema Marchal1, Kelsie Van Deman4, Hema Bajaj Misra3, Michael Duah5, Moran Ambar2,Avi Caciularu2, Sandra Lefdal1, Chris Summerfield7, James An1, Pierre-Alexandre Kamienny1, Abhinit Mohdi3,Theofilos Strinopoulous3, Annie Hale5, Wayne Anderson5, Luis C. Cobo1, Niv Efron†,2, Muktha Ananda3,Shakir Mohamed1, Maureen Heymans3, Zoubin Ghahramani1, Yossi Matias2, Ben Gomes3and Lila Ibrahim1*Equal contributions,†Technical lead,‡Research lead,§Workstream lead,1Google DeepMind,2Google Research,3Google,4GoogleCreative Lab,5Arizona State University,6Lund University,7University of Oxford,8Anthropic, work carried out while employed atGoogle DeepMind A major challenge facing the world is the provision of equitable and universal access to quality education.Recent advances in generative AI (gen AI) have created excitement about the potential of new technologiesto offer a personal tutor for every learner and a teaching assistant for every teacher. The full extentof this dream, however, has not yet materialised. We argue that this is primarily due to the difficultieswith verbalising pedagogical intuitions into gen AI prompts and the lack of good evaluation practices,reinforced by the challenges in defining excellent pedagogy. Here we present our work collaboratingwith learners and educators to translate high level principles from learning science into a pragmaticset of seven diverse educational benchmarks, spanning quantitative, qualitative, automatic and humanevaluations; and to develop a new set of fine-tuning datasets to improve the pedagogical capabilities ofGemini, introducingLearnLM-Tutor. Our evaluations show thatLearnLM-Tutoris consistently preferredover a prompt tuned Gemini by educators and learners on a number of pedagogical dimensions. Wehope that this work can serve as a first step towards developing a comprehensive educational evaluationframework, and that this can enable rapid progress within the AI and EdTech communities towardsmaximising the positive impact of gen AI in education. 1. Introduction The roughly 70 year history of Artificial Intelligence (AI) has been one of paradigm shifts: fromsymbolic systems, to Bayesian approaches, to deep learning, and in the last few years, generative AI(gen AI)—large foundational models trained on huge swaths of media available on the internet togain an impressive set of general capabilities, whereby they are (most of the time) able to providea useful response to any user prompt or enquiry. Each paradigm shift brought with it a unique setof hopes, opportunities, and challenges. Yet the current gen AI era is unprecedented: AI is moreaccessible than ever (because it only requires prompting through natural language), more capablethan ever, and appears to be improving faster than ever. Questions naturally arise about how toharness this technology for maximal social benefit. One of the key challenges facing the world is the lack of universal and equitable access toquality education [2]. Education is a key economic driver [3] and a facilitator of upward social mobility [4]; however, even before the COVID-19 pandemic, 53% of all ten-year-old children in low-to middle-income countries were experiencing learning poverty [5], and 40% of US school districtleads described their teacher shortages as “severe” or “very severe” [6]. The long-standing problemswith educational attainment and teacher retention have been further exacerbated by the pandemic,disproportionately affecting those from less privileged backgrounds [5, 6]. The rise in gen AI that followed the pandemic has been met with mixed reactions. On the one hand,it appears to hold some promise to democratise access to knowledge and education: students are earlyadopters and top users of the technology [7], and gen AI is dominating the EdTech landscape [8]. Onthe other hand, several concerns have been raised about the misuse of this technology in educationalsettings [7,9]. For example, the gen AI models that power most of the latest EdTech systems arenot explicitly optimised for pedagogy. Instead, models are trained to be “helpful” [10–14], but thisspecific definition of helpfulness may often be at odds with