教育生成式人工智能开发技术报告英

goo.gle/LearnLM2024-05-14 TowardsResponsibleDevelopmentofGenerativeAIforEducation: AnEvaluation-DrivenApproach IrinaJurenka*,‡,1,MarkusKunesch*,†,1,KevinMcKee§,1,DanielGillick§,1,ShaojianZhu†,1,SaraWiltberger§,1,ShubhamMilindPhal1,KatherineHermann1,DanielKasenberg§,1,AvishkarBhoopchand1,AnkitAnand1,MirunaPîslar1,StephanieChan§,1,LisaWang§,1,JenniferShe1,ParsaMahmoudieh1,AliyaRysbek1,Wei-JenKo3,AndreaHuber1,BrettWiltshire1,GalElidan‡,2,RoniRabin2,JasminRubinovitz†,4,AmitPitaru4,MacMcAllister3,JuliaWilkowski3,DavidChoi8,RoeeEngelberg2,LidanHackmon2,AdvaLevin2,RachelGriffin5,MichaelSears5,FilipBar6,MiaMesar3,ManaJabbour3,ArslanChaudhry1,JamesCohan3,SridharThiagarajan1,NirLevine1,BenBrown1,DilanGorur§,1,SvetlanaGrant1,RachelHashimoshoni3,LauraWeidinger1,JieruHu1,DawnChen3,KubaDolecki3,CanferAkbulut1,MaxwellBileschi1,LauraCulp1, Wen-XinDong3,NahemaMarchal1,KelsieVanDeman4,HemaBajajMisra3,MichaelDuah5,MoranAmbar2,AviCaciularu2,SandraLefdal1,ChrisSummerfield7,JamesAn1,Pierre-AlexandreKamienny1,AbhinitMohdi3,TheofilosStrinopoulous3,AnnieHale5,WayneAnderson5,LuisC.Cobo1,NivEfron†,2,MukthaAnanda3,ShakirMohamed1,MaureenHeymans3,ZoubinGhahramani1,YossiMatias2,BenGomes3andLilaIbrahim1 *Equalcontributions,†Technicallead,‡Researchlead,§Workstreamlead,1GoogleDeepMind,2GoogleResearch,3Google,4Google CreativeLab,5ArizonaStateUniversity,6LundUniversity,7UniversityofOxford,8Anthropic,workcarriedoutwhileemployedatGoogleDeepMind Amajorchallengefacingtheworldistheprovisionofequitableanduniversalaccesstoqualityeducation.RecentadvancesingenerativeAI(genAI)havecreatedexcitementaboutthepotentialofnewtechnologiestoofferapersonaltutorforeverylearnerandateachingassistantforeveryteacher.Thefullextentofthisdream,however,hasnotyetmaterialised.WearguethatthisisprimarilyduetothedifficultieswithverbalisingpedagogicalintuitionsintogenAIpromptsandthelackofgoodevaluationpractices,reinforcedbythechallengesindefiningexcellentpedagogy.Herewepresentourworkcollaboratingwithlearnersandeducatorstotranslatehighlevelprinciplesfromlearningscienceintoapragmaticsetofsevendiverseeducationalbenchmarks,spanningquantitative,qualitative,automaticandhumanevaluations;andtodevelopanewsetoffine-tuningdatasetstoimprovethepedagogicalcapabilitiesofGemini,introducingLearnLM-Tutor.OurevaluationsshowthatLearnLM-TutorisconsistentlypreferredoveraprompttunedGeminibyeducatorsandlearnersonanumberofpedagogicaldimensions.Wehopethatthisworkcanserveasafirststeptowardsdevelopingacomprehensiveeducationalevaluationframework,andthatthiscanenablerapidprogresswithintheAIandEdTechcommunitiestowardsmaximisingthepositiveimpactofgenAIineducation. 1.Introduction Theroughly70yearhistoryofArtificialIntelligence(AI)hasbeenoneofparadigmshifts:fromsymbolicsystems,toBayesianapproaches,todeeplearning,andinthelastfewyears,generativeAI(genAI)—largefoundationalmodelstrainedonhugeswathsofmediaavailableontheinternettogainanimpressivesetofgeneralcapabilities,wherebytheyare(mostofthetime)abletoprovideausefulresponsetoanyuserpromptorenquiry.Eachparadigmshiftbroughtwithitauniquesetofhopes,opportunities,andchallenges.YetthecurrentgenAIeraisunprecedented:AIismoreaccessiblethanever(becauseitonlyrequirespromptingthroughnaturallanguage),morecapablethanever,andappearstobeimprovingfasterthanever.Questionsnaturallyariseabouthowtoharnessthistechnologyformaximalsocialbenefit. Correspondingauthor(s):learnlm-tech-report@google.com ©2024GoogleDeepMind.Allrightsreserved humanevaluations automaticevaluations participation model improvements deployment LearnLM-TuĞorDevelopment Evaluation:Teacherpreferences WhatdoesthiserrormeanMatchingChar.java:14:error:intcannotbedereferenced Learner LearnLM-Tutor Itlookslikeyou'retryingtocomparetwocharactersinastringusingtheif statement.However,you'reusingthe.|charAt()methodincorrectly.The .|charAt()methodisusedtoaccessacharacterataspecificindexinastring.Itreturnsacharacter,notaninteger. Therefore,theexpression(idx1.charAt((idx2))isinvalidbecause.|idx1isaninteger,notastring.Tofixthis,youcanchangetheexpressionto .|(userText.charAt((idx1)).ThiswillallowyoutocomparethecharactersatthespecifiedindicesintheuserTextstring. Doesthismakesense? if((idx1.charAt(idx2))&&(userText.charAt(idx2)))[System.out.printLn("Matchfound"); ] eLse[ System.out.printLn("Matchnotfound"); ] Learner AftersometrialanderrorIfinallygotthefollowingcodetoworkwithyourtip.Thatmadeabigdifferenceinmyudnerstanding- LearnLM-Tutor Great!Canyouexplaininyourownwordswhatthe.|charAt()methoddoes? if((userText.charAt(idx1))==(userText.charAt(idx2)))[System.out.printLn("Matchfound"); ] eLse[ System.out.printLn("Matchnotfound"); ] Learner Learner Deployment:ASUStudyHall Couldyoucopyyourcodehere?That'llhelpmedebugtheproblem. LearnLM-Tutor “ Participation:Learnerfeedback Iwoulddescribeitasahelpfulfriendthatknowsalotaboutonesubjectthatcanhelpyoulearntheclass. LearnLM-TutorStudyHallUser Learner LearnLM-Tutor thecharacteratmethodislookingforacharacterinaspecificlocation butitcannotdothatforintegers,onlyforstrings That'sexactlyrig