2024测试生成式人工智能：降低风险和最大化机会报告（英）

EBOOK TESTINGGENERATIVEAI:MITIGATINGRISKSANDMAXIMIZINGOPPORTUNITIES UNDSEERCSTAIONNDING 1 GETNITELREAHTIEVREEAI TheBasicsofGenerativeAI Anoveltechnologythatcreatescontentbasedonuserrequestsknownasprompts,GenerativeAI(GenAI)applicationscanproducenewtext,images,videoandaudiobysynthesizing,summarizing,orgeneratingcontent.WhileusersinteractwithGenAImuchliketheydoasearchengine,thewaythetwotechnologiesgenerateresponsesisverydifferent. Whereassearchenginesretrieveinformationintheformatinwhichitisstored,GenAIanalyzeshugeamountsofhuman-generateddata,knownastrainingdata,tolearnhowtorespondtouserrequestsinawaythatprovidesvalue.Theanalysisiscarriedoutbyalargelanguagemodel(LLM),atypeofneuralnetwork.Overtime,humansmustcontinue toprovideLLMswithfreshtrainingdatatofine-tunethemodelandkeeptheinformationitlearnsfromuptodate. GenAIcanautomateandenhanceanynumberofbusinesstasks,rangingfromdraftingemailsandgeneratingfinancialreportstocreatingmarketingcontentandanalyzingcustomerinteractions—andusecasesareexpandingdaily.Today,thetechnologyislargelyusedtoimproveproductivityandpersonalizeuserexperiences. PopularGenAIapplicationsincludeChatGPT(OpenAI),Gemini(Google)andCopilot(Microsoft).Whileeachapplicationcurrentlymainlyofferstext-basedinputs,voiceinteractionisontherise.Midjourney,StableDiffusionandDALL-Earepopularapplicationsintheimagegenerationcategory. TestingGenerativeAI:MitigatingRisksandMaximizingOpportunities Keytermsexplained Largelanguagemodels(LLMs) AIfoundationmodels,includinglargelanguagemodels(LLMs)anddiffusionmodels,powerthevastmajorityofGenAIapplications.Theseinnovationsrepresentarevolutionarydevelopmentinthehistoryofhumanitybecausetheyautomatetasksthathavehistoricallyreliedentirelyonhumaneffort,suchastextgeneration,summarizationandanalysis.JustastheIndustrialRevolutionautomatedlabor,thegenerativeAIrevolutionisautomatingintelligence.Infact,itislikelythatthecurrenterawillbereferredtoastheintelligencerevolutionbyfuturegenerations. LLMsarewidelyusedinapplicationsthatinterpret,transformorgeneratetext.However,theyarealsoemployedinusecasesthatrequireimage,audio,andcontentgeneration.Inaddition,AImodelsthatgeneratevisualandaudiocontentoftenemployLLMstointerprettheuser’sinputrequestandaresometimesusedtoevaluatepromptinputsoroutputstorespondtouserquestions. ItisworthclarifyingthatanLLMisnotanapplication.Itisatechnologythatsupportsanapplication.Forexample,ChatGPTisanapplicationpoweredbyamodelcalledGPT-4o.Claude,atext-basedGenAIapplication,ispoweredbytheLLMsHaiku,SonnetandOpus.MetaAI,whichformspartoftheFacebook,InstagramandWhatsAppapplications,isbasedontheLLMLama3. MultimodalLLMs MultimodalisatermemployedtodescribeLLMsthatcanprocessinformationprovidedbyusersinmultiplecontent‘modes,’suchastext,images,diagramsandvideos.Theycanalsogeneratenewcontentinthesemodes.Theleadingmultimodalmodelscananalyzeandgeneratevariouscontentmodesaspartofasingleprompt. HowGenerativeAIDiffersfromTraditionalAI GenerativeAIisoftenconfusedwithtwootherAItechnologies:traditionalmachinelearningandnaturallanguageprocessing. »Machinelearning(ML)istypicallyemployedtoanalyzelargequantitiesofdataandidentifypatterns,anomaliesorhiddeninsights.WhileGenAIalsoemploysML,itismainlyusedtogenerateortransformcontent,ratherthananalyzeit.UnliketraditionalML,whichoftenfunctionsasynchronouslyfordataanalysis,GenAItypicallyoperatesasaruntimetechnology. »Naturallanguageprocessing(NLP)ismostcommonlyusedinvoiceassistantsandchatbots.Itisusedtointerprethumanspeechortext(suchaswhenauserverballyasksavoiceassistantlikeAmazonAlexaaquestion)andthenmatchtheirquestionwithapredefinedanswer.WhileNLPcanprocessunboundeduserinput,itcanonlyprovidepre-determinedoutputs.ThismakesitlessversatilethanGenAIforrespondingtoinformation-relateduserintents. TestingGenerativeAI:MitigatingRisksandMaximizingOpportunities AdamCheyer,thecreatorofthetechnologybehindApple’sSiri,saidinapodcastthatNLP-basedsolutionslikeSiriwereoptimizedas‘doing’assistants,likeplayingmusicoropeningauser’scalendar.Bycontrast,manygenerativeAIapplicationsare‘knowing’assistants. Theyexcelatinformation-basedqueriesandcreativerequestsbutarenotasadvancedinconsistentlyexecutingproceduralrequests. ExamplesofNLP,MachineLearningandGenAI NaturalLearningProcessing »Alexa »GoogleAssistant »Siri »BankofAmerica’sErica »Mostwebsitechatbots TraditionalMachineLearning »Proprietarybusinessanalytics »Forecastingmodels »Nottypicallyusedbyconsumersbutsometimesexposedthroughalgorithms GenerativeAI »ChatGPT »MicrosoftCopilot »GoogleGemini »MetaAI »Midjourney »StableDiffusion SEC2TIONGENERTAITTLIVEEHAEIRTERENDS SeveralcurrenttrendscharacterizeGenAIadoptiontodayandwhereit'sheaded.Theyareexploredbelow. Threeusecasesarebyfarthemostpopular TounderstandGenAI,youneedtoconsideritsapplicationforconsumersandenterprisesalike.Consumersareinterestedinusingitforentertainmentpurposesandforassistancewithday-to-daytasks,whileenterprisesarefocusedonleveragingittobooststaff productivity.Allotherusecasescomeadistantsecondintermsofpriority. Text-driveninputsandoutputsarel