2024年AI安全报告

ZscalerThreatLabz2024AISecurityReport TheAIrevolutionhasarrived.Discoverkeytrends,risks,andbestpracticesinenterprise AIadoption,withinsightsintoAI-driventhreatsandkeystrategiestodefendagainstthem. ©2024Zscaler,Inc.Allrightsreserved. 03ExecutiveSummary 04KeyFindings 05KeyGenAIandMLUsageTrends 05AItransactionscontinuetoaccelerate 06EnterprisesareblockingmoreAItransactionsthanever 07IndustryAIbreakdown 09HealthcareandAI 10Finance 11Government 12Manufacturing 13EducationandAI 14ChatGPTusagetrends 15AIusagebycountry Regionalbreakdown:EMEARegionalbreakdown:APAC 18EnterpriseAIRiskandReal-WorldThreatScenarios 18EnablingAIintheenterprise:top3risks 20AI-driventhreatscenarios AIimpersonation:deepfakes,misinformation,andmore 21AI-generatedphishingcampaigns Fromquerytocrime:creatingaphishingloginpageusingChatGPT 22Darkchatbots:uncoveringWormGPTandFraudGPTonthedarkweb 23AI-drivenmalwareandransomwareacrosstheattackchain 24AIwormattacksand“viral”AIjailbreaking 25AIandUSelections 26AllEyesonAIRegulations 26UnitedStates 27EuropeanUnion 28AIThreatPredictions 31CaseStudy:HowtoSecurelyEnableChatGPTintheEnterprise 315StepstointegrateandsecuregenerativeAItools 33HowZscalerDeliversAI+ZeroTrustandSecuresGenerativeAI 33ThekeytoAI-drivencybersecurity:high-qualitydataatscale 34LeveragingAIacrosstheattackchain 35SummaryofZscaler’sAI-infusedofferings 36EnablingtheenterpriseAItransition:thecontrolisinyourhands 37Appendix 37ThreatLabzresearchmethodology 37AboutZscalerThreatLabz AIismorethanapioneeringinnovation—it’snowbusinessasusual.AsgenerativeAItoolslikeChatGPTtransformbusinessinlargeandsmallways,AIisbeingwovendeepintothefabricofenterpriselife.However,questionsabouthowtosecurelyadopttheseAItoolswhiledefendingagainstAI-driventhreatsarenotsettled. EnterprisesarerapidlyadoptingAIandMLtoolsacrossdepartmentslikeengineering,ITmarketing,finance,customersuccess,andmore.Yet,theymustbalancethenumerousrisksthatcomewithAItoolstoreaptheirfullestrewards.Indeed,tounlockthetransformativepotentialofAI,enterprisesmustenablesecurecontrolstoprotecttheirdata,preventtheleakageofsensitiveinformation,mitigate‘ShadowAI’sprawl,andensurethequalityofAIdata. TheseAIriskstoenterprisesarebidirectional:outsideenterprisewalls,AIhasbecomeadrivingforceforcyberthreats.Indeed,AItoolsareallowingcybercriminalsandnationstate-sponsoredthreatactorstolaunchsophisticatedattacks,morequickly,andatgreaterscale.Despitethis,AIholdspromiseasakeypieceofthecyberdefensepuzzleasenterprisesgrapplewithadynamicthreatlandscape. TheThreatLabz2024AISecurityReportofferskeyinsightsintothesecriticalAIchallengesandopportunities. Drawingonmorethan18billiontransactionsfromApril2023toJanuary2024acrosstheZscalerZeroTrustExchange™,ThreatLabzanalyzedhowenterprisesareusingAIandMLtoolstoday.TheseinsightsrevealkeytrendsacrossbusinesssectorsandgeographiesinhowenterprisesareadaptingtotheshiftingAIlandscapeandsecuringtheirAItools. Throughout,you’llfindinsightsintotop-of-mindAItopicsincludingbusinessrisk,AI-driventhreatscenariosandadversarytactics,regulatoryconsiderations,andpredictionsfortheAIlandscapein2024andbeyond. Justascritically,thisreportoffersbestpracticesontwofronts:howenterprisescansecurelyembracegenerativeAItransformationwhileprotectingcriticaldata,andhowAI-poweredtoolsareworkingtodeliverlayered,zerotrustsecuritytofacethenewlandscapeofAI-driventhreats. AI/MLtoolusageskyrocketedby594.82%,risingfrom521millionAI/ML-driventransactionsinApril2023to3.1billionmonthlybyJanuary2024. Enterprisesareblocking18.5%ofallAI/MLtransactions—a577%increaseinblockedtransactionsoverninemonths—reflectinggrowingconcernsaroundAIdatasecurityandcompanies’reluctancetoestablishAIpolicies. ManufacturinggeneratesthemostAItrafficwith20.9%ofallAI/MLtransactionsintheZscalercloud,followedbyFinanceandInsurance(19.9%)andServices(16.8%). ChatGPTusagecontinuestosoar,with634.1%growth,eventhoughitisalsothemost-blockedAIapplicationbyenterprises,basedonZscalercloudinsights. ThemostwidelyusedAIapplicationsbytransactionvolumeareChatGPT,Drift,OpenAI*,Writer,andLivePerson.ThetopthreeblockedapplicationsbytransactionvolumeareChatGPT,OpenAI,andFraud.net. Thetop5countriesgeneratingthemostAIandMLtransactionsaretheUS,India,theUK,Australia,andJapan. EnterprisesaresendingsignificantvolumesofdatatoAItools,withatotalof569TBexchangedbetweenAI/MLapplicationsbetweenSeptember2023andJanuary2024. AIisempoweringthreatactorsinunprecedentedways,includingforAI-drivenphishingcampaigns,deepfakesandsocialengineeringattacks,polymorphicransomware,enterpriseattacksurfacediscovery,automatedexploitgeneration,andmore. NOTE:TheZscalerZeroTrustExchangetracksChatGPTtransactionsindependentlyfromotherOpenAItransactionsatlarge. AIandMLTransactionTrends 4000M 3000M 2000M 1000M 0M May JuL Sep Month Nov Jan FIGURE1AItransactionsfromApril2023toJanuary2024 TopAIApplications ChatGPTDriftOpenAI Writer 52.23% 18.51% 7.82% 3.86% FIGURE2TopAIapplicationsbytransactionvolume ZSCALERTHREATLABZREPORT2024 05 Transactions UsageTrends TheenterpriseAIrevolutionisfarfromits