BISWorkingPapers No1188 Findinganeedleinahaystack:amachinelearningframeworkforanomalydetectioninpaymentsystems byAjitDesai,AnnekeKosseandJacobSharples MonetaryandEconomicDepartment May2024 JELclassification:C45,C55,D83,E42. Keywords:paymentsystems,transactionmonitoring,anomalydetection,machinelearning. BISWorkingPapersarewrittenbymembersoftheMonetaryandEconomicDepartmentoftheBankforInternationalSettlements,andfromtimetotimebyothereconomists,andarepublishedbytheBank.Thepapersareonsubjectsoftopicalinterestandaretechnicalincharacter.TheviewsexpressedinthemarethoseoftheirauthorsandnotnecessarilytheviewsoftheBIS. ThispublicationisavailableontheBISwebsite(www.bis.org). ©BankforInternationalSettlements2024.Allrightsreserved.Briefexcerptsmaybereproducedortranslatedprovidedthesourceisstated. ISSN1020-0959(print) ISSN1682-7678(online) FindingaNeedleinaHaystack:AMachineLearningFrameworkforAnomalyDetectioninPaymentSystems* AjitDesai1,‡,AnnekeKosse2andJacobSharples11BankofCanada 2BankforInternationalSettlements May13,2024 Abstract Weproposeaflexiblemachinelearning(ML)frameworkforreal-timetransactionmonitoringinhigh-valuepaymentsystems(HVPS),whichareacentralpieceofacountry’sfinancialinfras-tructure.Thisframeworkcanbeusedbysystemoperatorsandoverseerstodetectanomaloustransactions,which—ifcausedbyacyberattackoranoperationaloutageandleftundetected—couldhaveseriousimplicationsfortheHVPS,itsparticipantsandthefinancialsystemmorebroadly.GiventhesubstantialvolumeofpaymentssettledeachdayandthescarcityofactualanomaloustransactionsinHVPS,detectinganomaliesresemblesanattempttofindaneedleinahaystack.Therefore,ourframeworkusesalayeredapproach.Inthefirstlayer,asupervisedMLalgorithmisusedtoidentifyandseparate‘typical’paymentsfrom‘unusual’payments.Inthesecondlayer,onlythe‘unusual’paymentsarerunthroughanunsupervisedMLalgorithmforanomalydetection.WetestthisframeworkusingartificiallymanipulatedtransactionsandpaymentsdatafromtheCanadianHVPS.TheMLalgorithmemployedinthefirstlayerachievesadetectionrateof93%,markingasignificantimprovementovercommonly-usedeconometricmodels.Moreover,theMLalgorithmusedinthesecondlayermarkstheartificiallymanipulatedtransactionsasnearlytwiceassuspiciousastheoriginaltransactions,provingitseffectiveness. Keywords:PaymentSystems,TransactionMonitoring,AnomalyDetection,MachineLearning JELCodes:C45,C55,D83,E42 *TheviewsexpressedinthispaperaresolelythoseoftheauthorsanddonotnecessarilyreflectthoseoftheBankofCanada,theBankforInternationalSettlements(BIS),theBISCommitteeonPaymentsandMarketInfrastructures(CPMI),oritsmembers.WewouldliketothankLeonardSabettiforhiscontributionduringearlierstagesofthework.WealsothankSegunBewaji,AlessioBrini,NarayanBulusu,RicardoDeAvillez,LauraFelber,MarcGlowka,TarushGupta,ConstanzaMartinez,andEllenvanderWoerdfortheirdetailedcomments.Inaddition,wethankparticipantsofthefollowingconferencesfortheircommentsandsugges-tions:BISResearchWebinarSeries(2022),EconomicsofPaymentsXIConference(2022),DNBCentralBankersGoDataDrivenConference(2022),CanadianEconomicsAssociationAnnualConference(2022),RBIGlobalConferenceonFinancialResilience(2023),InternationalConferenceonEconomicModelingandDataScience(2023),theBankofCanadaandtheBankforInterna-tionalSettlement’sSeminaronGranularData(2023),theSecondCEMLARegionalConferenceofPaymentsandFinancialMarketInfrastructures(2023),andtheAEAAnnualMeetingPosterSession(2024).‡Correspondingauthor:adesai@bankofcanada.ca. 1Introduction High-valuepaymentsystems(HVPSs),suchasLynxinCanada,FedwireintheUS,ChapsintheUK,andTarget2intheEurozone,arevitalcomponentsofjurisdictions’financialsystems.Typically,thesearereal-timegrosssettlement(RTGS)systemsthatprocesslarge-valuetransactionsbetweenfinancialinstitutions,oftenrequiringsettlementbyaparticulartime.Assuch,thesafetyandefficiencyofHVPSsarekeytofinancialstabilityandeconomicgrowth.Ifnotproperlymanaged,anHVPScanbeasourceofashock,suchaspaymentsfraud,acyberattack,marketstress,oroperationalproblems(Chapmanetal.2015;BIS-Report2019;FED-Report2019;KotidisandSchreft2023).Moreover,asHVPSsprovidealinkbetweentheirparticipatingfinancialinstitutions,theycouldbecomeachannelthroughwhichshocksaretransmittedacrossdomesticoreveninternationalfinancialmarkets(KosseandLu2022;KotidisandSchreft2023). Inparticular,cyberattacksposeagrowingrisktofinancialinstitutionsandHVPSs.1Forinstance,in2016,theCentralBankofBangladesh(CBB)fellvictimtoacyberheist,wherehackersattemptedtostealnearlyonebilliondollarsfromtheCBBreservesaccountattheFederalReserveBankofNewYork(BukthandHuda2017).Similarly,cyberattacksonMexico’sinterbankpaymentnetworkandBancodeChilein2018resultedinlossesamountingtomillionsofdollars(NishandNaumaan2019).2Recently,KotidisandSchreft(2023)hasdocumentedtheeffectsofacyberattackonaserviceprovidertothebanksparticipatinginFedwireandhighlightedtheimportanceofoperationalresilience.Moreover,simulation-basedstudieshaveshownthatcyberattacks,evenwhentargetingindividualHVPSparticipants,canhaveasignificantimpactonthesysteminwhichtheyparticipate(Eisenbachetal.2021;KosseandLu202