The dark face of digital commerce generalization is the increase of fraud attempts. To prevent any type of attacks, state-of-the-art fraud detection systems are now embedding Machine Learning (ML) modules. The conception of such modules is only communicated at the level of research and papers mostly focus on results for isolated benchmark datasets and metrics. But research is only a part of the journey, preceded by the right formulation of the business problem and collection of data, and followed by a practical integration. In this paper, we give a wider vision of the process, on a case study of transfer learning for fraud detection, from business to research, and back to business

Bontempi, Gianluca

Coter, Guillaume

Fabry, Remy

He-Guelton, Liyun

Le Borgne, Yann-Aël

Lebichot, Bertrand

Oble, Frederic

Siblini, Wissam

arXiv

peer reviewedThe dark face of digital commerce generalization is the increase of fraud attempts. To prevent any type of attacks, state-of-the-art fraud detection systems are now embedding Machine Learning (ML) modules. The conception of such modules is only communicated at the level of research and papers mostly focus on results for isolated benchmark datasets and metrics. But research is only a part of the journey, preceded by the right formulation of the business problem and collection of data, and followed by a practical integration. In this paper, we give a wider vision of the process, on a case study of transfer learning for fraud detection, from business to research, and back to business

Open Repository and Bibliography - Luxembourg

Transfer Learning for Credit Card Fraud Detection:A Journey from Research to ProductionWissam Siblini, Guillaume Coter, Rémy FabryLiyun He-Guelton & Frédéric ObléWorldlineBrussels, Belgium & Lyon, FranceEmail: wissam.siblini@worldline.comBertrand Lebichot, Yann-Aël Le Borgne& Gianluca BontempiUniversite Libre de BruxellesBrussels, BelgiumEmail: bertrand.lebichot@ulb.ac.beAbstract—The dark face of digital commerce generalization isthe increase of fraud attempts. To prevent any type of attacks,state-of-the-art fraud detection systems are now embedding Ma-chine Learning (ML) modules. The conception of such modulesis only communicated at the level of research and papers mostlyfocus on results for isolated benchmark datasets and metrics.But research is only a part of the journey, preceded by the rightformulation of the business problem and collection of data, andfollowed by a practical integration. In this paper, we give a widervision of the process, on a case study of transfer learning for frauddetection, from business to research, and back to business.I. INTRODUCTIONFinancial institutions that process payment have to fight atremendous amount of fraudulent activities. In 2018, the levelof card fraud losses amounted to 1.8 billion euros in the SingleEuropean Payment Area [1]. On a global scale, it is expectedto reach more than 34 billion by 2022. Payment processorsalso face indirect repercussions as fraud jeopardizes the trustof customers, namely banks and cardholders. It is thereforeobvious for them to resort to Fraud Detection Systems (FDS).Most FDS has a rule-based basis (e.g. ”raise an alert if thelast transaction from cardholder occurred less than 5 secondsbefore”) [2], designed by experts, which can be specific (lownumber of false positives) but not very sensitive nor flexible.The cat-and-mouse game played by fraudsters makes themquickly obsolete, and their update requires a lot of manualwork. Adding an incremental ML-based system can providecomplementarity (from experience, combining expert rules andML models provides from 20% to 100% better results thaneach approach alone) and adaptability to make an overallpowerful FDS [3], [4]. ML also opens the door to underlyingtechniques like Transfer Learning (TL) [5], which is essentialwhen starting working on a new payment dataset/domain (e.g.new bank) with very few annotations. TL allows adaptingan existing efficient model to use it on the new domain.This provides a competitive business advantage in terms ofcustomer prospects by promising a less perilous start.In this paper, we give an overview of a 3-year projectbetween Worldline and ULB, involving a dozen researchers.We focus on a TL study for FDS, carried out over two yearsfor research and one year for industrialization, with a 6-month overlap. Rather than detailing our research findings(available in [5]), we describe the back-and-forth journeybetween business and production: from the formulation ofpractical business constraints to the research study definitionand conclusions, and finally the transformation into concreteassets for deployment, including our return on experienceabout reproducibility issues and how to anticipate them.II. THE BUSINESS CONSTRAINTS AND CHALLENGESML research for FDS starts with open questions (e.g. howto detect new fraud patterns quickly?). Then, one selectsan appropriate dataset, proposes an algorithm to optimizeadequate performance metrics. But having to implement theresults in production imposes a lot of additional constraints:(1) Data collection must be done with caution (e.g. propertrain-test temporal split with verification gap [3], denoisedannotations, etc.). Such data-centric considerations are nowacknowledged by experts to be key in the overall ML process.[6] gives a non-exhaustive list of those related to the fraudproblem. (2) Metrics: Apart from detection rate, metricslike memory/time consumption are primordial for efficientreal-time fraud detection [7]. Contracts can also impose aminimum level of performance (e.g a precision above 1/3).Finally, interpretable indicators are required for continuousimprovement [8]. (3) Compliance and interoperability: theML pipeline should comply with regulations (e.g. limits indata storage/usage), be integrable in the payment process, andinteract properly with the production platform and services.(4) Relatively to transfer learning, specific constraints canarise like having to homogenize the variables encoding andthe processing pipeline between source and target domains.III. THE RESEARCH PROBLEM AND ITS CONCLUSIONSIn Transfer Learning, we consider domains D (describedby variables X and distributions P (X)) and tasks T (e.g.predict fraud y by modeling P (y|X) with a function f ).Given a source (Ds,Ts) with a rich dataset, TL adapts theassociated solution fs to compute a solution ft for a target(Dt,Tt) where the dataset is poor in annotations. In our project,there are two use-cases of TL: reusing an FDS trained ononline payment (resp. payment in country A), for paymenton physical terminals (resp. in country B). These cases fallinto a category called ”Homogeneous domain adaptation”:same tasks Ts = Tt and close sets of variables, but differentarXiv:2107.09323v2  [cs.LG]  4 Nov 2021distributions since customers (resp. fraudsters) do not have thesame habits (resp. techniques). In our experiments, the source(resp. target) set was made of around 140 (resp. 60) milliontransactions over 183 operating days, with a fraud ratio in theorder of 0.1%. A transaction is described by around 20 features(e.g. amount, country, payment method, merchant, cardholder).Our work consisted in adapting, implementing, and evaluatinga large number of approaches from the literature: standardbase models (NB, RF, NN, Xgboost) with several transferstrategies (model trained on the source or target or both,transfer with domain-invariant representations, etc.). For thesake of conciseness, we refer the reader to [9], [5] for moredetails and results. We concluded that models trained on sourceperform better when the target domain is poorly labeled.When target labels get less scarce, semi-supervised domainadaptation methods take over the reins. Overall, a combinationof a model trained only on source and a model trained onboth target and source performs well in most cases. Thiscombination is under integration into production, as it allows aglobal improvement of detection by around 15% compared toa zero-shot transfer baseline. With thousands of fraud attemptsevery day, this allows significant financial savings.IV. THE INTEGRATION METHODOLOGY: MAKE IT HAPPENFigure 1 shows the steps to industrialize findings. Researchis followed by Pre-industrialization, during which (1) thework is adapted to be integrated into production-compatibletools (libraries, versioning tools) and (2) results are confirmedon larger datasets. Then comes Industrialization, where meth-ods are adapted and deployed to the production environment. Proposals Initial results Toolbox Research Refactored Toolbox Extended results Production code Final model Pre-industrialization Industrialization Fig. 1. A cyclic industrialization process.As the steps are usually carried one after another (withdelays of several months) and have their own requirements,reproducibility issues arise. Some are related to data. Usually,a rather small subset of data is selected for the researchpart. Meanwhile, new data is collected so when the studyis over, one has to check if results still hold. Sometimes,it is necessary to change the data format and integrate newfeatures. Other issues are related to the code. Research andpre-industrialization use different environments (also differentfrom the actual production environment) due (i) to their differ-ent objectives (research can require a flexible environment likeJupyter) and (ii) due to collaboration between different entities(e.g. academics and industrials). As a consequence, libraryversions require adaptation either in the delivered code orin the platform welcoming it. Additionally, a research imple-mentation contain code snippets that are already implementedin production. In this case, the code needs to be refactoredaccordingly. Lastly, versioning tools (Git, DVC, wandb, ...) forcode, data, and results are essential in later steps. To properlyapply them, the code needs adaptations as well.To tackle reproducibility challenges, our main return onexperience is that all steps of the project should not be seenas linear and successive. It is important (1) to anticipate:providing pieces of the production environment as a basis forthe research implementation allows an easier later integration(savings counted in months) and even a cleaner formulationof the problem; and (2) to do several quick rounds with allthe steps instead of a single long one: integrating code soonallows to capture hidden issues and redefining formulationactively to reorientate the study. However, it remains importantto maintain flexibility because when too restricted by businessconstraints, findings can remain limited to incremental inno-vations with moderate added value [10].V. CONCLUSIONResearch and development are important for competitive-ness in the industry. It demonstrates expertise and createsvalue. It is also at the core of the ML revolution. But weusually tend to only communicate on the research part of theprocess. Yet the complete path from emergence to deploymenthides many other challenges. A good practice is to anticipatethem to go further faster. Promising formalized directions startto emerge for instance with MLOps, the set of recipes/tools forML deployment. This however does not mean that businessesshould constrain research. It allows efficient integration ofcourse but can prevent disruptive innovations [10].REFERENCES[1] E. C. Bank, “Sixth report on card fraud,” 2018.[2] Y. Kou, C.-T. Lu, S. Sirwongwattana, and Y.-P. Huang, “Survey of frauddetection techniques,” in IEEE International Conference on Networking,Sensing and Control, 2004, vol. 2. IEEE, 2004, pp. 749–754.[3] A. Dal Pozzolo, O. Caelen, Y.-A. Le Borgne, S. Waterschoot, andG. Bontempi, “Learned lessons in credit card fraud detection froma practitioner perspective,” Expert systems with applications, vol. 41,no. 10, pp. 4915–4928, 2014.[4] B. Lebichot, G. M. Paldino, G. Bontempi, W. Siblini, L. He-Guelton,and F. Oblé, “Incremental learning strategies for credit cards frauddetection,” in 2020 IEEE 7th International Conference on Data Scienceand Advanced Analytics (DSAA). IEEE, 2020, pp. 785–786.[5] B. Lebichot, T. Verhelst, Y.-A. Le Borgne, L. He-Guelton, F. Oblé,and G. Bontempi, “Transfer learning strategies for credit card frauddetection,” IEEE Access, 2021.[6] A. Alazizi, A. Habrard, F. Jacquenet, L. He-Guelton, F. Oblé, andW. Siblini, “Anomaly detection, consider your dataset first an illustrationon fraud detection,” in 2019 IEEE 31st International Conference onTools with Artificial Intelligence (ICTAI). IEEE, 2019, pp. 1351–1355.[7] P. H. Tran, K. P. Tran, T. T. Huong, C. Heuchenne, P. HienTran, andT. M. H. Le, “Real time data-driven approaches for credit card frauddetection,” in Proceedings of the 2018 international conference on e-business and applications, 2018, pp. 6–9.[8] W. Siblini, J. Fréry, L. He-Guelton, F. Oblé, and Y.-Q. Wang, “Masteryour metrics with calibration,” in International Symposium on IntelligentData Analysis. Springer, 2020, pp. 457–469.[9] B. Lebichot, Y.-A. Le Borgne, L. He-Guelton, F. Oblé, and G. Bontempi,“Deep-learning domain adaptation techniques for credit cards frauddetection,” in INNS Big Data and Deep Learning conference. Springer,2019, pp. 78–88.[10] M. Assink, “Inhibitors of disruptive innovation capability: a conceptualmodel,” European journal of innovation management, 2006.

Transfer learning for credit card fraud detection : A journey from research to production.

https://orbilu.uni.lu/bitstream/10993/50240/1/A%20Journey%20from%20Research%20to%20Production.pdf

Transfer learning for credit card fraud detection : A journey from research to production.

Abstract

Similar works

Full text

Available Versions

Open Repository and Bibliography - Luxembourg