International Journal of Science, Engineering and TechnologyAuthors: Abdullahi Mohammed Ibrahim, Naja’atu Basiru Sanusi
Abstract: The rapid growth of digital ecosystems has intensified challenges in authentication and fraud detection, as traditional mechanisms such as static passwords, rule-based systems, and CAPTCHA are increasingly inadequate against sophisticated threats like credential stuffing, phishing, and AI-driven deepfakes. This study presents a systematic literature review (SLR) of machine learning (ML) applications in authentication and fraud detection, synthesizing advances across four key domains: biometric authentication, financial fraud detection, password security, and behavioral biometrics. Following the PRISMA framework, the review encompassed publications from 2013–2025 across SpringerLink, ScienceDirect, Scopus, Web of Science, and Google Scholar. From an initial set of 1,520 studies, 146 peer-reviewed articles were selected through rigorous screening and quality assessment. Data were extracted and coded into thematic categories, enabling both quantitative and qualitative synthesis. The findings reveal that deep learning models such as CNNs, RNNs, and Transformers dominate biometric authentication, significantly improving accuracy in face, voice, and keystroke recognition, though challenges persist with spoofing and privacy. In financial fraud detection, ensemble methods (e.g., XGBoost, LightGBM) and hybrid ML–rule-based systems offer robust performance, while graph neural networks are emerging as powerful tools for detecting fraud rings. Password security research has advanced through ML-based strength estimation and generative models, though ethical risks remain. Behavioral biometrics increasingly leverage multimodal fusion and sequential deep learning, enabling continuous authentication but raising privacy concerns. Across domains, systemic challenges include adversarial attacks on ML models, extreme class imbalance in fraud datasets, limited data sharing due to privacy constraints, lack of standardized benchmarks, and the pressing need for explainable AI (XAI) to meet regulatory and trust requirements. Emerging directions emphasize federated learning, adversarial robustness, hybrid AI frameworks combining symbolic reasoning with ML, and user-centric privacy-preserving approaches. This review concludes that while ML has redefined authentication and fraud detection with adaptive, intelligent, and scalable solutions, its sustainable deployment requires addressing these systemic barriers to ensure future digital ecosystems are not only secure but also transparent, privacy-preserving, and user-friendly.
DOI: http://doi.org/10.5281/zenodo.17374548
