by James Hurren
It’s no longer enough for an artificial intelligence (AI) model to perform well. Regulators, customers, and compliance ask not just “does it work?” but “how does it work?”
Explainable AI (XAI) is becoming essential to competitiveness, customer trust, and compliance. The proliferation of AI among financial services firms has only furthered the case for explainability. AI-powered fraud detection is now mainstream, with vendors selling machine learning (ML) fraud detection systems and established players building their own in-house models.
Alex Kelly, VP of product at Tribe, is conscious of the downsides—both material and reputational—when AI fraud risk models get it wrong: “False positives can have real consequences, including financial exclusion, reputational harm, and loss of trust; all of which disproportionately impact certain customer groups.”
This concern is widely shared, but understanding how these systems work also matters beyond avoiding losses. Explainability can be a competitive advantage. Chris Elliott, director of data governance at ComplyAdvantage, argues that this shift is as much about business performance as regulation: “Compliance becomes a free bonus if you can embed the notion that strong model risk management equates to product excellence. Responsibly developed AI drives better customer outcomes.”
Under pressure from regulators, users, and competition, AI models must now be not only accurate but explainable.
Mastercard, DBS, and JP Morgan Chase have all deployed AI fraud risk monitoring, with positive early results. They report both higher detection accuracy and fewer false positives; typically, an improvement in one of these factors leads to deterioration in the other. Public perception data adds another dimension.
An EY survey of over 1,000 people found 63% were comfortable with AI in fraud protection and detection—the highest approval of any use case tested. Conversely, just 31% were comfortable with it being used for claim evaluation and the automation of decisions, such as insurance and fraud.
The two use cases differ, but a 30 percentage-point gap in support is striking. Public understanding of AI partly explains the gap, but it also highlights branding: people support ‘fraud protection’, not ‘decision automation’.
Generative AI (GenAI) is more widely known—models that produce new content, such as text or code, from large datasets. Famous models like ChatGPT and Gemini fall into this category, but their infrastructure differs significantly from the AI used in fraud detection.
Compared with the vast, general-purpose large language models (LLMs) used for natural-language tasks, fraud-detection models are smaller, more specialised, and built for speed. LLMs are trained on trillions of words of text, require enormous GPU clusters (i.e. vastly greater compute), and thousands of terabytes of storage, and are updated relatively infrequently.
Fraud-detection models, by contrast, are trained on structured transaction data— amounts, merchants, device IDs, and timestamps—and are often retrained daily or weekly to catch emerging scams.
They utilise lighter architectures, such as gradient-boosted trees or compact neural networks, which run in real-time payment systems with millisecond-level latency. These models are designed to be explainable for regulatory compliance purposes.
In short, AI is a broad field, and advances in GenAI do not automatically translate to fraud detection models.
When a company says it’s “using AI”, it could mean almost anything.
Regulators increasingly expect companies to provide explanations for a rating or decision that affects one of their customers.
The EU’s AI Act classifies “high-risk” AI systems, such as those used in credit scoring, hiring, and healthcare, as requiring transparency, with clear information on how they work, their limitations, and capabilities.
In May, the UK announced similar legislation for payment service providers (PSPs) on consumer rights and debanking. If passed, it would extend the notice period for account closures from 60 to 90 days, require PSPs to give specific reasons beyond “commercial decision” or “risk appetite”, and oblige them to inform customers of their right to complain to the Financial Ombudsman Service (FOS).
False positives can have real consequences, including financial exclusion, reputational harm, and loss of trust; all of which disproportionately impact certain customer groups.
Regulatory developments such as these mean demand for XAI has risen. Sam Gilbert, sales account manager at Mobius Networks, says “clients want to know not just whether the AI works, but how it works,” especially in payments, where networks like Mastercard and Visa are paying “close attention.”
Shubnem Marcout, chief innovation partner at REIopay, warns that opacity can erode user trust and, crucially, “automation doesn’t absolve responsibility.”
The case for explainable AI now extends beyond compliance or ethics: it directly affects revenue, operating costs, and competitive positioning.
Operationally, more transparent models can reduce the burden on investigation teams. Clearer alerts let teams focus on the riskiest transactions, cutting manual reviews and speeding resolution. Iain Armstrong, FCC strategy executive director at ComplyAdvantage, notes that “smart” alerts, informed can help teams “avoid the blanket application issues of more straightforward rulebased systems.” The result is greater efficiency without compromising coverage.
Customer retention is another consideration. False positives can drive attrition, especially when account freezes or payment declines are not clearly explained. With XAI, firms can provide reason codes and supporting evidence in plain language, helping to preserve relationships. Such reports can be embedded into the models themselves, as advocated for by Sara West, commercial director at ID-PAL. She advises these reports can consolidate the evidence used in automated checks, such as document data, facial match, and liveness test results, and address verification outcomes.
From a commercial perspective, explainability can be a differentiator in merchant acquisition and corporate client onboarding. In request for proposal (RFP) processes, PSPs and acquirers that can demonstrate auditable, biasmitigated decisioning gain an advantage, particularly in sectors with tighter regulation or with compliance-sensitive merchants. There is also a costbenefit case: while XAI requires upfront investment in models, governance, or external providers, the savings can be significant. Avoiding enforcement penalties, reducing disputehandling costs, and lowering reputational risk can outweigh these initial costs over the model’s lifecycle.
There is a tension between powerful AI models and explainability. The efficacy of these AI models stems from the vast quantities of data they are trained on and the multilayered statistical reasoning they apply to produce their results. Inevitably, the results of this process are not given in short, simple answers.
One solution is building explainability into the output, notes Gilbert: “What’s helped in keeping the output simple is things like scorecards that make it clear what’s going on without needing a data science degree.” He also suggests leaving AI to do the “heavy lifting” while teams use their judgment to interpret results.
Armstrong adds that segment-based monitoring and “smart” alerts, which learn from user actions via ML, help focus on the highest-risk cases without drowning in false positives.
Kelly points to the use of SHAP (shapley additive explanations) and LIME (local interpretable model-agnostic explanations), alongside behavioural biometrics and anomaly detection, to provide tangible, evidence-based justifications for flagged activity. “When combined with transactional data like merchant type or geographic location, fraud systems can build more credible, context-aware views of risk. The challenge remains in translating these findings into language that regulators and customers can understand and trust,” notes Kelly.
To be both accurate and explainable, AI fraud models require robust governance frameworks, high-quality data, and human oversight.
Explainable, auditable AI fraud systems need a formal structure to guide design, deployment, and monitoring. That structure should come from widely accepted governance frameworks. For example:
Elliott cites ISO42001 and the British Standards Institution’s new standard for AI audits as important steps toward consistent oversight. Importantly, these frameworks should not be considered after the fact but embedded from the start: “If you bake these principles into normal business processes, you end up with responsible AI by design, rather than retrofitted compliance questionnaires,” notes Elliott.
Kelly agrees, stressing AI should fit within fraud logic that is controllable, understandable, and auditable, giving teams clear operational boundaries as risks evolve.
West says, “One of the most effective safeguards for minimising false positives is ensuring high-quality, reliable input data.” Elliott and Kelly echo this point.
Additionally, West warns of risks when systems fail to distinguish impersonation victims from first-party fraudsters. She cites UK fraud prevention body Cifas as a good example of an organisation making this distinction, contrasting it with weaker US practices. Kelly agrees that poor data quality increases the risk of bias and overreach, undermining trust in AI decisions.
So what counts as good data for AI fraud models? Four foundations stand out:
Human-in-the-loop (HITL) AI integrates human expertise into ML to improve accuracy, reliability, and ethical alignment. This can be done throughout the training process or following implementation, for example, data annotation, model evaluation, and complex decision-making.
HITL systems benefit both model accuracy and explainability. Humans can translate technical outputs—like risk scores, anomaly flags, or SHAP value plots—into business and compliance terms.
The outputs from these algorithms can themselves be counterintuitive. In some cases, this is the system functioning as designed, identifying fraud that traditional risk models would have missed. In others, they are simply hallucinating. Human review helps separate genuine findings from errors and provides clear explanations. These human reviews add an extra layer of documented decision-making, which is valuable for regulatory audits. Marcout advises regulator audits, particularly where false positives appear concentrated across demographics like expats, digital nomads, or individuals with limited local credit footprints.
Smart alerts, informed by past investigator decisions, can help teams avoid the blanket application issues of more straightforward rule-based systems.
“In high-friction moments, trust hinges on responsiveness, not opacity”, observes Marcout. Human reviewers can ensure that decisions, especially those with severe consequences like account freezes, are backed by a defensible explanation before they’re actioned.
standardisation Across payments, there is growing recognition that fragmented regulation of XAI increases compliance costs and operational friction, particularly for cross-border providers.
Elliott states, “Regulators and industry sharing best practices to establish common standards is happening to an extent, but I would love to see more of it.” Without alignment, firms must juggle divergent rules such as the EU AI Act, the UK’s proposed consumer rights reforms, and the Monetary Authority of Singapore’s FEAT principles.
One potential area for collaboration is the creation of shared, anonymised datasets for bias testing. These could help smaller PSPs and fintechs validate the representativeness of their training data without breaching confidentiality or data protection laws. Independent bodies or trade associations could manage such datasets, ensuring quality and fairness benchmarks are met.
Interoperability is another consideration. In multi-party payments flows—where issuers, acquirers, PSPs, and fraud vendors each have their own detection systems—explainability suffers if outputs are inconsistent or non-transferable. Establishing common data fields and reasoning structures for flagged transactions could make it easier to trace and justify decisions end-to-end.
The alternative is a patchwork of regional and organisational standards that risks undermining both compliance and customer trust. Harmonised audit and explainability protocols, developed through industry working groups and supported by regulators, could help ensure that XAI adoption strengthens rather than fragments the global payments landscape.
XAI is shifting from a desirable bonus to a core requirement in fraud detection, cutting operational friction, strengthening client trust, and giving firms a competitive edge.
Explainability is achievable without sacrificing performance. Tools like scorecards, SHAP/ LIME outputs and behavioural biometrics, combined with strong governance frameworks, high-quality data, and human-in-the-loop review, are helping firms meet both compliance demands and commercial objectives.
However, these gains will be limited if the industry adopts a fragmented approach. Shared datasets, interoperability standards, and harmonised audit protocols will be essential to avoid duplicated effort and ensure explainability scales across borders. The value of AI in payments will be judged not just by its ability to detect fraud, but by its capacity to explain its reasoning to regulators, customers, and internal teams. In a competitive and regulated market, opening the black box may be the most important step payment providers can take.
