Introduction
A 2025 McKinsey analysis of regulatory technology found that a US-based bank's legacy compliance system met just 75 percent of its regulatory requirements. After adopting an automated RegTech solution that streamlined data mapping, that figure rose to above 95 percent. It is a genuine, measurable improvement, and the kind of result compliance leaders are right to weigh seriously when deciding whether automation belongs in their document review process.
It is also, on its own, an incomplete answer to the question compliance teams are really asking. "Is AI accurate enough" doesn't have one answer. Everything depends on what the AI is being asked to do. Extracting a registration number from a scanned filing and determining whether a beneficial ownership structure represents an acceptable risk are different problems, and the evidence available in 2025 and 2026 points in different directions for each. Conflating the two is where a lot of overconfidence, in both directions, tends to come from.
The distinction matters beyond the compliance function itself. Procurement teams evaluating an AI-enabled due diligence vendor, auditors checking whether a decision trail can withstand scrutiny, and senior management signing off on a technology budget are, in effect, all asking a version of the same question. A vendor pitch that leads with a single headline accuracy figure, without saying which category of task that figure describes, isn't giving compliance and risk teams what they need to make the call.
Where the evidence supports high accuracy: structured extraction and consistency checks
The McKinsey RegTech example above sits squarely in the category where automated compliance tools perform most reliably: tasks with a defined correct answer that can be checked against a source document or a rule set. Pulling a company name, a registration number, or a filing date out of a scanned document falls here. So does confirming a data field is present, correctly formatted, or consistent across filings.
What tends to get overlooked is that the source data fed into the model matters as much as the model itself in this category. An extraction tool run against a clean, well-formatted filing behaves differently from the same tool run against a low-resolution scan or a document in a script it wasn't primarily trained on. McKinsey's RegTech result reflects better data mapping and structured processing, not a general claim that automation stays accurate regardless of what it's shown. Compliance teams working across the Middle East and Africa should read that caveat closely: document quality, language, and formatting vary considerably by jurisdiction in this region, so an extraction tool's real-world accuracy will vary with it, even when the underlying model doesn't change.
Consistency checking belongs in the same reliable category as extraction, and it's worth separating out because vendor marketing tends to bundle the two together more readily than the evidence supports. Confirming a registration number matches across two filings, or that a reported address holds up within a single document set, is a comparison task with a defined correct answer, not a judgement call. Automation does this well because the answer doesn't depend on interpreting context, only on comparing what's already there.
Where the evidence shows accuracy breaking down: judgement and unverifiable outputs
A 2025 McKinsey and IACPM survey of chief risk officers on generative AI in credit portfolio management found a specific, recurring limitation. Hallucination issues with large language models pose a genuine challenge in business scenarios that require accuracy close to 100 percent. The same study found institutions often struggle to validate model performance, because there's no reliable way to back test the outputs against a known correct answer.
A meaningful share of compliance document review isn't extraction. It's judgement. Deciding whether a beneficial ownership structure is unusual for its jurisdiction, whether an adverse media hit is relevant to the entity being screened, or whether a documentation gap is material all require weighing context a model wasn't built to hold, and producing an answer that's often hard to independently verify. That's exactly where the McKinsey and IACPM finding bites, not that generative AI is unreliable everywhere, but that its reliability is hardest to establish where compliance teams most need confidence in the answer.
The gap between a politically exposed person designation and a positive sanctions match illustrates why the distinction holds up in practice. Both can surface from an automated screening tool as a name match against a reference list. Which classification applies, and what response it calls for, depends on the regulatory context the tool wasn't designed to interpret. Treating that call as equivalent to a document extraction task, one where the model simply got the field right or wrong, misstates what kind of confidence the output deserves.
The MidFirst Bank lesson: accuracy also depends on what the automation was configured to do
A 2022 Finding of Violation issued by the US Department of the Treasury's Office of Foreign Assets Control against MidFirst Bank shows a different failure mode altogether, one with nothing to do with model accuracy and everything to do with how automated systems get configured and understood. MidFirst's sanctions screening vendor was contracted to screen new customers, and existing customers with account changes, daily, but screened the full existing customer base only once a month. MidFirst believed the full customer base was screened daily. It wasn't.
When OFAC added two individuals to its Specially Designated Nationals list, MidFirst's systems missed the match for 14 days. OFAC's public enforcement release states that 98 percent of the transaction value MidFirst processed on behalf of the newly designated individuals occurred within six hours of their designation, long before the bank's monthly re-screening cycle would have caught it, and well inside a window that continuous, rather than periodic, screening would have flagged straight away.
The automation here wasn't inaccurate. It ran exactly the screening frequency it had been configured to run. The failure sat elsewhere: nobody at the institution had checked that the configuration matched what compliance actually required. That's a distinct dimension of "is AI accurate enough" in its own right. Accuracy only means something relative to a correctly understood, independently verified configuration, not just a well-performing model.
The same failure mode shows up well beyond sanctions screening. A KYB record verified once at onboarding and never re-checked carries the same structural risk as a customer base screened monthly instead of daily. The automation does exactly what it was set up to do, and what it was set up to do no longer matches what compliance needs. Any due diligence programme relying on automated re-verification should be able to say, specifically, how often that re-verification runs and against what triggers, rather than assuming continuous coverage because a vendor contract uses the word.
What "good enough" looks like in practice
Put together, the evidence points to a specific, practical standard rather than a blanket yes or no. Automation earns high confidence on structured extraction and consistency checks, provided the source data quality holds up and outputs are spot-checked against source documents as routine practice, not exception. It earns lower confidence, and needs a documented human decision point, on judgement calls where the correct answer can't be independently verified against a known ground truth, exactly the category the 2025 McKinsey and IACPM study flagged as hardest to validate. And whatever the underlying model's accuracy, the configuration of any automated system, screening frequency, data scope, what triggers a re-check, needs independent verification against actual compliance requirements rather than an assumption drawn from a vendor agreement. That gap is what produced the MidFirst violation.
Turning this into an evaluation checklist gives compliance and risk teams something more useful than a single accuracy number to interrogate. Before accepting a headline accuracy claim, it's worth asking which of the three categories a given feature falls into. It's worth asking what mechanism exists to verify a judgement-based output against something other than the model's own confidence in it. And it's worth asking who inside the organisation is responsible for confirming an automated system's configuration, its screening frequency, scope, and update cadence, still matches what compliance requires, rather than what a vendor agreement assumed a year or two ago.
None of this is an argument against using AI in compliance document review. McKinsey's RegTech result is a genuine, substantial improvement, and reviewing the volume of documentation involved in modern KYB and AML programmes by hand isn't a realistic alternative at scale. The point is narrower: knowing which category a task falls into- extraction, judgement, or configuration- determines what kind of confidence is actually warranted. Treating all three as solved by the same tool is where accuracy claims tend to outrun the evidence. Cedar Rose's white paper, Faster Verification, Same Decision: AI and the Future of KYB in MEA , sets out the same distinction at greater length, including why the judgement category in particular tends to stay with human reviewers even as extraction and monitoring become more automated.
The three categories also travel well across compliance functions beyond document review specifically. Transaction monitoring, adverse media screening, and credit risk scoring all split along a similar line: parts that come down to comparing structured data against a rule, and parts that come down to a human weighing context a model wasn't built to hold. Teams that map their own workflow against this split before selecting a vendor tend to end up with clearer expectations and fewer surprises than teams that start from a headline accuracy number and work backwards.
Conclusion
The evidence available through 2025 and 2026 supports a specific, bounded answer to whether AI is accurate enough for compliance document review. For structured extraction and consistency checking, the evidence holds up well, and the RegTech result McKinsey documented shows the scale of improvement on offer. For judgement calls that can't be independently verified, the evidence points the other way, with hallucination and back-testing difficulty flagged by risk officers themselves as an open problem. And whichever category applies, an automated system's accuracy is only as good as whether its configuration was checked against what compliance requires, a lesson the MidFirst Bank case makes plainly.
Compliance and risk teams evaluating AI-enabled document review tools are better served asking which of these three categories a given task falls into than asking for one accuracy figure covering all of them. A vendor able to answer that question specifically, rather than defaulting to an aggregate accuracy percentage, has usually thought harder about where its own tool's limits sit. Cedar Rose's approach to business verification across the Middle East and Africa reflects this distinction directly, applying automation to structured extraction and continuous monitoring at scale while keeping judgement calls and configuration verification with reviewers who know the region.
Sources & References :
McKinsey & Company, Ushering in a New Era of Trusted AI (2025). https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/tech-forward/ushering-in-a-new-era-of-trusted-ai
McKinsey & Company and IACPM, Gen AI in Credit Portfolio Management Study (2025). https://iacpm.org/wp-content/uploads/2025/03/IACPM-McKinsey-Gen-AI-Webinar-2025.pdf
U.S. Department of the Treasury, Office of Foreign Assets Control, Finding of Violation to MidFirst Bank (2022). https://ofac.treasury.gov/recent-actions/20220721
Faster Verification, Same Decision AI and the Future of KYB in MEA. https://www.cedar-rose.com/insights/faster-verification-same-decision-ai-and-the-future-of-kyb-in-mea
Business Verification Challenges in the Middle East & Africa. https://www.cedar-rose.com/blog/business-verification-challenges-in-the-middle-east-africa
