17 June 2025
Moorfields Eye Hospital and University College London Institute of Ophthalmology researchers have examined 36 artificial intelligence (AI) tools for eye care approved by regulators in Europe, Australia and the US.
The review, which was published in npj Digital Medicine, found there was significant variation in the amount of evidence that was provided on the clinical performance of the devices.
The analysis also highlighted a lack of transparency around training data – including details of gender, age and ethnicity.
Of the devices included in the analysis, close to one in five (19%) had no published peer-reviewed data on accuracy or outcomes.
Among the 131 clinical evaluations of the remaining devices, only around half reported patient age (52%) and gender (51%) while one in five reported ethnicity (21%).
The review outlined how validation was mostly completed using archival image sets, with limited diversity or inadequate reporting of basic demographic characteristics.
Of the devices, more than two-thirds targeted diabetic retinopathy in a screening context.
The authors highlighted that a greater emphasis should be placed on accurate and transparent reporting of datasets.
“This is critical to ensuring equitable performance as some populations may be underrepresented in the training data,” they emphasised.
Lead author, Dr Ariel Ong, highlighted that AI has potential to fill a global gap in eye care.
“In many parts of the world, there simply aren’t enough eye specialists, leading to delayed diagnoses and preventable vision loss,” she said.
She added that while AI screening can help to identify disease earlier and inform clinical management, this technology must be built on “solid foundations.”
“We must hold AI tools to the same high standards of evidence as any medical test or drug. Facilitating greater transparency from manufacturers, validation across diverse populations, and high-quality interventional studies with implementation-focused outcomes are key steps towards building user confidence and supporting clinical integration,” Ong emphasised.
The authors have called for regulatory frameworks to take a more standardised approach to evidence reporting for medical devices incorporating AI.
They also highlighted that new guidance, such as the EU AI Act, could set a higher benchmark for data diversity and real-world trials.
Comments (1)
You must be logged in to join the discussion. Log in
Don Williams19 June 2025
Having read the recent Moorfields and UCL scoping review, it’s striking (and a little concerning) to see that a significant proportion of AI tools already approved for ophthalmic use have little to no published, peer-reviewed evidence about their accuracy or outcomes. Nearly one in five of these tools apparently entered the market without peer-reviewed validation and even among those that did, many lack transparency around crucial details like patient demographics or real-world performance.
This raises an interesting dilemma: while peer review is essential to build trust and ensure safety, I wonder how feasible it is in practice for certain AI medical devices. AI tools are often continuously evolving (updates, retraining), and commercial sensitivities can limit how much data companies are willing to share publicly. This makes traditional peer review more complicated than for static drugs or surgical techniques.
It’s clear that regulators and developers need to find better ways to balance innovation with rigorous, ongoing validation perhaps by requiring more real-world trials, better dataset reporting and independent auditing over time. I’d be interested to hear how others think we can make peer review practical for AI that doesn’t stand still..! Tricky.
ReportLike2