UK researchers test performance of AI algorithms for the detection of diabetic eye disease

Researchers have assessed the performance of eight commercially-available automated retinal image analysis systems on 1.2 million images collected through one of the largest and most diverse diabetic screening programmes in the NHS – the North East London Diabetic Eye Screening Programme.

A total of 25 companies with CE-marked systems for diabetic retinopathy detections were invited to take part in the research, and eight accepted the invitation.

Writing in The Lancet Digital Health, the researchers concluded that the eight systems that agreed to take part in the study demonstrated high sensitivity for medium to high-risk diabetic retinopathy with equitable performance across population sub-groups.

Study lead, Professor Alicja Rudnicka, of City St Georges, University of London, highlighted: “We’ve shown that these AI systems are safe for use in the NHS by using enormous data sets, and most importantly, showing that they work well across different ethnicities and age groups.”

Co-principal investigator, Adnan Tufail, of Moorfields Eye Hospital, shared that there are four million people with diabetes in the UK who require regular eye checks.

“This groundbreaking study sets a new benchmark by rigorously testing AI systems to detect sight threatening diabetic eye disease before potential mass rollout. The approach we have developed paves the way for safer, smarter AI adoption across many healthcare applications,” he said.

Rudnicka added that the process used to examine the different AI systems provides a transferable framework for evaluating clinical AI.

“Our vision is to deliver centralised AI infrastructure that hosts approved algorithms, enabling all screening centres to upload retinal images securely for analysis. The AI-generated results would be returned to the centre and integrated directly into the patient’s electronic health record. This approach eliminates the need for duplicating infrastructure across multiple sites, reducing setup costs and ensuring consistent, equitable service delivery nationwide,” Rudnicka said.

Professor Sarah Barman, of Kingston University, highlighted that the research provides a clear approach that could be used in other medical domains to ensure that AI is fair and equitable.

“This large-scale evaluation of the effectiveness of AI algorithms has allowed us to demonstrate how different algorithms perform across subgroups of the population,” she said.