“Jaw-dropping” DeepMind algorithm on par with retinal specialists
Preliminary results indicate AI technology used to analyse OCT scans can diagnose eye disease with the same skill as world-leading clinicians
Early results suggest that an algorithm developed through a partnership between DeepMind and Moorfields Eye Hospital can diagnose common eye conditions with the same skill as experienced clinicians.
Speaking at the Moorfields AGM ahead of the publication of the research in Nature, consultant ophthalmologist Pearse Keane highlighted promising results from a project applying artificial intelligence technology to ocular coherence tomography (OCT) scans.
He shared that an algorithm developed by researchers can assess OCT scans for more than 50 different retinal diseases, diagnosing conditions with a similar degree of accuracy to Moorfields retinal specialists.
“It is very exciting that Moorfields can lead the way in applying this exciting technology to healthcare…I think ophthalmology as a specialty will be one of the first that is transformed by these new technologies,” Dr Keane observed.
Growing popularity of OCT
Hospital Eye Services in the UK see close to 10 million patients annually – a figure that is second only to orthopaedics and growing each year, Dr Keane said.
“We struggle to see and treat those patients in a timely manner,” he added.
More than 1000 ocular coherence scans were performed each day across Moorfields’ sites in 2017, Dr Keane shared.
He added that OCT is increasing being used in the community by High Street optometrists.
While Dr Keane shared that this is a positive development, there are challenges associated with the increased number of referrals into the Hospital Eye Service.
Dr Keane highlighted that in 2016 there were 7000 urgent referrals to Moorfields Eye Hospital with possible wet age-related macular degeneration (AMD).
Only 800 of these patients actually had the sight-threatening condition, Dr Keane added.
“Some of those patients didn’t get seen and treated as quickly as we would have liked in an ideal world,” he said.
Dr Keane shared the story of his patient, Elaine Manna, who lost sight completely in one eye from wet AMD more than 10 years ago.
In 2013, she started to lose sight in her good eye. Her optometrist made an urgent referral to the hospital and she received an appointment six weeks later.
“Can you imagine if you are sitting at home, waiting, and the sight in the good eye is going?” Dr Keane emphasised.
“If it was my mother, I would probably want her to be seen and treated in six days, rather than six weeks,” he added.
Technology as a solution
By applying artificial intelligence technology to OCT scans, Moorfields and DeepMind hope that patients with be diagnosed and treated more efficiently.
The next step is for the research to go through clinical trials to explore how the technology might improve patient care in practice. The technology would then require regulatory approval before it can be used in hospitals and other clinical settings.
Dr Keane said the initial focus would be on trialing the technology within the hospital system at rapid access clinics.
“That is in the short term, I think in the medium term and the longer term we are interested in applying this technology within the community,” he shared.
Dr Keane observed that this could be within optometry practices, general practice or even using OCT devices within patients’ homes as technology advances.
“As OCT is increasingly being adopted in the community, I think that every OCT device will have some form of AI associated with it that will help make the diagnosis,” he said.
He shared his belief that humans would always be needed to confirm the diagnosis suggested by AI.
“This is a way of getting people with sight threatening disease in front of the healthcare professional with the expertise to make the final diagnosis,” Dr Keane explained.
Optometrists played an important role in developing the technology. Dr Keane shared that a “rising star” of optometry, Reena Chopra, led a group of optometrists who were responsible for labelling data to make sure it was of sufficient quality to train the neural networks.
“They delineated all of the disease features on the OCT scans, pixel by pixel, manually drawing all the cysts and sub retinal fluid…That was crucial,” he said.
After the algorithm was developed, it was assessed by working out the gold-standard diagnosis for 1000 OCT scans and seeing how specialists performed compared to the algorithm.
Four specialist optometrists with experience working in Moorfields retinal clinics and four consultant ophthalmologist retinal specialists assessed the scans, Dr Keane highlighted.
“Some of the specialist optometrists were as good as some of the consultant ophthalmologists,” he observed.
The algorithm had an accuracy of 94% – as good or better than all of the clinicians involved.
Dr Keane shared that this percentage related to spotting the most urgent cases. He emphasised that while there were false positives, the algorithm did not make any false negative diagnoses.
“There was no case where there was something serious and the algorithm said don’t worry about it,” Dr Keane said.
Dr Keane emphasised the exceptional capabilities of the algorithm that has been developed.
“The performance of this algorithm is jaw-dropping,” he concluded.
Image credit: Moorfields Eye Hospital