The truth about false positive referrals

Introduction

Identifying eye diseases such as glaucoma is a core purpose of the sight test, and referring suspected cases of disease to hospital for further tests and treatment is a key part of the primary care optometrist’s role. 

Sometimes these cases are found not to need treatment after further testing at the hospital. And sometimes people argue that the existence of these ‘false positive’ referrals shows that optometrists in primary care need to be better trained.

This is a mistake. 

This page explains why a one-off test for low prevalence diseases like glaucoma will, by definition, lead to high false positive rates. It shows that, far from being incompetent, community optometrists are already very good at using a combination of tests, information from the patient, and professional judgment, to keep unnecessary referrals to a minimum.

It also explains how the NHS in England could facilitate a significant further reduction in the number of false positive referrals to hospital if it funded more checks, such as repeat measures and referral refinement schemes, in community practice.

Some false positives are necessary if we want to identify all cases of disease

First, if referral criteria in medicine are set conservatively so that no patient without the disease will ever be referred, then that will also mean that more patients with the disease will not be identified and referred. Quite simply, some over-referral is necessary if disease is to be effectively identified and treated.

For most low prevalence eye diseases, a 50% false positive rate is good

Disease detection is often described in terms of sensitivity, specificity and positive predictive value (PPV). Sensitivity is how good we are at detecting the condition. Specificity is how good we are detecting those without the condition – in other words, avoiding false positive referrals. PPV is the simple proportion of those referred who have the condition, and it is this that can appear low at times. This ignores the problem of a low prevalence condition, which is that if the proportion of those without a condition is very large, then even a tiny percentage false positive rate will lead to a high number of false positives.

Thankfully, some common eye diseases only affect a small proportion of the population. For example, glaucoma, which accounts for over 10% of registrations of visual impairment and around a quarter of ophthalmology outpatient visits, has a prevalence of only around 2% in the UK population over 40¹. The prevalence of age-related macular degeneration (AMD), the most common cause of sight loss among older people, varies by age group. NICE quotes 2.4% among 50+, 4.8% in 65+ and 12.2% in people aged 80 or over².

In diseases with such low prevalence rates, in order to get as low as a 50% false referral rate, optometrists in primary care actually need to achieve around 98% accuracy in their testing procedures. Given that no single diagnostic test is that accurate, it is clear that optometrists are using a number of tests and their professional judgment to achieve such high accuracy.

This is how that maths works in a disease with 2% prevalence:

• In a sample population of 10,000 people, 200 people will have the disease and the other 9,800 won’t
• If this population is checked for the disease using a test with 98% sensitivity and specificity, the test will identify 196 of the 200 patients who actually have the disease
• The test will also accurately identify 98% of the 9,800 people who don’t have the disease (9,604)
• But that means that it will incorrectly identify signs of disease in 2% of those people who don’t actually have it. In this example, that’s 196 people
• So overall the test will lead to about 392 people testing positive, but about 196 of those don’t have the disease
• That is a 50% false positive rate
• It is also a 50% PPV (i.e. 196 expressed as a percentage of 392)

 

There are proven ways further to reduce false positive rates, but these are not routinely funded in England

Further investigation (also known as repeat measures) by optometrists in primary care before they refer a patient to hospital has been shown to make false positive referrals less likely. Parkins and Edgar³ showed that it meant 76% of suspect glaucoma patients were not referred, and produced a 62% financial saving in Bexley. Davey et al⁴ found patients were 2.7 times less likely to be referred in Bradford. Ang et al⁵ found that in the six months after the introduction of the new GOS in Scotland, which includes these repeat measures, there was a statistically significant increase in true-positive referrals (from 18.0% to 31.7%) and a decrease in false-positive referrals (from 36.6% to 31.7%).

Referral refinement schemes are another way to reduce false positive referrals. In these schemes, patients who show signs of glaucoma in an initial test are reviewed in the community by specially trained optometrists. Ratnarajan et al⁶ compared two places with referral refinement schemes (Manchester and Gloucestershire) and two areas that did not have them. The first-visit discharge rate (FVDR) (patients who were false positives) was 14.1% where the schemes were in place compared with 36.1% where there was no scheme. Henson et al⁷ also describes the Manchester scheme, which has been in place since 2002, and reports that the number of false positive referrals was reduced by 40%.

Notes

1. www.bmj.com/bmj/section-pdf/187924?path=/bmj/346/7912/Clinical_Review.full.pdf But note that prevalence is different in white, black and Asian populations.
2. www.nice.org.uk/guidance/ng82/chapter/Context
3. Parkins DJ & Edgar DF. Comparison of the effectiveness of two enhanced glaucoma referral schemes. Ophthalmic Physiol Opt 2011, 31, 343–352. doi: 10.1111/j.1475‐1313.2011.00853.x
4. Davey CJ, Scally AJ, Green C, Mitchell ES and Elliott DB (2016) Factors influencing accuracy of referral and the likelihood of false positive referral by optometrists in Bradford, United Kingdom. Journal of Optometry. 9(3): 158-165.
5. Ang GS, Ng WS and Azuara-Blanco, A. The influence of the new general ophthalmic services (GOS) contract in optometrist referrals for glaucoma in Scotland Eye volume 23, pages 351–355 (2009)
6. Ratnarajan G, Newsom W, Vernon SA, et al The effectiveness of schemes that refine referrals between primary and secondary care—the UK experience with glaucoma referrals: the Health Innovation & Education Cluster (HIEC) Glaucoma Pathways Project  BMJ Open 2013;3:e002715. doi: 10.1136/bmjopen-2013-002715
7. Henson DB, Spencer F, Harper R & Cadman EJ, Community refinement of glaucoma referrals Eye volume 17, pages 21–26 (2003)