Screening of Diabetes Retinopathy in Optometry : Assessment of a Web-based Training Protocol

Diabetic retinopathy is a sight threatening complication of diabetes mellitus. Regular eye examinations by trained health professionals can help prevent this. Although ophthalmologists have screening responsibility in Norway today, Norwegian optometrists regularly examine the ocular health of a large part of the population. The exact number of people with diabetes in Norway is unknown, and many are likely undiagnosed. Optometrists should be able to detect and grade diabetic retinopathy and ensure proper management of these patients. Previous studies in Norway have shown that optometrists need improved diagnostic skills to provide screening according to recommended standards. This study investigated the effect of web-based targeted training on the optometrists’ ability to detect, classify and manage patients with diabetic retinopathy. The study had an experimental prospective design. Eighteen optometrists working in optometric practice in Norway participated in a web-based survey ”Visual Identification and Management of Ocular Conditions” (VIMOC) related to diabetic retinopathy before and after a minimal web-based training protocol. In the VIMOC, the optometrists assessed 14 retinal digital photographs of people with known diabetes. An ophthalmologist’s assessment and grading of the images was considered as the gold standard. The prevalence of retinopathy in the sample was set to 50% to prevent false high specificity. The webbased training significantly improved the optometrists’ diagnostic sensitivity, but did not significantly improve specificity. The diagnostic sensitivity before training was 71.4% (SD = 19.6). After training, the sensitivity was 85.71% (SD = 12.9). However, only six (33%) of the optometrists achieved the recommended screening standard; sensitivity of 80% or better and specificity of 95% of better. Web-based training in screening for diabetic retinopathy significantly improved optometrists’ screening and grading skills. Specific training in diabetes and screening for diabetic retinopathy are of great importance for detection and management of patients with diabetes by optometrists.


Background
Diabetic retinopathy is a microvascular late complication of diabetes mellitus, and is the most common cause of visual impairment in the working-age population (Porta & Bandello, 2002). Diabetic retinopathy causes 6.3 to 9.7% of the cases of visual impairment in the western world (Bamashmus, Matlhaga, & Dutton, 2004). Diabetic retinopathy develops gradually and symptoms do not necessarily occur before the advanced stage, as retinopathy often does not affect the macula in the initial phase (Bek, 2012). Regular retinal examination and timely treatment reduce the incidence of vision loss (Backlund, Algvere, & Rosenqvist, 1997;Kristinsson, Hauksdottir, Stefansson, Jonasson, & Gislason, 1997;Stefansson et al., 2000;Zoega et al., 2005). The increased prevalence of diabetes in the population predicts an increase in diabetic retinopathy (Delcourt, Massin, & Rosilio, 2009). The Norwegian Diabetes Association estimates that 375,000 Norwegians have diabetes. The prevalence of diagnosed diabetes in Norwegian population is 4%, that is about 200,000 people (Diabetesforbundet, 2014). In Norway, mainly ophthalmologists carry out screening for diabetic retinopathy. A report published by the Norwegian Ophthalmological Association (2012) stated a yearly total of 6116 diabetes related ophthalmologist consultations in Norway in 2009 and a total of 4007 laser-treatments of diabetic retinopathy (Norsk Oftalmologisk Forening, 2012). These figures suggest that many patients with diabetes do not receive eye examinations according to the recommended screening program. Further, the report estimates a 20% increase in consultations related to diabetes towards 2030 (Norsk Oftalmologisk Forening, 2012). This will lead to major challenges in the screening for and treatment of diabetic retinopathy in patients with diabetes.
The aim of this study is to evaluate the effect of a minimal web-based training protocol for screening of diabetic retinopathy by optometrists in terms of diagnostic sensitivity and specificity.

Methods
The population studied was authorised optometrists working in private practice in Norway. The optometrists were recruited through The Norwegian Association of Optometry and the head offices of the optical chains in Norway. Participation was voluntary. Only optometrists working in Norwegian optometric practice and having a bachelor degree in optometry or equivalent were included in the study. Optometrists working in ophthalmologist practices and hospitals were excluded. The training protocol consisted of three parts, and included two questionnaires, a study guide, two journal articles, three digital learning resources and three internet-based assessments Visual Identification and Management of Ocular Conditions (VIMOC) related to diabetes and screening for diabetic retinopathy.
The initial part included an online questionnaire asking the optometrists about key points regarding their optometric background and clinical experience with emphasis on patients with diabetes and an assessment of screening skills. The assessment of screening skills was a pre-training VIMOC assessment containing 14 retinal images. The optometrists had to assess the manifestations of diabetic retinopathy, identify clinical signs of diabetic retinopathy, grade the severity of diabetic retinopathy and state how they would manage the patient. The optometrists did not have information about the patients' history of general or ocular health. However, they had the information that all retinal images were of people with diabetes not regularly examined by an ophthalmologist. The optometrists did not use a grading scale for assessment. When assessing the retinal images, the optometrists could assess the images in both colour and in black/white. Further, the optometrists had the opportunity to go back and forth between the images if needed. There was no time limit on how long the optometrists could view the images, but the time spent on the assessment was recorded. Half of the retinal images did not have diabetic retinopathy. The prevalence of diabetic retinopathy of 50% was chosen to reduce the possibility of false high specificity.
The second part of the protocol was the web-based training protocol. The Study Guide had links to three online video lectures and two review articles about diabetes mellitus, diabetic retinopathy and screening for diabetic retinopathy (Sundling, 2012;2013). After completing each video lecture, the optometrists had to answer five multiple-choice questions to allow for self-assessment of understanding. There were no restrictions to how many times the lectures could be viewed within the twoweek time-window allowed to complete the training.
The final part included a post-training VIMOC. After completing the training, the optometrists assessed the same 14 retinal images as presented in the initial assessment however, the images presentation was in a different order and the optometrist did not know that the images were the same. After completing the assessment, the optometrist received the VIMOC with correct answers to allow for personal feed-back on grading and assessment. Additionally, the optometrists assess the web-based training protocol and the value for their clinical practice. Informed consent in written form was obtained from all the participants. The Norwegian Social Science Data Services (NSD) approved the study.
Data collection took place February to March 2015. Google docs was used to collect data for the online questionnaires, and Question Writer HTML5 was used to assess the optometrist screening using a VIMOC format with multiple-choice questions (Aakre & Svarverud, 2011). The retinal images used in the study were selected from a database of a population study of people with diabetes with and without diabetic retinopathy (Sundling, 2012). An ophthalmologist had assessed and graded all images and we considered the ophthalmologist grading as the gold standard. All images were 45-degree-field images focusing on the macula and optic nerve.
The VIMOC and digital learning resources had been developed by one of the authors (VS). The VIMOC and teaching materials were available online to facilitate a low threshold for participation. The participants were encouraged to use computers with Windows 7 or newer, and recommended to optimise their screen with regard to lighting, colour and contrast by using the operating system calibration solution. The recommended screen resolution was 1024×768 pixels and above, and a minimum screen size of 15" was recommended.
Microsoft Excel (2010) and (IBM) SPSS version 22.0 were used for statistical analysis. The data was analysed in frequency and summation tables. Group association and training effect on sensitivity and specificity were calculated with two-tailed student t-test, considering a p-value < 0.05 as significant. Cases with non-normally distributed data were analysed by Wilcoxon signed rank test and McNemar test.

Results
Twenty-one optometrists responded and wanted to take part in the study. However, three optometrists withdrew during the study period, two due to illness and one for other reasons. In all 18 optometrists completed the study (age 25-61 years), 12 (67%) were women. The participants had a mean of 20 years professional experience (range 4-39) and examined a mean of six patients per day (range 2-12). The majority of the optometrists (89%) had legal rights to requisite and use ocular diagnostic drugs. Four participants (22%) reported diabetes as a field of academic interest. Table 1 describes the characteristics of the optometrists. Most optometrists (89%) had access to a fundus camera, indirect and direct ophthalmoscope and the majority used more than one examination method for examining the retina. Moreover, 61% had access to wide-field ophthalmoscopy (Optomap) and 17% had access to optical coherence tomography (OCT). All optometrists used fundus photography as a method of retinal examination. Undilated fundus photography was the preferred method for retinal examination (72%), followed by undilated indirect ophthalmoscopy using slit lamp and Volk-lens (28%). In total, 67% reported to use mydriatic drugs; however, the optometrists rarely performed dilated retinal examination, Table 2. Only 12 of the 16 optometrists (75%) with legal rights for use of diagnostic drugs used the opportunity to perform dilated retinal examination. The optometrists' mean score of self-confidence in dealing with patients with diabetes were 5.4 (range 4-8) on an 11-point Likert scale, where 0 = very unsure and 10 = very sure. For retinal examination of patients with known diabetes, the majority of optometrist (89%) reported undilated fundus photography as the preferred screening method. Although, most optometrists stated that they used more than one screening method, Table 3. Nine (50%) of the optometrists reported that patient management and decision to refer patients depended on the degree of diabetic retinopathy and whether the patient was under a follow-up regime by an ophthalmologist. Three optometrists reported that they would always refer patients with diabetes to an ophthalmologist, while five reported that referral to ophthalmologist was dependent on the degree of retinopathy. The optometrists had significantly better sensitivity for identifying retinopathy post-training than pre-training, 71.4% (SD = 19.6) versus 85.7% (SD = 12.9), t-test p = 0.011. The optometrists showed a significant improvement in correct classification of images with diabetic retinopathy after web-based training, improving from 18.3% (SD = 11.8) to 45.2% (SD = 13.2), t-test p < 0.001. There were no statistically significant changes in specificity or ability to classify non-retinopathy images correctly. In total, the percentage of correctly classified images on VIMOC assessment of diabetic retinopathy increased by 27% (95% CI, 20 to 34) from pre-training to post-training, Table 4.
The recommended screening standard, sensitivity of at least 80% combined with a specificity of at least 95% (British Diabetic Association, 1997), was met by respectively 2 and 6 of the optometrists before and after the completing the web-based training protocol, Figure 1. Figure 1: The number of optometrists who met the screening standard with sensitivity higher than 80% and specificity higher than 95% before and after the online course.
The optometrists referred when they found diabetic retinopathy. In cases of both true and false positives findings, the patients were usually referred to their general practitioner (GP) or ophthalmologist, Table 5. Because of the number of false positives, respectively 17 and 34 of 252 cases would have been unnecessarily referred to a GP or ophthalmologist in VIMOC 1 and VIMOC 2. In cases where the optometrists did not detect diabetic retinopathy, that is false negative, few of the optometrists would have referred the patients to an ophthalmologist, respectively 5 of 36 and 2 of 18 cases in VIMOC 1 and VIMOC 2, Table 5. In cases where the optometrists would not have referred despite findings of diabetic retinopathy, the patients had mild non-proliferative diabetic retinopathy or laser treated diabetic retinopathy. In cases of false negatives, which the optometrist would not have referred, the cases included both moderate nonproliferative retinopathy, diabetic macular oedema and laser treated diabetic retinopathy.

Discussion
The aim of this study was to assess whether optometrists achieve standards for screening of diabetic retinopathy by use of a web-based training programme. There were several optometrists who reached the requirements for sensitivity after having undergone training, but for specificity the number optometrist who met the requirement criteria was unchanged. However, higher specificity may come at the cost of a low sensitivity, which can result in major consequences for patients if sight-threatening retinopathy is undetected and left untreated. This may further have economic consequences for the society in a long-term perspective. Screening and preventive treatment of diabetic retinopathy is very cost-effective measured in Quality Adjusted Life-Years (Javitt & Aiello, 1996). For Norwegian optometrists to be able to take on a role in screening responsibility, the web-based training programme needs to be extended to ensure that clinical skills and diagnostic sensitivity and sensitivity meet the required screening standard. The optometrists in our study showed significantly improved sensitivity after training. This suggests that the optometrists improved their skills in correctly identifying diabetic retinopathy. However, the specificity fell slightly, but this change was not significant. This may indicate that the optometrists became better at detecting diabetic retinopathy at the expense of an increased over-diagnosis. This is not an unusual effect when screening is performed (Wilson & Jungner, 1968). A high number of false positives may also indicate that optometrists are careful when managing patients with diabetes assuming that they are afraid to miss sight-threatening retinopathy. Studies of optometrists in other countries have shown that optometrists can achieve good sensitivity and specificity for screening for diabetic retinopathy through extensive training. In an Australian study, optometrists achieved a screening sensitivity and specificity of respectively 94% and 93.6% and they correctly classified retinopathy in 69% of cases (Schmid et al., 2002). A reason why the results of the Australian study are better than our study may be differences in study design and training protocol. The Australian optometrists were gathered in one study centre where they underwent clinical training. This ensured virtually identical conditions for examination and grading for all participants. Guidelines were also available during the examinations. In our study the optometrists did the grading in their own office on their own computer and computer screen, therefore equal screening conditions were difficult to secure. Moreover, the optometrist in our study did not use grading scales.
Our study also showed an improvement in the optometrists' ability to grade diabetic retinopathy after web-based training. Being able to classify diabetic retinopathy correctly is essential, as it will secure proper follow-up and timely treatment. This is especially important in patients with potentially sightthreatening diabetic retinopathy and for patients who need medical treatment to preserve their vision. Optometrists' ability to classify diabetic retinopathy is also essential in ensuring good and precise communication between optometrists and ophthalmologists, and other health professionals. The optometrists' ability to grade and assess diabetic retinopathy has in previous studies shown to be good, but in these studies, the optometrists were specially trained. A Norwegian study using the same study design and method as our study, but without the web-based training protocol, showed a slightly lower sensitivity and specificity of 67% and 84%, respectively (Sundling, 2013) compared to the sensitivity of 71.4% and specificity of 85.7% achieved after completion of web-based training in our study. Another difference in our study design is the implemented redfree images as well as colour images. This may have improved the detection of retinopathy as the use of red-free images enhances detection of retinal microaneurysms and haemorrhages.
It has also been shown that assessment of red-free retinal images is better than retinal slit-lamp examination, but the best results were achieved by grading colour images (Olson et al., 2003).
Optometrists in Norwegian optometric practice examine mainly healthy patients, as ophthalmologists take care of patients with retinal findings. Therefore, optometrists do not get the same amount of experience as they would have acquired if they assessed pathology more frequently. A UK study showed that optometrists working in hospitals have a higher sensitivity and specificity than optometrists working in optometric practice (Hulme et al., 2002). The improved diagnostic skills may be explained by more experience in number and varying degrees of diabetic retinopathy. Hospital settings experience will not be achieved in ordinary Norwegian optometric practice. However, proper training and regular examination of a substantial number of patients could secure high quality of screening in optometric practice.
Mydriasis or pupillary dilation provides a better view of the retina and improves image quality, positively affecting the optometrists' diagnostic abilities to detect diabetic retinopathy (Murgatroyd et al., 2004;Scanlon, Foy, Malhotra, & Aldington, 2005). In a previous Norwegian study, only 23% of optometrists reported that they performed dilated retinal examination in patients with diabetes (Sundling, 2013), which was consistent with our own findings. This practice conflicts with current clinical guidelines issued by The Norwegian Association of Optometry (Norges Optikerforbund, 2010). As our study used retinal photos of high quality, the sensitivity achieved in our study is likely higher than the actual sensitivity for detecting diabetic retinopathy in Norwegian optometric practice. This is supported by a previous Norwegian study, which reported a considerably lower number of patients with suspected diabetic retinopathy in optometric practice compared to the prevalence found in population-based studies (Sundling et al., 2008).
The optometrists included in our study represent a heterogeneous group of optometrists, reflecting Norwegian optometrists in general (Sundling et al., 2007). This may indicate that some optometrist chose to take part in the study because they feel they have little knowledge about diabetes or took part because they were already interested in diabetes or education in general. The medium mean self-confidence score (5.4 on a 0-10 Likert scale) in management of people with diabetes may reflect such heterogeneity.
The training protocol could have been improved to increase optometrists' competency in screening for diabetic retinopathy. For instance, introducing practical demonstrations, discussions and clinical workshops in screening for diabetic retinopathy in addition to the web-based training programme may have improved the diagnostic skills. However, such a training protocol may not have recruited the same population as the web-based training protocol, as long-distance travel would have been required for some of the optometrists to participate. Similar studies in countries like the UK had clinical practice as part of their curriculum. The studies showed sensitivity between 72% and 76% and specificity between 77% and 99% (Hulme et al., 2002;Olson et al., 2003;Prasad et al., 2001). These results were somewhat poorer in sensitivity, but better in specificity than our results. However, the studies are not directly comparable as they utilized different screening methods. In particular, the specificity was higher, which may be explained by clinical practice being included in the training programme. In a centralized and clinical screening setting like in the UK studies, the optometrists have the opportunity to ask questions directly and discuss cases; this may improve clinical reasoning and screening competency.

Conclusions
Our study showed a potential for Norwegian optometrists to improve their diagnostic skills for detection of diabetic retinopathy. Targeted training in diabetes screening and diabetic retinopathy for practicing optometrists significantly improved diagnostic sensitivity and correct classification of diabetic retinopathy. Future studies should assess whether implementation of clinical workshops and use of grading scales in the training program will further improve diagnostic quality and meet the screening standard for diabetic retinopathy.