Delhi Journal of Ophthalmology

Comparison of Angiography Performance of Low-cost portable Fundus camera with standard FFA camera – Validation Study

Deependra Vikram Singh, Gaurav Mathur, Rishi Bhardwaj,  Ajay Sharma 
Retina Department of Eye-Q Superspecialty Eye Hospitals, Sushant Lok, Gurugram, India

Corresponding Author:

Deependra Vikram Singh, MD
Eye-Q Superspecialty Eye Hospitals
H.NO – 522, Sec-27. Sushant Lok, Gurugram, India, 122002
Email - deependravsingh@hotmail.com

Received: 30-MAR-2020

Accepted: 07-JUN-2020

Published Online: 20-DEC-2020

DOI:http://dx.doi.org/10.7869/djo.594

Abstract
Study Design: In an observational cross-sectional study, 91 eyes of 46 consecutive patients underwent simultaneous Fundus Fluorescein Angiography (FFA) for common retinal disorders. The FFA images captured by both cameras were graded by 3 Retinologists (DVS, GM, RB) in a structured format. 7 characteristics of FFA images that were used for grading were Foveal Avascular Zone (FAZ) visualization, identifying branch retinal vein, Identification of any abnormal hyper fluorescence at macula, Identification of Micro aneurysms, Identification of capillary non perfusion areas and identification of leakage from retinal neovascularization, Image quality and Impact on Clinical decision. Results were captured and statistically analyzed for interobserver and intercamera agreements. 

Results: Out of total 7 features in FFA images graded by 3 Retinologists, 4 characteristics were found to have statistically significant intercamera agreement for 3 retinologists. These were Identification of any abnormal hyper fluorescence at macula, Identification of Micro aneurysms, Identification of capillary non perfusion areas and identification of leakage from retinal neovascularization. These 4 characteristics also had very good agreement between 3 retinologists. 

Conclusion: Based on this pilot study, we can conclude that the Fluorescein Angiography performance of this low-cost portable Fundus camera for detecting clinically relevant findings was comparable to a standard Fundus camera. We conclude that in resource limited settings, this low-cost portable Fundus camera can be a useful alternative to standard Fundus camera for treating common retinal disorders..

Keywords :Low-cost Fundus Fluorescein Angiography, Portable Fluorescein Angiography, Diabetic Retinopathy, Hyperfluorescence

Introduction
Globally, retinal diseases are a significant cause of visual impairment and blindness. A recent metanalysis projected that by 2020, Age-related macular degeneration (AMD) and Diabetic Retinopathy will be responsible for moderate to severe vision loss in 8.8 million and 3.2 million cases respectively with 2.0 million people estimated to be blind by AMD.1 Prevalence of Diabetes2 and Diabetic retinopathy3 is increasing worldwide and global burden of macular diseases is also increasing.4 Another systematic review estimated the global prevalence of any retinal vein occlusion (RVO) in people aged between 30 to 89 years and reported that 56.11 million people to be affected by any RVO.5
Asia is now home to approximately 80% of the world's diabetic population, including more than 60 million Indians, and the total number of diabetic persons is expected to increase to more than 100 million by 2030.6 
India being a developing country where majority of population lives in rural area, impact of retinal disorders can be far higher and devastating due to constraints in manpower and infrastructure.7,8 Home eye care and Ambulatory Retina clinics can provide essential care to patients with retinal disorders in this low resource settings. A low-cost Fundus camera for Fluorescein angiography seems to be a useful addition to this model.
While few low-cost portable imaging devices are now available for screening retinal diseases,9,10 A low-cost portable fundus fluorescein angiography (FFA) camera is not widely available. 
Fluorescein angiography performance of Low-cost portable Fundus camera has not been evaluated and has not been compared with standard Fundus camera. In an observational cross-sectional study, we evaluated fluorescein angiography performance of low-cost portable Fundus camera (Smartscope® Pro, Optomed Inc, Oulu, Finland) with standard fundus camera (Visucam® 500 by Carl Zeiss Inc., San Leandro, CA). FFA images from both cameras were compared and analyzed for quality and image parameters that are important in making clinical decisions for common retinal disorders.

Material and Methods
A single-site (tertiary eye care hospital, Gurugram India), prospective, clinic-based comparative instrument validation study evaluated the FA performance of low-cost portable Fundus camera and compared it with standard fundus camera during a period of 6 months (January 2017 to June 2017). The FA module of low-cost fundus camera (Smartscope® Pro, Optomed Inc, Oulu, Finland) has camera sensor resolution of 5 megapixels and clicks images with field of view of 40 degrees. The standard camera (Visucam 500® Carl Zeiss Inc., San Leandro, CA) used in this study has capture sensor resolution of 5.0 mega pixels Charged Coupled Device (CCD) with 2 fields of view 45° and 30° available. All images in this study were captured with 45° option only. Total 91 eyes of 46 patients with Diabetic Retinopathy, retinal vascular occlusions or central serous retinopathy were included. Eyes with neovascular AMD were not included because clinical utility of FFA has reduced for these cases. Eyes with hazy media due to cataract or posterior capsule opacification were excluded if treating surgeon felt that a good quality FFA images wouldn’t be obtained. All patients underwent simultaneous FFA on both low-cost Portable Fundus camera and standard Fundus camera. The study was approved by institutional review board and adhered to the tenets of the Declaration of Helsinki. Detailed Informed consent informing about nature of study was signed by all subjects before the angiography procedures. The identity of all patients and the diagnosis of each image were masked before grading. Each patient underwent a detailed ophthalmologic examination that included dilated fundus examination, color fundus photograph, and FFA using both fundus cameras. 

FFA protocol
All patients were randomly assigned to one of the 2 groups – Z (Zeiss) group & O (Optomed) group. Patients in Z group had early phase (First 1 minute) of FFA captured on standard (Zeiss) camera and subsequent mid phase captured on Slit-lamp mounted portable (Optomed) camera for next 2 minutes and finally late phases captured on both cameras. Reverse sequence was followed for patients in O group. All FFA images were captured by experienced retinologists and each eye got seven 45° photographs (ensuring that all ETDRS fields are covered) captured. We stored all photographs as Joint Photographic Experts Group (JPEG) files after removing all patient
Demographic details and assigning a unique ID number linked to the participant’s study ID number. Images from low-cost Fundus camera (Optomed) were stored in native state with JPEG quality of 1536 × 1156 pixels and those from Standard Fundus camera (Zeiss) were stored with JPEG quality of 1064 × 1028 pixels. 
These Images were arranged in pairs with folders named by unique patient IDs by a trained technician who saved Zeiss images and Optomed images of each eye in respective folders. 

Grading of Images
Before grading was performed, all the participating retinal specialists attended a discussion session to align the interpretation of different parameters to be graded that included: FAZ visualization, identifying branch retinal vein, Identification of any abnormal hyper fluorescence at macula, Identification of Micro aneurysms, Identification of capillary non perfusion areas and identification of leakage from retinal neovascularization and Image quality. 
Table 1 shows the different Grading scales assigned to these parameters.



Some of these features have already been described in earlier studies evaluating performance of FFA devices.12
Both sets of images (91 eyes from each camera) were graded by three senior retinal specialists experienced in assessment and management of retinal vascular disorders (D.V.S., G.M., R.B.).
As a protocol, for each ID, Images from Folder O were graded first followed by images from Folder Z. The results of grading were captured in an excel sheet (Microsoft® Excel® 2016 MSO. Version 2004) for statistical analysis.

Statistical Analysis
Statistical analysis was performed using SPSS on-line Subscription – Version 23 (SPSS version 23; Statistical Package for Social Science, Chicago, IL). For each Grading feature the statistical analysis was carried out to find out correlation and agreement between both devices. 
Cohen’s Kappa analysis was also done for each feature to test the agreement between 3 Retinologists for that feature on each device. These analyses were carried out on an eye-specific basis. We used following interpretation for Kappa Analysis.13
Poor agreement = Less than 0.20, Fair agreement = 0.20 to 0.40, Moderate agreement = 0.40 to 0.60, Good agreement = 0.60 to 0.80, Very good agreement = 0.80 to 1.00.

Results
Total of 91 eyes of 46 patients were included for Image analysis. Total of 1647 (18 per eye) FFA images from low-cost Fundus camera and 1596 (17.5 per eye) FFA images from standard camera were graded. Grading of FFA images captured by both devices was done for each eye by 3 Retinologists – DVS (PI), GM (CI) & RB (CI). The Demographic data and Ocular characteristics of these 46 patients is listed in (Table - 2).



Results of kappa analysis for intra-observer agreement for grading these 7 parameters on FFA images from both devices by 3 retinologists are discussed here. Results of interobserver agreement are shown in (Table 3). Detailed statistical results are available as supplementary material (Annexure -1).



1. FAZ (Percentage of FAZ Visible) – 
FAZ was evaluated and graded into 4 scales ranging from A to D with Grade D showing best visualization (Table–1). Grading of FFA Images from Standard Camera revealed FAZ visualization to be Grade D (75% to 100% FAZ visualized) in 85/91 (93.41%) eyes as compared to FFA images from low-cost Camera where FAZ visualization was Grade D (75% to 100% FAZ visualized) in 69/91 (75.82%) eyes when graded by PI. The agreement between the two devices calculated by Kappa analysis was 0.302 (fair agreement) (p = 0.000). The p value here signifies the validity of the test (Cohen’s Kappa), yet, bears no implication on the test value itself.
Similar Analysis for agreement between two devices was done for 2 CIs (GM and RB). The agreement on kappa analysis was 0.339 (fair agreement) (p = 0.00) for GM and 0.105 (poor agreement) (p = 0.020) for RB. Figure 1 shows a case where FAZ in nicely visualized in FFA images from both devices.


Figure 1: FFA images from case ID 31 showing OD with Non-Proliferative Diabetic Retinopathy. FAZ is nicely visualized in FFA images from both low-cost (1A) and Standard Fundus (1B) Cameras. Please note timing displayed (white solid arrow) in second image (1B) is irrelevant because FFA was done sequentially on two cameras

2. Branch Retinal Vessel Identification – 
All 1st, 2nd and 3rd order Branch Retinal vessels were visible in 89/91 (97.8%) eyes on Zeiss FFA Images and in 82/91 (92.11%) eyes on Optomed FFA Images. The agreement between two devices calculated by Kappa analysis was 0.347 (Fair agreement) with P value of 0.000. 
Similar Analysis for agreement between two devices for branch vessel identification was done for 2 CIs (GM and RB). The agreement on kappa analysis was 0.175 (poor agreement) (with p = 0.018) for GM and 0.128 (poor agreement) (with p = 0.003) for RB.

3. Identification of any abnormal hyper fluorescence at macula –
Abnormal Hyper fluorescence at macula was detected in 80/91 eyes on Zeiss Images and 78/91 eyes on Optomed Images by PI. The agreement between two devices calculated by kappa analysis was 0.768 (good agreement) with P value of 0.000. Similar Analysis for agreement between two devices on identification of any abnormal hyper fluorescence was done for 2 CIs (GM and RB). The agreement on kappa analysis was 0.950 (very good agreement) (p = 0.00) for GM and 0.724 (good agreement) (p = 0.00) for RB. Figure 2 shows variety of abnormal hyper fluorescences visualized in images from both devices.


Figure 2: FFA images from case ID 22 showing OD with Central serous chorioretinopathy (CSCR) with stippled hyper fluorescence due to chronic RPE alterations (Solid white arrows), and ink blot leakage (hollow white arrows). Also, seen is bright hyper fluorescence suggestive of Pigment epithelial detachment (PED) (White arrow heads). Both Images are showing all abnormal hyper fluorescence at macula.

4. Identification of Micro aneurysms –
60 eyes with diabetic retinopathy or RVO were eligible for detection of possible microaneurysms. Microaneurysyms could be identified in 51/60 eyes on FFA Images from Standard Camera and 47/60 eyes on FFA Images from low-cost Camera by PI. The agreement between two devices calculated by Kappa analysis was 0.924 (very good agreement) with P value of 0.000. Similar Analysis for agreement between two devices on identification of Microaneurysyms was done for 2 CIs (GM and RB). The agreement on kappa analysis was 0.899 (good agreement) (p = 0.00) for GM and 0.961 (very good agreement) (p = 0.00) for RB. Figure 3 shows multiple MAs nicely visible as hyperfluorescent dots of uniform size in FFA images from both devices.


Figure 3: FFA images from case ID 29 showing OD with PDR with microaneurysms (MAs) visible as hyperfluorescent dots (white arrows) scattered over macula and leakage from neovascularization at disc.

5. Identification of capillary non perfusion areas
32 eyes with severe NPDR and RVO were eligible for evaluation of Capillary non perfusion (CNP). CNP areas could be identified in 31/32 eyes on Images from Standard Camera and 29/32 eyes on Images from low-cost Camera by PI. The agreement between two devices calculated by Kappa analysis was 0.976 (very good agreement) with P value of 0.000. Similar Analysis for agreement between two devices on identification of capillary non perfusion areas was done for 2 CIs (GM and RB). The agreement on kappa analysis was 0.977 (very good agreement) (p = 0.00) for GM and 0.954 (very good agreement) (p = 0.00) for RB.

6. Leakages from RNV 
32 eyes with severe NPDR and RVO were eligible for evaluation of Leakage from Retinal Neovascularization (RNV). Leakage from RNV could be identified in 24/32 eyes on Standard Fundus Camera (Zeiss) Images and 21/32 eyes on Low-cost Fundus Camera (Optomed) Images by PI. The agreement between two devices calculated by Kappa analysis was 0.935 (very good agreement) with P value of 0.000. Similar Analysis for agreement between two devices on identification of leakages from RNV was done for 2 CIs (GM and RB). The agreement on kappa analysis was 0.936 (very good agreement) (p = 0.00) for GM and 0.957 (very good agreement) (p = 0.00) for RB.

7. Image Quality
Image quality was graded as excellent (details of All vessels seen and FAZ seen clearly) in 84/91 eyes on Standard Fundus Camera (Zeiss) Images and 68/91 eyes on Low-cost Fundus Camera (Optomed) Images by PI. Image quality was graded as poor (only Disc and Macula seen) in 1/91 eyes on Zeiss Images and 5/91 eyes on Optomed Images by PI. The agreement between two devices calculated by Kappa analysis was 0.268 (fair agreement) (p = 0.00). Overall, 89/91 (97.8%) eyes in Zeiss group and 83/91 (91.2%) eyes in Optomed group had excellent to good image quality FFA images. Similar Analysis for agreement between two devices on Image Quality was done for 2 CIs (GM and RB). The agreement on kappa analysis was 0.137 (poor agreement) (p = 0.014) for GM and - 0.013 (poor agreement) (p = 0.769) for RB. Figure 4 shows a case with (Branch Retinal Vein Occlusion) BRVO and retinal edema seen as equally well delineated in FFA images from both devices.


Figure 4: FFA images from case ID 39 showing OD with fresh BRVO with macular edema. Extent of involvement is well delineated in both images.

Discussion
In this clinic-based comparative instrument validation study, we evaluated the performance of a low cost portable FFA camera (Smartscope®, Optomed Inc, Oulu, Finland) by comparing its FFA images with standard camera (Carl Zeiss Inc., San Leandro, CA). We performed Kappa analysis on grading on Images from 2 devices done by 3 Retinologists and found a very high agreement (value of k > 0.6) between two devices for 4 out of 7 parameters. These parameters were Identification of any abnormal hyper fluorescence at macula, Identification of Micro aneurysms, Identification of capillary non perfusion areas and identification of leakage from retinal neovascularization. Inter-observer agreement for 3 Retinologists (DVS, RB & GM) in grading FFA images from two devices was found to be good or better (Kappa >0.6) for 3 features out of these 7 parameters.
However, for parameters like FAZ visualization, identifying branch retinal vein and Image quality, the agreement between 2 devices was fair to poor. Results for these parameters were same for interobserver agreement between graders. We feel that these categories could have been more objectively defined to make grading for these 3 features less subjective and this could explain the lack of agreement. The fact that the “Interobserver agreement between 3 graders on these 3 features was poor even with standard (Zeiss) camera that is known to produce high quality FFA images:” supports our hypothesis that these were too subjective parameters.
Identification of Micro aneurysms, Identification of capillary non perfusion areas and identification of leakage from retinal neovascularization are 3 important parameters for which a retinologists would order a FFA test and these FFA findings are utilized for managing eyes with diabetic retinopathy, RVO, Retinal vasculitis and same applies to detection of abnormal hyper fluorescence in eyes with CSCR. 
This study has found very good inter-device (Zeiss Vs Optomed) agreement and very good interobserver agreement (for Optomed) while grading FFA images for these parameters implying that this low-cost (˜US $8600.00) Fundus camera can substitute for a (higher cost) (˜US $34500.00) Standard Fundus Camera. 
In India, more than 62 million individuals are currently diagnosed with Diabetes11 and the ophthalmologist to population ratio is 1:107 000. Although 70% of the population lives in rural areas, 70% of ophthalmologists practice in urban areas.14,15 This discrepancy creates a significant need for outreach programs. This also means that affordability and accessibility to retina care is lacking significantly in rural areas. Most tier-2 and tier-3 cities get retina care by visiting retinologists who travel from tier-1 cities and deliver retina services for different retinal disorders including management of Diabetic retinopathy. Importance of using a low cost portable FFA camera with acceptable performance on Image quality and good performance on relevant (decision influencing) parameters in this setting cannot be over emphasized. Current study has provided useful insights into decision influencing parameters-based assessment of a low-cost FFA Camera. Recently published guidelines from International Council of Ophthalmology has acknowledged the prohibitive cost of OCT in low and intermediate resource settings.16 In this scenario a low cost FFA can assist retinologists in detecting and documenting findings, like macular edema and serous detachment and leaks in CSCR, that could impact and also guide the clinical treatment decisions for these retinal diseases.
Merits of this study include; it was an observational validation study with Simultaneous FFA done on both devices, both cameras were randomly used to click early phases in subjects and FFA images were graded by 3 experienced retinologists and relevant Image parameters for grading were used to assess angiography performance. We are also unaware of any other study evaluating angiography performance of a portable FFA Camera. However, there were some limitations of this study including a small sample size, Eyes with significant cataract or PCO were excluded, Graders could not be blinded because of the different characteristics of Images from both devices and also, we didn’t analyze the performance of this low-cost camera on peripheral FFA imaging but same was not possible with Standard FFA camera either. Need to evaluate performance of this low-cost portable camera for other retinal diseases like AMD and in a larger population cannot be over emphasized.

Conclusion
Over all, we found that Fluorescein Angiography performance of Low cost (Optomed) Fundus camera was comparable to Standard (Zeiss) Fundus camera especially for objective features like identification of abnormal hyper fluorescence, MAs, CNP areas and leakages from RNV in eyes with diabetic retinopathy, RVO, CSCR and Retinal vasculitis. Because these FFA findings influence clinical decision for managing these retinal disorders, we can safely conclude that this camera is a useful low-cost alternative to gold standard high cost (Zeiss) camera in low resource settings. 

References
  1. Flaxman SR, Bourne RRA, Resnikoff S et al. Global causes of blindness and distance vision impairment 1990-2020: a systematic review and meta-analysis. Lancet Glob Health. 2017 Dec;5(12):e1221-e1234
  2. Whiting DR, Guariguata L, Weil C, Shaw J. IDF diabetes atlas: global estimates of the prevalence of diabetes for 2011 and 2030. Diabetes Res Clin Pract. 2011; 94:311–21. [PubMed: 22079683].
  3. Yau JWY, Rogers SL, Kawasaki R, et al. Global prevalence and major risk factors of diabetic retinopathy. Diabetes Care. 2012; 35:556–64. [PubMed: 22301125].
  4. Jonas JB, Bourne RR, White RA, Flaxman SR. Visual Impairment and Blindness Due to Macular Diseases Globally: A Systematic Review and Meta-Analysis. Am J Ophthalmol 2014;158:808–815.
  5. Song P, Xu Y, Zha M, Zhang Y, Rudan I. Global epidemiology of retinal vein occlusion: a systematic review and meta-analysis of prevalence, incidence, and risk factors. J Glob Health. 2019 Jun;9(1):010427. 
  6. Anjana RM, Pradeepa R, Deepa M, et al. Prevalence of diabetes and prediabetes (impaired fasting glucose and/or impaired glucose tolerance) in urban and rural India: phase I results of the Indian Council of Medical Research-India Diabetes (ICMRINDIAB) study. Diabetologia. 2011; 54:3022–7.
  7. Bressler NM, Varma R, Doan QV, et al. Underuse of the health care system by persons with diabetes mellitus and diabetic macular edema in the United States. JAMA Ophthalmol. 2014; 132:168–73.
  8. Fletcher AE, Donoghue M, Devavaram J, et al. Low uptake of eye services in rural India: a challenge for programs of blindness prevention. Arch Ophthalmol. 1999; 117:1393–9.
  9. Sengupta S, Sindal MD, Besirli CG, Upadhyaya S, et al. Screening for vision-threatening diabetic retinopathy in South India: comparing portable non-mydriatic and standard fundus cameras and clinical exam. Eye (Lond). 2018 Feb;32(2):375-383.
  10. Bursztyn L, Woodward MA, Cornblath WT, et al. Accuracy and Reliability of a Handheld, Nonmydriatic Fundus Camera for the Remote Detection of Optic Disc Edema. Telemed J E Health. 2018 May;24(5):344-350.
  11. Joshi SR, Parikh RM. India - diabetes capital of the world: now heading towards hypertension. J Assoc Physicians India. 2007;55:323–4.
  12. Garcia CR, Rivero ME, Bartsch DU, et al. Oral fluorescein angiography with the confocal scanning laser ophthalmoscope. Ophthalmology. 1999 Jun;106(6):1114-8.
  13. Altman DG. Practical Statistics for Medical Research. (1991) London England: Chapman and Hall. Page No. 404.
  14. Kumar R. Ophthalmic manpower in India—need for a serious review. Int Ophthalmol. 1993; 17(5): 269–275. 
  15. Samandar R, Kleefield S, Hammel J, Mehta M, Crone R. Privately funded quality health care in India: a sustainable and equitable model. Int J Qual Heal Care 2001; 13(4): 283–288.
  16. Wong TY, Sun J, Kawasaki R, Ruamviboonsuk P, Guidelines on Diabetic Eye Care: The International Council of Ophthalmology Recommendations for Screening, Follow-up, Referral, and Treatment Based on Resource Settings. Ophthalmology. 2018 Oct;125(10):1608-1622.

CITE THIS ARTICLE

Singh DV, Mathur G, Bhardwaj R, Sharma AComparison of Angiography Performance of Low-cost portable Fundus camera with standard FFA camera – Validation Study.DJO 2020;31:65-70

CITE THIS URL

Singh DV, Mathur G, Bhardwaj R, Sharma AComparison of Angiography Performance of Low-cost portable Fundus camera with standard FFA camera – Validation Study.DJO [serial online] 2020[cited 2021 Jan 27];31:65-70. Available from: http://www.djo.org.in/articles/31/2/Comparison-of-Angiography-Performance-of-Low-cost-portable-Fundus-camera-with-standard-FFA-camera-Validation-Study.html