Multi-Class Retinopathy classification in Fundus Image using Deep Learning Approaches


Nisha Wankhade
Kishor Bhoyar


Retinopathy classification from fundus images put a lot of issues in front of ophthalmologists. Convolution and deep neural network models open the doors to handle such challenges and achieve great success in computer vision, but it is reaching its computational limits. This leads to the rethinking of less computationally intensive network architectures for computer vision problems. In this work we have used a RFMiD dataset, which is challenging for machine learning researchers due its multiclass, multi-labelled, and imbalanced nature. In the proposed work three models are developed to classify the retinopathy from fundus images. The first model inherits the properties of the VGG Net and Inception Net. This results in significant reduction in computational complexity compared with VGG Net and Inception net models. The second model is an improvised version of the previous one with an increase in depth that yields notable improvement in results, while maintaining the lower number of computations. The third model uses a bidirectional LSTM model as a classifier with 192 hand-crafted features. This model gives 0.985 AUC, with a precision of 0.98, and recall of 0.9 respectively.


How to Cite
Wankhade, N., & Bhoyar, K. (2021). Multi-Class Retinopathy classification in Fundus Image using Deep Learning Approaches. International Journal of Next-Generation Computing, 12(5).


  1. Neil Thompson, Shuning Ge, and Gabriel Filipe. The importance of (exponentially more) computing power. Mimeo, 2020.
  2. Thompson, Greenewald, Lee,Manso, “ The Computation limits of Deep learning”, arXiv:2007.05558v1 [cs.LG] 10 Jul 2020
  3. Samiksha Pachade, Prasanna Porwal, Dhanshree Thulkar, Manesh Kokare, November 25, 2020, "Retinal Fundus Multi-disease Image Dataset (RFMiD)", IEEE Dataport, doi:
  4. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al., Imagenet large scale visual recognition challenge, International Journal of Conflict and Violence (IJCV) 115 (3) (2015) 211–252.
  5. R. Banu, V. Arun, N. Shankaraiah and V. Shyam, “Meta-cognitive neural network method for classification of diabetic retinal images,” in Proc. 2nd Int. Conf. on Cognitive Computing and Information Processing, CCIP 2016, Mysuru, India, 2016.
  6. U. Raghavendra, H. Fujita, S. V. Bhandary, A. Gudigar, J. H. Tan et al., “Deep convolution neural network for accuratediagnosis of glaucoma using digital fundus images,” Information Sciences, vol. 441, pp. 41–49, 2018
  7. U. Raghavendra, H. Fujita, S. V. Bhandary, A. Gudigar, J. H. Tan et al., “Deep convolution neural network for accurate diagnosis of glaucoma using digital fundus images,” Information Sciences, vol. 441, pp. 41–49, 2018.
  8. C. S. Lee, D. M. Baughman and A. Y. Lee, “Deep learning is effective for classifying normal versus age-related macular degeneration OCT images,” Ophthalmology Retina, vol. 1, no. 4, pp. 322–327, 2017.
  9. P. Burlina, D. E. Freund, N. Joshi, Y. Wolfson and N. M. Bressler, “Detection of age-related macular degeneration via deep learning,” in Proc. 13th IEEE Int. Sym. on Biomedical Imaging (ISBI), Prague, Czech Republic, pp. 184–188, 2016.
  10. J. Y. Choi, T. K. Yoo, J. G. Seo, J. Kwak, T. T. Um et al., “Multi-categorical deep learning neural network to classify retinal images: a pilot study employing small database,” PLoS One, vol. 12, no. 11, e0187336, 2017.
  11. Rajpurkar P, Irvin J, Zhu K, Yang B, Mehta H, Duan T, Ding D, Bagul A, Langlotz C, Shpanskaya K, et al. Chexnet: radiologist-level pneumonia detection on chest X-rays with deep learning; 2017. arXiv preprint arXiv:1711.05225.
  12. Li Z, He Y, Keel S, Meng W, Chang RT, He M. Effiffifficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs. Ophthalmology. (2018) 125:1199–206. doi: 10.1016/j.ophtha.2018. 01.023
  13. Treder M, Lauermann JL, Eter N. Automated detection of exudative age-related macular degeneration in spectral domain optical coherence tomography using deep learning. Graefes Arch Clin Exp Ophthalmol. (2018) 256:259–65.
  14. Schlegl T, Waldstein SM, Bogunovic H, Endstraßer F, Sadeghipour A, Philip AM, et al. Fully automated detection and quantifification of macular fluid in OCT using deep learning. Ophthalmology. (2018) 125:549– 58. doi: 0.1016/j.ophtha.2017.10.031
  15. Ting DSW, Cheung CY, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic population with diabetes. JAMA. (2017) 318:2211–23. doi: 10.1001/ jama.2017.18152
  16. Fauw J, Ledsam JR, Romera-Paredes B, Nikolov S, Tomasev N, Blackwell S, et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med. (2018) 24:1342–50. doi: 10.1038/s41591-018-0107-6
  17. Li W, Yang Y, Zhang K, Long E, He L, Zhang L, et al. Dense anatomical annotation of slit-lamp images improves the performance of deep learning for the diagnosis of ophthalmic disorders. Nature Bio Eng. (2020) 4:767– doi: 10.1038/s41551-020-0577-y
  18. Jing Wang et. al., “Multi-Label Classification of Fundus Images with EfficientNet”, IEEE Access, vol. 8, p.p. 212499-212508, 2020.
  19. Neha Gour and Pritee Khanna, “Multi-class multi-label ophthalmological disease detection using transfer learning based convolutional neural network”, Biomedical Signal Processing and Control, (Article in Press).
  20. Ping Jiang, Quansheng Dou and Li Shi, “Ophthalmologist-Level Classification of Fundus Disease with Deep Neural Networks”, Trans Vis Sci Tech., vol. 9(2):39, 2020.
  21. Hong J, Liu X, Guo Y, Gu H, Gu L, Xu J, Lu Y, Sun X, Ye Z, Liu J, Peters BA and Chen J (2021) A Novel Hierarchical Deep Learning Framework for Diagnosing Multiple Visual Impairment Diseases in the Clinical Environment. Front. Med. 8:654696.
  22. Alexander Selvikvåg Lundervold, ArvidLundervold, “An overview of deep learning in medical imaging focusing on MRI”, Zeitschrift für Medizinische Physik, Volume 29, Issue 2, May 2019, Pages 102-127
  23. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition, arXiv:1409.1556 (2014).
  24. K. He, X. Zhang, “Deep residual learning for image recognition”, Proceedings of the IEEE conference on computer vision and pattern recognition (2015), pp. 770-778
  25. G. Huang, Z. Liu, L. Van Der Maaten, K.Q. Weinberger, “Densely connected convolutional networks”, CVPR, vol. 1 (2016), p. 3
  26. Tan C, Sun F, Kong T, Zhang W, Yang C, Liu C. A survey on deep transfer learning. In: International conference on artificial neural networks. Springer; 2018. p. 270–9.
  27. Weiss K, Khoshgoftaar TM, A survey of transfer learning. J Big Data. 2016;3(1):9.
  28. Mingxing Tan, Quoc V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”, arXiv:1905.11946v3 [cs.LG] 23 Nov 2019
  29. Chollet F. Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 1251–8.
  30. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1–9, 2015.
  31. C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, “Inception-v4, inception-resnet and the impact of residual connections on learning,” in International Conference Learning Representations (ICLR) Workshop, 2016.
  32. Szegedy, Vanhoucke, loffe, Shlens, Wojna, “Rethinking the Incpetion Architecutre for computer Vision” arXiv:1512.00567v3 [cs.CV] 11 Dec 2015
  33. D. Erhan, C. Szegedy, A. Toshev, and D. Anguelov. Scalable object detection using deep neural networks. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 2155–2162. IEEE, 2014.
  34. Ioffe, S. and Szegedy, C., 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167
  35. Santurkar, S., Tsipras, D., Ilyas, A., 2018. How does batch normalization help optimization?. In Advances in Neural Information Processing Systems (pp. 2483–2493)
  36. Lin M, Chen Q, Yan S. Network in network; 2013. arXiv preprint arXiv:1312.4400.
  37. Alzubaidi, L., Zhang, J., Humaidi, A.J. et al. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data 8, 53 (2021).
  38. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, et al. Imagenet large scale visual recognition challenge. 2014.
  39. Hassim Sak, Andrew Senior, Françoise Beaufays, ”Long Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling”, arXiv:1402.1128v1 [cs.NE] 5 Feb 2014
  40. Alex Graves, Abdel-rahman Mohamed, Geoffrey Hinton,” Speech recognition with deep recurrent neural networks”,IEEE International Conference on Acoustics, Speech and Signal Processing, 2013
  41. Alex Graves, J. Schmidhuber ,” Framewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures”, IEEE International Joint Conference on Neural Networks, 2005.
  42. Hossin M, Sulaiman M. A review on evaluation metrics for data classification evaluations. Int J Data Min Knowl Manag Process. 2015;5(2):1.
  43. Hand DJ, Till RJ. A simple generalisation of the area under the ROC curve for multiple class classification problems. Mach Learn. 2001;45(2):171–86