Incorporating Transfer Learning in CNN Architecture

##plugins.themes.academic_pro.article.main##

Aparna Gurjar
Preeti Voditel

Abstract

Machine learning (ML) is a data intensive process. For training of ML algorithms huge datasets are required.
There are times when enough data is not available due to multitude of reasons. This could be due to lack of
availability of annotated data in a particular domain or paucity of time in data collection process resulting in
non-availability of enough data. Many a times data collection is very expensive and in few domains data collection
is very difficult. In such cases, if methods can be designed to reuse the knowledge gained in one domain having
enough training data, to some other related domain having less training data, then problems associated with lack
of data can be overcome. Transfer Learning (TL) is one such method. TL improves the performance of the target
domain through knowledge transfer from some different but related source domain. This knowledge transfer can
be in form of feature extraction, domain adaptation, rule extraction for advice and so on. TL also works with
various kinds of ML tasks related to supervised, unsupervised and reinforcement learning. The Convolutional
Neural Networks are well suited for the TL approach. The general features learned on a base network (source)
are shifted to the target network. The target network then uses its own data and learns new features specific to
its requirement.

##plugins.themes.academic_pro.article.details##

How to Cite
Gurjar, A., & Voditel, P. (2023). Incorporating Transfer Learning in CNN Architecture. International Journal of Next-Generation Computing, 14(1). https://doi.org/10.47164/ijngc.v14i1.1052

References

  1. Afridi, M., Ross, A., and Shapiro, E. 2017. On automated source selection for transfer learning in convolutional neural networks. Pattern Recognition, Elsevier. DOI: https://doi.org/10.1016/j.patcog.2017.07.019
  2. Alom, M. Z. and et.al. 2022. The history began from alexnet: A comprehensive survey on deep learning approaches”, rep. Online:https://arxiv.org/ftp/arxiv/papers/1803/1803.01164.pdf.
  3. Google-Colab. 2022. Google colab runtime environment. Online:https://colab.research.google.com.
  4. Howard, A., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861v1 [cs.CV].
  5. IBM. 2020. Convolutional neural networks. Online:https://www.ibm.com/cloud/learn/convolutionalneural-networks.
  6. Kalavathy, G., Revanth, S., and Venkat, S. 2022. Artificial intelligence based integrated and distributed system for preventing covid-19 spread using deep learning. International Journal of Next-Generation Computing 13(3). https://doi.org/10.47164/ijngc.v13i3.853. DOI: https://doi.org/10.47164/ijngc.v13i3.853
  7. Kouw, W. M. and Loog, M. 2019. An introduction to domain adaptation and transfer learning. arXiv:1812.11806v2 [cs.LG] , Tech. Rep.
  8. Krizhevsky, A., Sutskever, I., and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems.
  9. Liu, X., Wang, G., Cai, Z., and Zhang, H. 2016. Bagging based ensemble transfer learning. J Ambient Intell. Human Comput 7:29–36, DOI 10.1007/s12652-015-0296-5 Springer. DOI: https://doi.org/10.1007/s12652-015-0296-5
  10. Pan, S. and Yang, Q. 2009. A survey on transfer learning. IEEE transactions on knowledge and data engineering.
  11. Rawat, W. and Wang, Z. 2017. Deep convolutional neural networks for image classification: A comprehensive review. Neural Computation 29, 2352–2449 © Massachusetts Institute of Technology doi:10.1162/NECO-a-00990. DOI: https://doi.org/10.1162/neco_a_00990
  12. Salaken, S. M., Khosravi, A., Nguyen, T., and Nahavandi, S. 2017. Extreme learning machine based transfer learning algorithms: A survey. Neurocomputing, Elsevier, pp.516–524. DOI: https://doi.org/10.1016/j.neucom.2017.06.037
  13. Schmidhuber, J. 2014. Deep learning in neural networks: An overview. Neural Networks, Elsevier. DOI: https://doi.org/10.1016/j.neunet.2014.09.003
  14. Shin, H., Roth, H., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., and Summers, R. M. 2016. Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE transactions on medical imaging Vol. 35, No. 5. DOI: https://doi.org/10.1109/TMI.2016.2528162
  15. TensorFlow. 2022. Transfer learning. Online:https://www.tensorflow.org/tutorials/images/transferlearning.
  16. Wang, Y., Zhai, J., Li, Y., Chen, K., and Xue, H. 2018. Transfer learning with partial related “instance-feature”knowledge. Neurocomputing, Elsevier . DOI: https://doi.org/10.1016/j.neucom.2018.05.029
  17. Weiss, K., Khoshgoftaar, T. M., andWang, D. 2016. A survey of transfer learning. Journal of Big Data DOI 10.1186/s40537-016-0043-6. DOI: https://doi.org/10.1186/s40537-016-0043-6
  18. Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. 2014. How transferable are features in deep neural networks. arXiv:1411.1792v1 [cs.LG].
  19. Zhu, Y., Chen, Y., Lu, Z., Pan, S. J., Xue, G., Yu, T., and Yang, Q. 2011. Heterogeneous transfer learning for image classification. Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence. DOI: https://doi.org/10.1609/aaai.v25i1.8090
  20. Zhuang, F., Qi, Z., Duan, K., Xi, D., Zu, Y., Zhu, H., Xiong, H., and He, Q. 2020. A comprehensive survey on transfer learning. arXiv:1911.02685v3 [cs.LG].