Incorporating Transfer Learning in CNN Architecture
##plugins.themes.academic_pro.article.sidebar##
##plugins.themes.academic_pro.article.main##
Abstract
Machine learning (ML) is a data intensive process. For training of ML algorithms huge datasets are required.
There are times when enough data is not available due to multitude of reasons. This could be due to lack of
availability of annotated data in a particular domain or paucity of time in data collection process resulting in
non-availability of enough data. Many a times data collection is very expensive and in few domains data collection
is very difficult. In such cases, if methods can be designed to reuse the knowledge gained in one domain having
enough training data, to some other related domain having less training data, then problems associated with lack
of data can be overcome. Transfer Learning (TL) is one such method. TL improves the performance of the target
domain through knowledge transfer from some different but related source domain. This knowledge transfer can
be in form of feature extraction, domain adaptation, rule extraction for advice and so on. TL also works with
various kinds of ML tasks related to supervised, unsupervised and reinforcement learning. The Convolutional
Neural Networks are well suited for the TL approach. The general features learned on a base network (source)
are shifted to the target network. The target network then uses its own data and learns new features specific to
its requirement.
##plugins.themes.academic_pro.article.details##
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
- Afridi, M., Ross, A., and Shapiro, E. 2017. On automated source selection for transfer learning in convolutional neural networks. Pattern Recognition, Elsevier. DOI: https://doi.org/10.1016/j.patcog.2017.07.019
- Alom, M. Z. and et.al. 2022. The history began from alexnet: A comprehensive survey on deep learning approaches”, rep. Online:https://arxiv.org/ftp/arxiv/papers/1803/1803.01164.pdf.
- Google-Colab. 2022. Google colab runtime environment. Online:https://colab.research.google.com.
- Howard, A., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861v1 [cs.CV].
- IBM. 2020. Convolutional neural networks. Online:https://www.ibm.com/cloud/learn/convolutionalneural-networks.
- Kalavathy, G., Revanth, S., and Venkat, S. 2022. Artificial intelligence based integrated and distributed system for preventing covid-19 spread using deep learning. International Journal of Next-Generation Computing 13(3). https://doi.org/10.47164/ijngc.v13i3.853. DOI: https://doi.org/10.47164/ijngc.v13i3.853
- Kouw, W. M. and Loog, M. 2019. An introduction to domain adaptation and transfer learning. arXiv:1812.11806v2 [cs.LG] , Tech. Rep.
- Krizhevsky, A., Sutskever, I., and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems.
- Liu, X., Wang, G., Cai, Z., and Zhang, H. 2016. Bagging based ensemble transfer learning. J Ambient Intell. Human Comput 7:29–36, DOI 10.1007/s12652-015-0296-5 Springer. DOI: https://doi.org/10.1007/s12652-015-0296-5
- Pan, S. and Yang, Q. 2009. A survey on transfer learning. IEEE transactions on knowledge and data engineering.
- Rawat, W. and Wang, Z. 2017. Deep convolutional neural networks for image classification: A comprehensive review. Neural Computation 29, 2352–2449 © Massachusetts Institute of Technology doi:10.1162/NECO-a-00990. DOI: https://doi.org/10.1162/neco_a_00990
- Salaken, S. M., Khosravi, A., Nguyen, T., and Nahavandi, S. 2017. Extreme learning machine based transfer learning algorithms: A survey. Neurocomputing, Elsevier, pp.516–524. DOI: https://doi.org/10.1016/j.neucom.2017.06.037
- Schmidhuber, J. 2014. Deep learning in neural networks: An overview. Neural Networks, Elsevier. DOI: https://doi.org/10.1016/j.neunet.2014.09.003
- Shin, H., Roth, H., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., and Summers, R. M. 2016. Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE transactions on medical imaging Vol. 35, No. 5. DOI: https://doi.org/10.1109/TMI.2016.2528162
- TensorFlow. 2022. Transfer learning. Online:https://www.tensorflow.org/tutorials/images/transferlearning.
- Wang, Y., Zhai, J., Li, Y., Chen, K., and Xue, H. 2018. Transfer learning with partial related “instance-feature”knowledge. Neurocomputing, Elsevier . DOI: https://doi.org/10.1016/j.neucom.2018.05.029
- Weiss, K., Khoshgoftaar, T. M., andWang, D. 2016. A survey of transfer learning. Journal of Big Data DOI 10.1186/s40537-016-0043-6. DOI: https://doi.org/10.1186/s40537-016-0043-6
- Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. 2014. How transferable are features in deep neural networks. arXiv:1411.1792v1 [cs.LG].
- Zhu, Y., Chen, Y., Lu, Z., Pan, S. J., Xue, G., Yu, T., and Yang, Q. 2011. Heterogeneous transfer learning for image classification. Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence. DOI: https://doi.org/10.1609/aaai.v25i1.8090
- Zhuang, F., Qi, Z., Duan, K., Xi, D., Zu, Y., Zhu, H., Xiong, H., and He, Q. 2020. A comprehensive survey on transfer learning. arXiv:1911.02685v3 [cs.LG].