Realtime Hand Gesture Recognition System for Human Computer Interaction
##plugins.themes.academic_pro.article.sidebar##
##plugins.themes.academic_pro.article.main##
Abstract
Humans are using various devices for interacting with the system like mouse, keyboard, joystick etc. We have developed a real time human computer interaction system for virtual mouse based on the hand gestures. The system is designed in 3 modules as detection of hand, recognition of gestures and human computer interaction with control of mouse events to achieve the higher degree of gesture recognition. We first capture the video using the built-in webcam or USB webcam. Each frame of hand is recognized using a media Pipe palm detection model and using opencv fingertips. The user can move the mouse cursor by moving their fingertip and can perform a click by bringing two fingertips to close. So, this system captures frames using a webcam and detects the hand and fingertips and clicks or moves of the cursor. The system does not require a physical device for cursor movement. The developed system can be extended in other scenarios where human-machine interaction is required with more complex command formats rather than just mouse events.
##plugins.themes.academic_pro.article.details##
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
- Faisal, M. A. A., Abir, F. F., and Ahmed, M. U. 2021. Sensor dataglove for real-time static and dynamic hand gesture recognition. In 2021 Joint 10th International Conference on Informatics, Electronics Vision (ICIEV) and 2021 5th International Conference on Imaging, Vision Pattern Recognition (icIVPR). 1–7. DOI: https://doi.org/10.1109/ICIEVicIVPR52578.2021.9564226
- Grif, H.-S. and Turc, T. 2018. Human hand gesture based system for mouse cursor control. Procedia Manufacturing 22, 1038–1042. DOI: https://doi.org/10.1016/j.promfg.2018.03.147
- Haria, A., Subramanian, A., Asokkumar, N., Poddar, S., and Nayak, J. S. 2017. Hand gesture recognition for human computer interaction. Procedia Computer Science 115, 367– 374. 7th International Conference on Advances in Computing Communications, ICACC- 2017, 22-24 August 2017, Cochin, India. DOI: https://doi.org/10.1016/j.procs.2017.09.092
- Islam, M. M., Siddiqua, S., and Afnan, J. 2017. Real time hand gesture recognition using different algorithms based on american sign language. In 2017 IEEE International Conference on Imaging, Vision Pattern Recognition (icIVPR). 1–6. DOI: https://doi.org/10.1109/ICIVPR.2017.7890854
- Kumari, P., Singh, S., and Pasi, V. K. 2016. Cursor control using hand gestures. IJCA Proceedings on Recent Trends in Future Prospective in Engineering and Management Technology, 25–29.
- Sung, G., Sokal, K., Uboweja, E., Bazarevsky, V., Baccash, J., Bazavan, E. G., Chang, C., and Grundmann, M. 2021. On-device real-time hand gesture recognition. CoRR abs/2111.00038.
- Tran, D.-S., Ho, N.-H., Yang, H.-J., Kim, S.-H., and Lee, G. S. 2021. Real-time virtual mouse system using rgb-d images and fingertip detection. Multimedia Tools and Applications 80, 7, 10473–10490. DOI: https://doi.org/10.1007/s11042-020-10156-5
- Zhang, R., Jiang, C., Wu, S., Zhou, Q., Jing, X., and Mu, J. 2022. Wi-fi sensing for joint gesture recognition and human identification from few samples in human-computer interaction. IEEE Journal on Selected Areas in Communications 40, 7, 2193–2205. DOI: https://doi.org/10.1109/JSAC.2022.3155526