Efficient load aware scheduler for map reduce applications in cloud environment

##plugins.themes.academic_pro.article.sidebar##

Published Jul 1, 2018
https://doi.org/10.47164/ijngc.v9i2.144
Download
PDF
Statistic

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...
Volume 9, Issue 2, July 2018

##plugins.themes.academic_pro.article.main##

Sree Lakshmi Adepu
Geethanjali College of Engineering and Technology
BalRaju M
Krishna Murthy Institute of Technology, Hyderabad, Telangana
Subhash Chandra N
CVR college of Engineering,Hyderabad, Telangana

Abstract

Most of the current day applications are compute and data intensive which laid a platform for invention of technologies like Hadoop. Hadoop uses a Map Reduce paradigm to solve the problem by using parallelism. Cloud computing environments have provided more flexibility in using Hadoop to solve Big Data problems without any investment on infrastructure procurement and maintenance and take the advantage of parallelism with required scalability. The current schedulers of Map Reduce tasks can be improved for virtual environments to reduce the cost of the services used in the cloud. This work proposes an efficient scheduler for Map reduce applications in cloud environment.

##plugins.themes.academic_pro.article.details##

How to Cite
Sree Lakshmi Adepu, BalRaju M, & Subhash Chandra N. (2018). Efficient load aware scheduler for map reduce applications in cloud environment. International Journal of Next-Generation Computing, 9(2), 151–161. https://doi.org/10.47164/ijngc.v9i2.144
Crossref
Scopus
Google Scholar
Europe PMC

References

  1. A.Sree Lakshmi , Dr.M.BalRaju , Dr.N.Subhash Chandra,’Towards optimization of hadoop map reduce jobs on cloud’ IEEE International Conference on Computing, Analytics and Security Trends (CAST 2016), Dec 2016, ISBN: 978-1-5090-1338-8
  2. A.Sree Lakshmi, Dr.M.BalRaju, Dr.N.SubhashChandra : “Scheduling of Parallel Applications Using Map Reduce On Cloud: A Literature Surveyâ€, International Journal of Computer Science and Information Technologies, Vol. 6 (1) , 2015, Page(s): 112-115
  3. Andrew Wylie, Wei Shi, Jean-Pierre Corriveau “A Scheduling Algorithm for Hadoop MapReduceWorkflows with Budget Constraints in the Heterogeneous Cloudâ€, 2016 IEEE International Parallel and Distributed Processing Symposium Workshops
  4. Hadoop Distributed File System: http://hadoop.apache.org/docs/r1.2.1/hdfs_design.html
  5. H. Kang, Y. Chen, J. L. Wong, R. Sion, and J. Wu, “Enhancement of Xen’s scheduler for MapReduce workloads,†in Proceedings of the 20th international symposium on High performance distributed computing, New York, NY, USA, 2011, pp. 251–262
  6. http://nikgrozev.com/2014/06/08/cloudsim-and-cloudsimex-part-1/
  7. http://www.cloudbus.org/cloudsim/
  8. https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/
  9. Lenar Yazdanov, Maxim Gorbunov, Christof Fetzer “EHadoop: network I/O aware scheduler for elastic MapReduce cluster†, 2015 IEEE 8th International Conference on Cloud Computing, DOI 10.1109/CLOUD.2015.113
  10. Moussa Ehsan, Karthiek Chandrasekaran, Yao Chen, Radu Sion , “Cost-efficient tasks and data co-scheduling with affordhadoop†, IEEE transactions on cloud computing, april 2016, , DOI 10.1109/tcc.2017.2702661
  11. Quan Chen, Daqiang Zhang, Minyi Guo, Qianni Deng, and Song Guo. “Samr: A self-adaptive mapreduce scheduling algorithm in heterogeneous environment.†Computer and Information Technology, International Conference, 0:2736– 2743, 2010.
  12. Qutaibah Althebyan,Yaser Jararweh, Qussai Yaseen, Omar AlQudah and Mahmoud Al-Ayyoub “Evaluating map reduce tasks scheduling algorithms over cloud computing infrastructureâ€, Concurrency and Computation: practice and experience 2015; 27:5686–5699 ,31 July 2015 , Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/cpe.3595
  13. Rajdeep Das, Rohit Pratap Singh, Ripon Patgiri ,â€MAPREDUCE SCHEDULER: A 360-DEGREE VIEWâ€, International journal of current engineering and scientific research (IJCESR), ,issn (print): 2393-8374, (online): 2394-0697, volume-3, issue-11, 2016
  14. Sewoog Kim, Dongwoo Kang and Jongmoo Choi I/O Characteristics and Implications of Big Data Processing on Virtualized Environments, Applied Mathematics & Information Sciences An International Journal, 9, No. 2L, 591-598 (2015)
  15. Shyam Deshmukh, Dr. J. V. Aghav, Rohan Chakravarthy “Job Classification for MapReduce Scheduler in Heterogeneous Environmentâ€, 2013 International Conference on Cloud & Ubiquitous Computing & Emerging Technologies
  16. S. Kim, D. Kang, J. Choi and J. Kim, Burstiness-aware I/O scheduler for MapReduce framework on virtualized environments'', 2014 International Conference on Big Data and Smart Computing (BIGCOMP), 2014, Pages: 305- 308, doi:10.1109/BIGCOMP.2014.
  17. W. Tian, G. Li, W. Yang and R. Buyya,``HScheduler: an optimal approach tominimize the makespan of multiple MapReduce jobs'', The Journal of Supercomputing June 2016,Volume 72, Issue 6, pp 2376–2393, doi:10.1007/s11227-016-1737-4
  18. X. Wang, D. Shen, G. Yu, T. Nie and Y.Kou, ``A ThroughputDriven Task Scheduler for Improving MapReduce Performance in Job-Intensive Environments'', 2013 IEEE International Congress on Big Data, 2013, Pages: 211 -218
  19. Yi Yao,Jiayin Wang, Bo Sheng, Jason Lin, Ningfang Mi , “HaSTE: Hadoop YARN Scheduling Based on Task-Dependency and Resource-Demandâ€, 2014 IEEE International Conference on Cloud Computing , 978-1-4799-5063-8/14