Input/output optimization scheduler for cloud-based map reduce framework
Abstract
Hadoop MapReduce (HMR) provides the most common MapReduce (MR) framework, and it is available as open source. MR is a famous computational framework for evaluating unstructured, and semi-structured big data and executing applications in the past ten years. Memory and input/output (I/O) overhead are just two of the many problems affecting the current HMR scheduler system. This study aims to improve systems resource use including the processing of data in real-time by creating a memory I/O optimized scheduler (MIOOS) for HMR. The disk I/O seek can be reduced by using MIOOS, which analyzes the entire memory management. Additionally, the MIOOS makespan approach is used to reduce the occurrence of problems in intermediary tasks. Both the MIOOS approach and the current approach are assessed by using complex scientific workflow applications with extreme task inter-dependencies. Further, the comparison study demonstrates that the MIOOS framework outdoes the current approach regarding makespan and overall memory usage.
Keywords
Cloud computing; Hadoop MapReduce; MapReduce; Memory; Quality of services; Scientific workflow
Full Text:
PDFDOI: http://doi.org/10.11591/ijeecs.v35.i3.pp1765-1772
Refbacks
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Indonesian Journal of Electrical Engineering and Computer Science (IJEECS)
p-ISSN: 2502-4752, e-ISSN: 2502-4760
This journal is published by the Institute of Advanced Engineering and Science (IAES) in collaboration with Intelektual Pustaka Media Utama (IPMU).