Efficiency of Flat File Database Approach in Data Storage and Data Extraction for Big Data

Abstract

Big data is the latest industry buzzword to describe large volume of structured and unstructured data that can be difficult to process and analyze. Most of organization looking for the best approach to manage and analyze the large volume of data especially in making a decision. XML and JSON are chosen by many organization because of powerful approach during retrieval and storage processes. However, these approaches, the execution time for retrieving large volume of data are still considerably inefficient due to several factors. In this contribution, three databases approaches namely Extensible Markup Language (XML), Java Object Notation (JSON) and Flat File database approach were investigated to evaluate their suitability for handling thousands records of publication data. The results showed flat file is the best choice for query retrieving speed and CPU usage. These are essential to cope with the characteristics of publication’s data. Whilst, XML, JSON and Flat File database approach technologies are relatively new to date in comparison to the relational database. Indeed, Text File Format technology demonstrates greater potential to become a key database technology for handling huge data due to increase of data annually.