Indonesian news classification using convolutional neural network

Muhammad Ali Ramdhani, Dian Sa’adillah Maylawati, Teddy Mantoro

Abstract


Every language has unique characteristics, structures, and grammar. Thus, different styles will have different processes and result in processed in Natural Language Processing (NLP) research area. In the current NLP research area, Data Mining (DM) or Machine Learning (ML) technique is popular, especially for Deep Learning (DL) method. This research aims to classify text data in the Indonesian language using Convolutional Neural Network (CNN) as one of the DL algorithms. The CNN algorithm used modified following the Indonesian language characteristics. Thereby, in the text pre-processing phase, stopword removal and stemming are particularly suitable for the Indonesian language. The experiment conducted using 472 Indonesian News text data from various sources with four categories: ‘hiburan’ (entertainment), ‘olahraga’ (sport), ‘tajuk utama’ (headline news), and ‘teknologi’ (technology). Based on the experiment and evaluation using 377 training data and 95 testing data, producing five models with ten epoch for each model, CNN has the best percentage of accuracy around 90,74% and loss value around 29,05% for 300 hidden layers in classifying the Indonesian News data.

Keywords


Classification; Convolutional Neural Network; Deep Learning; Indonesian Language Process; GloVe; Natural Language Processing; Text Mining

References


Alyosha, “Jumlah Bahasa di Dunia,” WordPress Blog2, 2016. [Online]. Available: https://alyoshaminded.wordpress.com/tag/jumlah-bahasa-di-dunia/. [Accessed: 09-Apr-2019].

Anonymous, “Terpetakan! Jumlah Bahasa di Seluruh Dunia, Di mana Posisi Indonesia?,” Republika Online, 2016. [Online]. Available: http://portalsatu.com/read/budaya/terpetakan-jumlah-bahasa-di-seluruh-dunia-di-mana-posisi-indonesia-5960. [Accessed: 09-Apr-2019].

P. M. Nadkarni, L. Ohno-Machado, and W. W. Chapman, “Natural language processing: An introduction,” J. Am. Med. Inform. Assoc., 2011.

G. G. Chowdhury, “Natural language processing,” Annu. Rev. Inf. Sci. Technol., 2005.

J. Hirschberg and C. D. Manning, “Advances in natural language processing,” Science. 2015.

H. Jiawei, M. Kamber, J. Han, M. Kamber, and J. Pei, Data Mining: Concepts and Techniques. 2012.

Y. E. Zohar, “Introduction to Text Mining,” Automated Learning Group, University of Illinois, 2002. [Online]. Available: http://www.docstoc.com/docs/25443990/Introduction-to-TextMining.

L. Deng, “Deep Learning: Methods and Applications,” Found. Trends® Signal Process., 2014.

J. Schmidhuber, “Deep Learning in neural networks: An overview,” Neural Networks. 2015.

J. Ahmad, H. Farman, and Z. Jan, “Deep Learning Methods and Applications,” in SpringerBriefs in Computer Science, 2019.

A. Kulkarni and A. Shivananda, “Deep Learning for NLP,” in Natural Language Processing Recipes, 2019.

T. Young, D. Hazarika, S. Poria, and E. Cambria, “Recent trends in deep learning based natural language processing [Review Article],” IEEE Comput. Intell. Mag., vol. 13, no. 3, pp. 55–75, 2018.

M. Yousefi-Azar and L. Hamey, “Text summarization using unsupervised deep learning,” Expert Syst. Appl., 2017.

G. Rossiello, “Neural abstractive text summarization,” in CEUR Workshop Proceedings, 2016.

M. Patel, A. Chokshi, S. Vyas, and K. Maurya, “Machine Learning Approach for Automatic Text Summarization Using Neural Networks,” Int. J. Adv. Res. Comput. Commun. Eng., vol. 7, no. 1, pp. 194–202, 2018.

M. Kalash, M. Rochan, N. Mohammed, N. D. B. Bruce, Y. Wang, and F. Iqbal, “Malware Classification with Deep Convolutional Neural Networks,” in 2018 9th IFIP International Conference on New Technologies, Mobility and Security, NTMS 2018 - Proceedings, 2018.

M. Jaderberg, K. Simonyan, A. Vedaldi, and A. Zisserman, “Reading Text in the Wild with Convolutional Neural Networks,” Int. J. Comput. Vis., vol. 116, no. 1, pp. 1–20, 2016.

K. Yao, L. Zhang, T. Luo, and Y. Wu, “Deep reinforcement learning for extractive document summarization,” Neurocomputing, 2018.

M. Day and C. Chen, “Artificial Intelligence for Automatic Text Summarization,” 2018 IEEE Int. Conf. Inf. Reuse Integr., pp. 478–484, 2018.

Y. Zhang, Q. Liu, and L. Song, “Sentence-State LSTM for Text Representation,” ACL2018, 2018.

J. T. Behrens, “Principles and Procedures of Exploratory Data Analysis,” Psychol. Methods, 1997.

M. Stuart, D. C. Hoaglin, F. Mosteller, and J. W. Tukey, “Understanding Robust and Exploratory Data Analysis.,” Stat., 2006.

D. S. Maylawati and G. A. P. Saptawati, “Set of Frequent Word Item sets as Feature Representation for Text with Indonesian Slang,” in International Conference on Computing and Applied Informatics, 2016, pp. 1–6.

J. W. Tukey, “The Future of Data Analysis,” Ann. Math. Stat., 1962.

D. C. Hoaglin, “John W. Tukey and Data Analysis,” Stat. Sci., 2004.

D. S. Maylawati, H. Aulawi, and M. A. Ramdhani, “Flexibility of Indonesian text pre-processing library,” Indones. J. Electr. Eng. Comput. Sci., 2019.

B. S. Harish, D. S. Guru, and S. Manjunath, “Representation and classification of text documents: A brief review,” IJCA, Spec. Issue Recent Trends Image Process. Pattern Recognit., no. 2, pp. 110–119, 2010.

W. Jin and R. K. Srihari, “Graph-based text representation and knowledge discovery,” SAC, p. 807, 2007.

W. Pu, N. Liu, S. Yan, J. Yan, K. Xie, and Z. Chen, “Local word bag model for text categorization,” in Proceedings - IEEE International Conference on Data Mining, ICDM, 2007, pp. 625–630.

H. M. Wallach, “Topic Modeling: Beyond Bag-of-Words,” ICML, no. 1, pp. 977–984, 2006.

A. Sethy and B. Ramabhadran, “Bag-of-word normalized n-gram models,” in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 2008, pp. 1594–1597.

D. S. Maylawati, M. A. Ramdhani, A. Rahman, and W. Darmalaksana, “Incremental technique with set of frequent word item sets for mining large Indonesian text data,” 2017 5th Int. Conf. Cyber IT Serv. Manag. CITSM 2017, pp. 1–6, 2017.

M. Kageback, O. Mogren, N. Tahmasebi, and D. Dubhashi, “Extractive Summarization using Continuous Vector Space Models,” Proc. 2nd Work. Contin. Vector Sp. Model. their Compos., 2014.

Y. Zhang, M. Lease, and B. C. Wallace, “Active Discriminative Text Representation Learning,” Proc. Thirty-First AAAI Conf. Artif. Intell., pp. 3386–3392, 2016.

K. Zupanc and Z. Bosnić, “Automated essay evaluation with semantic analysis,” Knowledge-Based Syst., 2017.

S. T. Dumais, “Latent semantic analysis,” Annu. Rev. Inf. Sci. Technol., vol. 38, no. 1, pp. 188–230, 2005.

J. Y. Yeh, H. R. Ke, W. P. Yang, and I. H. Meng, “Text summarization using a trainable summarizer and latent semantic analysis,” Inf. Process. Manag., vol. 41, no. 1, pp. 75–95, 2005.

H. Schuetze, H. Adel, and E. Asgari, “Nonsymbolic Text Representation,” Proc. 15th Conf. Eur. Chapter Assoc. Comput. Linguist. Comput. Linguist., vol. 1, pp. 785–796, 2017.

K. N. Singh and H. M. Devi, “Document representation techniques and their effect on the document Clustering and Classification : A Review,” Int. J. Adv. Res. Comput. Sci., vol. 8, no. 5, pp. 1780–1784, 2017.

L. A. Gatys, A. S. Ecker, and M. Bethge, “Image Style Transfer Using Convolutional Neural Networks,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016.

S. Sena, “Pengenalan Deep Learning Part 7 : Convolutional Neural Network (CNN),” Medium.com, 2017. [Online]. Available: https://medium.com/@samuelsena/pengenalan-deep-learning-part-7-convolutional-neural-network-cnn-b003b477dc94. [Accessed: 02-Aug-2019].

I. W. Suartika E. P, “Klasifikasi Citra Menggunakan Convolutional Neural Network (Cnn) Pada Caltech 101,” J. Tek. ITS, vol. 5, no. 1, p. 76, 2016.

D. S. Maylawati, H. Aulawi, and M. A. Ramdhani, “Flexibility of Indonesian text pre-processing library,” Indones. J. Electr. Eng. Comput. Sci., vol. 13, no. 1, pp. 420–426, 2019.

D. S. Maylawati, W. B. Zulfikar, C. Slamet, and M. A. Ramdhani, “An Improved of Stemming Algorithm for Mining Indonesian Text with Slang on Social Media,” in The 6th International Conference on Cyber and IT Service Management (CITSM 2018), 2018.

H. Zamani and W. B. Croft, “Relevance-based Word Embedding,” in Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR ’17, 2017.

M. Naili, A. H. Chaibi, and H. H. Ben Ghezala, “Comparative study of word embedding methods in topic segmentation,” in Procedia Computer Science, 2017.

M. Kamkarhaghighi and M. Makrehchi, “Content Tree Word Embedding for document representation,” Expert Syst. Appl., 2017.

X. Wu, X. Zhu, G. Q. Wu, and W. Ding, “Data mining with big data,” IEEE Trans. Knowl. Data Eng., 2014.

I. H. Witten, E. Frank, M. A. Hall, and C. J. Pal, Data Mining: Practical Machine Learning Tools and Techniques, Fourth Edi. United States of America: Morgan Kaufman, 2012.

F. A. Hermawati, Data Mining. Yogyakarta: Andi, 2013.

A. Lamani, B. Erraha, M. Elkyal, and A. Sair, “Data mining techniques application for prediction in OLAP cube,” Int. J. Electr. Comput. Eng., 2019.




DOI: http://doi.org/10.11591/ijeecs.v19.i2.pp%25p
Total views : 13 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

shopify stats IJEECS visitor statistics