Bangla language textual image description by hybrid neural network model
Md. Asifuzzaman Jishan, Khan Raqib Mahmud, Abul Kalam Al Azad, Mohammad Rifat Ahmmad Rashid, Bijan Paul, Md. Shahabub Alam
Abstract
Automatic image captioning task in different language is a challenging task which has not been well investigated yet due to the lack of dataset and effective models. It also requires good understanding of scene and contextual embedding for robust semantic interpretation of images for natural language image descriptor. To generate image descriptor in Bangla, we created a new Bangla dataset of images paired with target language label, named as Bangla Natural Language Image to Text (BNLIT) dataset. To deal with the image understanding, we propose a hybrid encoder-decoder model based on encoder-decoder architecture and the model is evaluated on our newly created dataset. This proposed approach achieves significance performance improvement on task of semantic retrieval of images. Our hybrid model uses the Convolutional Neural Network as an encoder whereas the Bidirectional Long Short Term Memory is used for the sentence representation that decreases the computational complexities without trading off the exactness of the descriptor. The model yielded benchmark accuracy in recovering Bangla natural language and we also conducted a thorough numerical analysis of the model performance on the BNLIT dataset.
Keywords
Convolutional Neural Network; Hybrid Recurrent Neural Network; Long Short-Term Memory; Bi-directional Recurrent Neural Network; Bangla Natural Language Descriptors
DOI:
http://doi.org/10.11591/ijeecs.v21.i2.pp757-767
Refbacks
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Indonesian Journal of Electrical Engineering and Computer Science (IJEECS)
p-ISSN: 2502-4752, e-ISSN: 2502-4760
This journal is published by the Institute of Advanced Engineering and Science (IAES) in collaboration with Intelektual Pustaka Media Utama (IPMU).
IJEECS visitor statistics