Depression recognition over fusion of visual and vocal expression using artificial intelligence

Chandan Gautam, Aaradhya Raj, Bhargavee Nemade, Vinaya Gohokar


Depression is a mental illness that usually goes untreated in people and can have catastrophic consequences, including suicidal thoughts. Counselling services are widely available, but because depression is a stigmatized illness, many people who are depressed decide not to seek help. Therefore, it is essential to develop an automated system that can recognize depression in individuals before it worsens. In this study, a novel approach is proposed for identifying depression using a combination of visual and vocal emotions. Long short-term memory (LSTM) is used to assess verbal expressions and convolutional neural networks (CNN) to analyze facial expressions. The proposed system is trained using features of depression from the distress analysis interview corpus (DAIC) dataset and tested on videos of college students with frontal faces. The proposed approach is effective in detecting depression in individuals, with high accuracy and reliability.


CNN; DAIC dataset; Depression scale recognition; LSTM; Visual expression; Vocal expression

Full Text:




  • There are currently no refbacks.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

The Indonesian Journal of Electrical Engineering and Computer Science (IJEECS)
p-ISSN: 2502-4752, e-ISSN: 2502-4760
This journal is published by the Institute of Advanced Engineering and Science (IAES) in collaboration with Intelektual Pustaka Media Utama (IPMU).

shopify stats IJEECS visitor statistics