International Journal of Science, Engineering and TechnologyAuthors: Ms. Shrutika Suresh More, Ms. Shridevi Amol Nandi
Abstract: Emotion recognition is a pivotal component in the evolution of Human-Computer Interaction (HCI) and digital mental health. Traditional unimodal systems, which rely solely on text or facial expressions, often suffer from low reliability due to the subjective and complex nature of human expression. This paper presents a robust, computationally efficient multimodal framework for Mood Swing Analysis by integrating Natural Language Processing (NLP), Computer Vision, and Spectral Audio Analysis. Our architecture utilizes a multinomial Logistic Regression model for textual sentiment, alongside two distinct 18-layer Deep Residual Convolutional Neural Networks (ResNet18) optimized for facial expressions and Mel-spectrogram-based voice analysis. The unified system is deployed via a Django web framework, offering a centralized interface for triple-modality emotion detection. Experimental validations demonstrate that the late fusion of these distinct modalities mitigates context-ignorance, resolves cross-modal conflicts, and significantly enhances overall classification accuracy on standard hardware footprints.
DOI: http://doi.org/10.5281/zenodo.21308797
