Human Mood Prediction Using Landscape Photographs with VGG16 and CuDNN- LSTM 5 folds Attention Algorithms

Abstract

Mood's state depicts human personality traitsdsdsd and behavior in every situation. Predicting mood is a complex problem that deals with interactions of human psychology, subconscious mind thoughts, and their connection with the environment. This research proposes a deep learning-based solution to deal with such problems of the functional aspects of facial expressions and body gestures. Still, the scope of connecting psychology and the environment is yet unsolved. Mood prediction using landscape photographs is a unique computer vision problem that solves abstract image analysis issues. Human moods are always dynamic and dependent on the surroundings and their interactions. Moods can easily predict the relevance and requirement of any situation and its improvements, though very few computer vision architectures have solved this problem in a detailed and accurate manner. This work uses a unique and robust architecture of Image captioning with sentiment analysis to solve the problem correctly and production-ready. To analyze the model performance, 7440 landscape pictures were collected from different sources mentioned in the dataset section. After the data cleaning and annotation, 6912 landscape pictures for Image captioning were sent to the model, eliminating 528 images due to duplicates. As a result, VGG 16 LSTM trained on an unseen dataset, which attained training and testing accuracy of 94.5% and 87.9%, respectively. Further, CuDNN LSTM (5-fold attention) is used for the sentiment analysis on the same dataset and achieved training and testing accuracy of 93% and 81.1%, respectively