Recognizing Human Emotions in Images Using Deep Learning and the KDEF Dataset

Project Objective

To build a deep learning model capable of identifying human facial expressions based on images, with the goal of supporting applications in psychology, human-computer interaction, and emotionally aware systems.

Notebook: https://www.kaggle.com/code/paolaandreaortiz/final

The Problem

Recognizing emotions from facial expressions is usually a subjective task, especially when done by humans. This makes it hard to scale or integrate into automated systems. The challenge was to design a model that could accurately classify emotions from facial images and provide consistent results.

The Proposed Solution

A convolutional neural network (CNN) trained on grayscale facial images from the KDEF dataset. The model was designed to classify each image into one of seven core emotions:

Anger (AF), Disgust (DI), Fear (FE), Happiness (HA), Neutral (NE), Sadness (SA), Surprise (SU).

Methodology

5.1 Dataset: KDEF (Karolinska Directed Emotional Faces)

• Labeled facial images from actors expressing seven distinct emotions

• Each emotion category contains multiple images from different angles and genders

5.2 Preprocessing Steps

• Converted all images to grayscale to simplify input

• Resized to 48x48 pixels for consistency

• Normalized pixel values to a [0, 1] scale

• One-hot encoded the emotion labels for training

5.3 Data Split

• 70% training

• 15% validation

• 15% testing

• Performed with stratification to maintain emotion class balance

5.4 Model Architecture

• Custom CNN built using TensorFlow/Keras

• Stacked Conv2D and MaxPooling layers, with Dropout for regularization

• Activation functions: ReLU (hidden layers), Softmax (output layer)

• Optimizer: Adam

• Loss Function: Categorical Crossentropy

5.5 Evaluation Metrics

• Overall accuracy

• Class-wise confusion matrix to identify emotion-specific performance

• Observed trends in misclassification (e.g., fear vs. surprise)

Results

• Achieved strong classification accuracy across most emotions

• Best performance on distinct expressions like happiness and surprise

• Lower precision on subtle emotions like fear and disgust

• The model proved capable of generalizing to new faces with minimal tuning

Challenges and Learnings

Challenges: Differentiating visually similar emotions, limited dataset size, class imbalance

Learnings: The impact of preprocessing on performance, importance of dropout layers to reduce overfitting, and how facial structure affects classification

Tools and Technologies

• Language: Python

• Libraries: TensorFlow, Keras, NumPy, OpenCV, Matplotlib

• Dataset: KDEF

Potential Applications

Emotion-aware virtual assistants

Educational tools for emotional development

Feedback systems for UX testing

Adaptive gaming environments

Mental health screening platforms

Conclusion

This project shows how deep learning can be used to interpret human emotion through facial analysis. The model demonstrated that CNNs are capable of capturing emotional cues in facial expressions with reasonable accuracy, making it a useful foundation for emotionally responsive systems in tech, education, and healthcare.