Facial expression databases


A facial expression database is a collection of images or video clips with facial expressions of a range of emotions.
Well-annotated media content of facial behavior is essential for training, testing, and validation of algorithms for the development of expression recognition systems. The emotion annotation can be done in discrete emotion labels or on a continuous scale. Most of the databases are usually based on the basic emotions theory which assumes the existence of six discrete basic emotions. However, some databases include the emotion tagging in continuous arousal-valence scale.
In posed expression databases, the participants are asked to display different basic emotional expressions, while in spontaneous expression database, the expressions are natural. Spontaneous expressions differ from posed ones remarkably in terms of intensity, configuration, and duration. Apart from this, synthesis of some AUs are barely achievable without undergoing the associated emotional state. Therefore, in most cases, the posed expressions are exaggerated, while the spontaneous ones are subtle and differ in appearance.
Many publicly available databases are categorized here. Here are some details of the facial expression databases.
DatabaseFacial expressionNumber of SubjectsNumber of images/videosGray/ColorResolution, Frame rateGround truthType
FERG-3D-DB for stylized charactersangry, disgust, fear, joy, neutral, sad, surprise439574 annotated examplesColorEmotion labelsFrontal pose-
Ryerson Audio-Visual Database of Emotional Speech and Song Speech: Calm, happy, sad, angry, fearful, surprise, disgust, and neutral.
Song: Calm, happy, sad, angry, fearful, and neutral.
Each expression at two levels of emotional intensity.		24 7356 video and audio filesColor1280x720 Facial expression labels
Ratings provided by 319 human raters		Posed-
Extended Cohn-Kanade Dataset neutral, sadness, surprise, happiness, fear, anger, contempt and disgust123 593 image sequences Mostly gray640* 490Facial expression labels and FACS Posed; spontaneous smiles-
Japanese Female Facial Expressions neutral, sadness, surprise, happiness, fear, anger, and disgust10213 static imagesGray256* 256Facial expression labelPosed-
MMI Database431280 videos and over 250 imagesColor720* 576AU label for the image frame with apex facial expression in each image sequencePosed and Spontaneous-
Belfast DatabaseSet 1 114570 video clipsColor720*576Natural Emotion-
Belfast DatabaseSet 2 82650 video clipsColorNatural Emotion-
Belfast DatabaseSet 3 60180 video clipsColor1920*1080Natural Emotion-
iSAFE Happy, Sad, Fear, Surprise, Angry, Neutral, Disgust44395 clipsColor1920x1080
Emotion labelsSpontaneous-
DISFA-274,845 video framesColor1024*768; 20 fpsAU intensity for each video frame Spontaneous-
Multimedia Understanding Group neutral, sadness, surprise, happiness, fear, anger, and disgust861462 sequencesColor896*896, 19fpsEmotion labelsPosed-
Indian Spontaneous Expression Database sadness, surprise, happiness, and disgust50428 videos Color1920* 1080, 50 fpsEmotion labelsSpontaneous-
Radboud Faces Database neutral, sadness, contempt, surprise, happiness, fear, anger, and disgust67Three different gaze directions and five camera angles Color681*1024Emotion labelsPosed-
Oulu-CASIA NIR-VIS databasesurprise, happiness, sadness, anger, fear and disgust80three different illumination conditions: normal, weak and dark Color320×240Posed-
FERG -DB for stylized charactersangry, disgust, fear, joy, neutral, sad, surprise655767Color768x768Emotion labelsFrontal pose-
AffectNetneutral, happy, sad, surprise, fear, disgust, anger, contempt~450,000 manually annotated
~ 500,000 automatically annotated		ColorVariousEmotion labels, valence, arousalWild setting-
IMPA-FACE3Dneutral frontal, joy, sadness, surprise, anger, disgust, fear, opened, closed, kiss, left side, right side, neutral sagittal left, neutral sagittal right, nape and forehead 38534 static imagesColor640X480Emotion labelsPosed-
FEI Face Databaseneutral,smile2002800 static imagesColor640X480Emotion labelsPosed-
Aff-Wildvalence and arousal200~1,250,000 manually annotatedColorVarious Valence, ArousalIn-the-Wild setting-
Aff-Wild2neutral, happiness, sadness, surprise, fear, disgust, anger + valence-arousal + action units 1,2,4,6,12,15,20,25458~2,800,000 manually annotatedColorVarious Valence, Arousal, 7 basic expressions, action units for each video frameIn-the-Wild setting-