Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
90
datasets available to search
ShareScore release 0.9.0
Dataset results
90 results for “facial expressions”
EEG Data for Emotive Response to Robot Facial Expressions
<p>This dataset consists of EEG recorded during visual human-robot interaction from 10 healthy participants to investigate the emotive response in EEG to different robot facial expressions. Participants observed four different facial expressions (angry, happy, sad and surprised along with neutral expression) displayed by the social robot Miko on its digital screen. EEG was recorded from 16 unipolar channels in frontal, central, temporal, parietal, and occipital locations . During each trial, an emotion stimulus was displayed for approximately 4s followed by 4s break during which the Miko robot displayed neutral expression and blinked regularly. Emotions were displayed in random order. Total of 240 EEG trials were recorded from each participant with 60 trials per emotion. The dataset provides raw minimally filtered EEG along with cleaned EEG with artefacts removal using ICA with sampling frequency of 128 Hz, and corresponding stimulus onset markers. Please refer to README file for further details and example code.</p> <p><em>Please cite the original publication:</em></p> <p>M. Wairagkar et al., "Emotive Response to a Hybrid-Face Robot and Translation to Consumer Social Robots," <em>IEEE Internet of Things Journal</em>, DOI: <a href="https://doi.org/10.1109/JIOT.2021.3097592">10.1109/JIOT.2021.3097592</a>.</p> <p><em>Preprint: </em></p> <p>M. Wairagkar et al., "Emotive Response to a Hybrid-Face Robot and Translation to Consumer Social Robots," <a href="https://arxiv.org/abs/2012.04511">arXiv:2012.04511</a></p>
Identification of emotional facial expressions in a lab and over the internet
<p>This dataset includes the data used for the analyses presented in the paper published (under the same title as this dataset) in the journal Psychology, Journal of the Higher School of Economics. </p> <p>Abstract of the publication:</p> <p>Collecting data over the internet is an approach that allows researchers to vastly expand the possible sample sizes of their studies, and enables the study of populations that may otherwise be difficult to access. However, to ensure that data collected over the internet is of the same level of quality as data collected in a lab, the comparability of internet-collected data with lab-collected data must first be assessed for individual areas of research and experimental approaches. To answer the question of whether internet data collection is suitable for experiments involving facial expressions, we conducted a deliberately difficult facial emotion-identification experiment where participants completed the same task either under supervision in our lab, or at an unsupervised location over the internet. Stimuli consisted of sad faces that participants were asked to identify as resembling either anger, fear, or disgust. Regardless of belonging to either the group tested in the lab or over the internet, participants showed highly similar response distributions, while differences between the groups were non-significant and of very low magnitude. We can therefore conclude from our findings that internet data collection is a viable method for experiments requiring the identification of emotional facial expressions, being able to produce similar results to those which can be obtained in a lab.</p>
Data and supplementary material in support of article "Can robots express facial emotions dominantly enough for use in dementia care?"
<p>Data and supplementary material in support of article "Vlachos, E. and Tan, Z. H. (2020). Can robots express facial emotions dominantly enough for use in dementia care?, <em>International Psychogeriatrics</em>, Cambridge University Press". </p> <p>Our objective is to evaluate the recognition, and denomination of the six basic emotional facial expressions as displayed by a social robot to persons with dementia, and to compare it with the results from the evaluation of static photographs of humans from the Paul Ekman database in order to investigate the differences in recognition rates among the two stimuli.</p>
EEG and eye-tracking data from a go/no-go saccadic task based on facial expression cues
<p>Electroencephalographic (EEG) and eye-tracking data from 20 healthy individuals who performed a go/no-go saccadic task based on facial expression cues aimed at studying error monitoring processes.</p> <p> </p> <p>Version 2 includes only the EEG data, but with triggers of correct and erroneous actions. Version 2 has an error that was corrected for version 3.</p> <p> </p> <p>EEG Triggers</p> <table> <tbody> <tr> <td> <p><span>19</span></p> </td> <td> <p><span>Eye tracker starts recording</span></p> </td> </tr> <tr> <td> <p><span>1</span></p> </td> <td> <p><span>Beginning of each trial</span></p> </td> </tr> <tr> <td> <p><span>200</span></p> </td> <td> <p><span>Gap between neutral and instruction</span></p> </td> </tr> <tr> <td> <p><span>99</span></p> </td> <td> <p><span>Fixation cross between instruction and saccade</span></p> </td> </tr> <tr> <td> <p><span>2</span></p> </td> <td> <p><span>Instruction no-go happy</span></p> </td> </tr> <tr> <td> <p><span>21</span></p> </td> <td> <p><span>Target left no-go happy</span></p> </td> </tr> <tr> <td> <p><span>22</span></p> </td> <td> <p><span>Target right no-go happy</span></p> </td> </tr> <tr> <td> <p><span>121</span></p> </td> <td> <p><span>Response period for no-go happy after target left</span></p> </td> </tr> <tr> <td> <p><span>122</span></p> </td> <td> <p><span>Response period for no-go happy after target right</span></p> </td> </tr> <tr> <td> <p><span>3</span></p> </td> <td> <p><span>Instruction no-go sad</span></p> </td> </tr> <tr> <td> <p><span>31</span></p> </td> <td> <p><span>Target left no-go sad</span></p> </td> </tr> <tr> <td> <p><span>32</span></p> </td> <td> <p><span>Target right no-go sad</span></p> </td> </tr> <tr> <td> <p><span>131</span></p> </td> <td> <p><span>Response period for no-go sad after target left</span></p> </td> </tr> <tr> <td> <p><span>132</span></p> </td> <td> <p><span>Response period for no-go sad after target right</span></p> </td> </tr> <tr> <td> <p><span>4</span></p> </td> <td> <p><span>Instruction pro-right</span></p> </td> </tr> <tr> <td> <p><span>42</span></p> </td> <td> <p><span>Target right pro-right</span></p> </td> </tr> <tr> <td> <p><span>142</span></p> </td> <td> <p><span>Response period for pro-right</span></p> </td> </tr> <tr> <td> <p><span>5</span></p> </td> <td> <p><span>Instruction pro-left</span></p> </td> </tr> <tr> <td> <p><span>51</span></p> </td> <td> <p><span>Target left pro-left</span></p> </td> </tr> <tr> <td> <p><span>151</span></p> </td> <td> <p><span>Response period for pro-left</span></p> </td> </tr> <tr> <td> <p><span>6</span></p> </td> <td> <p><span>Instruction anti-left</span></p> </td> </tr> <tr> <td> <p><span>61</span></p> </td> <td> <p><span>Target left anti-left</span></p> </td> </tr> <tr> <td> <p><span>161</span></p> </td> <td> <p><span>Response period for anti-left</span></p> </td> </tr> <tr> <td> <p><span>7</span></p> </td> <td> <p><span>Instruction anti-right</span></p> </td> </tr> <tr> <td> <p><span>72</span></p> </td> <td> <p><span>Target right anti-right</span></p> </td> </tr> <tr> <td> <p><span>172</span></p> </td> <td> <p><span>Response period for anti-right</span></p> </td> </tr> <tr> <td> <p><span>199</span></p> </td> <td> <p><span>Final fixation period (1.5s black screen after last saccade)</span></p> </td> </tr> <tr> <td> <p><span>190</span></p> </td> <td> <p><span>Eye tracker stops recording</span></p> </td> </tr> <tr> <td> <p><span>proCorr</span></p> </td> <td> <p><span>Beginning of saccade in the correct direction following a pro-saccade intruction</span></p> </td> </tr> <tr> <td> <p><span>proErr</span></p> </td> <td> <p><span>Beginning of saccade in the erroneous direction following a pro-saccade intruction</span></p> </td> </tr> <tr> <td> <p><span>antiCorr</span></p> </td> <td> <p><span>Beginning of saccade in the correct direction following a anti-saccade intruction</span></p> </td> </tr> <tr> <td> <p><span>antiErr</span></p> </td> <td> <p><span>Beginning of saccade in the erroneous direction following a anti-saccade intruction</span></p> </td> </tr> <tr> <td> <p><span>nogoErr</span></p> </td> <td> <p><span>Beginning of saccade in the erroneous direction following a no-go intruction</span></p> </td> </tr> </tbody> </table>
UIBVFEDPlus-Light: Virtual facial expression dataset with lighting
<p>This database, named UIBVFEDPlus-Light, is an extension of the previously published UIBVFED virtual facial expression dataset. It includes 100 characters, four lighting configurations and 13200 images. Images are in png format with a resolution of 1080x1920 RGB, without alpha channel and an average size of 2.0 MB.<br>The images represent virtual characters reproducing FACS-based facial expressions. Expressions are classified based on the six universal emotions (Anger, Disgust, Fear, Joy, Sadness, and Surprise) labeled according to Faigin’s classification.<br>The dataset aims to give researchers access to data they may use to support their research and generate new knowledge. In particular, to study the effect of lighting conditions in the fields of facial expression and emotion recognition.</p>
Research Data: Facial Expression Recognition under Visual Field Restriction
<p>This dataset contains the following files:</p> <p><strong>-</strong> <strong>view_trial.xlsx:</strong> Excel spreadsheet containing data from individual trials.<br><strong>-</strong> <strong>view_participant.xlsx:</strong> Excel spreadsheet containing data aggregated at the participant level.<br><strong>- consensus.xlsx:</strong> Excel spreadsheet containing consensus data analysis.<br><strong>- image_id_list.txt:</strong> Text file listing the IDs of the images used in the study from The Karolinska Directed Emotional Faces (KDEF); https://kdef.se/.</p> <p>These files provide comprehensive data used in the research project titled "Exploring the Visual Field Restriction in the Recognition of Basic Facial Expressions: A Combined Eye Tracking and Gaze Contingency Study" conducted by M. B. Urtado, R. D. Rodrigues, and S. S. Fukusima. The dataset is intended for analysis and replication of the study's findings.</p> <p>Please, when using these data, we kindly request citing the following article:<br>Urtado, M.B.; Rodrigues, R.D.; Fukusima, S.S. <strong>Visual Field Restriction in the Recognition of Basic Facial Expressions: A Combined Eye Tracking and Gaze Contingency Study</strong>. <em>Behavioral Sciences</em> <strong>2024</strong>, <em>14</em>, 355. <a href="https://doi.org/10.3390/bs14050355">https://doi.org/10.3390/bs14050355</a></p> <p>The study was approved by the Research Ethics Committee (CEP) of the University of São Paulo (protocol code 41844720.5.0000.5407). </p>
Data and trained models for: Human-robot facial co-expression
<p>Large language models are enabling rapid progress in robotic verbal communication, but nonverbal communication is not keeping pace. Physical humanoid robots struggle to express and communicate using facial movement, relying primarily on voice. The challenge is twofold: First, the actuation of an expressively versatile robotic face is mechanically challenging. A second challenge is knowing what expression to generate so that they appear natural, timely, and genuine. Here we propose that both barriers can be alleviated by training a robot to anticipate future facial expressions and execute them simultaneously with a human. Whereas delayed facial mimicry looks disingenuous, facial co-expression feels more genuine since it requires correctly inferring the human's emotional state for timely execution. We find that a robot can learn to predict a forthcoming smile about 839 milliseconds before the human smiles, and using a learned inverse kinematic facial self-model, co-express the smile simultaneously with the human. We demonstrate this ability using a robot face comprising 26 degrees of freedom. We believe that the ability co-express simultaneous facial expressions could improve human-robot interaction.</p>
A Facial Motion Capture System Based on Neural Network Classifier Using RGB-D Data-Figure 8. 3D model of some facial expressions
<p>Face region is separated precisely from video frames by using a segmentation method based on skin color. The depth data corresponding to this separated area is taken for a 3D representation from depth data corresponding to each frame. At the end, a file is prepared for each frame consisting of face points with 6 features: X, Y, depth, red, green and blue color. These data are used for producing a 3D model and a graphical avatar for each frame (Figure 7). Figure 8 shows 3D model of some facial expressions.</p>
A Facial Motion Capture System Based on Neural Network Classifier Using RGB-D Data-Figure 3. Feature vectors of facial expression in database
<p> Figure 3 shows feature vectors of facial expression of our database. Matrices ‘U’ and ‘V’ values that are obtained from this algorithm are used as feature vectors. The ‘U’ matrix represents the position and the ‘V’ matrix represents the change of direction. In the following, the proposed method is combined with some other feature extraction methods (LBP uniform approach and LBP circular approach) and the obtained results will be mentioned.</p>
A Facial Motion Capture System Based on Neural Network Classifier Using RGB-D Data-Figure 2. Facial expression recognition in proposed method
<p>In this stage, a video is prepared using the color data captured from Kinect camera. The face region in each frame is obtained from the video using Viola-Jones algorithm (Figure 2). Because of different distance from the Kinect camera, the obtained images from the face must be re-sized, in order to have the same size. At the end, the colored images are converted to gray-scaled images.</p>
A Facial Motion Capture System Based on Neural Network Classifier Using RGB-D Data-Figure 1. Peak of facial expression in database
<p>In this study, data are obtained from the Kinect camera that benefits from colorful images and depth data. Kinect can record colorful and depth data simultaneously at 30 frames per second. The data are collected from the person who initially pose in front of the camera with normal face mode and then the various modes are represented. It should be noted that data are obtained at different distances from the Kinect camera and in different lighting conditions. Figure 1 shows various facial modes in our database.</p>
Facial Expression and Landmark Tracking (FELT) dataset
<p><strong>Contact Information</strong></p> <p>If you would like further information about the Facial expression and landmark tracking data set, or if you experience any issues downloading files, please contact us at <a href="mailto:ravdess@gmail.com?subject=RAVDESS%20feedback%20from%20Zenodo">ravdess@gmail.com</a>. </p> <p><strong>Facial Expression examples</strong></p> <p>Watch a sample of the <a href="https://youtu.be/OA5gZ6iycPc">facial expression tracking results</a>.</p> <p><strong>Commercial Licenses</strong></p> <p>Commercial licenses for this dataset can be purchased. For more information, please contact us at <a href="mailto:ravdess@gmail.com?subject=RAVDESS%20Commercial%20License">ravdess@gmail.com</a>.</p> <p><strong>Description</strong></p> <p>The <a href="https://affectivedatascience.com/datasets.html#felt">Facial Expression and Landmark Tracking (FELT) dataset</a> dataset contains tracked facial expression movements and animated videos from the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) [<a href="../record/1188976">RAVDESS Zenodo page</a>]. Tracking data and videos were produced by Py-Feat 0.6.2 (2024-03-29 release) (Cheong, J.H., Jolly, E., Xie, T. <em>et al.</em> Py-Feat: Python Facial Expression Analysis Toolbox. <em>Affec Sci</em> <strong>4</strong>, 781–796 (2023). https://doi.org/10.1007/s42761-023-00191-4) and custom code (<a href="https://github.com/harveyliao/Py-feat-RAVDESS">github repo</a>). Tracked information includes: facial emotion classification estimates, facial landmark detection (68 points), head pose estimation (yaw, pitch, roll, x, y), and facial Action Unit (AU) recognition. Videos include: landmark overlay videos, AU activation animations, and landmark plot animations.</p> <p>The FELT dataset was created at the <a href="https://affectivedatascience.com">Affective Data Science Lab</a>.</p> <p>This dataset contains tracking data and videos for all 2452 RAVDESS trials. Raw and smoothed tracking data are provided. All tracking movement data are contained in the following archives: raw_motion_speech.zip, smoothed_motion_speech.zip, raw_motion_song.zip, and smoothed_motion_song.zip. Each actor has 104 tracked trials (60 speech, 44 song). Note, there are no song files for Actor 18.</p> <p>Total Tracked Files = (24 Actors x 60 Speech trials) + (23 Actors x 44 Song trials) = 2452 CSV files.</p> <p>Tracking results for each trial are provided as individual comma separated value files (CSV format). File naming convention of raw and smoothed tracked files is identical to that of the RAVDESS. For example, smoothed tracked file "01-01-01-01-01-01-01.csv" corresponds to RAVDESS audio-video file "01-01-01-01-01-01-01.mp4". For a complete description of the RAVDESS file naming convention and experimental manipulations, please see the <a href="../record/1188976">RAVDESS Zenodo page</a>. </p> <p>Landmark overlays, AU activation, and landmark plot videos for all trials are also provided (720p h264, .mp4). Landmark overlays present tracked landmarks and head pose overlaid on the original RAVDESS actor video. As the RAVDESS does not contain "ground truth" facial landmark locations, the overlay videos provide a visual 'sanity check' for researchers to confirm the general accuracy of the tracking results. Landmark plot animations present landmarks only, anchored to the top left corner of the head bounding box with translational head motion removed. AU activation animations visualize intensity of AU activations (0-1 normalized) as a heatmap over time. The file naming convention of all videos also matches that of the RAVDESS. For example, "Landmark_Overlay/01-01-01-01-01-01-01.mp4", "Landmark_Plot/01-01-01-01-01-01-01.mp4", "ActionUnit_Animation/01-01-01-01-01-01-01.mp4", all correspond to RAVDESS audio-video file "01-01-01-01-01-01-01.mp4".</p> <p><strong>Smoothing procedure<br></strong></p> <p>Raw tracking data were first low-pass filtered with a 5th order butterworth filter (cutoff_freq = 6, sampling_freq = 29.97, order = 5) to remove high-frequency noise. Data were then smoothed with a Savitzky-Golay filter (window_length = 11, poly_order = 5). Scipy.signal (v 1.13.1) was used for both procedures.</p> <p><strong>Landmark Tracking models</strong></p> <p>Six separate machine learning models were used by Py-Feat to perform various aspects of tracking and classification. Video outputs generated by different combinations of ML models were visually compared, with final model choice determined by voting of first and second authors. Models were specified in the call to <code>Detector</code> class (<a href="https://py-feat.org/basic_tutorials/02_detector_imgs.html?highlight=facescore">described here</a>). Exact function call as follows:</p> <p><code> Detector(face_model='img2pose',</code><br><code> landmark_model='mobilenet',</code><br><code> au_model='xgb',</code><br><code> emotion_model='resmasknet',</code><br><code> facepose_model='img2pose-c',</code><br><code> identity_model='facenet',</code><br><code> device='cuda',</code><br><code> n_jobs=1,</code><br><code> verbose=False,</code><br><code> )</code></p> <p>Default Py_feat parameters to each model were used in most cases. Non-defaults were specified in the call to <code>detect_video</code> function (<a href="https://py-feat.org/pages/api.html?highlight=detect_video">described here</a>). Exact function call as follows:<br><br><code> (video_path,</code><br><code> skip_frames=None,</code><br><code> output_size=(720, 1280),</code><br><code> batch_size=5,</code><br><code> num_workers=0,</code><br><code> pin_memory=False,</code><br><code> face_detection_threshold=0.83,</code><br><code> face_identity_threshold=0.8</code><br><code> )</code></p> <p><strong>Tracking File Output Format</strong></p> <p>This data set retained Py-Feat's data output format. The resolution of all input videos was 1280x720. Tracking output units are in pixels, their range of values is (0,0) (top left corner) to (1280,720) (bottom right corner). </p> <p><em>Column 1 = Timing information</em></p> <ul> <li>1. frame - The number of the frame (source videos 29.97 fps), range = 1 to n</li> </ul> <p><em>Columns 2-5 = Head bounding box<br></em></p> <ul> <li>2-3. FaceRectX, FaceRectY - X and Y coordinates of top-left corner of head bounding box (pixels)</li> <li>4-5. FaceRectWidth, FaceRectHeightF - Width and Height of head bounding box (pixels)</li> </ul> <p><em>Column 6 = Face detection confidence<br></em></p> <ul> <li>FaceScore - Confidence level that a human face was deteceted, range = 0 to 1</li> </ul> <p><em>Columns 7-142 = Facial landmark locations in 2D<br></em></p> <ul> <li>7-142. x_0, ..., x_67, y_0,...y_67 - Location of 2D landmarks in pixels. A figure describing the landmark index <a href="https://raw.githubusercontent.com/TadasBaltrusaitis/OpenFace/master/imgs/landmark_scheme_68.png">can be found here</a>.</li> </ul> <p><em>Columns 143-145 = Head pose</em></p> <ul> <li>143-145. Pitch, Roll, Yaw - Rotation of the head in degrees (<a href="https://py-feat.org/basic_tutorials/02_detector_imgs.html?highlight=facerect#:~:text=single_face_prediction.poses%20%23%20(in%20degrees)">described here</a>). The rotation is in world coordinates with the camera being located at the origin.</li> </ul> <p><em>Columns 146-165 = </em><em>Facial Action Units</em></p> <p>Facial Action Units (AUs) are a way to describe human facial movements (Ekman, Friesen, and Hager, 2002) [<a href="https://en.wikipedia.org/wiki/Facial_Action_Coding_System">wiki link</a>]. More information on Py-Feat's implementation of AUs <a href="https://py-feat.org/pages/au_reference.html">can be found here</a>.</p> <ul> <li>145-150, 152-153, 155-158, 160-165. AU01, AU02, AU04, AU05, AU06, AU09, AU10, AU12, AU14, AU15, AU17, AU23, AU24, AU25, AU26, AU28, AU43 - Intensity of AU movement, range from 0 (no muscle contraction) to 1 (maximal muscle contraction).</li> <li>151, 154, 159. AU07, AU11, AU20 - Presence or absence of AUs, range 0 (absent, not detected) to 1 (present, detected).</li> </ul> <p><em>Columns 166-172 = Emotion classification confidence<br></em></p> <ul> <li>162-172. anger, disgust, fear, happiness, sadness, surprise, neutral - Confidence of classified emotion category, range 0 (0%) to 1 (100%) confidence.</li> </ul> <p><em>Columns 173-685 = Face identity score<br></em></p> <p>Identity of faces contained in the video were classified using the FaceNet model (<a href="https://arxiv.org/pdf/1503.03832">described here</a>). This procedure generates at 512 dimension Euclidean embedding space.</p> <ul> <li>173. Identity - Predicated individual identifyed in the RAVDESS video. Note, value is always Person_0, as each video only contains a single actor at all times (categorical).</li> <li>174-685. Identity_1, ..., Identity_512 - Face embedding vector used by FaceNet to perform facial identity matching.</li> </ul> <p><em>Column 686 = Input video<br></em></p> <ul> <li>686. frame - The number of the frame (source videos 29.97 fps), range = 1 to n</li> </ul> <p><em>Columns 687-688 = Timing information<br></em></p> <ul> <li>687. frame.1 - The number of the frame (source videos 29.97 fps), duplicated column, range = 1 to n</li> <li>688. approx_time - Approximate time of current frame (0.0 to x.x, in seconds)</li> </ul> <p><strong>Tracking videos<br></strong></p> <p>Landmark Overlay and Landmark Plot videos were produced with <code>plot_detections</code> function call (<a href="https://py-feat.org/pages/api.html?highlight=plot_detections#feat.data.Fex.plot_detections">described here</a>). This function generated invidual images for each frame, which were then compiled into a video using the imageio library (<a href="https://pypi.org/project/imageio/">described here</a>).</p> <p>AU Activation videos were produced with plot_face function call (<a href="https://py-feat.org/pages/api.html?highlight=plot_face#feat.plotting.plot_face">described here</a>). This function also generated invidual images for each frame, which were then compiled into a video using the imageio library. Some frames could not be correctly generated by Py-Feat, producing only the AU heatmap but failing to plot/locate facial landmarks. These frames were dropped prior to compositing the output video. Drop rate was approximately 10% of all frames, in each video. Dropped frames were distributed evenly across the video timeline (i.e. no apparent clustering).</p> <p><strong>License information</strong></p> <p>The RAVDESS Facial expression and landmark tracking data set is released under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC BY-NA-SC 4.0</a>.</p> <p><strong>How to cite the RAVDESS Facial Tracking data set</strong></p> <p><em>Academic citation </em><br>If you use the RAVDESS Facial Tracking data set in an academic publication, please cite both references: </p> <ol> <li>Liao, Z., Livingstone, SR., & Russo, FA. (2024). RAVDESS Facial expression and landmark tracking (Version 1.0.0) [Data set]. Zenodo. http://doi.org/10.5281/zenodo.13243600</li> <li>Livingstone SR, Russo FA (2018). The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English. PLoS ONE 13(5): e0196391. <a href="https://doi.org/10.1371/journal.pone.0196391">https://doi.org/10.1371/journal.pone.0196391</a>.</li> </ol> <p><em>All other attributions </em><br>If you use the RAVDESS Facial expression and landmark tracking dataset in a form other than an academic publication, such as in a blog post, data science project or competition, school project, or non-commercial product, please use the following attribution: "<a href="../records/13243600">RAVDESS Facial expression and landmark tracking</a>" by Liao, Livingstone, & Russo is licensed under <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC BY-NA-SC 4.0</a>.</p> <p><strong>Related Data sets</strong></p> <ul> <li>The Ryerson Audio-Visual Database of Emotional Speech and Song [<a href="../record/1188976">Zenodo project page</a>].</li> </ul> <p> </p>
IFEED: Interactive Facial Expression and Emotion Detection Dataset
<p>Interactive Facial Expression and Emotion Detection (IFEED) is an annotated dataset that can be used to train, validate, and test Deep Learning models for facial expression and emotion recognition. It contains pre-filtered and analysed images of the interactions between the six main characters of the Friends television series, obtained from the video recordings of the Multimodal EmotionLines Dataset (MELD).</p> <p>The images were obtained by decomposing the videos into multiple frames and extracting the facial expression of the correctly identified characters. A team composed of 14 researchers manually verified and annotated the processed data into several classes: Angry, Sad, Happy, Fearful, Disgusted, Surprised and Neutral.</p> <p>IFEED can be valuable for the development of intelligent facial expression recognition solutions and emotion detection software, enabling binary or multi-class classification, or even anomaly detection or clustering tasks. The images with ambiguous or very subtle facial expressions can be repurposed for adversarial learning. The dataset can be combined with additional data recordings to create more complete and extensive datasets and improve the generalization of robust deep learning models.</p>
Data and trained models for: Human-robot facial co-expression
Open the record for dataset details and reuse information.
Dateset for article: Pupillary responses to infant facial expressions
<p>Pupil data related to study: <strong>Pupillary responses to infant facial expressions </strong></p> <p>Dataset contains "raw" eye-tracking data including timeseries of pupil diameter (.gazedata, SA pupil gazefiles anon.zip). The output of automated pupil analyses are included as Excel files (.xls, SA pupil xls.zip). These have been further restructured for statistical analyses into SPSS data files (.sav, SA pupil spss.zip). </p> <p>Matlab code and SPSS syntax for the analyses have been published and documented in GitHub repository: yrttiahoS/<strong>sa-pupil-analysis</strong></p>
UIBVFED: Virtual facial expression dataset
<p>The dataset is composed of 660 facial images (1080x1920) from 20 virtual characters each creating 32 facial expressions. The avatars represent 10 men and 10 women, aged between 20 and 80, from different ethnicities. Expressions are classified by the six universal expressions according to Gary Faigin classification.</p>
Videos accompanying article "Facial Expressions for Sign Language Synthesis using FACSHuman and AZee"
<p>Accompanying videos for the article "Facial Expressions for Sign Language Synthesis using FACSHuman and AZee".</p> <ul> <li>all_action_units.mp4 - Shows mesh deformations for all the action units used.</li> <li>all_expressions.mp4 - Shows all the expressions synthesized using the study.</li> <li>big_threatening_hot.mp4 - Shows comparison of a sign with and without facial expressions.</li> </ul>
A fMRI dataset in response to large number of short natural dynamic facial expression videos
<pre>#A fMRI dataset in response to large number of short natural dynamic facial expression videos<br>Natural facial expressions dataset (NFED),a dataset of functional magnetic resonance imaging (fMRI) responses to 1,320 short (3s) facial expression video clips.NFED offers researchers fMRI data that enables them to investigate the neural mechanisms involved in processing emotional information communicated by facial expression videos in real-world environments.<br>The dataset contains raw data, pre-processed volume data,pre-processed surface data and suface-based analyzed data.<br>To get more details, please refer to the paper at {website} and the dataset at https://openneuro.org/datasets/ds005047<br><br>## Preprocess procedure<br>The MRI data were preprocessed by using Kay et al, combining code written in MATLAB and certain tools from FreeSurfer, SPM,and FSL(http://github.com/kendrickkay). We used FreeSurfer software (http://surfer.nmr.mgh.harvard.edu) to construct the pial and white surfaces of participants from the T1 volume. Additionally, we established an intermediate gray matter surface between the pial and the white surfaces for all participants.<br>**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/volume_pre-process/**<br>Detailed usage notes are available in codes, please read carefully and modify variables to satisfy your customed environment.<br>## GLM of main experiment<br>We utilized a single-trial General Linear Model (GLMsingle) (https://github.com/cvnlab/GLMsingle) approach, an advanced denoising approach in MATLAB R2019a, to model the pre-processed fMRI data from main experiment. For single trials, the method of GLM was developed to offer estimations of BOLD response magnitudes ('betas'). GLMsingle requires only fMRI time series data and a design matrix as inputs, integrating three techniques to enhance the accuracy of experimental GLM beta estimates. Firstly, for each voxel, a custom HRF is identified from a library of candidate functions. Secondly, cross-validation is utilized to derive a set of noise regressors from voxels unrelated to the experimental paradigm. Thirdly, to improve the stability of beta estimates for closely spaced trials, ridge regression is employed on a voxel-wise basis to regularize the betas. In this study, three betas were calculated by analyzing the BOLD response corresponding to individual video onset ranging from 1 to 3 seconds with 1-second intervals. We produced individual GLMsingle models for each session (consisted of 4 training runs and 2 test runs). In general, for each video within the training set, 2 (repetitions) x 3 (seconds) betas were acquired. Similarly, for each video within the testing set, 10 (repetitions) x 3 (seconds) betas were acquired. The utilization of repetitions enabled us to acquire video-evoked responses with a high signal-to-noise ratio (SNR).<br><br>The regressors in the GLMsingle toolbox mainly include the following categories:<br>1.Experimental Design Matrix: This is constructed based on the experimental design, including experimental conditions, task events, etc., and is used to estimate the BOLD (Blood - Oxygen - Level - Dependent) signal response.<br>2.Data-driven nuisance regressors: The data-driven nuisance regressors used in the GLMdenoise technique. These are identified by analyzing the data itself and are used to remove noise and improve the accuracy of beta estimation.<br>3.Physiological noise regressors: May include indicators of physiological signals such as heart rate and respiration, and are used to correct the influence of physiological noise on the BOLD signal.<br>4.Movement parameter regressors: Usually include head movement parameters, such as translation and rotation, and are used to correct signal changes caused by head movement.<br>5.Polynomial regressors: Polynomial terms used to o characterize the baseline signal level.<br><br><br>**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/GLMsingle-main-experiment/matlab/NFED_GLMsingle.m**<br><br>#### retinotopic mapping<br><br>The fMRI data from the the population receptive field experiment were analyzed by a pRF model implemented in the analyzePRF toolbox (http://cvnlab.net/analyzePRF/) to characterize individual retinotopic representation. Make sure to download required software mentioned in the code.<br><br>**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/Functional-localizer-experiment-analysis/s4a_analysis_prf.m**<br><br>#### fLoc experiment<br><br>We used GLMdenoise,a data-driven denoising method,to analyze the pre-processed fMRI data from the fLoc experiment.We used a "condition-split" strategy to code the 10 stimulus categories, splitting the trials related to each category into individual conditions in each run. Six response estimates (beta values) for each category were produced by using six condition-splits.To quantify selectivity for various categories and domains,we computed t-values using the GLM beta values after fitting the GLM.The regions of interest with category selectivity for each participant were defined by using the resulting maps.<br>**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/Functional-localizer-experiment-analysis/s4a_analysis_floc.m**<br><br><br>## Validation<br>### Basic quality control<br>The fundamental quality control suggests that the data displays good quality. To evaluate the quality of the structural data obtained from NFED, we employed four crucial metrics:Coefficient of Joint Variation(CJV), Contrast-to-Noise Ratio (CNR), Signal-to-Noise Ratio in Grey Matter (SNR_GM), and Signal-to-Noise Ratio in White Matter (SNR_WM). Specifically, the CJV is between white matter(WM) and grey matter(GM).The CNR assesses the relationship between the contrast of GM and WM with the noise present in the image. The SNR assesses the connection between the mean signal measurements and the noise present in the image. The SNR assessment is conducted individually for GM and WM.For quality control of the NFED’s functional scans, we assessed the amount of head motion for each participant and the temporal signal-to-noise ratio (tSNR) of the time-series data, separately.<br>**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/validation/T1_Image-quality-metrics/T1data/noiseeval/cal_SNRindex.m**<br>**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/validation/FD/FD.py**<br>**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/validation/tSNR/tSNR.py**<br>### The visual cortex exhibits reliable BOLD responses to natural facial expression videos stimuli in main experiment.<br>In the main experiment, there were 5 participants who accomplished 10 sessions, each consisting of of 4 training runs and 2 test runs, making a total of 60 runs(40 training and 20 test runs) in all. For each run, 30 (videos)*2 (repetitions) x 3 (seconds) betas were acquired. Z-score normalization was performed on the raw betas at each voxel for each run.The betas were then averaged across stimulus repetitions to generate a vector of betas. In general, 44 runs (40 training runs and 4 test runs)*90 betas were acquired. Hence, the test-retest reliability of responses to these videos in main experiment were evaluated through computing the Pearson correlation between the 90 betas obtained from the even runs and odd runs on each vertex.<br>**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/validation/reliability_face/reliability_face.py**<br>### noise celling<br>The code are available at https://openneuro.org/datasets/ds005047/validation/code/noise_celling/sub-xx" store the intermediate files required for running the program<br>**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/noise_celling/Noise_Ceiling.py**<br><br>### Correspondence between human brain and DCNN<br>The code are available at https://openneuro.org/datasets/ds005047. We combined the data from main experiment and functional localizer experiments to build an encoding model to replicate the hierarchical correspondences of representation between the brain and the DCNN. The encoding models were built to map artificial representations from each layer of the pre-trained VideoMAEv2 to neural representations from each area of the human visual cortex as defined in the multimodal parcellation atlas.<br>**code: https://openneuro.org/datasets/ds005047/derivatives/validation/code/dnnbrain/**<br><br>### Semantic metadata of action and expression labels reveal that NFED can encode temporal and spatial stimuli features in the brain<br>The code are available at https://openneuro.org/datasets/ds005047/validation/code/semantic_metadata/xx_xx_semantic_metadata" store the intermediate files required for running the program.<br>**code:https://openneuro.org/datasets/ds005047/derivatives/validation/code/semantic_metadata/**<br><br>## results<br> The results can be viewed at "https://openneuro.org/datasets/ds005047/derivatives/validation/results/brain_map_individual".<br><br>## Whole-brain mapping<br>The whole-brain data mapped to the cerebral cortex as obtained from the technical validation.<br>**code: https://openneuro.org/datasets/ds005047/derivatives/validation/results/show_results_allbrain/Showresults.m**<br><br>## Mannually prepared environment<br>We provide the *requirements.txt* to install python packages used in these codes. However, some packages like *GLM* and *pre-processing* require external dependecies and we have provided the packages in the corresponding file.<br><br>## stimuli<br>The video stimuli used in the NFED experiment are saved in the "stimuli" folders.<br><br></pre>
Video clips related to the paper "Humanoid facial expressions as a tool to study human behavior
<p>During the experiment, participants observed video-clips of the iCub robot performing a gentle or rude arm movement (giving request) with a happy or angry facial expressions.</p>
Data from: Functional decline in facial expression generation in older women: a cross-sectional study using three-dimensional morphometry
Open the record for dataset details and reuse information.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.