Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
18
datasets available to search
ShareScore release 0.9.0
Dataset results
18 results for “Genea”
GENEA Challenge 2023 Human-Likeness subjective evaluation data
<p>This repository contains human-likeness user-study response data and associated analyses from the GENEA Challenge 2023. The code was written by Gustav Eje Henter and is released under a Creative Commons Attribution 4.0 International (CC BY 4.0) license.</p> <p><strong>The archive contains:</strong><br> * This README.txt file.<br> * A LICENSE.txt file.<br> * A JSON file containing all human-likeness user-study responses included in the analyses.<br> * MATLAB .m files for reproducing the above analyses (both tables and figures).<br> * A folder called "output" that contains all files written by the analysis script, including .mat files with all the results computed by each analysis, as well as files created when compiling the final, pretty-looking pdf figures.<br> * A file output.txt that contains all output written to the MATLAB Command Window when running the analyses.</p> <p><strong>Requirements:</strong><br> * MATLAB and MATLAB Image Processing Toolbox<br> * distinguishable_colors.m by Timothy E. Holy (https://www.mathworks.com/matlabcentral/fileexchange/29702-generate-maximally-perceptually-distinct-colors)<br> * fragmaster by Augustin Martin Domingo and Tilman Vogel (https://ctan.org/pkg/fragmaster), a LaTeX utility<br> This was last tested on 2023-10-08 on MATLAB version 9.10.0.1739362 (R2021a) Update 5, running on macOS version 13.2 build 22D49.</p> <p><strong>To re-generate/reproduce</strong> the contents in the "output" folder:<br> 0. Ensure that all requirements are met.<br> 1. Run through the entire analysis script "run_analyses.m".<br> 2. Run the fragmaster command in the "output" folder.</p> <p><strong>Attribution:</strong><br> If you use this material in a scientific publication, please cite our latest paper on the GENEA Challenge 2023. At the time of writing (2023-10-08) this is our ACM ICMI 2023 paper. You can use the following BibTeX code to cite that work:<br> @inproceedings{kucherenko2023genea,<br> author={Kucherenko, Taras and Nagy, Rajmund and Yoon, Youngwoo and Woo, Jieyeon and Nikolov, Teodor and Tsakov, Mihail and Henter, Gustav Eje},<br> title={The {GENEA} {C}hallenge 2023: {A} large-scale evaluation of gesture generation models in monadic and dyadic settings},<br> booktitle = {Proceedings of the ACM International Conference on Multimodal Interaction},<br> publisher = {ACM},<br> series = {ICMI '23},<br> year={2023}<br> }</p> <p><strong>To find more</strong> GENEA Challenge 2023 material on the web, please see:<br> * https://svito-zar.github.io/GENEAchallenge2023/<br> * https://genea-workshop.github.io/2023/challenge/</p> <p><strong>If you have any questions</strong> or comments, please contact:<br> * Gustav Eje Henter <ghe@kth.se><br> * The GENEA Challenge organisers <genea-challenge@googlegroups.com></p>
GENEA Challenge 2022 subjective evaluation data
<p>This zip archive contains user-study response data and associated analyses and analysis results from the GENEA Challenge 2022 (<a href="https://youngwoo-yoon.github.io/GENEAchallenge2022/">https://youngwoo-yoon.github.io/GENEAchallenge2022/</a>). The code was written by Gustav Eje Henter and is released under a Creative Commons Attribution 4.0 International license.</p> <p>If you use this material, please cite the following paper:</p> <p>Youngwoo Yoon, Pieter Wolfert, Taras Kucherenko, Carla Viegas, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2022. The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI '22). ACM</p>
GENEA Challenge 2022 submitted BVH files
<p>This Zenodo repository contains 3D motion in the Biovision Hierarchy (BVH) format for all test-set motion submitted by teams participating in the GENEA Challenge 2022.</p> <p> </p> <p>Contents:</p> <p>The "full-body" zip file corresponds to the full-body tier of the challenge and the "upper-body" file to the upper-body tier.</p> <p>The repository also contains the time stamps for cutting out the evaluation segments we used (matched and mismatched) in the file "segment_selection_info.csv".</p> <p> </p> <p>Attribution:</p> <p>If you use this material, please cite our latest paper on the GENEA Challenge 2022. At the time of writing (2022-08-08) this is our ACM ICMI 2022 paper:</p> <p>Youngwoo Yoon, Pieter Wolfert, Taras Kucherenko, Carla Viegas, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2022. The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI '22). ACM.</p> <p>You can find the latest information and a BibTeX file on the project website:</p> <p><a href="https://youngwoo-yoon.github.io/GENEAchallenge2022/">https://youngwoo-yoon.github.io/GENEAchallenge2022/</a></p> <p> </p> <p>Conditions FNA and UNA in the data contain motion from the Talking With Hands 16.2M dataset at <a href="https://github.com/facebookresearch/TalkingWithHands32M/">https://github.com/facebookresearch/TalkingWithHands32M/</a>. These are licensed under a CC BY NC 4.0 international license. The remaining material is available under a CC BY 4.0 international license, with the text provided in LICENSE.txt.</p> <p> </p> <p>To find more GENEA Challenge 2022 material on the web, please see:</p> <p>*<a href="https://youngwoo-yoon.github.io/GENEAchallenge2022/"> https://youngwoo-yoon.github.io/GENEAchallenge2022/</a></p> <p>*<a href="https://genea-workshop.github.io/2022/challenge/"> https://genea-workshop.github.io/2022/challenge/</a></p> <p> </p> <p>If you have any questions or comments, please contact:</p> <p>* The GENEA Challenge & Workshop organisers <genea-contact@googlegroups.com</p>
GENEA Challenge 2022 objective evaluation data
<p>This Zenodo repository contains objective evaluation results for all test-set motion submitted by teams participating in the GENEA Challenge 2022.</p> <p> </p> <p>We caution the user that objective metrics are known to have poor correlation with actual, perceived motion quality. For definitions and explanations of the "Average jerk magnitude", "Average acceleration magnitude", and "Average Hellinger distance", please see the paper "Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation" by Kucherenko et al., published in the International Journal of Human–Computer Interaction in 2021. For a definition and explanation of the "Canonical correlation analysis (CCA) coefficient", see the paper "Speech-driven animation with meaningful behaviors" by <a href="https://www.sciencedirect.com/science/article/abs/pii/S0167639318300013?via%3Dihub#!">Sadoughi and Busso</a>, published in <a href="https://www.sciencedirect.com/journal/speech-communication">Speech Communication</a> in 2019. Code for computing these objective metrics is available through the challenge webpage.</p> <p> </p> <p>Attribution:</p> <p>If you use this material, please cite our latest paper on the GENEA Challenge 2022. At the time of writing (2022-08-10) this is our ACM ICMI 2022 paper:</p> <p>Youngwoo Yoon, Pieter Wolfert, Taras Kucherenko, Carla Viegas, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2022. The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI '22). ACM.</p> <p>You can find the latest information and a BibTeX file on the project website:</p> <p><a href="https://youngwoo-yoon.github.io/GENEAchallenge2022/">https://youngwoo-yoon.github.io/GENEAchallenge2022/</a></p> <p> </p> <p>The material is available under a CC BY 4.0 international license, with the text provided in LICENSE.txt.</p> <p> </p> <p>To find more GENEA Challenge 2022 material on the web, please see:</p> <p>*<a href="https://youngwoo-yoon.github.io/GENEAchallenge2022/"> https://youngwoo-yoon.github.io/GENEAchallenge2022/</a></p> <p>*<a href="https://genea-workshop.github.io/2022/challenge/"> https://genea-workshop.github.io/2022/challenge/</a></p> <p> </p> <p>If you have any questions or comments, please contact:</p> <p>* The GENEA Challenge & Workshop organisers <genea-contact@googlegroups.com</p>
GENEA Challenge 2023 user-study video stimuli
<p>This Zenodo repository contains video stimuli in mp4 format from the user studies in the GENEA Challenge 2023.</p> <p> </p> <p><strong>Contents:</strong></p> <p>The file "monadic_videos.zip" contains the video stimuli used in the monadic evaluations of the challenge (human-likeness and speech appropriateness) and the file "dyadic_videos.zip" the video stimuli from the dyadic evaluation (appropriateness for the interlocutor).</p> <p>Video stimuli with mismatched motion are in the corresponding subfolders with the prefix "mismat_".</p> <p>The file "attention_check_examples.zip" contains examples of the attention-check videos used in the challenge.</p> <p> </p> <p>The release contains all videos used for the challenge evaluation, except for the attention-checks, where only a few examples are provided. Except for the attention-check examples for the human-likeness studies, all videos contain speech audio. This audio needs to be removed to replicate the human-likeness evaluation.</p> <p> </p> <p><strong>Attribution:</strong></p> <p>If you use this material, please cite our latest paper on the GENEA Challenge 2023. At the time of writing (2023-08-01) this is our ACM ICMI 2023 paper:</p> <p>Taras Kucherenko, Rajmund Nagy, Youngwoo Yoon, Jieyeon Woo, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2023. The GENEA Challenge 2023: A large-scale evaluation of gesture generation models in monadic and dyadic settings. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI ’23). ACM.</p> <p>Also, please cite the paper about the original dataset from Meta Research:</p> <p>Gilwoo Lee, Zhiwei Deng, Shugao Ma, Takaaki Shiratori, Siddhartha S. Srinivasa, and Yaser Sheikh. 2019. Talking With Hands 16.2M: A large-scale dataset of synchronized body-finger motion and audio for conversational motion analysis and synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV ’19). IEEE, 763–772.</p> <p> </p> <p>Condition NA in the data contains motion from the Talking With Hands 16.2M dataset at <a href="https://github.com/facebookresearch/TalkingWithHands32M/">https://github.com/facebookresearch/TalkingWithHands32M/</a>. These stimuli, and the audio, are licensed under a CC BY NC 4.0 international license. The motion for all other conditions is released under a CC BY 4.0 international license, whose license text is reproduced in the file LICENSE.txt.</p> <p> </p> <p><strong>More information:</strong></p> <p>To find more GENEA Challenge 2023 material on the web, please see:</p> <ul> <li> <p><a href="https://genea-workshop.github.io/2023/challenge/">https://genea-workshop.github.io/2023/challenge/</a></p> </li> </ul> <p>If you have any questions or comments, please contact:</p> <ul> <li> <p>The GENEA Challenge organisers <genea-challenge@googlegroups.com></p> </li> </ul>
GENEA Challenge 2022 3D coordinates of submitted motion
<p>This Zenodo repository contains Cartesian 3D coordinates in npy format for all test-set motion submitted by teams participating in the GENEA Challenge 2022. The npy format is easily readable using the NumPy package in Python.</p> <p> </p> <p>Attribution:</p> <p>If you use this material, please cite our latest paper on the GENEA Challenge 2022. At the time of writing (2022-08-04) this is our ACM ICMI 2022 paper:</p> <p>Youngwoo Yoon, Pieter Wolfert, Taras Kucherenko, Carla Viegas, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2022. The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI '22). ACM.</p> <p>You can find the latest information and a BibTeX file on the project website:</p> <p><a href="https://youngwoo-yoon.github.io/GENEAchallenge2022/">https://youngwoo-yoon.github.io/GENEAchallenge2022/</a></p> <p> </p> <p>Conditions FNA and UNA in the data contain motion from the Talking With Hands 16.2M dataset at <a href="https://github.com/facebookresearch/TalkingWithHands32M/">https://github.com/facebookresearch/TalkingWithHands32M/</a>. These are licensed under a CC BY NC 4.0 international license. The remaining material is available under a CC BY 4.0 international license, with the text provided in LICENSE.txt.</p> <p> </p> <p>To find more GENEA Challenge 2022 material on the web, please see:</p> <p>*<a href="https://youngwoo-yoon.github.io/GENEAchallenge2022/"> https://youngwoo-yoon.github.io/GENEAchallenge2022/</a></p> <p>*<a href="https://genea-workshop.github.io/2022/challenge/"> https://genea-workshop.github.io/2022/challenge/</a></p>
GENEA Challenge 2022 3D coordinates of submitted motion
<p>This Zenodo repository contains Cartesian 3D coordinates in npy format for all test-set motion submitted by teams participating in the GENEA Challenge 2022. The npy format is easily readable using the NumPy package in Python.</p> <p> </p> <p>Attribution:</p> <p>If you use this material, please cite our latest paper on the GENEA Challenge 2022. At the time of writing (2022-08-04) this is our ACM ICMI 2022 paper:</p> <p>Youngwoo Yoon, Pieter Wolfert, Taras Kucherenko, Carla Viegas, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2022. The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI '22). ACM.</p> <p>You can find the latest information and a BibTeX file on the project website:</p> <p><a href="https://youngwoo-yoon.github.io/GENEAchallenge2022/">https://youngwoo-yoon.github.io/GENEAchallenge2022/</a></p> <p> </p> <p>Conditions FNA and UNA in the data contain motion from the Talking With Hands 16.2M dataset at <a href="https://github.com/facebookresearch/TalkingWithHands32M/">https://github.com/facebookresearch/TalkingWithHands32M/</a>. These are licensed under a CC BY NC 4.0 international license. The remaining material is available under a CC BY 4.0 international license, with the text provided in LICENSE.txt.</p> <p> </p> <p>To find more GENEA Challenge 2022 material on the web, please see:</p> <p>*<a href="https://youngwoo-yoon.github.io/GENEAchallenge2022/"> https://youngwoo-yoon.github.io/GENEAchallenge2022/</a></p> <p>*<a href="https://genea-workshop.github.io/2022/challenge/"> https://genea-workshop.github.io/2022/challenge/</a></p>
GENEA Challenge 2022 User Study Raw Responses
<p>This archive contains user-study response data in JSON format from the GENEA Challenge 2022 (https://youngwoo-yoon.github.io/GENEAchallenge2022/). Included in this data are the raw text responses on the questionnaire that was presented to participants after finishing the evaluation.</p> <p>If you use this material, please cite the following paper:</p> <p>Youngwoo Yoon, Pieter Wolfert, Taras Kucherenko, Carla Viegas, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2022. The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI '22). ACM</p> <p> </p>
GENEA Challenge 2022 Dataset Files
<p>This Zenodo repository contains the main dataset for the GENEA 2022 challenge, which is based on the Talking With Hands 16.2M data.</p> <p><strong>Contents:</strong></p> <p>The "trn" and "val" zip files contain audio files (in WAV format), time-aligned transcriptions (in TSV format), and motion files (in BVH format) for the training and validation datasets, respectively.</p> <p>The "tst" zip file contains audio files (in WAV format) and transcriptions (in TSV format) for the test set, but no motion. The corresponding test motion is available at:</p> <p>https://doi.org/10.5281/zenodo.6976463</p> <p>Each zip file also contains a "metadata.csv" file that contains information for all files regarding the speaker ID and whether or not the motion files contain finger motion.</p> <p>Note that the speech audio in the data sometimes has been replaced by silence for the purpose of anonymisation.<br> <br> <strong>Data processing scripts:</strong></p> <p>We provide a number of optional scripts for encoding and processing the challenge data:</p> <p>Audio: Scripts for extracting basic audio features, such as spectrograms, prosodic features, and mel-frequency cepstral coefficients (MFCCs) can be found <a href="https://github.com/genea-workshop/Speech_driven_gesture_generation_with_autoencoder/blob/GENEA_2022/data_processing/tools.py#L105">at this link</a>.</p> <p>Text: A script to encode text transcriptions to word vectors using FastText is available. <a href="https://gist.github.com/youngwoo-yoon/c9e5d5b04994478011911f88eb15caa4"><code>tsv2wordvectors.py</code></a></p> <p>Motion: If you wish to encode the joint angles from the BVH files to and from an exponential map representation, you can use scripts by <a href="https://github.com/simonalexanderson">Simon Alexanderson</a> based on the <a href="https://omid.al/projects/pymo/">PyMo library</a>, which are available here:</p> <ul> <li><a href="https://github.com/genea-workshop/Speech_driven_gesture_generation_with_autoencoder/blob/GENEA_2022/data_processing/bvh2features.py"><code>bvh2features.py</code></a></li> <li><a href="https://github.com/genea-workshop/Speech_driven_gesture_generation_with_autoencoder/blob/GENEA_2022/data_processing/features2bvh.py"><code>features2bvh.py</code></a></li> </ul> <p><strong>Attribution:</strong></p> <p>If you use this material, please cite our latest paper on the GENEA Challenge 2022. At the time of writing (2022-08-16), that is our ACM ICMI 2022 paper:</p> <p>Youngwoo Yoon, Pieter Wolfert, Taras Kucherenko, Carla Viegas, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2022. The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI '22). ACM.</p> <p>You can find the latest information and a BibTeX file on the project website:</p> <p>https://youngwoo-yoon.github.io/GENEAchallenge2022/</p> <p>Also cite the paper about the original dataset from Meta Research:</p> <p>Gilwoo Lee, Zhiwei Deng, Shugao Ma, Takaaki Shiratori, Siddhartha S. Srinivasa, and Yaser Sheikh. 2019. Talking With Hands 16.2M: A large-scale dataset of synchronized body-finger motion and audio for conversational motion analysis and synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV ’19). IEEE, 763–772.</p> <p> </p> <p>The motion and audio files are based on the Talking With Hands 16.2M dataset at https://github.com/facebookresearch/TalkingWithHands32M/. All material is available under a CC BY NC 4.0 international license, with the text provided in LICENSE.txt.</p> <p> </p> <p>To find more GENEA Challenge 2022 material on the web, please see:</p> <p>* <a href="https://youngwoo-yoon.github.io/GENEAchallenge2022/">https://youngwoo-yoon.github.io/GENEAchallenge2022/</a></p> <p>* <a href="https://genea-workshop.github.io/2022/challenge/">https://genea-workshop.github.io/2022/challenge/</a></p> <p>If you have any questions or comments, please contact:</p> <p>* The GENEA Challenge & Workshop organisers <genea-contact@googlegroups.com</p>
FIGURES 29‑30 in Revision Of Neotropical Genea Rondani (Diptera, Tachinidae, Tachininae, Leskiini)
FIGURES 29‑30: Genea tenuirostris (James), 29. Head, lateral view; 30. Wing, dorsal view. Figures 31-35: Genea trifaria (Wiedemann), 31. Head, lateral view; 32. Wing, dorsal view; 33. Epandrium, cerci and surstyli, posterior view; 34. Epandrium, cerci and surstyli, lateral view; 35. Aedegal complex, lateral view.
FIGURES 12‑16 in Revision Of Neotropical Genea Rondani (Diptera, Tachinidae, Tachininae, Leskiini)
FIGURES 12‑16: Genea jaynesi (Townsend), 12. Head, lateal view; 13. Wing, dorsal view; 14. Epandrium, cerci and surstyli, posterior view; 15. Epandrium, cerci and surstyli, lateral view; 16. Aedegal complex, lateral view. Figures 17-21: Genea major (Townsend), 17. Head, lateral view; 18. Sternite 5; 19. Epandrium, cerci and surstyli, posterior view; 20. Epandrium, cerci and surstyli, lateral view; 21. Aedegal complex, lateral view.
FIGURES 22‑26 in Revision Of Neotropical Genea Rondani (Diptera, Tachinidae, Tachininae, Leskiini)
FIGURES 22‑26: Genea paulistana sp. nov. 22. Head, lateral view; 23. Sternite 5; 24. Epandrium, cerci and surstyli, posterior view; 25. Epandrium, cerci and surstyli, lateral view; 26. Aedegal complex, lateral view. Figures 27-28: Genea pellucens (Curran), 27. Head, lateral view; 28. Wing, dorsal view.
FIGURES 1‑5 in Revision Of Neotropical Genea Rondani (Diptera, Tachinidae, Tachininae, Leskiini)
FIGURES 1‑5: Genea australis (Townsend) 1. Head, lateral view; 2. Wing, dorsal view; 3. Epandrium, cerci and surstylus, posterior view; 4. Epandrium, cerci and surstyli, lateral view 5. Aedegal complex, lateral view. Figures 6-7: Genea brasiliensis (Townsend) 6. Head, lateral view; 7. Abdomen, dorsal view. Figures 8-11: Genea gracilis James 8. Head, lateral view; 9. Epandrium, cerci and surstyli, posterior view; 10. Epandrium, cerci and surstyli, lateral view; 11. Aedegal complex, lateral view.
GENEA Challenge 2023 submitted BVH files
<p>This Zenodo repository contains 3D motion in the Biovision Hierarchy (BVH) format for all test-set motion submitted by teams participating in the GENEA Challenge 2023.</p> <p> </p> <p><strong>Contents:</strong></p> <p>The "genea_2023_test_bvh" zip file corresponds to the BVH files themselves (2GB).</p> <p>The repository also contains the time stamps for cutting out the evaluation segments we used (matched and mismatched) for the two kinds of studies we conducted (monadic and dyadic) in the files "monadic_segment_selection_info.csv" and "dyadic_segment_selection_info.csv".</p> <p> </p> <p><strong>Attribution:</strong></p> <p>If you use this material, please cite our latest paper on the GENEA Challenge 2023. At the time of writing (2023-07-11) this is our ACM ICMI 2023 paper:</p> <p>Taras Kucherenko, Rajmund Nagy, Youngwoo Yoon, Jieyeon Woo, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2023. The GENEA Challenge 2023: A large-scale evaluation of gesture generation models in monadic and dyadic settings. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI ’23). ACM.</p> <p> </p> <p>Condition NA in the data contains motion from the Talking With Hands 16.2M dataset at <a href="https://github.com/facebookresearch/TalkingWithHands32M/">https://github.com/facebookresearch/TalkingWithHands32M/</a>. These are licensed under a CC BY NC 4.0 international license. The remaining material is available under a CC BY 4.0 international license, with the text provided in LICENSE.txt.</p> <p> </p> <p>To find more GENEA Challenge 2023 material on the web, please see:</p> <p>* <a href="https://genea-workshop.github.io/2023/challenge/">https://genea-workshop.github.io/2023/challenge/</a></p> <p> </p> <p>If you have any questions or comments, please contact:</p> <p>* The GENEA Challenge organisers <genea-challenge@googlegroups.com</p>
GENEA Challenge 2023 Dataset Files
<p>This Zenodo repository contains the main dataset for the GENEA Challenge 2023, which is based on the Talking With Hands 16.2M dataset.</p> <p><strong>Notation:</strong></p> <p>Please take note of the following nomenclature when reading this document:</p> <ul> <li> <p><em>main agent</em> refers to the speaker in the dyadic interaction for which the systems generated motions. </p> </li> </ul> <ul> <li> <p><em>interlocutor</em> refers to the speaker in front of the main agent. </p> </li> </ul> <p><strong>Contents:</strong></p> <p>The “genea2023_trn" and "genea2023_val" zip files contain audio files (in WAV format), time-aligned transcriptions (in TSV format), and motion files (in BVH format) for the training and validation datasets, respectively.</p> <p>The "genea2023_test" zip file contains audio files (in WAV format) and transcriptions (in TSV format) for the test set, but no motion. The corresponding test motion is available at:</p> <p><a href="https://zenodo.org/record/8146028">https://zenodo.org/record/814602</a><a href="https://zenodo.org/record/8146027">7</a></p> <p>Each zip file also contains a "metadata.csv" file that contains information for all files regarding the speaker ID and whether or not the motion files contain finger motion.</p> <p>Note that the speech audio in the data sometimes has been replaced by silence for the purpose of anonymisation.</p> <p>In the test set, files with indices from 0 to 40 correspond to "matched" interactions (the core test set), where main agent and interlocutor data come from the same conversation, whilst file indices from 41 to 69 correspond to "mismatched" interactions (the extended test set), where main agent and interlocutor data come from different conversations.</p> <p><strong>Folder structure:</strong></p> <ul> <li>main-agent/ (main agent): Encapsulates BVH, TSV, WAV data subfolders for the main agent.</li> <li>interloctr/ (interlocutor): Encapsulates BVH, TSV, WAV data subfolders for the interlocutor.</li> <li>bvh/ (motion): Time-aligned 3D full-body motion-capture data in BVH format from a speaking and gesticulating actor. Each file is a single person, but each data sample contains files for both the main agent and the interlocutor.</li> <li>wav/ (audio): Recorded audio data in WAV format from a speaking and gesticulating actor with a close-talking microphone. Parts of the audio recordings have been muted to omit personally identifiable information.</li> <li>tsv/ (text): Word-level time-aligned text transcriptions of the above audio recordings in TSV format (tab-separated values). For privacy reasons, the transcriptions do not include references to personally identifiable information, similar to the audio files.</li> </ul> <p><strong>Data processing scripts:</strong></p> <p>We provide a number of optional scripts for encoding and processing the challenge data:</p> <p>Audio: Scripts for extracting basic audio features, such as spectrograms, prosodic features, and mel-frequency cepstral coefficients (MFCCs) can be found <a href="https://github.com/genea-workshop/Speech_driven_gesture_generation_with_autoencoder/blob/GENEA_2022/data_processing/tools.py#L105">at this link</a>.</p> <p>Text: A script to encode text transcriptions to word vectors using FastText is available here: <a href="https://gist.github.com/youngwoo-yoon/c9e5d5b04994478011911f88eb15caa4">tsv2wordvectors.py</a></p> <p>Motion: If you wish to encode the joint angles from the BVH files to and from an exponential map representation, you can use scripts by <a href="https://github.com/simonalexanderson">Simon Alexanderson</a> based on the <a href="https://omid.al/projects/pymo/">PyMo library</a>, which are available here:</p> <ul> <li> <p><a href="https://github.com/genea-workshop/Speech_driven_gesture_generation_with_autoencoder/blob/GENEA_2022/data_processing/bvh2features.py">bvh2features.py</a></p> </li> <li> <p><a href="https://github.com/genea-workshop/Speech_driven_gesture_generation_with_autoencoder/blob/GENEA_2022/data_processing/features2bvh.py">features2bvh.py</a></p> </li> </ul> <p><strong>Attribution:</strong></p> <p>If you use this material, please cite our latest paper on the GENEA Challenge 2023. At the time of writing (2023-07-25) this is our ACM ICMI 2023 paper:</p> <p>Taras Kucherenko, Rajmund Nagy, Youngwoo Yoon, Jieyeon Woo, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2023. The GENEA Challenge 2023: A large-scale evaluation of gesture generation models in monadic and dyadic settings. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI ’23). ACM.</p> <p>Also, please cite the paper about the original dataset from Meta Research:</p> <p>Gilwoo Lee, Zhiwei Deng, Shugao Ma, Takaaki Shiratori, Siddhartha S. Srinivasa, and Yaser Sheikh. 2019. Talking With Hands 16.2M: A large-scale dataset of synchronized body-finger motion and audio for conversational motion analysis and synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV ’19). IEEE, 763–772.</p> <p> </p> <p>The motion and audio files are based on the Talking With Hands 16.2M dataset at <a href="https://github.com/facebookresearch/TalkingWithHands32M/">https://github.com/facebookresearch/TalkingWithHands32M/</a>. The material is available under a CC BY NC 4.0 Attribution-NonCommercial 4.0 International license, with the text provided in LICENSE.txt.</p> <p> </p> <p>To find more GENEA Challenge 2023 material on the web, please see:</p> <ul> <li><a href="https://genea-workshop.github.io/2023/challenge/">https://genea-workshop.github.io/2023/challenge/</a></li> </ul> <p> </p> <p>If you have any questions or comments, please contact:</p> <ul> <li>The GENEA Challenge organisers <genea-challenge@googlegroups.com></li> </ul>
GENEA Challenge 2022 user-study video stimuli
<p>This Zenodo repository contains user-study video stimuli in mp4 format for all test-set motion submitted by teams participating in the GENEA Challenge 2022.</p> <p> </p> <p>Contents:</p> <p>The "full-body_videos" zip file corresponds to the full-body tier of the challenge and the "upper-body_videos" file to the upper-body tier.</p> <p>Video stimuli with mismatched motion are in the corresponding folders with the prefix "mismatched_".</p> <p>The "attention_check_examples" zip file contains examples of the attention-check videos used in the challenge.</p> <p> </p> <p>The release contains all videos used for the challenge evaluation, except for the attention-checks, where only a few examples are provided. Except the attention-check examples for the human-likeness studies, all videos contain speech audio. This audio needs to be removed to replicate the human-likeness evaluation.</p> <p> </p> <p>Attribution:</p> <p>If you use this material, please cite our latest paper on the GENEA Challenge 2022. At the time of writing (2022-08-16) this is our ACM ICMI 2022 paper:</p> <p>Youngwoo Yoon, Pieter Wolfert, Taras Kucherenko, Carla Viegas, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2022. The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI '22). ACM.</p> <p>You can find the latest information and a BibTeX file on the project website:</p> <p><a href="https://youngwoo-yoon.github.io/GENEAchallenge2022/">https://youngwoo-yoon.github.io/GENEAchallenge2022/</a></p> <p> </p> <p>Conditions FNA and UNA in the data contain motion from the Talking With Hands 16.2M dataset at <a href="https://github.com/facebookresearch/TalkingWithHands32M/">https://github.com/facebookresearch/TalkingWithHands32M/</a>. These are licensed under a CC BY NC 4.0 international license. The remaining material is available under a CC BY 4.0 international license, with the license text provided in LICENSE.txt.</p> <p> </p> <p>To find more GENEA Challenge 2022 material on the web, please see:</p> <p>*<a href="https://youngwoo-yoon.github.io/GENEAchallenge2022/"> https://youngwoo-yoon.github.io/GENEAchallenge2022/</a></p> <p>*<a href="https://genea-workshop.github.io/2022/challenge/"> https://genea-workshop.github.io/2022/challenge/</a></p> <p> </p> <p>If you have any questions or comments, please contact:</p> <p>* The GENEA Challenge & Workshop organisers <genea-contact@googlegroups.com></p>
GENEA Challenge 2020 user-study video stimuli
<p>This Zenodo repository contains all video stimuli used in the two crowdsourced user studies (appropriateness and human-likeness) of the GENEA Challenge 2020.</p>
GENEA Challenge 2020 3D coordinates of submitted motion
<p>This Zenodo repository contains all 3D coordinates for the all the motions submitted by the participants of the GENEA Challenge 2020.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.